The amount of open source Application Performance Monitoring (APM) tools has grown dramatically in the last four years. The following eight tools appeared publicly starting from early 2014:
If you miss some Monitoring tools in this overview, please note the following:
- With APM tools we refer to tools that allow you to analyze individual transaction traces. With „transaction trace“ we refer to the complete transaction execution flow including a timing breakdown for internal application components and external systems such as databases.
- System monitoring tools that just gather and display aggregated data (e.g., response time or resource utilization statistics) such as Grafana, Prometheus, or ZMON are not analyzed.
- Open source means not just „free“ but also that the code is publicly available for adaptations.
- Hawkular APM is not mentioned as they discontinued their own solution in favor of Jaeger (http://www.hawkular.org/hawkular-apm/)
If we have missed an open source APM tool that fits these criteria and was open-sourced since 2014, please let us know at info@retit.de!
Reasons for the growth of Open Source APM Tools
This huge growth of open source APM tools (and thus commoditization of APM) is driven by multiple trends:
- The overall complexity increase in modern software systems and the resulting need for real time views on the overall system behaviour. Especially as organizations move to more loosly coupled (micro-)services which need to interact with each other in ways that might not be obvious at the time of development or deployment.
- The growing importance of IT for the business due to which downtimes or bad software performance has a direct impact on revenue.
- The development of tracing standards which allow to easily exchange the tracing tool in use.
- The movement of commercial APM vendors from „2nd Gen“ to „3rd Gen“ features, which means that they see tracing as a commodity and not as a main competitive advantage anymore and thus focus on advanced features such as automatic root cause analysis and anomaly detection.
The main standards that helped to increase the community contributions to these tools are OpenTracing and OpenCensus:
Especially OpenTracing helped to build a community of tracing providers that can be used by multiple tools. OpenCensus is driven by Google and provides more features than OpenTracing, but we will dive into this topic a bit more in an upcoming blog post.
Another aspect that helped during the development of the aforementioned tools is a paper by some Google employees from 2010 called: „Dapper, a Large-Scale Distributed Systems Tracing Infrastructure (2010)“ (https://research.google.com/pubs/pub36356.html). This paper outlines how they have build a tracing system within Google and lays the foundation for at least three out of the eight aforementioned tools. Pinpoint, Jaeger and Zipkin officially state that they have build their tools based on the concepts described in this paper. In this regard it is also important to mention that Jaeger is again based on Zipkin (more details about Jaeger’s architecture can be found here https://eng.uber.com/distributed-tracing/).
Choosing an Open Source APM Tool
So how do you choose one of these tools for your own requirements? And what would be reasons to rather choose a commercial, closed-source alternative? In this blog series we will try to answer some of these questions for you.
For each of these tools we will look at the following dimensions:
- Age
- Popularity
- Contributers
- Supported Technologies
- Setup Effort
- Standards Support
- Integration Capabilities
- License
In order to already give you a sense of how these tools compare with each other, we will cover the age, popularity and contributer dimensions in this overview post. The remaining dimensions (Supported Technologies, Setup Effort, Standards Support, Integration Capabilities, License) will be covered in an individual blog post for each tool. In the next section we will start with the tool age in order to give you an idea of how many years each tool has been (publicly) available already.
Open Source APM Tool Age
In the following figure you can see the year in which each of these tools appeared on the market. This means, that the years below do not denote the year when the open source project was announced, but the year in which we could find the first release tag in their respective Github repositories. It especially noticeable that the standards creation in 2016 followed immediately after the first wave of open source tools in 2014 and 2015:
Open Source APM Tool Popularity
In order to get a sense of the popularity of each tool we looked at two metrics – Github stars and Google Trends. First, lets look at the Github stars:
In this chart it is clearly visible that the Dapper-based tools Zipkin, Pinpoint and Jaeger are far more popular than the others. When looking at the Google Trends this picture is confirmed – at least for Zipkin, as we could not include Pinpoint and Jaeger in the Search terms as they are too generic for a fair comparison:
Open Source APM Tool Contributers
The popularity is also supported by the amount of contributers for each of these tools. Except for InspectIT and Elastic APM we see that InspectIT is more popular while having less contributers than Elastic APM. However, for Elastic APM it is important to note that Elastic APM was created based on an acquisition of a company called opBeat in 2017 (https://techcrunch.com/2017/06/22/elastic-enters-apm-space-with-opbeat-acquisition/) and was just recently released publicly.
When looking at these graphs it seems as if Zipkin, Pinpoint and Jaeger give the other tools a hard time but when you look at these tools you also need to notice that some of those only try to focus on a specific niche and some are fairly new and are more recent runner ups that are currently catching up.
As you should not only draw your conclusions from such popularity metrics, we will give each of these tools a more detailed review in the following articles of this blog series.
Next Articles in this Blog Series
In the following Blog posts of this series we will dig into the following topics. We will link each article here once it becomes available:
- Open APM Standards (OpenTracing, OpenCensus)
- Open Source APM Tool Overview – ZipKin
- Open Source APM Tool Overview – Pinpoint
- Open Source APM Tool Overview – Jaeger
- Open Source APM Tool Overview – Apache Skywalking
- Open Source APM Tool Overview – AppDash
- Open Source APM Tool Overview – Stagemonior
- Open Source APM Tool Overview – InspectIT
- Open Source APM Tool Overview – Elastic APM
- APM Tools Open Source vs. Commercial – Reasons for commercial alterantives
