You probably already know that Kubernetes is the leading container orchestration system. And according to the most recent CNCF study, you’re likely already using it for production workloads or considering it for the year ahead. The 2021 study found that an astounding 96% of respondents are using Kubernetes or are planning to use it in the near future — and that 69% of respondents are using Kubernetes in production today.
Kubernetes offers many benefits for both large organizations and small ones: it improves developer productivity, reduces costs, increases efficiency, and ultimately leads to a better experience for end-users.
From our partners:
But while Kubernetes has many advantages, it comes with some challenges, too. Implementing a comprehensive monitoring stack is an important early step for teams running workloads on K8s. In this post, we’ll explore four open-source tools and technologies that you can use to reduce downtime, troubleshoot more efficiently, and get a full picture of everything happening inside a cluster.
4 Open-Source Tools and Technologies
The Cloud Native Computing Foundation, or CNCF, has incubated and graduated a number of amazing technologies and tools used for monitoring and observability. Four of these tools and technologies stand out as especially helpful and can be leveraged by organizations of all sizes. Let’s jump in.
Prometheus | Metrics and Alerting
Prometheus, accepted to CNCF on May 9, 2016, is a powerful and 100% open-source tool and time-series database. With Prometheus, engineering teams are able to collect metrics and configure alerts on a large scale. Prometheus is used by nascent startups, as well as some of the largest companies in the world, such as Digital Ocean, Ericsson, and Docker.
With Prometheus, teams are able to write queries and create ad-hoc tables, graphs, and alerts using PromQL. And with Alertmanager, teams are able to use preconfigured and customizable alerts to identify important issues as they happen. Using the alerting rules, users are able to define alert conditions using the Prometheus expression language, and then send notifications to an external service.
Prometheus has a built-in toolset for visualization but is often paired with another visualization tool, such as Grafana or ContainIQ. Connecting Prometheus to a visualization tool is easy; there are plenty of prebuilt dashboards available in Grafana.
Because Prometheus has many integrations and existing exporters, bridging third-party metrics into Prometheus is very easy with the official exporters, as well as externally maintained options.
Prometheus is a CNCF Graduated project. On GitHub, Prometheus has more than 42,000 stars and contributions from more than 700 contributors.
Jaeger | Tracing
Jaeger, accepted to CNCF on September 13, 2017, is an open-source platform for distributed tracing. With Jaeger, engineers can monitor and troubleshoot with distributed transaction monitoring, or tracing. Like Prometheus, Jaeger is used by teams both large and small and was designed to be used on a massive scale. Companies like Uber use Jaeger to process billions of spans per day.
Jaeger is particularly helpful for analyzing performance and latency and for making optimizations. And with Jaeger, it is much easier to perform root cause analysis and research service dependencies. For example, Jaeger can be used to identify spikes in latency for particular microservices, including those that impact end-user experience.
Prometheus and Jaeger are often used together: Prometheus provides a toolset for detecting issues within your infrastructure, and Jaeger helps you fix them by digging into the individual requests.
Jaeger has a native UI called Jaeger Web UI, which is implemented in Javascript.
Getting started with Jaeger and Kubernetes is a straightforward process. The Jaeger Operator can be installed on a Kubernetes cluster and can be enabled for a specific namespace or across the entire cluster.
Jaeger is a CNCF Graduated project. On GitHub, Jaeger has more than 15,000 stars and contributions from more than 200 contributors.
OpenTelemetry | Standardizing Metrics, Logs, Traces
OpenTelemetry, accepted to CNCF on May 17, 2019, is a collection of tools, APIs, and SDKs that instrument, generate, collect, and export telemetry data. By using OpenTelemetry, engineers are able to collect metrics, logs, and traces, so they can dig deeper into the performance of their infrastructure and applications.
OpenTelemetry is open-source, vendor neutral, and supported by many of the largest companies in observability, as well as by the cloud providers themselves.
There are many benefits to microservices architectures, but when they are deployed at scale, it can become more difficult for engineering teams to view how services are performing and how they are affecting other services. Metrics, logs, and traces give teams a full picture of what’s happening, but gathering this data requires running, operating, and maintaining multiple agents/collectors, which can be a challenge.
OpenTelemetry solves this by standardizing the format for sending data to an observability backend, whether it be an open-source tool or a paid solution. And it removes the risk of vendor lock-in, as teams are now able to switch between backends easily with a standard format.
OpenTelemetry is a CNCF Graduated project and is on GitHub.
Guest post by Matt Lenhard, co-founder and CTO of ContainIQ
Source CNCF
For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!
Our humans need coffee too! Your support is highly appreciated, thank you!
