Monitoring Kubernetes with Panopta
Kubernetes, Google’s open-source container management platform, offers one of the most robust toolsets for facilitating container configuration and automation. However, the level of abstraction that is native to Kubernetes presents a lot of challenges to monitoring and, while the level of abstraction and auto-scaling Kubernetes offers is incredibly powerful, without proper monitoring, it can create even larger pain points for your team and customers.
When we began to build out our Kubernetes support, we wanted to take some of the abstraction away, and ensure that we could provide full visibility into both the Kubernetes core competencies, but also application-level metrics. This more holistic view of Kubernetes gives teams a better look at how their containers are functioning and end-user experience.
Monitoring Kubernetes with Panopta: How it works
As shown in the diagram, there are two main components that Panopta adds to your Kubernetes cluster. First is the Panopta OnSight vCollector, which sits on a single node in the cluster, and takes up the same space any other pod would. The OnSight gives you the power to do synthetic checks within the Kubernetes infrastructure and specifically behind your firewall, monitors the health of the cluster, and sends the data it collects securely to Panopta. The Panopta Agent is also deployed on each individual node and collect application and container performance metrics, which are routed through the Panopta OnSight.
Working together, the Agent and OnSight are able to deliver metrics on both the core competencies of your Kubernetes cluster and application performance to give your team the data and tools needed to ensure an end-to-end view of your cluster.
Installed using the Helm tool, Panopta offers a Helm chart which will automatically download all of the necessary Panopta monitoring infrastructure and deploy the Panopta Agent and OnSight vCollector. Once the Agent and vCollector are on every node, and one node respectively, Panopta will be able to see your cluster and can begin monitoring it immediately.
Benefits of Monitoring Kubernetes with Panopta
One of the major benefits of working with Kubernetes is that it allows teams to ignore the health of individual instances in favor of focusing on the Kubernetes core competencies and whether those are running. However, this doesn’t mean that you should neglect application performance, and on top of that, the abstraction makes it difficult to make sense of the metrics coming back from any monitoring service.
Our focus for Kubernetes monitoring is to remove some of this abstraction and give teams the ability to display and visualize Kubernetes instances alongside traditional bare metal and cloud infrastructure. Displaying all of these instances together gives a more cohesive look at infrastructure, exposing the impact Kubernetes services have on other infrastructure in your environment.
The Panopta Kubernetes details page, shown below, offers individual instance metrics, richer visualizations, service-level metrics, and coverage for Kubernetes first-class objects.
Displaying Kubernetes alongside traditional and cloud infrastructure has a couple of very simple benefits:
- Teams who are newer to Kubernetes will more intuitively understand the metrics they’re receiving
- Exposes pod-level incidents which could affect end-user experience
- Support teams receive performance-based metrics, making it easier for them to find correlation and resolve incidents
With the added visibility and effective way that data is communicated within the Kubernetes details page, teams can more efficiently monitor their Kubernetes cluster and infrastructure as a whole.
In addition to revealing data effectively, Panopta brings down the level of noise that is inherent to the auto-scaling in Kubernetes. Teams will never receive an alert for native functions of Kubernetes such as adding or removing a pod that isn’t functioning correctly. This reduces alert fatigue and ensures that your team is seeing and responding to the incidents which actually matter.
Coupled with clear, outcome-based data, teams are able to more efficiently handle incidents and easily find correlations. When using Kubernetes, it’s easier for a team to respond to an alert such as “users can’t log in” than alerts like “slow queries”, “High CPU usage”, or “slow response times”. These descriptive alerts offer systems admins more information about which services might have caused the incident.
Panopta’s incident details page also consistently presents performance and outcome-based data. Support teams can smoothly transition from monitoring the instances they have deployed in the cloud or on bare metal infrastructure to those deployed on Kubernetes, making it easier for them to find correlation and do root cause analysis.
The Google SRE book highly recommends using synthetic checks when monitoring any distributed system, and when deploying in Kubernetes, we feel that this is especially important to simulate the end-user experience with synthetic checks. Panopta offers a system of public monitoring probes, the Panopta OnSight, and the Panopta Agent, all of which work together to complete synthetic checks both from the public space and behind your firewall.
Panopta’s OnSight vCollector is a powerful, lightweight pod that deploys directly into your Kubernetes cluster. By deploying within the cluster, the OnSight is able to run synthetic checks, just as a public probe would, from inside the cluster. This lets you securely gather performance metrics on your services that are not deployed publically, and exposes the health of the services you’re running within the cluster.
These metrics are then pulled into a customizable dashboard, creating a single-pane-of-glass for you to cohesively monitor your infrastructure. As we mentioned previously, any Kubernetes infrastructure you monitor with Panopta displays alongside your cloud and traditional infrastructure. Having all your infrastructure in one place means data from synthetic checks performed within your Kubernetes cluster and synthetic checks done on your legacy infrastructure will appear together, and metrics can be drawn into visualizations from both sources. correlation and root cause analysis easier for your team.
- A SaaS-based solution means pricing scales with Kubernetes, you will only pay for monitoring instances as long as they exist.
- Panopta requires no additional hardware, just a simple software installment
- Unlimited Access to the Panopta support team
One of our customers, a large data processing company that has created several massive data management (MDM) tools, reached out to us in mid-2018 to let us know that they decided any new tools they created from that moment forward would deploy in Kubernetes. At the time, several of their teams already had tools that had been deployed on Kubernetes clusters, but they used different monitoring services for those tools. Wanting to consolidate their monitoring solutions, they came to us looking for Kubernetes support, and we began developing features to support monitoring Kubernetes.
How we achieved full Visibility for our Customer’s new environment
We developed our Kubernetes support knowing that both the customer we were working with, and future customers, would be in varying stages of moving to Kubernetes deployments even after we’d finished building out this support. To ensure that we would be able to meet any customer no matter where they were in their journey with Kubernetes and effectively monitor all of their instances, regardless of how much of their infrastructure has moved into a Kubernetes deployment, we worked with a hybrid environment in mind.
Updating the Panopta OnSight vCollector
Once we knew more about the environment and what it would take to ensure end-to-end visibility, our focus moved to how we could best progress Panopta tools to integrate with Kubernetes. We decided early on that the Panopta OnSight vCollector would become the centerpiece of our Kubernetes support, maturing it to integrate with Kubernetes rather than building out a new tool to support Kubernetes or using a third party agent to ingest the native Kubernetes metrics.
By adapting the Panopta OnSight, not only could our customer monitor their clusters without overburdening them, but also they had the same out-of-box monitoring available that their cloud infrastructure used. This meant they could immediately begin doing synthetic checks on their Kubernetes deployments as soon as they had the Panopta OnSight installed.
As we mentioned previously, monitoring outcomes and the end-user experience is incredibly important, so we wanted to make sure that our customer could immediately have that up and running. Additionally, this meant they could monitor both from the public probes, and from behind their firewall, making it easier for them to test new tools and deployments without creating a security risk.
With the Panopta OnSight vCollector and Agent deployed within their Kubernetes cluster, our customer began monitoring their Kubernetes seamlessly. Adding in checks from Panopta public probes, they were able to quickly get metrics on the end-user experience, and adapt their tools as necessary.