AWS First Steps
LanguageEnglish
New feature

Managed Prometheus collectors now support Amazon MSK consumer-group metrics

Managed Prometheus collectors for Amazon Managed Service for Prometheus and Amazon CloudWatch now support Amazon MSK consumer-group metrics, collecting Kafka exporter and consumer-group lag metrics to help detect processing bottlenecks in streaming applications.

The managed Prometheus collector now supports Amazon MSK consumer group metrics, enabling collection of Kafka export and consumer group lag metrics from Amazon MSK clusters. It automatically detects and collects Prometheus metrics from workloads without user-managed agents. These metrics are ingested into Amazon Managed Service for Prometheus workspaces or Amazon CloudWatch for querying and analysis, allowing monitoring of Kafka stream health and consumer performance to identify streaming application bottlenecks. This feature is available in all AWS regions where the managed Prometheus collector is generally available.

Why it matters

This update is useful for users who want to automate monitoring of Amazon MSK clusters, enabling easy monitoring of Kafka stream health and consumer performance.

Read the original AWS announcement

Related articles and terms

Container Monitoring with Amazon Managed Service for Prometheus - Collecting and Analyzing EKS MetricsEKS/ECS metrics are collected in a Prometheus-compatible way and queried with PromQL. This article introduces a method for building a container monitoring platform through integration with Managed Grafana.Managed Kafka Streaming - Building Large-Scale Real-Time Data Pipelines with Amazon MSKLearn how to build a fully managed Kafka cluster with Amazon MSK (Managed Streaming for Apache Kafka) and when to choose it over Kinesis. This article covers design patterns for large-scale real-time data streaming infrastructure.Getting Started with Event Streaming on Amazon MSK Serverless - Building a Zero-Ops Kafka EnvironmentThis article introduces a method for building a Kafka environment that needs no cluster management with IAM authentication, and simplifying event streaming through integration with EventBridge Pipes.Amazon MSKA fully managed streaming service that provides Apache Kafka cluster provisioning and operations on AWS, with Kafka-compatible API supportBuilding an Apache Kafka Streaming Platform with Amazon MSK - Cluster Design and OperationsThis article covers designing managed Apache Kafka clusters, choosing between them and MSK Serverless, and data integration patterns with MSK Connect.