Kafka Tools: AKHQ, Kafka UI, Kowl, Offset Explorer
Introduction
Apache Kafka's distributed architecture makes it powerful but complex to manage. The right tooling is essential for monitoring consumer lag, inspecting messages, managing topics, and debugging production issues. This article covers four Kafka management tools: AKHQ, Kafka UI, Kowl, and Offset Explorer.
AKHQ (formerly KafkaHQ)
A comprehensive Kafka web UI with topic management, consumer group monitoring, and schema registry support:
# docker-compose.yml
version: "3.8"
services:
akhq:
image: tchiotludo/akhq:latest
environment:
AKHQ_CONFIGURATION: |
akhq:
connections:
local:
properties:
bootstrap.servers: "kafka:9092"
schema-registry:
url: "http://schema-registry:8081"
connect:
- name: connect
url: "http://kafka-connect:8083"
ports:
- "8080:8080"
depends_on:
- kafka
**Key features**:
* Topic browser with search and filter
* Consumer group monitoring with lag visualization
* Schema registry viewer (Avro, Protobuf, JSON Schema)
* Kafka Connect management
* Message producer (publish test messages)
* Partition reassignment viewer
* Node and broker monitoring
# AKHQ advanced configuration
akhq:
connections:
production:
properties:
bootstrap.servers: "broker1:9092,broker2:9092,broker3:9092"
security.protocol: SASL_SSL
sasl.mechanism: SCRAM-SHA-512
sasl.jaas.config: "org.apache.kafka.common.security.scram.ScramLoginModule required username='admin' password='${SECRET_AKHQ_PASSWORD}';"
schema-registry:
url: "https://schema-registry:8081"
basic-auth-username: admin
basic-auth-password: "${SECRET_SCHEMA_REGISTRY_PASSWORD}"
Kafka UI (by Provectus)
A lightweight, open-source Kafka web UI:
# docker-compose.yml
services:
kafka-ui:
image: provectuslabs/kafka-ui:latest
ports:
- "8081:8080"
environment:
KAFKA_CLUSTERS_0_NAME: local
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092
KAFKA_CLUSTERS_0_SCHEMAREGISTRY: http://schema-registry:8081
KAFKA_CLUSTERS_0_KAFKACONNECT_0_NAME: connect
KAFKA_CLUSTERS_0_KAFKACONNECT_0_ADDRESS: http://kafka-connect:8083
SERVER_PORT: 8080
**Key features**:
* Dashboard with cluster overview and broker health
* Topic browser with partition visualization
* Consumer groups with lag chart
* Schema registry viewer
* Kafka Connect management
* Message filtering with custom predicates
* Dark and light themes
# Multi-cluster configuration
KAFKA_CLUSTERS_0_NAME: staging
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: staging-kafka:9092
KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL: SASL_SSL
KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM: SCRAM-SHA-512
KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG: "org.apache.kafka.common.security.scram.ScramLoginModule required username='user' password='pass';"
KAFKA_CLUSTERS_1_NAME: production
KAFKA_CLUSTERS_1_BOOTSTRAPSERVERS: prod-kafka:9092
Kowl (now Redpanda Console)
Developer-friendly Kafka browser with a focus on data exploration:
# docker-compose.yml
services:
redpanda-console:
image: docker.redpanda.com/redpandadata/console:latest
ports:
- "8082:8080"
environment:
KAFKA_BROKERS: kafka:9092
SCHEMA_REGISTRY_ENABLED: "true"
SCHEMA_REGISTRY_URLS: "http://schema-registry:8081"
**Key features**:
* Fast message search and filtering with custom predicates
* Consumer group monitoring with lag alerts
* Schema registry with compatibility checking
* Message publish with schema validation
* Kafka Connect management
* Data masking for sensitive fields
* Role-based access control
# Data masking configuration
server:
maskedFields:
- fieldName: "password"
maskingStrategy: "redact"
- fieldName: "email"
maskingStrategy: "hash"
- fieldName: "ssn"
maskingStrategy: "hash_last_four"
# Message search predicate examples
# Find messages with specific JSON field:
# $.user.email == "test@example.com"
# $.order.total > 100.0
# $.status in ["pending", "processing"]
Offset Explorer (formerly Kafka Tool)
A desktop GUI application for Kafka (Windows/macOS/Linux):
**Key features**:
* Tree-based cluster browser
* Topic partition viewer with message content
* Consumer group offset management
* Message producer
* Import/export topics
* SSL/SASL authentication support
* Time-based message search
CLI Tools
The Kafka distribution includes essential CLI tools:
# Topic management
kafka-topics.sh --bootstrap-server localhost:9092 --list
kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders
kafka-topics.sh --bootstrap-server localhost:9092 --create --topic events --partitions 6 --replication-factor 3 --config retention.ms=604800000
# Consumer group monitoring
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group order-processor
# Message inspection
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders --from-beginning --max-messages 10
# Performance testing
kafka-producer-perf-test.sh --topic test --num-records 100000 --record-size 1024 --throughput 10000 --producer-props bootstrap.servers=localhost:9092
kafka-consumer-perf-test.sh --broker-list localhost:9092 --topic test --messages 50000
Comparison
| Feature | AKHQ | Kafka UI | Kowl (Console) | Offset Explorer |
|---------|------|----------|----------------|-----------------|
| Type | Web UI | Web UI | Web UI | Desktop |
| Message search | Basic | Basic | Advanced | Basic |
| Schema registry | Yes | Yes | Yes | No |
| Kafka Connect | Yes | Yes | Yes | No |
| Data masking | No | No | Yes | No |
| RBAC | No | No | Yes | No |
| Multi-cluster | Yes | Yes | Yes | Yes |
Recommendations
* **Full-featured management**: AKHQ for comprehensive cluster administration.
* **Quick monitoring**: Kafka UI for lightweight cluster health checks.
* **Message debugging**: Kowl (Redpanda Console) for powerful message search and filtering.
* **Desktop preference**: Offset Explorer for a native desktop experience.
* **Scripting**: Kafka CLI tools for automation and CI/CD tasks.
For development and staging environments, Kafka UI or Kowl provide excellent web-based management with minimal setup. For production, AKHQ offers the most comprehensive feature set, while Redpanda Console provides the best message debugging experience.