Observability
Monitoring, logging, and performance tracking for LLM Proxy.
What’s in this section
- Instrumentation - Event middleware, async event bus, and dispatcher integrations
- Distributed Rate Limiting - Redis-backed rate limiting across instances
- HTTP Response Caching - Cache configuration and performance
- Coverage Reports - Live test coverage report
- Coverage Setup - Setting up coverage reporting
Grafana Dashboards
Ready-to-import Grafana dashboards are available for visualizing LLM Proxy metrics:
- Location:
deploy/helm/llm-proxy/dashboards/ - Documentation: See the dashboards README for import instructions
- Datasource: Prometheus
The operational dashboard (llm-proxy.json) includes:
- Request rate, error rate, and uptime monitoring
- Cache performance metrics (hits, misses, bypass, stores)
- Memory usage and Go runtime metrics
- Garbage collection statistics
Event Flow
Request → Proxy → Event Bus → Dispatcher → Backends
↓
[Lunary, Helicone, File]
For detailed event flow documentation, see the Instrumentation Guide.