Airbnb migrated its high-volume metrics infrastructure to adopt the OpenTelemetry Protocol (OTLP) and Prometheus. To do so without massive disruption, they implemented a dual-emit strategy in their shared metrics libraries. They encountered and solved specific performance bottlenecks regarding high-cardinality data and replaced their legacy Veneur aggregator with a custom-sharded vmagent setup. Crucially, they developed a "zero injection" technique to solve systemic undercounting issues when translating StatsD-style counters into Prometheus cumulative counters.

https://medium.com/airbnb-engineering/building-a-high-volume-metrics-pipeline-with-opentelemetry-and-vmagent-c714d6910b45