Working premise
OpenTelemetry provides a portable foundation for traces, metrics, context propagation, and collectors. AI applications still need a deliberate schema for prompts, models, tools, retrieval, evaluation, and privacy.
Operational context
OpenTelemetry provides correlation, propagation, collection, and a shared vocabulary across services. It does not decide the product-specific evidence needed for AI quality. Teams still need an internal contract for task, release, prompt, model, evaluator, cost, outcome, and sensitive-payload handling.
Generative AI semantic conventions continue to evolve. Instrumentation should record the convention and SDK versions it emits, distinguish stable from experimental fields, and place provider-specific details behind a translation layer. Otherwise an SDK upgrade can silently split historical dashboards or break evaluation joins.
A good rollout begins with questions the telemetry must answer. Tracing every local operation creates volume and cardinality without necessarily improving diagnosis. Spans should represent meaningful operations with duration; events should capture point-in-time facts; metrics should support population-level monitoring.
Questions to resolve first
Which production questions require traces, metrics, logs, or events?
Which GenAI attributes are stable enough for shared reporting and which need an internal alias?
How will trace context cross queues, tools, gateways, and asynchronous jobs?
Where are payload collection, redaction, sampling, and retention enforced?
Recommended method
- P1
Begin with W3C trace context and consistent service, environment, and release attributes.
- P2
Adopt GenAI semantic conventions where stable and document product-specific extensions.
- P3
Instrument framework boundaries without losing application-level task context.
- P4
Route telemetry through collectors that can redact, sample, enrich, and export.
Common failure modes
Auto-instrumentation produces spans but omits the release, prompt, task, and outcome needed for AI decisions.
Dashboards and stored queries depend directly on changing convention names without migration support.
Unbounded prompt, user, document, or tool values become metric dimensions and destabilize the telemetry system.
Control points
- Validate attribute cardinality and payload size before high-volume rollout.
- Separate sensitive content from metadata needed for operations.
- Keep exporter and collector failures observable.
- Test trace continuity across queues, jobs, gateways, and external tools.
Implementation sequence
- S1
Define priority use cases and the minimal evidence contract before selecting instrumentation libraries.
- S2
Map internal attributes to OpenTelemetry and provider conventions with explicit stability and version metadata.
- S3
Validate propagation, required fields, redaction, and cardinality in CI and staging.
- S4
Roll out by workflow, measure collector loss and overhead, and maintain a controlled convention-migration plan.
Measures worth reviewing
Instrumentation quality should include coverage of expected operations, trace continuity, required-attribute conformance, collector failure, overhead, and cardinality growth. Ingestion volume alone does not indicate useful observability.
Primary references
This note is an editorial synthesis of public standards, primary institutional guidance, and common engineering control patterns. It is intended to support engineering design and review. It is not a substitute for legal, security, safety, audit, or other professional advice for a specific system.