Skip to content

TechnicalPractitioner6 min read

Module 2: OpenTelemetry GenAI conventions

The standard vocabulary that describes an LLM call: gen_ai.* attributes, what a span contains, the metrics to emit by default and the precautions to take with a standard that is still moving.

For: engineers and SREsPrerequisites: Have read module 1 of the course.

Picture two teams in the same company (an illustrative scenario). One calls Mistral and records the model in a model field. The other calls a different provider and records it in llm_name. The day management asks for total consumption per model, someone first has to translate one into the other, then rewrite the dashboards. Semantic conventions exist to avoid that kind of project.

OpenTelemetry, the open collection standard, produces three kinds of data. A trace tells the journey of a request, like parcel tracking: it is split into spans, timed steps nested inside one another, each carrying attributes (key and value pairs) and timestamped events. A metric is a count or a measurement followed over time (a time series): a counter that only goes up, a gauge that goes up and down, or a histogram that sorts values into buckets. It is exported over OTLP (the OpenTelemetry protocol) and stays compatible with Prometheus remote write. A log is a structured message, linked to traces through two identifiers, trace_id and span_id.

The fourth notion is the one that matters here: semantic conventions. They are a shared vocabulary, like the standard format of a postal address. There are families for the web (http.*), databases (db.*), message queues (messaging.*) and generative AI (gen_ai.*). This vocabulary is what makes instrumentation portable from one tool to another.

These are the attributes that describe a model call. They answer simple questions: who provides the model, which operation, which model, with which settings, how many tokens and why generation stopped. Temperature, for example, sets how much randomness goes into the answer: the lower it is, the more the model sticks to the most likely words.

AttributeDescriptionExample
gen_ai.provider.namemodel providerwell-known values: openai, anthropic, mistral_ai, azure.ai.openai, etc.; a custom value is allowed when none applies (the kit uses ollama)
gen_ai.operation.nametype of operationchat, embeddings, text_completion
gen_ai.request.modelrequested modelmistral:7b
gen_ai.request.temperaturerequested temperature0.7
gen_ai.usage.input_tokensprompt tokens245
gen_ai.usage.output_tokensgenerated tokens128
gen_ai.response.finish_reasonsfinish reasonstop, length, tool_call

Here is what a production span should contain:

span chat mistral:7b
trace_id 5b8efff798038103d269b633813fc60c
duration 1.847s
attributes
gen_ai.provider.name = ollama
gen_ai.request.model = mistral:7b
gen_ai.operation.name = chat
gen_ai.usage.input_tokens = 245
gen_ai.usage.output_tokens = 128
app.tenant.id = acme-corp
app.feature = support-chat
app.quality.score = 0.78
events
[t+0.000s] request.start
[t+0.182s] first_token_received

The value ollama is not in the list of well-known values of gen_ai.provider.name. The convention requires a well-known value when one applies and otherwise allows a custom value: that is the case for a model served locally by Ollama.

The last three attributes are not standard: they are in-house extensions. Put them in your own namespace (app.*, or the name of your organization) rather than in gen_ai.*, so they do not collide with a future version of the convention. The lab kit puts its own under mttl.* (mttl.tenant.id, mttl.feature, mttl.quality.score).

The first_token_received event gives the time to first token. In streaming, when the answer appears word by word, this is the latency the user perceives. The convention also measures it as a metric, below.

Three metrics are standard; the other three are kit extensions, under mttl.*.

MetricTypeContent
gen_ai.client.token.usagehistogram (standard)tokens consumed per operation, by input or output type; being replaced, see the box above
gen_ai.client.operation.durationhistogram (standard)duration of an LLM operation, including failures with the error.type attribute
gen_ai.client.operation.time_to_first_chunkhistogram (standard)time until the first chunk is received in streaming
estimated cost (mttl.client.cost in the kit)counter (extension)euros, computed from tokens and a dated price list you supply; without a price list, the metric is not emitted
errors (mttl.client.errors in the kit)counter (extension)timeouts, refusals, provider errors
quality score (mttl.quality.score in the kit)histogram (extension)score from 0 to 1 coming from evaluation

Always break these metrics down by model, by customer and by feature. Those are the three questions finance, product and operations will ask: which model costs the most, which customer consumes the most, which feature is degrading.

For technical teams: bucket boundaries for duration histograms

For duration histograms, set bucket boundaries suited to seconds: the convention recommends some (0.01; 0.02; 0.04 … 81.92 s). Without them, the Python SDK applies default boundaries designed for milliseconds and the computed quantiles no longer make sense.

Each kind of data has its use. The trace is for diagnosing one specific request: which document was retrieved, which prompt went out, which tool was called. The metric is for aggregating and alerting: cost per hour, p95 latency, average score. The event (or log) carries bulky or sensitive content, such as the prompt and the response, provided you have explicitly enabled its capture; it then gets a shorter retention and sensitive data is masked.

The GenAI conventions give providers and frameworks a common vocabulary. Since they are still moving, I recommend translating between versions in the Collector, never in dashboards. Before opening the lab, list the in-house attributes your application will need and choose their namespace. Also check that every LLM metric you plan can be broken down at least by model, customer and feature.

Revised on 2 October 2026: move of the GenAI conventions to the semantic-conventions-genai repository (version 1.42) and ongoing rework of token metrics noted, standard time to first chunk metric, known values of gen_ai.provider.name, names of the kit extensions, kit cost emitted only with a price list supplied by the user.

Revised on 4 October 2026: history of the conventions replaced by a pointer to §2 of the article Observing an LLM system; content capture rule aligned with the GenAI conventions (off by default, opt-in, outside indexed attributes in production), checked against the semantic-conventions-genai repository; ollama presented as an allowed custom value of gen_ai.provider.name.