Own the Telemetry Namespace for Non-Model GenAI Systems

Context

The LLM Workflow Router is a topology-enforcement layer, not a model-selection router. Given structured metadata about an interaction, it answers one question, "is this workflow transition permitted", independently of model choice or content. It validates workflow topology at load time, enforces invocation limits, and returns one of three terminal decisions: PROCEED, REFUSE, or PAUSE. Its defining property for instrumentation is that it is content-blind: it never sees a prompt, never calls a provider, and never generates a token.

Instrumenting it with the emerging OpenTelemetry GenAI semantic conventions technically worked, but pinched in six specific places.

Problem

Each friction exposed the same hidden assumption: that a GenAI system is, by definition, a model invoker.

1. No operation name for validation. The defined gen_ai.operation.name values all describe runtime inference (chat, generate_content, invoke_agent, retrieval, execute_tool). None describe static structural validation.

2. provider.name assumes a provider. Every GenAI span is modeled as calling a model, but this system calls none.

3. No vocabulary for topology. The unit of interest is assumed to be a conversation, not a position in a graph.

4. A refusal reads as an error. The conventions treat the only non-success outcome as failure, but a REFUSE is a correct, expected decision.

5. Content observability is the default. Instrumentation is assumed to mean watching prompts and completions, which a content-blind system must not do.

6. No trace-parenting for standalone runs. Workflow spans are assumed to be children of a model span that does not exist here.

Encoding a refusal as an error is the most damaging of these, because it corrupts every downstream success-rate and alerting signal.

Decision

The system owns its own telemetry namespace and reuses OpenTelemetry conventions only where they genuinely fit, following four rules:

1. Reuse the standard GenAI attributes wherever they apply, and record every place they do not.

2. Define a dedicated namespace (for example wfrouter.*) for the concepts the standard lacks: a validation operation, workflow topology, and graph position.

3. Encode a refusal as a first-class, non-error outcome rather than mapping it onto error status.

4. Do not emit prompt or completion content, keeping the content-blind property visible in the telemetry itself.

The mismatches are published as a friction log rather than a formal proposal, which is enough to inform the live semantic-conventions discussion.

Alternatives Considered

Force-fit the GenAI conventions as written

Rejected. It requires encoding a refusal as an error and inventing model-shaped attributes for a system with no model, which produces misleading telemetry.

Wait for the standard to cover orchestration layers

Rejected. The system needs observability now, and a documented friction log is a faster and more honest contribution than blocking on a future revision.

Ship no instrumentation

Rejected. A governance layer that makes PROCEED, REFUSE, and PAUSE decisions is exactly the kind of component that must be observable.

Consequences

Positive

• Refusals are recorded as correct outcomes, keeping success and error signals meaningful.

• Validation and topology gain first-class vocabulary.

• The content-blind guarantee is expressed in the telemetry.

• The friction log offers a reusable pattern for instrumenting orchestration, governance, and policy layers that do not call models, and it feeds the standards discussion.

Trade-offs

• A custom namespace is not automatically understood by off-the-shelf GenAI dashboards.

• The namespace must be maintained and reconciled as the conventions evolve.

Evidence & Related Work

Implementation Evidence

• LLM Workflow Router (PyPI):

• Related article:

Relationship to the Broader Engineering Approach

The router is the enforcement engine behind ADR-002. This ADR ensures its decisions are observable without distorting them: a refusal is a designed outcome of a validation boundary, and the telemetry must represent it as such rather than as a failure.

Decision Outcome

Observability for a non-model system is modeled on what the system actually does, not on an assumed model call:

Reuse where it fits + own namespace where it does not + refusal as first-class outcome

This ADR should be revisited when the OpenTelemetry GenAI conventions add first-class support for validation, topology, and non-error refusals, at which point parts of the custom namespace can be retired.