A trace is the hierarchical view of a single run — agent → step → tool call → LLM call — with a duration and token count on each node. It renders as the Trace tab on the run detail page (shown whenever a run has spans).
A span is one timed node in that tree: a name, a start/end, a parent, and attributes (model, tokens, …). Unlike a flat event, a span has a duration and a place in the hierarchy, so you can see where the time and tokens went.
Telemetry reaches AgentOS three ways. You don't have to pick one.
Agents that run on AgentOS are traced for you. Every run records a root agent span with an LLM span per model call and a tool span per tool call (with durations and token usage). Open the run → Trace. Nothing to configure.
If you run your agent yourself and report runs via the SDK, wrap work in spans. Spans are timed and nestable; call .end() to record one.
import { AgentOS } from "@agentos-sdk/core";
const os = new AgentOS({ apiKey: process.env.AGENTOS_KEY! });
const run = await os.startRun({ agentId: AGENT_ID });
const step = run.startSpan("classify email", { kind: "step" });
const llm = step.child("gpt-4.1", { kind: "llm" });
// ... call the model ...
llm.end({ input_tokens: 1240, output_tokens: 90 }); // tokens show on the span
step.end();
await run.complete();
from agentos import AgentOS
os = AgentOS(api_key=os.environ["AGENTOS_KEY"])
run = os.start_run(agent_id=AGENT_ID)
step = run.start_span("classify email", kind="step")
llm = step.child("gpt-4.1", kind="llm")
# ... call the model ...
llm.end({"input_tokens": 1240, "output_tokens": 90})
step.end()
run.complete()
Span kinds (agent, step, llm, tool, retrieval) drive the chip color and grouping in the trace view. Kind is open text — any value is accepted; unknown kinds render neutrally. Put input_tokens / output_tokens in the .end() payload to get the per-span token label.
If your agent is already instrumented with OpenTelemetry (LangChain, the OpenAI Agents SDK, Vercel AI SDK, LlamaIndex, and others emit GenAI spans natively), you don't need any AgentOS-specific code — just send your traces here.
Endpoint (OTLP/HTTP, JSON encoding):
POST https://agentos-ai.dev/api/ingest/v1/traces
Authorization: Bearer <your AgentOS key>
Content-Type: application/json
Set the standard OTLP env vars on your app:
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://agentos-ai.dev/api/ingest/v1/traces"
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=Bearer <your AgentOS key>"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/json"
Which agent does a trace belong to?
With an agent-scoped key (aos_agent_…), every trace is attributed to that agent — nothing else to set.
With a workspace key (aos_ws_…), set the agent on the OTel resource:
agentos.agent_id = <your agent uuid>
How it maps:
| OpenTelemetry | AgentOS |
|---|---|
one trace (trace_id) | one run |
span (span_id / parent_span_id) | a node in the trace tree |
gen_ai.request.model | the LLM span's name |
gen_ai.usage.input_tokens / output_tokens | per-span + run token totals |
gen_ai.operation.name = execute_tool / gen_ai.tool.name | a tool span |
span status = ERROR | the run is marked failed |
Notes:
OTEL_EXPORTER_OTLP_PROTOCOL=http/json (protobuf isn't accepted).Spans are additive — a run with no spans simply has no Trace tab, and the flat event stream is unaffected.