feat(dev): collect local OTEL traces in project dev - #1980
Open
tejaskash wants to merge 5 commits into
Open
Conversation
tejaskash
had a problem deploying
to
e2e-testing
August 12, 2026 14:44 — with 
GitHub Actions
Failure
GitHub Actions
Failure
github-actions
Bot
added
agentcore-harness-reviewing
tejaskash
had a problem deploying
to
e2e-testing
August 12, 2026 14:47 — with
GitHub Actions
Failure
GitHub Actions
Failure
github-actions
Bot
removed
the
agentcore-harness-reviewing
Adds an in-process OTLP/HTTP collector (protobuf + JSON ingest, JSONL persistence per trace) started by the dev handler unless --no-traces or instrumentation.enableOtel is false. Spawned agents receive OTEL env pointing at the collector; container runtimes get a host.docker.internal endpoint, and Python CodeZip agents get sitecustomize-based auto-instrumentation so uvicorn --reload workers stay traced.
tejaskash
force-pushed
the
feat/dev-otel-collector
branch
from
August 14, 2026 20:38
b4d0349 to
e2e696c
Compare
tejaskash
had a problem deploying
to
e2e-testing
August 14, 2026 20:39 — with
GitHub Actions
Failure
GitHub Actions
Failure
…findings - Partition each OTLP export batch by trace id before persistence: a batch routinely carries spans from several traces, and writing it whole to the first trace's file corrupted trace identity (get() missed every other trace in the batch). Files are now keyed <traceId>.otlp.jsonl, making get() a direct path lookup. - Set signal-specific OTEL_EXPORTER_OTLP_TRACES_ENDPOINT/_LOGS_ENDPOINT (and protocols) alongside the generic variables: signal-specific values take SDK precedence, so a stray value from the shell or .env.local could silently redirect traces away from the local collector. The container rewrite now covers every OTLP endpoint variable. - Build the PYTHONPATH test expectation with node:path.delimiter so it passes on Windows.
tejaskash
had a problem deploying
to
e2e-testing
August 14, 2026 20:59 — with
GitHub Actions
Failure
GitHub Actions
Failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
When you run
agentcore project dev, your agent now gets local tracing out of the box. The CLI starts a small local trace receiver, tells the agent's OpenTelemetry SDK to send traces to it (via environment variables), and saves everything to files underagentcore/.cli/traces/. So while you develop, every invocation of your agent leaves a trace on disk you can inspect — model calls, tool calls, timings.Nothing is sent anywhere: the receiver only listens on 127.0.0.1 and writes to your project folder. If you don't want it, pass
--no-tracesor setinstrumentation.enableOtel: falseon the runtime.Why we need it
This is the plumbing for the Agent Inspector (the local web UI from the current CLI). The Inspector's trace view reads exactly these files — without this PR there is nothing for it to show. It also stands alone: even before the Inspector lands, you get trace files you can cat/jq while debugging an agent.
Where the code comes from
This is a port of what the current agentcore-cli already ships in
src/cli/operations/dev/otel/— same mechanism, same on-disk format, same env vars. Differences from the original are structural, to fit the rewrite's layering:OtelCollectorclass (HTTP + persistence + queries in one) is split into an HTTP receiver, aTraceStore, and pure transform functions — each independently testable.--reloadrespawns workers, so instrumentation is injected viasitecustomize.pyonPYTHONPATHrather than theopentelemetry-instrumentwrapper (which would only instrument the parent process, and traces would silently stop after the first hot reload).host.docker.internal— same as the original.One dependency is added, pinned exactly:
@opentelemetry/otlp-transformer@0.213.0. Agents send traces in protobuf (the Python SDK supports nothing else over HTTP), and this is the last version that ships the protobuf request decoders — the same version the current CLI uses. Don't let dependabot bump it.Stack
Targets #1966 (
feat/project-dev-handler); retarget torefactorafter it merges. PR B (the Agent Inspector web UI) builds on this.Verification
bun test(1487 pass), typecheck, lint, format all cleanbun run build+ smoke test:node dist/index.js project dev --helpshows--no-tracesbunfrom source, the Node npm bundle, and a compiled standalone binary