Skip to content

fix(sanitization): secret exposure in function and agent trace spans - #6000

Open
BillLeoutsakosvl346 wants to merge 15 commits into
stagingfrom
fix/secrets-sanitization-trace-spans
Open

fix(sanitization): secret exposure in function and agent trace spans#6000
BillLeoutsakosvl346 wants to merge 15 commits into
stagingfrom
fix/secrets-sanitization-trace-spans

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Added an environment-aware sanitizer that restores configured {{ENV_VAR}} references in trace/display values, including nested tool calls, timing segments, errors, object keys, and URL-encoded values.
  • Preserved unresolved environment references in Function display/error source while continuing to resolve plaintext runtime code.
  • Applied sanitization at the BlockExecutor observability boundary without changing handler inputs, runtime state, handled errors, retries, or resume execution.
  • Registered the sanitizer at the workflow logging boundary so Overview final output, synthetic errors, fallbacks, final spans, and telemetry copies are also sanitized.
  • Prevented streaming completion and human-in-the-loop resume paths from reintroducing raw values into already-sanitized logs.
  • Kept raw outputs available for API responses, downstream execution, retry/resume state, and billing calculations.

Root cause

Block inputs and provider/tool outputs were logged after environment references had been resolved. Function display code also resolved environment values, and post-executor paths could overwrite sanitized logs with raw streaming, pause-state, or workflow final-output data immediately before persistence.

Impact

New trace spans and Overview workflow output show configured references such as {{AWS_SECRET_ACCESS_KEY}} instead of plaintext values. Runtime tool/function execution, API responses, downstream block behavior, billing, retries, pause/resume, and persisted runtime state remain unchanged. Existing historical logs are not rewritten.

Validation

  • Independent senior review with a second review after addressing all findings
  • 216 targeted Vitest tests across sanitizer, resolver, BlockExecutor, execution core, logging completion, trace spans, Function handler, snapshots, streaming, and HITL resume
  • App TypeScript check
  • Targeted Biome lint/format on all changed files
  • API validation contract audit
  • git diff --check

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 1, 2026 7:08pm

Request Review

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor

cursor Bot commented Jul 28, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Touches secret handling, authentication boundaries for private provenance metadata, and broad execution/logging paths; incorrect masking or provenance could leak secrets or break observability, though design fails closed when provenance is incomplete.

Overview
Introduces execution log protection: when a value from Settings → Secrets is successfully substituted through {{KEY}}, exact case-sensitive occurrences are masked in log-facing content (live block console, Logs Overview, stored traces, v1 log APIs, export, Logs block Get Run Details), usually as {{KEY}}. Runtime behavior is unchanged—blocks, tools, and workflow responses still receive real values; only observability projections and read APIs get a protected copy.

Provenance is tracked so masking knows which plaintexts to replace: the Function execute API reports __resolvedSecretNames only for successful {{KEY}} resolution from scoped env vars (not direct environmentVariables or shell $VAR alone), gated by an internal request header. MCP tool execution and Mothership execute attach encrypted __resolvedSecretTraceProvenance only to internal JWT callers that request it; session and external callers never see it. Workflow execute can return the same provenance on JSON responses for internal tool callers.

Workflow execute projects masked display fields on SSE block events, stream chunks, and terminal errors; materializes logs for display via materializeExecutionDataForDisplay. Session users can retry with inputFromExecutionId (server loads stored input without returning it to the client) and run from block using trusted stored execution state with sourceExecutionId. Legacy workflow log completion restores trusted provenance from stored execution state before building traces.

UI retry uses executionId instead of extracting input from log detail; the editor honors server display for agent thinking and reconciles final block logs to masked terminal content.

Docs add Execution log protection under Secrets and update agents, MCP, logs, Function, and variables pages with scope and limitations (not a general redactor; encoded/transformed values not matched).

Reviewed by Cursor Bugbot for commit c9302a5. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stops resolved environment secrets from appearing in workflow trace spans and Overview logs while keeping runtime execution and API payloads unchanged.

  • Adds ResolvedSecretTraceRegistry plus Aho–Corasick-style projection (trace-secret-projection) to restore {{ENV_VAR}} (or anonymous redaction) in display traces.
  • Activates secrets only on successful {{NAME}} resolution; carries encrypted provenance across internal boundaries (MCP, function execute, mothership, nested workflows, HITL resume).
  • Applies projection at logging completion / execution-core boundaries; avoids re-injecting raw values from streaming completion and HITL resume into already-sanitized logs.
  • Leaves handler inputs, retries, billing, and downstream block state on raw resolved values; adds extensive Vitest coverage across sanitizer, registry, executor, logging, and streaming paths.

Confidence Score: 5/5

No prior Greptile findings remain to re-score and no eligible follow-up inline comments were identified, so the PR appears safe to merge from this pass.

No blocking failure remains from outstanding prior threads or newly eligible incomplete-fix findings in this follow-up review.

Important Files Changed

Filename Overview
apps/sim/executor/utils/resolved-secret-trace-registry.ts New catalog/active-match registry with encrypted provenance, scope checks, and bounded export-for-value filtering for cross-boundary secret tracking.
apps/sim/lib/logs/execution/trace-secret-projection.ts Terminal projection that rewrites secret literals in trace trees (including large-value refs) without mutating runtime block state.
apps/sim/lib/logs/execution/logger.ts Completes logs with prepared/projected spans, preserves private provenance on PII-masked execution state, and adjusts size compaction of tool-call content.
apps/sim/executor/variables/resolvers/env.ts Records resolved env secrets into the trace registry only when the name exists on execution environment variables.
apps/sim/lib/workflows/executor/execution-core.ts Wires registry creation and trace projection into the main workflow execution completion path.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  EnvResolve["EnvResolver.recordResolved"] --> Registry["ResolvedSecretTraceRegistry"]
  Nested["MCP / function / mothership / nested WF"] --> Prov["Encrypted provenance import/export"]
  Prov --> Registry
  Runtime["Raw block I/O for execution"] --> Logs["buildTraceSpans / completeWorkflowExecution"]
  Registry --> Project["projectTraceSpansForSecrets"]
  Logs --> Project
  Project --> Persist["Persisted executionData.traceSpans"]
  Project --> Overview["Overview / export / UI logs"]
Loading

Reviews (5): Last reviewed commit: "secrets sanitization correctness" | Re-trigger Greptile

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 95d4fd1. Configure here.

@BillLeoutsakosvl346
BillLeoutsakosvl346 marked this pull request as ready for review July 28, 2026 05:44
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

Comment thread apps/sim/lib/workflows/streaming/streaming.ts Outdated
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile please review the latest commit, including the workflow-overview sanitization follow-up and the streaming token-accounting correction.

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e73afbe. Configure here.

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

Demo:

Screen.Recording.2026-07-28.at.12.00.01.AM.mov

@icecrasher321 icecrasher321 changed the title Fix secret exposure in function and agent trace spans fix(sanitization): secret exposure in function and agent trace spans Jul 31, 2026
@gitguardian

gitguardian Bot commented Aug 1, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
35187658 Triggered Username Password 64f791f apps/desktop/src/main/browser-credentials/vault.test.ts View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Comment thread apps/sim/app/api/workflows/[id]/execute/route.ts
Comment thread apps/sim/hooks/queries/logs.ts
@icecrasher321

Copy link
Copy Markdown
Collaborator

bugbot run

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

Comment thread apps/sim/app/api/mcp/tools/execute/route.ts
Comment thread apps/sim/app/api/workflows/[id]/execute/route.ts
@icecrasher321

Copy link
Copy Markdown
Collaborator

@cursor review

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 2eb706a. Configure here.

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c9302a5. Configure here.

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile please review the final commit c9302a5, including the execution-state preservation follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants