🐛 Fix: agent evaluation execution in the runtime sandbox

[cj2026-bit]

Fix Agent Evaluation Execution in the Runtime Sandbox

Bug Fixed

When the sandbox is enabled, Agent Evaluation runs could fail with
Run Agent Error, produce no result, or remain blocked by stale locks.

The evaluation task was started directly in the Config process. However, the
runtime workspace, Agent run manager, and sandbox-related resources are owned
by the Runtime process.

Root Cause

There were three problems:

1. Config directly executed the evaluation worker instead of dispatching it to
   Runtime.
2. Config does not own the Runtime sandbox workspace required by
   prepare_agent_run.
3. Every evaluation case used conversation_id=0, so concurrent cases could
   share the same run-manager lock key.

Retries could also start the same evaluation more than once because the run
status was not claimed atomically.

Solution

Config-to-Runtime Dispatch

Config now sends an internal request to Runtime:

POST /agent-evaluations/internal/run

The request is authenticated with the internal runtime JWT.
Config remains responsible for creating and preparing the evaluation task.
Runtime is responsible for executing the Agent and scoring the cases.
Runtime-Side Execution
Runtime now:

• Validates the internal JWT.
• Checks the tenant and evaluation run.
• Atomically changes the run status from PENDING to RUNNING.
• Submits the evaluation worker to the Runtime thread pool.
• Returns an idempotent response for already-running or already-finished runs.
  Per-Case Run Isolation
  Each evaluation case now receives a stable, evaluation-specific conversation ID.
  This prevents different cases from sharing the same Agent run-manager lock.
  The Agent run is also unregistered in a finally block so that locks and
  runtime state are cleaned up after both successful and failed executions.
  Main Changes
• Added the internal Runtime evaluation dispatch endpoint.
• Registered the endpoint in the Runtime application.
• Added atomic evaluation-run claiming in the database layer.
• Added Config-to-Runtime dispatch through the runtime proxy.
• Moved sandbox-dependent evaluation execution into Runtime.
• Added stable per-case conversation IDs.
• Added guaranteed Agent run cleanup.
• Added handling for Runtime timeout, unavailable service, and upstream errors.
• Added unit tests for dispatch, authorization, idempotency, atomic claiming,
  proxy errors, and run cleanup.

Validation

The service starts in Docker and enables sandbox verification for agent evaluation

[codecov]

Codecov Report

❌ Patch coverage is 94.06780% with 7 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
backend/apps/agent_evaluation_runtime_app.py	92.15%	3 Missing and 1 partial ⚠️
backend/services/agent_evaluation_service.py	92.68%	0 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!