feat(network): enable policy DNS and transparent TCP egress

[johntmyers]

Problem Statement

Applications inside a sandbox can use HTTP CONNECT and forward HTTP egress, but native clients that perform ordinary DNS resolution and open direct TCP sockets cannot reach policy-approved services. This blocks database drivers, gRPC clients without proxy support, WebSocket libraries that ignore proxy variables, and other non-HTTP protocols even when the destination and process are explicitly authorized.

OpenShell needs a native path where getaddrinfo("db.example") followed by connect(address, 5432) works without weakening the sandbox boundary. Allowing unrestricted DNS, authorizing raw resolved IPs, or bypassing the shared proxy policy pipeline would create DNS exfiltration, confused-deputy, rebinding, and policy-generation risks.

This issue tracks the complete multi-PR implementation of policy-gated DNS and transparent TCP under the shared egress adapter architecture approved in RFC #2155. It supersedes #1107's earlier static-host-resolution proposal with an RFC-aligned synthetic-address correlation design.

Proposed Design

User-facing policy contract

Use the existing endpoint protocol field as the explicit opt-in, without adding a policy or protobuf field:

network_policies:
  postgres:
    name: postgres
    endpoints:
      - host: db.internal.example
        port: 5432
        protocol: tcp
    binaries:
      - path: /usr/bin/psql

• An omitted protocol retains existing explicit-proxy L4 behavior.
• Explicit protocol: tcp remains L4-only and works through CONNECT, while additionally making the endpoint eligible for native policy DNS and transparent TCP on capable runtimes.
• Eligibility requires a non-empty hostname and at least one concrete port.
• L7 protocols such as rest, graphql, json-rpc, mcp, and websocket remain explicit-proxy-only in this series.
• Once the native path is user-facing, policy activation must reject protocol: tcp when the selected runtime cannot provide policy DNS and transparent TCP. It must never silently broaden or partially activate the policy.

Request flow

1. The sandbox workload sends a normal DNS request.
2. Policy DNS normalizes the name and rejects names that are not explicitly eligible before making any upstream query.
3. Eligible names resolve through a trusted, operator-controlled resolver. Every A/AAAA answer passes the shared destination and SSRF controls.
4. Policy DNS allocates a supervisor-owned synthetic address and atomically publishes an exact, expiring mapping to the normalized name, policy endpoint, allowed ports, validated real addresses, policy generation, mapping generation, and opaque mapping ID.
5. The workload receives only the synthetic answer.
6. nftables redirects a TCP connection to that synthetic address into the transparent TCP adapter while preserving the original synthetic destination and port.
7. The adapter resolves process identity, requires the exact mapping, and runs the same authoritative egress authorization used by CONNECT and forward HTTP.
8. The unopened upstream connector dials only a validated real socket address pinned by the DNS mapping. It performs no connect-time DNS lookup.
9. The existing raw TCP relay carries bytes for protocol: tcp.

The sandbox never receives unrestricted external DNS or raw-IP egress. Existing CONNECT and forward HTTP paths remain supported.

Security and consistency invariants

• Policy before DNS: an ineligible name returns a local refusal, normally REFUSED, without an upstream lookup.
• DNS does not authorize a process: mappings are sandbox-scoped; binary, ancestor, and other process evidence is evaluated independently at connect time.
• Exact correlation is mandatory: authorization requires an unexpired mapping for the synthetic destination and requested port.
• No bare-IP inheritance: a real IP in a mapping never authorizes a direct connection to that IP.
• Pinned dialing only: transparent connections consume mapped, validated addresses and never resolve again at connect time.
• One policy generation: endpoint identity, process decision, destination plan, TLS/protocol enforcement, credentials, and middleware come from one authoritative generation.
• Separate mapping generation: DNS refresh/replacement has an opaque mapping ID and generation distinct from policy generation.
• Policy changes fail closed: stale mappings cannot authorize new connections after their endpoint contract changes or disappears.
• No partial publication: mapping and capture preparation complete before a synthetic answer is visible; failures publish nothing and return a local DNS error.
• Names remain distinct: two hostnames sharing the same real IP and port cannot inherit each other's endpoint policy.
• No proxy recursion: supervisor DNS and upstream TCP traffic are excluded from sandbox capture.
• Unmatched traffic remains fenced: real IPs, unknown synthetic addresses, wrong ports, UDP, and other bypass attempts retain reject/log behavior.
• No synthetic reassignment in v1: a synthetic address is not reused for a different normalized name/endpoint during a sandbox lifetime. Pool exhaustion fails closed.
• TCP latency contract: accepted and upstream TCP streams use the shared best-effort TCP_NODELAY helpers.

Runtime capability and lifecycle contract

Policy DNS, mapping state, capture rules, and the transparent listener must be ready before workload execution is released. A setup failure cannot leave DNS enabled without capture or capture enabled without correlation and authorization.

Docker and the Kubernetes combined topology are the initial integration targets. Podman and VM can join the same implementation when validated; otherwise they remain capability-gated. Kubernetes sidecar mode should install a static redirect for the reserved synthetic pool during init and enforce exact active mapping/port state in the unprivileged adapter, preserving the sidecar's current capability set. If maintainers require only active mappings in kernel state, that privilege model requires separate security review.

The implementation must validate IPv4 and IPv6 synthetic pools, DNS over UDP and TCP, nftables redirect behavior, original-destination recovery, resolver delivery, and capture exclusions on every supported runtime. Unsupported configurations fail policy validation or startup clearly rather than degrading to proxy-only behavior.

Implementation stack

1. Authoritative authorization and TCP policy contract — PR #2711

   • Accept and normalize explicit protocol: tcp as an L4 marker.
   • Make endpoint ambiguity and serialization behavior explicit.
   • Materialize egress authorization from one generation-consistent result.
   • Add dormant TransparentTcp, endpoint-correlation, and pinned-destination primitives while preserving all current egress behavior.

2. Policy DNS and resolved-endpoint store

   • Add bounded UDP/TCP DNS handling, name normalization, eligibility checks, trusted resolution, destination filtering, synthetic IPv4/IPv6 allocation, mapping refresh/expiry, and health metrics.
   • Expose a narrow lookup API from synthetic address plus port to the exact current mapping and pinned-address destination plan.
   • Do not enable kernel capture or expose the listener to userland in this PR.

3. Linux transparent TCP and embedded-runtime integration

   • Add namespace listener/capture helpers and nftables redirect before existing bypass rejection.
   • Recover original destination, resolve workload socket ownership, load the correlation, authorize the current process, consume the pinned connector, and use the shared relay.
   • Enable resolver injection and readiness ordering for Docker and supported combined runtimes.

4. Kubernetes sidecar and remaining runtimes

   • Wire policy DNS and static synthetic-pool capture into init/sidecar topology without granting unnecessary long-running privileges.
   • Validate missing process identity never broadens binary-scoped policy.
   • Add Podman and VM support or an explicit documented capability rejection.

Release documentation is maintained in the stacked branch docs/policy-dns-transparent-tcp/jm and should open only when the complete user-facing capability is ready. It must describe the final product, supported runtimes, failure behavior, operational events, and tested examples rather than intermediate stack states.

OCSF and test contract

Observable decisions and lifecycle changes must use structured OCSF events. Coverage includes listener readiness/failure, lookup eligibility and refusal, filtered/no-valid-address results, mapping publication/invalidation, capability availability, transparent TCP allow, and denials for missing/expired/stale mappings, wrong ports, identity, policy, or destination validation. Events must include useful domain, port, actor, matched policy, transport, and stable rationale without logging credentials, query parameters, full DNS payloads, or unbounded address history.

Stable denial details should include:

• policy_dns_ineligible
• policy_dns_no_valid_address
• transparent_tcp_mapping_missing
• transparent_tcp_mapping_expired
• transparent_tcp_mapping_stale_policy
• transparent_tcp_port_mismatch
• transparent_tcp_identity_denied
• transparent_tcp_destination_denied

Unit, concurrency, and e2e coverage must prove:

• ineligible names never reach an upstream resolver;
• A/AAAA answers obey existing destination controls, exact-host rules, and allowed_ips;
• mappings cannot cross-authorize names, ports, processes, or policy generations;
• refresh, expiry, reload, and pool exhaustion cannot expose partially initialized or stale state;
• the connector performs no connect-time DNS query;
• a native TCP client succeeds by hostname with proxy variables unset only for explicit protocol: tcp;
• omitted protocol retains current proxy-only behavior;
• wrong port, direct real-IP access, cached stale synthetic addresses, and disallowed binaries fail closed;
• Docker, Kubernetes combined/sidecar, Podman, and VM either pass dedicated coverage or reject the capability explicitly;
• existing CONNECT, forward HTTP, WebSocket, bypass detection, local-service, UDP, and policy-generation contracts remain green.

The series is complete when a native TCP client can resolve and connect to an explicitly allowed endpoint without proxy configuration; every connection is correlated, process-authorized, destination-validated, generation-consistent, and limited to pinned DNS answers; and every missing, stale, ambiguous, or unsupported state fails closed with a clear OCSF rationale.

Alternatives Considered

• Populate /etc/hosts from policy: simpler, but cannot safely represent TTL refresh, CNAME changes, exact mapping generations, wildcard/name normalization, or atomic publication. It also encourages IP-to-policy inheritance and makes hot reload semantics fragile.
• Allow direct UDP/53 to a cluster or external resolver: exposes unrestricted query capability and DNS exfiltration, and DNS answers still lack an authoritative connection correlation.
• SOCKS5 or mandatory application proxy support: preserves hostnames but does not work transparently for many database, gRPC, and compiled clients; it pushes language-specific configuration into every workload.
• Authorize resolved real IPs directly: loses hostname identity, lets unrelated names sharing an address inherit policy, and creates stale/rebinding ambiguity.
• Infer identity from TLS SNI or payload sniffing: does not cover non-TLS protocols reliably and makes payload inspection an authorization authority. The synthetic DNS mapping remains the sole correlation authority.
• Grant the Kubernetes network sidecar long-running NET_ADMIN: enables dynamic per-mapping rules but expands privilege unnecessarily. Static synthetic-pool capture plus exact user-space correlation is the preferred v1 design.
• Add provider-specific native protocol inspection: PostgreSQL, Redis, MySQL, and similar L7 inspection are separate follow-ups. This initiative provides L4 authorization and byte relay only.

Agent Investigation

The implementation plan was derived from the merged sandbox proxy egress adapter RFC #2155, its implementation plan, and the merged proxy compatibility refactor #2373. The refactor established CONNECT/forward-HTTP intent adapters, shared destination validation, unopened connectors, relay primitives, policy-generation guards, ambiguity rejection, and nftables bypass enforcement. Those primitives make policy DNS and transparent TCP an additional adapter rather than a second policy engine.

The first foundation increment is implemented in PR #2711: explicit TCP policy syntax, generation-consistent authorization snapshots, and dormant transparent/pinned-destination intent types without activating direct traffic.

Open and closed issues were searched for DNS, native/direct TCP, transparent proxying, and protocol: tcp. Issue #1107 reports the same user problem and proposes static hostname injection; this tracker intentionally supersedes it with the RFC-approved policy-DNS and synthetic-address correlation architecture. Related closed reports such as #387, #364, and #727 demonstrate the application compatibility and DNS symptoms but do not carry the complete security and runtime design.

Implementation still requires evidence-driven maintainer confirmation of concrete collision-checked IPv4/IPv6 synthetic pools, bounded DNS TTL and upstream timeout, original-destination recovery on each runtime, trusted resolver selection/delivery, immediate stale-mapping behavior, no synthetic reassignment in v1, and Kubernetes sidecar static-pool capture.