feat(kubernetes): add cni-sidecar supervisor topology - #2606
Open
russellb wants to merge 27 commits into
Open
Conversation
russellb
changed the title
Contributor
Author
|
I think this will need more review/fix iterations, but it's fine to review and try out now. |
russellb
marked this pull request as ready for review
August 4, 2026 23:29
russellb
requested review from
a team,
derekwaynecarr,
maxamillion and
mrunalp
as code owners
August 4, 2026 23:29
russellb
force-pushed
the
feat/kubernetes-cni-sidecar-topology
branch
from
August 11, 2026 22:23
5ce726a to
c4c8a85
Compare
3 tasks
Collaborator
|
I pointed [P1] Drain-gated enforcement misses most sandbox podsdriver.rs:2484-2489 adds openshell.ai/managed-by=openshell only when SPIFFE is enabled. However, the CNI drain logic relies on that label to retain namespaces containing live sandboxes (lib.rs:272-279). With the default provider_spiffe_enabled=false, removing a registration marker—during uninstall or sandbox namespace migration—canremove the namespace from the allowlist even while sandboxes remain. Recreated pods would then bypass CNI enforcement entirely. The managed-by label should be added for every CniSidecar pod, independently of SPIFFE. [P2] CNI CHECK accepts incomplete firewall ruleslib.rs:1131-1146 considers enforcement valid when the nft table or IPv4 iptables chain merely exists. It does not verify:
An empty or partially damaged ruleset therefore passes CNI CHECK while allowing policy bypass. The read-only check should validate the expected rule structure for the active backend and both required address families. [P2] cni.only can render a non-CNI registration resourceThe new mode promises to render “ONLY the CNI singleton,” but cni-registration.yaml is not guarded by not .Values.cni.only. If topology Reproduction: helm template openshell-cni deploy/helm/openshell \
-n openshell-cni \
--set cni.only=true \
--set cni.enabled=true \
--set supervisor.topology=cni-sidecarThis renders: That violates the mode’s contract and unnecessarily adds the CNI release namespace to the enforcement allowlist. cni-registration.yaml should require not .Values.cni.only, and a test should cover this value combination. |
russellb
added a commit
to russellb/OpenShell
that referenced
this pull request
Aug 13, 2026
Addresses the PR review (NVIDIA#2606) and adds the mixed-version guardrail. - Drain-gate label (P1): add openshell.ai/managed-by=openshell to every cni-sidecar pod, not only when SPIFFE is enabled. The node CNI's drain-gated allowlist keys on it to keep a namespace enforced while sandboxes are live; without it, removing a registration marker could drop the namespace from the allowlist with pods still running, letting recreated pods bypass enforcement. - CHECK validation (P2): CNI CHECK now validates rule STRUCTURE, not mere existence. nft: the OUTPUT hook, the proxy-UID exemption, and TCP/UDP rejection across both address families. iptables: the OUTPUT jump plus uid-owner and tcp/udp REJECT per family, with IPv6 gated on pod connectivity (mirrors install). An empty or partially damaged ruleset no longer passes. - cni.only registration guard (P2): guard cni-registration.yaml with not .Values.cni.only so a standalone CNI release renders no registration marker even if a cni-sidecar topology value is set. Adds a helm-unittest case. - Contract-version gate: the gateway stamps openshell.ai/cni-contract-version on each cni-sidecar pod; the node plugin fails closed (refuses CNI ADD) when it does not implement the required version, so a mixed-version cluster fails safe instead of mis-enforcing. It accepts any version at or below its own (backward compatible within a major) and treats an absent annotation as a pre-versioning gateway. Docs: topology.mdx and compute-runtimes.md updated for the version gate. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Add a cni-sidecar supervisor topology that keeps the sidecar runtime model but installs pod-network bypass-prevention rules through a privileged OpenShell chained CNI plugin instead of a pod-local network init container. A new openshell-cni crate provides the chained plugin and node installer. The Kubernetes driver emits openshell.ai/* pod annotations consumed by the plugin during CNI ADD, omits the network init container in this topology, and reuses the sidecar network-only process supervision path. Helm gains a privileged CNI installer DaemonSet gated on cni.enabled and a supervisor.topology=cni-sidecar option. Reconstructed on current main: uses the renamed topology config field and drops the abandoned proxy-pod topology. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
On OpenShift the binary-aware network sidecar must run as UID 0 with SYS_PTRACE and DAC_READ_SEARCH to inspect cross-UID /proc, which the restricted-v2 SCC forbids. Add a gated minimal SecurityContextConstraints (restricted-v2 baseline plus exactly those two capabilities and the image volume type) with a ClusterRole and ClusterRoleBinding granting it to the sandbox ServiceAccount. Controlled by sandboxServiceAccount.openshift.binaryAwareSCC (default false); the cni-sidecar OpenShift overlay enables it. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Remediate merge-blocking review findings on the cni-sidecar supervisor topology: - driver: mount the gateway client mTLS bundle directly onto the network sidecar (cni-sidecar omits network-init, so the bundle was never seeded and the sidecar crash-looped on policy fetch) - driver: annotate the effective sidecar proxy UID (0 in binary-aware mode) so the node CNI exempts the right identity instead of redirecting the sidecar's own egress - cni: fail closed when ip6tables is missing in the iptables fallback so a dual-stack sandbox cannot bypass policy over IPv6 - cni: emit a CNI-spec error object on stdout so the runtime surfaces plugin failures instead of an opaque crash - helm: re-patch the chained plugin on the reconcile tick when absent so a CNI config rewrite or restart cannot drop egress enforcement - helm: keep the component label out of the immutable gateway selector so existing releases can upgrade - helm: mirror supervisorImage repository/tag fallbacks for the CNI image - docs: document the sidecar privilege profile and cold-start window; trim RFC non-goals Signed-off-by: Russell Bryant <rbryant@redhat.com>
Address the second-round review of the cni-sidecar topology: - cni/driver: add a per-node readiness scheduling gate. The installer labels its node openshell.ai/cni-ready via a new `openshell-cni node-ready` subcommand (minimal cluster-scoped nodes get/patch RBAC), clearing it on shutdown or when a reconcile tick cannot restore the plugin; the driver sets a required nodeAffinity on that label for cni-sidecar sandbox pods, so a pod cannot schedule before per-node egress enforcement is active. - driver: run the untrusted workspace-init container as the sandbox UID under binary-aware policy instead of root, so it cannot inherit the CNI's UID-0 egress exemption and bypass policy before the sidecar enforces it. - helm: gate the conflist/chain-conf mv on jq success and non-empty output so a jq failure cannot truncate the host CNI config and break node networking. - helm: give the CNI DaemonSet a distinct app name (openshell-cni) and revert the gateway Service selector to base-only, so a helm upgrade no longer blanks gateway endpoints for pods that predate the component label. - cni: only require ip6tables in the iptables fallback when the pod has a routable IPv6 address (from prevResult), so IPv4-only nodes are not rejected. - build: add crates/openshell-cni/BUILD.bazel so Bazel and the aggregate rustfmt suite cover the crate. - docs: document the scheduling gate and corrected sidecar privilege profile across architecture, published docs, crate READMEs, values, AGENTS, and the debug-openshell-cluster skill. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Address the third review round of the cni-sidecar topology: - cni/helm: fence before repair and stop failing open on teardown. The reconcile loop clears the readiness label the instant enforcement is not verifiably present, then repairs and re-marks. Enforcement lives in the host CNI config and survives pod restarts, so preStop no longer strips it on an ordinary restart/rolling update: a new `openshell-cni daemonset-active` check removes it only when the owning DaemonSet is confirmed terminating, and fences the node first. Reconcile interval 300s -> 30s. - cni/helm: stamp an owner (<namespace>/<release>) on the chained plugin entry and refuse to overwrite an entry owned by a different release (fail closed). Only one OpenShell release per cluster is supported for cni-sidecar today; documented in the RFC. - helm: validate the multus chain conf content (type, owner, kubeconfig, non-empty namespaces) before marking the node ready, instead of accepting any existing file. - driver: harden the untrusted workspace-init container (allowPrivilegeEscalation=false, drop ALL capabilities, runAsNonRoot when non-root) so a setuid binary cannot regain the exempt UID 0; restrict the non-root init to the Sidecar/CniSidecar topologies so combined keeps root. - cni: determine IPv6 enforcement by probing /proc/net/if_inet6 in the pod netns (fail closed on indeterminate; link-local counts) instead of trusting prevResult, keeping IPv4-only nodes working without ip6tables. - build: stage openshell-cni in the prebuilt 'all' target. - rbac: add get on the installer's own DaemonSet (resourceNames-scoped). Signed-off-by: Russell Bryant <rbryant@redhat.com>
Address the fourth review round of the cni-sidecar topology: - cni/helm: convert the CNI installer to a cluster singleton. The chained plugin enforces any pod carrying the OpenShell annotations (set only by a gateway on its own sandbox pods), so a single installation serves every release across all namespaces. The plugin config uses an empty sandboxNamespaces allowlist and a fixed 'openshell' owner; the installer resources use release-independent names; pods get RBAC becomes cluster-scoped. This fixes the multi-release fail-open: a second release no longer relies on a namespace list that another release could overwrite. Additional gateway releases set cni.enabled=false + cni.external=true to share the singleton. - helm: bind node readiness to a configVersion stamp so a stale-version entry is repaired before the node is re-marked ready; fence before the initial patch on startup; retry mark_unready so a transient API error does not leave stale readiness. - helm: create the host SA token atomically with install -m 0600 (no briefly world-readable node-patch credential). - driver: keep default root capabilities for the combined-topology workspace init (UID 0 is not CNI-exempt there) so it retains DAC read access; apply the drop-ALL/no-privilege-escalation/runAsNonRoot hardening only to the non-root exempt (Sidecar/CniSidecar binary-aware) case. - docs: add a cni-sidecar enable-and-verify walkthrough to the OpenShift page, add user-facing example values files under deploy/helm/openshell/examples/ (replacing the ci/ fixture in docs), and update topology, compute-runtimes, and the RFC to the singleton model. Signed-off-by: Russell Bryant <rbryant@redhat.com>
… entry in place The singleton installer refused to overwrite an openshell-cni chained plugin entry whose owner value differed from the fixed "openshell" owner. That guard was redundant — the conflist patch already preserves every non-openshell-cni plugin and replaces only the openshell-cni entry, and the plugin type is itself the ownership signal — and it broke in-place upgrades from an earlier owner format (e.g. "<namespace>/<release>"), crash-looping the installer. Treat any existing openshell-cni entry as ours to upgrade in place. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Address the fifth review round of the cni-sidecar topology: - driver: harden the untrusted workspace-init in EVERY sidecar topology, not only the binary-aware case. A non-binary-aware Sidecar/CniSidecar init still needs UID 0 but now drops all capabilities and disables privilege escalation, so it cannot CAP_SETUID to the exempt proxy UID or use CAP_NET_RAW to bypass egress. Only the Combined topology (no separate exempt sidecar) keeps default root capabilities for DAC read access. - cni/helm: add cni.sandboxNamespaces (default: the release's sandbox namespace). The plugin passes through pods in unlisted namespaces WITHOUT a Kubernetes API lookup, so control-plane/RBAC problems no longer block unrelated workloads' pod creation. Multi-release deployments list every sandbox namespace on the singleton. Bound into configVersion. - cni/helm: close the node-reboot fail-open window. openshell-cni node-ready now removes a boot-time NoSchedule taint (openshell.ai/cni-not-ready) once enforcement is ready (optimistic-concurrency JSON patch); the DaemonSet tolerates it. Boot-time application is operator node config — an OpenShift MachineConfig example ships under examples/ with caveats. conflist mode is disk-backed and unaffected by reboot. - helm: make the CNI singleton name a genuinely fixed constant (openshell-cni), not derived from nameOverride, so releases cannot install competing singletons over shared host state. - docs: document cni.external and the singleton in setup.mdx and the debug-openshell-cluster skill; update the RFC and architecture docs. Signed-off-by: Russell Bryant <rbryant@redhat.com>
…ngleton docs Address the sixth review round (docs/examples only): - Fix the additional-release workflow: the singleton only enforces namespaces in its cni.sandboxNamespaces allowlist, so an extra cni.external release whose namespace is not added to the owner's allowlist runs unenforced. The extra-release example and the OpenShift docs now require (and show) the two-step flow — add the namespace to the singleton owner and let nodes reconcile, THEN install the additional gateway — with prominent warnings. - Harden the boot-taint MachineConfig example: retry-until-tainted loop, Restart=on-failure, kubelet --register-with-taints for the initial join, and an honest note that a small residual reboot race remains (no chart-shippable mechanism is fully race-free; conflist mode is unaffected). - Correct stale docs to match the implementation: topology and architecture docs described an empty all-namespaces allowlist and a foreign-owner refusal, but the plugin uses an explicit sandboxNamespaces allowlist and upgrades any openshell-cni entry in place. Add cni.sandboxNamespaces and boot-taint diagnosis to the debug-openshell-cluster skill. Signed-off-by: Russell Bryant <rbryant@redhat.com>
…owlist Round seven: make external-release enforcement machine-verifiable, and correct the remaining docs. - cni: add `openshell-cni label-namespace` and `list-sandbox-namespaces` subcommands. Each cni-sidecar gateway release runs a helm hook that labels its sandbox namespace openshell.ai/sandbox=true; the CNI singleton's reconcile aggregates every labeled namespace (unioned with the optional static cni.sandboxNamespaces) into the plugin allowlist and rebinds configVersion. An additional cni.external release is therefore discovered and enforced within one reconcile — no manual owner-allowlist edit, closing the silent-bypass footgun. - helm: add the namespace-label hook (Job + minimal namespaces get/patch RBAC scoped to the release's sandbox namespace) and grant the CNI ClusterRole namespaces list. On discovery failure the reconcile keeps the installed allowlist rather than shrinking it (never fails a running namespace open). - docs: stop claiming the boot taint 'closes' the reboot window (it narrows it; a residual race remains — conflist mode is unaffected). Correct topology, architecture, RFC, and the debug skill to describe the explicit auto-aggregated allowlist and in-place upgrade of any openshell-cni entry, and fix stale RFC claims (cni.enabled-or-external, 30s reconcile). Signed-off-by: Russell Bryant <rbryant@redhat.com>
Round eight: close the registration race and give registration a proper lifecycle. - helm/cni: replace the namespace-label hook with a Helm-owned marker ConfigMap (openshell.ai/cni-registration=true) in each release's sandbox namespace. The installer discovers namespaces by listing marker ConfigMaps (RBAC: configmaps list). Because the marker is a normal Helm resource, uninstalling a release or changing its sandboxNamespace removes the registration automatically — no orphaned allowlist entries. This also drops the hook Job, its namespace-patch RBAC, and the per-release ClusterRole name collision. - cni/helm: close the discovery-window race. The installer publishes on each node the CSV of namespaces it enforces (openshell.ai/cni-sandbox-namespaces annotation, via set-node-coverage); every gateway runs a wait-coverage init container that blocks serving until every cni-ready node acknowledges the gateway's namespace. A newly-registered release therefore cannot create sandboxes before enforcement is confirmed cluster-wide. - docs: update topology, architecture, RFC, values, and the debug skill to the marker + wait-for-ack model; fix the RFC boot-taint 'closes' -> 'narrows' and the design spec 300s -> 30s reconcile interval. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Three hardening changes to the cluster-singleton OpenShell CNI installer and its enforcement allowlist: - Drain-gated allowlist: the reconcile now unions registration-marker namespaces with namespaces that still contain an OpenShell-managed sandbox pod (openshell.ai/managed-by=openshell). Removing a release's marker no longer immediately drops enforcement for a namespace whose sandboxes are still running (which would fail-open their recreated pods); the namespace is pruned only once drained. Adds `pods list` to the CNI ClusterRole. - Read-only CNI CHECK: CHECK now verifies the bypass-prevention rules are present without reinstalling them, so a check never leaves a running pod momentarily unenforced. - Atomic nft install: ensure-exists, delete, and recreate run in a single `nft -f` transaction, so a failed apply never leaves the pod with the table deleted-but-not-recreated (unenforced). - Bounded kube-client timeouts on the CNI ADD path and the installer reconcile/coverage loops, so an API stall cannot wedge pod creation or stall reconciliation. Signed-off-by: Russell Bryant <rbryant@redhat.com>
- The k3s example and CI overlay set cni.binDir to /bin, which is not in k3s's containerd CNI search path. The patched conflist would then reference a plugin containerd cannot find, potentially blocking network setup for new pods. Point binDir at the stable, upgrade-safe k3s data CNI directory (/var/lib/rancher/k3s/data/cni; see k3s-io/k3s#10869). This path is not yet validated on a live k3s cluster — only openshift/multus has been exercised so far. - Update stale registration docs. The extra-release example and the CNI DaemonSet inline comments still described a helm hook labeling a namespace openshell.ai/sandbox=true. Registration now uses a Helm-owned marker ConfigMap (openshell.ai/cni-registration=true), unioned with namespaces that still have running openshell.ai/managed-by=openshell sandbox pods (drain-gated). Signed-off-by: Russell Bryant <rbryant@redhat.com>
The superpowers workflow writes local plan/spec working docs under docs/superpowers/. These are scratch, not versioned artifacts (design proposals belong in rfc/ and canonical docs in architecture/ or docs/). A design spec was accidentally committed because the directory was not ignored. Untrack it and add docs/superpowers/ to .gitignore, mirroring the existing architecture/plans entry. Signed-off-by: Russell Bryant <rbryant@redhat.com>
The cni-sidecar CNI installer is a cluster singleton, but its DaemonSet was only installable as part of a gateway release (cni.enabled=true). That couples node enforcement to one gateway's lifecycle: uninstalling the owning gateway strips egress enforcement for every other cni.external=true gateway still running, and there is no way to lifecycle the DaemonSet independently. Add cni.only: when set (with cni.enabled=true) the release renders only the CNI singleton — DaemonSet, RBAC, and (on OpenShift) SCC — and suppresses all gateway workload and gateway-support templates. Multi-gateway clusters can now install the CNI as its own release (examples/cni-standalone.yaml) and run every gateway as cni.external=true, so adding or removing a gateway never installs or strips node enforcement. - values.yaml: add cni.only (default false); cni-daemonset fails if cni.only is set without cni.enabled. - Gate every gateway-scoped template on (not cni.only); workload validation runs only from the gateway workload templates, so a CNI-only release skips it. - Add cni_only_test.yaml (renders only CNI resources; fails without enabled). - Docs: examples/cni-standalone.yaml, topology.mdx "Running multiple gateways" (recommend standalone release; note single-owner install-first/remove-last fallback and that the singleton image is not yet version-negotiated across releases), compute-runtimes.md, and the debug-openshell-cluster skill. Version-compatibility handling for a single singleton serving multiple gateway versions is deferred; it is coupled to the RFC 0012 contract-version design. Signed-off-by: Russell Bryant <rbryant@redhat.com>
The OpenShift additional-releases note only described reusing the singleton via cni.external=true. Add a pointer to the standalone CNI-only release (cni.only=true, examples/cni-standalone.yaml) as the preferred multi-gateway layout, linking the topology "Running multiple gateways" section, so node enforcement is not tied to any gateway's lifecycle. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Addresses the PR review (NVIDIA#2606) and adds the mixed-version guardrail. - Drain-gate label (P1): add openshell.ai/managed-by=openshell to every cni-sidecar pod, not only when SPIFFE is enabled. The node CNI's drain-gated allowlist keys on it to keep a namespace enforced while sandboxes are live; without it, removing a registration marker could drop the namespace from the allowlist with pods still running, letting recreated pods bypass enforcement. - CHECK validation (P2): CNI CHECK now validates rule STRUCTURE, not mere existence. nft: the OUTPUT hook, the proxy-UID exemption, and TCP/UDP rejection across both address families. iptables: the OUTPUT jump plus uid-owner and tcp/udp REJECT per family, with IPv6 gated on pod connectivity (mirrors install). An empty or partially damaged ruleset no longer passes. - cni.only registration guard (P2): guard cni-registration.yaml with not .Values.cni.only so a standalone CNI release renders no registration marker even if a cni-sidecar topology value is set. Adds a helm-unittest case. - Contract-version gate: the gateway stamps openshell.ai/cni-contract-version on each cni-sidecar pod; the node plugin fails closed (refuses CNI ADD) when it does not implement the required version, so a mixed-version cluster fails safe instead of mis-enforcing. It accepts any version at or below its own (backward compatible within a major) and treats an absent annotation as a pre-versioning gateway. Docs: topology.mdx and compute-runtimes.md updated for the version gate. Signed-off-by: Russell Bryant <rbryant@redhat.com>
The cni-sidecar RFC described the cluster-singleton and its lifecycle but not how one node plugin version coexists with gateways spanning OpenShell versions. Add a Risks entry documenting the versioned CNI-to-pod contract: the gateway stamps openshell.ai/cni-contract-version and the node plugin fails closed on a version it does not implement (backward compatible within a major, absent = pre-versioning), with independent CNI lifecycle (cni.only) and node pools as the operator mitigations and RFC 0012 as the long-term home. Also update the CNI CHECK note: it now validates ruleset structure, not mere table/chain existence. Signed-off-by: Russell Bryant <rbryant@redhat.com>
Rebase integration: main added an upstream-proxy credential Secret whose defaultMode is chosen by a match on SupervisorTopology that only handled Combined and Sidecar. Add the CniSidecar arm — it uses the same sidecar runtime model as Sidecar (the non-root network supervisor reads the credential via the pod fsGroup), so it takes the same 0o440 mode. Restores an exhaustive match after the rebase onto main. Signed-off-by: Russell Bryant <rbryant@redhat.com>
russellb
force-pushed
the
feat/kubernetes-cni-sidecar-topology
branch
from
August 14, 2026 19:27
a46d086 to
569f170
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
cni-sidecar, a third Kubernetes supervisor topology for OpenShell sandboxpods. It keeps the split-supervisor model of the
sidecartopology but movespod-network rule installation out of the sandbox pod: a privileged, node-level
OpenShell CNI DaemonSet installs a chained CNI plugin that programs the
bypass-prevention rules during CNI
ADD, removing the per-pod privileged networkinit container. Includes OpenShift/Multus enablement (multus-chain install mode,
purpose-built SCCs) and a per-node readiness scheduling gate.
Related Issue
Related: #899 (restricted SCC support for
managed Kubernetes). Design is captured in the in-branch RFC
(
rfc/cni-sidecar-topology-DRAFT.md).Changes
cni-sidecarSupervisorTopologyin the Kubernetes driver;omits the in-pod network-init container and annotates sandbox pods for the
chained plugin.
openshell-cni): chained plugin that installs nftables/iptablesbypass-prevention rules in the pod netns during CNI
ADD; fail-closed on jqand IPv6-enforcement errors; structured CNI-spec error objects on stdout.
openshell-cni node-readysubcommand labels thenode
openshell.ai/cni-ready(minimal cluster-scopednodes get;patchRBAC),cleared on shutdown/repair-failure; the driver sets a required
nodeAffinityso sandbox pods cannot schedule before per-node enforcement is active.
workspace-initcontainer runs as the sandboxUID (non-exempt) under binary-aware policy, so it cannot inherit the CNI's UID-0
egress exemption.
the gateway Service selector stays base-only (no upgrade endpoint gap);
OpenShift
multus-chainmode and cni-sidecar values overlay; minimalbinary-aware sandbox SCC.
compute-runtimes.md), published docs(
docs/kubernetes/topology.mdx), crate READMEs, and thedebug-openshell-clusterskill.Testing
Platform coverage so far: on-cluster E2E has been performed on OpenShift
(Multus / OVN-Kubernetes) using
multus-chainmode only. The defaultconflistmode (vanilla Kubernetes / k3s) is covered by unit tests but has notyet been validated on a live cluster.
mise run pre-commitpassestests) — covers both
conflistandmultus-chainpathsmultus-chainonly: node labeled
cni-ready, sandbox pod scheduled 2/2 with the requirednodeAffinity,
workspace-initrunning as the sandbox UID, sidecarauthenticated to the gateway, and default-deny egress enforced.
conflistmode) — not yet doneChecklist