Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .agents/skills/review-opensecret-security/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,12 @@ database, provider, client, build/artifact, and live evidence separately.

Local artifact builds and read-only PCR comparison are validation when in
scope. Root backend CI validates Rust, Nix checks/default binary, and SDK
compatibility; it does not build or publish EIFs or deploy the TEE service.
compatibility. A separate read-only ARM64 workflow compares dev/prod EIF
measurements on explicit approved-PCR JSON edits in PRs, relevant master
changes, and manual runs. Do not require ordinary backend PRs to update
approvals, and do not suppress meaningful master mismatches. CI never signs,
publishes EIFs, or deploys the TEE service. A passing comparison is not live
deployment evidence or proof that both public PCR locations are synchronized.
Use `docs/pcr-compatibility.md` for manual signed-PCR validation and legacy
publication. Require explicit authorization for PCR
reference/history mutation, signing, KMS/IAM changes, shared or remote
Expand Down
20 changes: 13 additions & 7 deletions .agents/skills/validate-opensecret/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: validate-opensecret
description: Validate OpenSecret changes with focused Rust tests, exact Rust CI parity, disposable PostgreSQL migration and ignored-test proof, separately authorized provider checks, encrypted SDK or Maple smoke tests, Nix checks, and release-only EIF/PCR evidence. Use before claiming backend work complete or when reviewing whether test evidence matches a changed API, provider, persistence, security, build, or deployment boundary.
description: Validate OpenSecret changes with focused Rust tests, exact Rust CI parity, disposable PostgreSQL migration and ignored-test proof, separately authorized provider checks, encrypted SDK or Maple smoke tests, Nix checks, and read-only EIF/PCR evidence. Use before claiming backend work complete or when reviewing whether test evidence matches a changed API, provider, persistence, security, build, or deployment boundary.
---

# Validate OpenSecret
Expand Down Expand Up @@ -170,10 +170,16 @@ nix flake check --no-write-lock-file --print-build-logs '.?submodules=1'
nix build --no-link --no-write-lock-file '.?submodules=1#default'
```

EIF construction, PCR comparison, and reference/history updates are
release-only work. Root backend CI runs applicable Nix checks and builds the
default backend binary; it does not build or publish EIFs or deploy the TEE
service. Ordinary pull-request completion does not update PCR references.
PCR reference/history updates remain operator-controlled release work.
Read-only EIF construction and PCR comparison are validation when in scope.
Root backend CI runs applicable Nix checks and builds the
default backend binary. The separate root EIF approval workflow builds dev/prod
and compares generated measurements on PRs that explicitly edit the four
approved PCR JSON files, relevant backend/TEE or approval changes to master,
and manual runs. Ordinary backend PRs do not require new PCR approvals; master
mismatches intentionally signal that the revision does not match its current
approvals. CI does not sign, publish EIFs, or deploy the TEE service.
Ordinary pull-request completion does not update PCR references.
Do not copy or sign values just to clear a validation failure; distinguish an
EIF build failure from a PCR mismatch. Use `docs/pcr-compatibility.md` for the
offline signed-history validation and manual legacy-publication procedure.
Expand All @@ -193,8 +199,8 @@ configuration, and every unrun or unavailable layer. For release evidence,
also record the target artifact and PCR source.

Use narrow labels: **static/unit**, **disposable DB**, **live provider**, or
**local encrypted full stack**. Use **Linux/Nitro/PCR** or **deployed** only for
authorized release/deployment evidence.
**local encrypted full stack**. Use **Linux/Nitro/PCR** for actual artifact
evidence and **deployed** only for authorized live deployment evidence.
Failed, skipped, ignored, interrupted, timing-dependent, and unavailable checks
remain exactly that; do not turn partial evidence into “fully tested” or
“production ready.”
51 changes: 44 additions & 7 deletions .github/workflows/opensecret-change-detection.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ on:
pcr:
description: Whether signed PCR compatibility checks are needed
value: ${{ jobs.detect.outputs.pcr }}
eif:
description: Whether EIF build inputs or approval references changed
value: ${{ jobs.detect.outputs.eif }}
pcr_approvals:
description: Whether the diff explicitly changes an approved dev/prod PCR JSON file
value: ${{ jobs.detect.outputs.pcr_approvals }}

permissions:
contents: read
Expand All @@ -32,6 +38,8 @@ jobs:
integration: ${{ steps.classify.outputs.integration }}
audit: ${{ steps.classify.outputs.audit }}
pcr: ${{ steps.classify.outputs.pcr }}
eif: ${{ steps.classify.outputs.eif }}
pcr_approvals: ${{ steps.classify.outputs.pcr_approvals }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
Expand All @@ -46,9 +54,18 @@ jobs:
run: |
set -euo pipefail
enable_all() {
for output in rust nix integration audit pcr; do
for output in rust nix integration audit pcr eif; do
printf '%s=true\n' "$output" >> "$GITHUB_OUTPUT"
done
# Unknown/manual input must not masquerade as an approval edit.
printf 'pcr_approvals=false\n' >> "$GITHUB_OUTPUT"
}
selection_failed() {
if [[ "$GITHUB_EVENT_NAME" == pull_request ]]; then
echo "::error::Cannot determine the PR's approved-PCR changes; refusing an ambiguous approval check."
exit 1
fi
enable_all
}
case "$GITHUB_EVENT_NAME" in
pull_request) separator='...' ;;
Expand All @@ -61,19 +78,39 @@ jobs:
;;
schedule)
# Scheduled advisories change independently of repository files.
printf 'rust=false\nnix=false\nintegration=false\naudit=true\npcr=false\n' >> "$GITHUB_OUTPUT"
printf 'rust=false\nnix=false\nintegration=false\naudit=true\npcr=false\neif=false\npcr_approvals=false\n' >> "$GITHUB_OUTPUT"
exit 0
;;
*) enable_all; exit 0 ;;
esac
if ! git cat-file -e "${BASE_SHA}^{commit}" || ! git cat-file -e "${HEAD_SHA}^{commit}"; then
enable_all
selection_failed
exit 0
fi
changed_paths="$(mktemp)"
trap 'rm -f "$changed_paths"' EXIT
selected_checks="$(mktemp)"
trap 'rm -f "$changed_paths" "$selected_checks"' EXIT
if ! git diff --no-renames --name-only -z "${BASE_SHA}${separator}${HEAD_SHA}" > "$changed_paths"; then
enable_all
elif ! python3 scripts/ci/opensecret_change_detection.py < "$changed_paths" >> "$GITHUB_OUTPUT"; then
enable_all
selection_failed
elif ! python3 scripts/ci/opensecret_change_detection.py < "$changed_paths" > "$selected_checks"; then
selection_failed
elif ! python3 - "$selected_checks" <<'PY'
from pathlib import Path
import sys
from scripts.ci.opensecret_change_detection import OUTPUTS

lines = Path(sys.argv[1]).read_text().splitlines()
expected = set(OUTPUTS)
keys = [line.partition("=")[0] for line in lines]
valid = (
len(lines) == len(expected)
and set(keys) == expected
and all(line.partition("=")[2] in {"true", "false"} for line in lines)
)
sys.exit(0 if valid else 1)
PY
then
selection_failed
else
cat "$selected_checks" >> "$GITHUB_OUTPUT"
fi
64 changes: 64 additions & 0 deletions .github/workflows/opensecret-eif.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: OpenSecret EIF approval checks

on:
push:
branches: [master]
pull_request:
branches: [master]
workflow_dispatch:

# This is measurement verification, never signing, publication, or deployment.
permissions:
contents: read

concurrency:
group: opensecret-eif-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: true

jobs:
changes:
uses: ./.github/workflows/opensecret-change-detection.yml

eif:
name: EIF/PCR approval match (${{ matrix.mode }})
needs: changes
# PR comparison requires an actual edit to one of the four approval files.
# A selector failure fails the changes job, not a speculative PCR comparison.
if: >-
${{
always() && !cancelled() &&
(
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'pull_request' &&
needs.changes.result == 'success' &&
needs.changes.outputs.pcr_approvals == 'true') ||
(github.event_name == 'push' && github.ref == 'refs/heads/master' &&
(needs.changes.result != 'success' || needs.changes.outputs.eif != 'false'))
)
}}
runs-on: ubuntu-24.04-arm
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
mode: [dev, prod]
env:
EIF_MODE: ${{ matrix.mode }}
OPENSECRET_DEV_POSTGRES: "0"
OPENSECRET_DEV_ENV: "0"
OPENSECRET_DEV_CONTAINERS: "0"
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
submodules: recursive
fetch-depth: 0

- name: Install pinned Nix
uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25 # v22
with:
github-token: ""

- name: Build EIF and compare approved measurements
shell: bash
run: bash scripts/ci/check_opensecret_eif.sh "$EIF_MODE"
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,10 @@ its table-driven tests when the dependency graph or component layout changes.
The backend has its own root `opensecret-ci.yml` workflow and change selector;
`sdk-integration.yml` tests both SDKs against `services/opensecret/` from the
same checkout. Backend changes do not imply Research or Agent packaging.
The separate `opensecret-eif.yml` compares dev/prod EIF measurements only on PRs
editing approved PCR JSON, relevant master changes, and manual runs. Preserve
ordinary backend PRs without fresh approvals and meaningful master mismatches;
these read-only checks never sign, publish, or authorize deployment.

For Pages, read [the deployment guide](docs/pages-deployments.md). Preserve
unprivileged preview builds and separate development/production profiles.
Expand Down
13 changes: 9 additions & 4 deletions docs/opensecret-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

OpenSecret lives in `services/opensecret/`, retaining its Rust package, Nix
toolchain, operator recipes, submodule revisions, and PCR filenames. This is a
source and development-workflow migration. Building and deploying the TEE
service remains an operator-controlled process.
source and development-workflow migration. Approval, signing, publication, and
deployment remain operator-controlled. Read-only EIF/PCR comparisons now also
run in CI under the [approval-check policy](../services/opensecret/docs/nitro-deploy.md#ci-approval-checks).

## Preserved source boundary

Expand Down Expand Up @@ -48,8 +49,12 @@ backend revision. This advances that lane from the former pinned commit

PR jobs have read-only credentials and use hosted runners. Backend CI has no
EIF publisher, signing credentials, OIDC permission, or deployment step. PCR
file changes have their own validation lane. Backend-only changes do not
select Research or Agent application packaging.
file changes retain their signature-validation lane. A separate ARM64 EIF
workflow compares dev/prod measurements on PRs explicitly editing approved PCR
JSON, relevant backend/TEE or approval changes to master, and manual runs.
Ordinary backend PRs do not fail merely because approvals have not been
updated; master mismatches are an intentional deployment-approval signal.
Backend-only changes do not select Research or Agent application packaging.

The companion OpenSecret Workspaces change supports Maple-only compositions
through `services/opensecret/`. An explicitly included standalone `opensecret`
Expand Down
46 changes: 46 additions & 0 deletions scripts/ci/check_opensecret_eif.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# Build and compare only. Never copy, sign, or publish approval files.
set -euo pipefail

if [[ $# -ne 1 ]]; then
echo "Usage: check_opensecret_eif.sh dev|prod" >&2
exit 2
fi
case "$1" in
dev) reference=pcrDev.json ;;
prod) reference=pcrProd.json ;;
*) echo "Expected dev or prod." >&2; exit 2 ;;
esac
mode=$1

if [[ "$(uname -s)" != Linux || "$(uname -m)" != aarch64 ]]; then
echo "EIF approval checks require a Linux ARM64 runner." >&2
exit 1
fi

repo_root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)
cd "$repo_root/services/opensecret"
if [[ ! -f "$reference" || ! -s "$reference" || -L "$reference" ]]; then
echo "Missing regular approved measurement file: $reference" >&2
exit 1
fi

# Direct Nix invocation avoids just's dotenv loader and development shell hooks.
# A fresh temporary link never replaces an operator's existing result symlink.
output_dir=$(mktemp -d "${TMPDIR:-/tmp}/opensecret-eif-${mode}.XXXXXX")
trap 'rm -rf -- "$output_dir"' EXIT
nix build --no-update-lock-file --print-build-logs \
--out-link "$output_dir/result" ".?submodules=1#eif-$mode"

if [[ ! -f "$output_dir/result/image.eif" || ! -s "$output_dir/result/image.eif" ||
! -f "$output_dir/result/pcr.json" || ! -s "$output_dir/result/pcr.json" ]]; then
echo "EIF build did not produce image.eif and pcr.json." >&2
exit 1
fi

if ! diff -u -- "$reference" "$output_dir/result/pcr.json"; then
echo "EIF/PCR approval mismatch ($mode): this build does not match $reference." >&2
echo "Review the measurements through the manual approval process; CI will not update or sign them." >&2
exit 1
fi
echo "EIF/PCR approval match ($mode). This does not authorize deployment."
37 changes: 24 additions & 13 deletions scripts/ci/opensecret_change_detection.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,24 @@
import sys


OUTPUTS = ("rust", "nix", "integration", "audit", "pcr")
ALL_OUTPUTS = frozenset(OUTPUTS)
CHECK_OUTPUTS = ("rust", "nix", "integration", "audit", "pcr", "eif")
OUTPUTS = (*CHECK_OUTPUTS, "pcr_approvals")
ALL_CHECKS = frozenset(CHECK_OUTPUTS)
BACKEND_PREFIX = "services/opensecret/"
BACKEND_INERT_FILES = frozenset({
"AGENTS.md", "README.md", "LICENSE", ".gitignore", ".gitmodules",
})
PCR_INPUTS = frozenset({
APPROVED_PCR_FILES = frozenset({
"pcrDev.json", "pcrDevHistory.json", "pcrProd.json", "pcrProdHistory.json",
})
PCR_INPUTS = frozenset({
"pcrPreview.json", "pcrPreviewHistory.json", "pcr_verify.js", "pcr_sign.js",
"scripts/pcr_compatibility.py", "scripts/test_pcr_compatibility.py",
})
EIF_CI_INPUTS = frozenset({
".github/workflows/opensecret-eif.yml",
"scripts/ci/check_opensecret_eif.sh",
})
# Shell test inputs consumed directly by the component flake, not Cargo.
NIX_TEST_INPUTS = frozenset({"tests/entrypoint_entropy_preflight.sh"})
BACKEND_INERT_PREFIXES = ("docs/", ".agents/", ".github/")
Expand Down Expand Up @@ -46,25 +53,29 @@

def classify_path(path: str) -> frozenset[str]:
if not path or path.startswith("/") or ".." in path.split("/"):
return ALL_OUTPUTS
return ALL_CHECKS
if path in SHARED_INPUTS:
return ALL_OUTPUTS
return ALL_CHECKS
if path in EIF_CI_INPUTS:
return frozenset({"eif"})
if path == ".github/workflows/opensecret-ci.yml":
return frozenset({"rust", "nix", "audit", "pcr"})
return frozenset({"rust", "nix", "audit", "pcr", "eif"})
if path in SDK_INTEGRATION_FILES or path.startswith(("sdk/src/", "sdk/rust/", "sdk/test/")):
return frozenset({"integration"})
if path.startswith(BACKEND_PREFIX):
relative = path.removeprefix(BACKEND_PREFIX)
if relative in BACKEND_INERT_FILES or relative.startswith(BACKEND_INERT_PREFIXES):
return frozenset()
if relative in APPROVED_PCR_FILES:
return frozenset({"pcr", "eif", "pcr_approvals"})
if relative in PCR_INPUTS:
return frozenset({"pcr"})
if relative == "deny.toml":
return frozenset({"audit"})
if relative in {"Cargo.toml", "Cargo.lock", "rust-toolchain.toml", "flake.nix", "flake.lock"}:
return ALL_OUTPUTS
return ALL_CHECKS
if relative.startswith(("src/", ".cargo/")) or relative == "build.rs":
return frozenset({"rust", "nix", "integration"})
return frozenset({"rust", "nix", "integration", "eif"})
if relative in NIX_TEST_INPUTS:
return frozenset({"nix"})
if relative.startswith(("tests/", "migrations/")):
Expand All @@ -74,13 +85,13 @@ def classify_path(path: str) -> frozenset[str]:
if relative.startswith(("nix/", "nitro-toolkit/", "privatemode-public/")) or relative in {
"entrypoint.sh", "continuum-proxy", "nitro-toolkit", "privatemode-public",
}:
return frozenset({"nix"})
return frozenset({"nix", "eif"})
# Unknown backend files could be build or runtime inputs.
return ALL_OUTPUTS
return ALL_CHECKS
if path.startswith("sdk/") or path in KNOWN_INDEPENDENT_FILES or path.startswith(KNOWN_INDEPENDENT_PREFIXES):
return frozenset()
# New roots must be classified explicitly before checks can be skipped.
return ALL_OUTPUTS
return ALL_CHECKS


def classify_paths(paths: Iterable[str]) -> dict[str, bool]:
Expand All @@ -92,10 +103,10 @@ def classify_paths(paths: Iterable[str]) -> dict[str, bool]:

def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--all", action="store_true", help="Select all checks when the diff is unavailable")
parser.add_argument("--all", action="store_true", help="Select all checks without claiming approvals changed")
args = parser.parse_args()
if args.all:
result = dict.fromkeys(OUTPUTS, True)
result = {name: name in ALL_CHECKS for name in OUTPUTS}
else:
paths = (path.decode("utf-8", errors="surrogateescape")
for path in sys.stdin.buffer.read().split(b"\0") if path)
Expand Down
Loading
Loading