Skip to content

Deterministic golden fixtures and a trace-format gate - #47

Merged
Jo5ta merged 1 commit into
mainfrom
feat/golden-version-gate
Jul 28, 2026
Merged

Deterministic golden fixtures and a trace-format gate#47
Jo5ta merged 1 commit into
mainfrom
feat/golden-version-gate

Conversation

@Jo5ta

@Jo5ta Jo5ta commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

What

Makes golden trace-file generation byte-deterministic and adds a CI gate
that grows the fixture corpus only when the trace format actually changes.

Why

Golden fixtures let us test the decoders against known trace files, but until
now a fresh generation was non-deterministic (timestamps, pid, tid, span ids
vary per run), so nothing could tell "did the format change?" from "this is
just a new run". The result was a manual, easy-to-forget policy.

How

  • frozen_info.c defines the three abstraction symbols the library uses to
    stamp entries (info_get_timestamp_ns/process_id/thread_id) with fixed
    constants. Linked ahead of the tracing static archive in the golden writer,
    it satisfies those references so the archive's info.o is never pulled —
    freezing timestamp, pid, tid, and (through the span-id salt derived from
    them) span ids. A generated trace is then byte-identical run to run. The
    core library is unchanged.
  • One C++ writer (writer.cpp) exercises every tracepoint kind — printf,
    dump, dynamic (CLLTK_DYN_TRACEPOINT), spans, and fmt — into a single
    GOLDEN fixture. It's C++ because fmt is C++20-only; the other kinds compile
    identically from C and the on-disk format is language-independent, so one
    writer covers the whole decoder surface. Built by the golden-generate cmake
    target behind the CLLTK_GOLDEN option and the golden preset;
    generate.sh runs it in a per-arch Fedora container.
  • The gate (scripts/ci-cd/step_golden.sh) regenerates the fixture at HEAD
    for both byte orders and byte-compares it against the newest committed
    golden, masking only the library-version field and its header crc
    (normalize.py, endianness-agnostic). Identical → the corpus already covers
    this format. Different → the format changed; regenerate_golden.sh writes a
    new version-named fixture (and refuses to overwrite an existing same-version
    baseline, so old-format coverage can't be silently dropped). Runs as its own
    CI job (qemu for le-aarch64 and be-s390x) and in run_all.sh
    (--skip-golden, auto-skipped when no container engine is present).
  • ELF metadata fixtures reproducible from the same source. The previous
    committed golden-1.3.0-be-s390x.{o,so} blobs (for the clltk meta ELF test)
    had no source. They're now regenerated from writer.cpp compiled as a
    relocatable object + shared object (generate.sh --elf, golden-1.7.7-be-s390x.{o,so}).
    A separate cmake target keeps the format gate from building them, so an ELF
    build issue can't masquerade as a trace-format change.
  • Deterministic 1.7.7 baselines for both byte orders. test_golden.py
    decodes every fixture with both decoders as the correctness oracle.

Testing

  • Generation verified byte-deterministic across runs for both byte orders
    (le-aarch64 native, be-s390x under qemu).
  • All fixtures decode to the expected messages with the Python and CLI
    decoders; the comprehensive fixture covers every tracepoint kind; clltk meta
    extracts the metadata from the committed s390x ELF objects cross-endian.
  • Gate validated end-to-end: passes when the corpus is current; version-only
    changes are masked; a real payload change is caught.
  • shellcheck (-x) clean; CMake presets / CI YAML validated.

@Jo5ta
Jo5ta force-pushed the feat/golden-version-gate branch 9 times, most recently from 05e645e to 790befb Compare July 28, 2026 13:39
Make golden trace-file generation byte-deterministic and add a CI gate that
grows the fixture corpus only on real format changes.

- frozen_info.c freezes timestamp/pid/tid (and the span-id salt derived from
  them); linked into the golden writer so a generated trace is byte-identical
  run to run. The core library is unchanged.
- One C++ writer (writer.cpp) exercises every tracepoint kind -- printf, dump,
  dynamic, spans, and fmt -- into a single GOLDEN fixture. The same source,
  compiled as a relocatable object and a shared object, provides the ELF
  metadata fixtures for the 'clltk meta' test (both byte orders; the old
  source-less 1.3.0 blobs are kept only as a backward-compat anchor). Built by
  cmake targets behind the CLLTK_GOLDEN option and the golden preset;
  generate.sh runs it in a per-arch Fedora container.
- step_golden.sh regenerates the fixture at HEAD for both byte orders and
  byte-compares it against the newest committed golden, masking only the
  library-version field (normalize.py, endianness-agnostic). A difference means
  the format changed; regenerate_golden.sh writes the new version's fixture and
  refuses to overwrite an existing same-version baseline. Wired as a CI job
  (qemu for le-aarch64 and be-s390x) and into run_all.sh.
- Fixtures are laid out one folder per library version, <version>/<arch>.<ext>;
  old big-endian traces backfilled (era-adaptive) so every trace folder has
  both byte orders.
- Deterministic 1.7.7 baselines for both byte orders; test_golden.py decodes
  every fixture with both decoders as the correctness oracle.

Signed-off-by: Jo5ta <jo5ta@mail.de>
@Jo5ta
Jo5ta force-pushed the feat/golden-version-gate branch from 790befb to 336bfff Compare July 28, 2026 13:51
@Jo5ta
Jo5ta merged commit f0f497e into main Jul 28, 2026
24 checks passed
@Jo5ta
Jo5ta deleted the feat/golden-version-gate branch July 28, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant