Skip to content

feat(mcp): add manage_adr mode='set_sections' and splice ADR sections instead of rebuilding - #1904

Merged
DeusData merged 3 commits into
mainfrom
feat/adr-section-update
Aug 29, 2026
Merged

feat(mcp): add manage_adr mode='set_sections' and splice ADR sections instead of rebuilding#1904
DeusData merged 3 commits into
mainfrom
feat/adr-section-update

Conversation

@DeusData

Copy link
Copy Markdown
Owner

Adds manage_adr mode='set_sections' — a retry-safe way to update named ADR sections — and fixes the document-rewriting it exposed. Distilled from #1243 with Co-authored-by: credit to @andis777, whose analysis and tests this builds on.

Why sections rather than append

#1243 proposed whole-document append. It is non-idempotent on client retry: a lost response means a silently duplicated chunk. Section updates are retry-safe by construction — the same request twice produces the same document.

The design that matters: splice, don't re-render

The first implementation merged by parsing the ADR, applying updates to the model, and re-rendering. That silently destroyed content, because parse → render is lossy today:

document shape old behaviour
preamble before the first heading dropped entirely
## Purpose (wrong case) not a heading → whole block deleted
## CUSTOM in prose absorbed into the section above
fenced ```md / ## Example absorbed
sections out of canonical order silently reordered
multi-blank separator runs, trailing newline collapsed / trimmed

An ADR beginning ## Purpose instead of ## PURPOSE — an ordinary typo — would have lost that entire section on the first set_sections call. mode='update' never had this exposure because it writes the caller's bytes verbatim; the merge introduced it.

So the merge no longer rebuilds anything. cbm_adr_splice_section locates the target heading's byte span and replaces only that span, appending a block when the heading is absent. Nothing outside the span is reconstructed, so untouched bytes are byte-identical by construction rather than by care. Every row in that table is now a regression test asserting the exact expected document.

cbm_adr_parse_sections is untouched and TEST(adr_parse_sections_non_canonical) passes unchanged — documents parse exactly as they did. That test was the tripwire: wanting to edit it would have meant the design had drifted back toward re-rendering.

Arbitrary headings now work

Custom headings are real sections, which is what the original request needed. ## DECISIONS is simply a span to locate or a block to append — no format migration, because stored ADRs are never rewritten wholesale.

  • Exact case matching: ## Purpose and ## PURPOSE are independently targetable in one document. Case-folding would silently merge two blocks a user deliberately kept apart.
  • Canonical sections become conventional, not privileged — still the documented default set, but no ordering privilege and no required-presence. cbm_adr_validate_section_keys now rejects only names that cannot round-trip (empty, #-leading, newline-bearing, edge-whitespace, over-long).

The fence locator, made refusable rather than destructive

A ## inside a fenced block is a code sample, not a heading — pinned by test. An unterminated fence hides every heading after it, so a write would append a duplicate instead of replacing: that is refused with write_error naming the open fence, leaving the document byte-identical, and mode='sections' reports unterminated_code_fence rather than answering with a quietly partial list.

One scanner, not two

mode='sections' previously used a different classifier from the store parser, so users were told ## CUSTOM was a section while the store treated it as body text. adr_list_sections_from_content now calls the same cbm_adr_scan_headings the splice locates with, and a test pins that they agree — listing exactly the real headings on a document containing a preamble, # Title, ### Sub and a fenced ## Fenced, then writing one and asserting every unlisted line survives byte-for-byte.

Three defects fixed along the way

  1. write_request classification (mcp.c) — a new write mode absent from that list runs through a query-only store handle with no mutation lease. Asserted by a mutation-guard probe (begin_count == 1), not by reasoning; rejection paths assert begin_count == 0 so a malformed write cannot block an index.
  2. Legacy-ADR data loss — adding the mode to write_request skips the migration block that populates legacy (it is gated !write_request so it cannot block on the lease). The write path therefore reads the legacy file itself under the lease it already holds, rather than merging onto an empty document.
  3. Lost-update raceBEGIN IMMEDIATE placed inside cbm_store_adr_update_sections, so it protects all three full-replace writers (MCP update, the UI at http_server.c:939, and the indexing pipeline at pipeline.c:1763), not just this call site.

Verification

  • mcp store_arch store_nodes store_edges complexity httpd pipeline700 passed, 7 skipped, 0 failed.
  • Idempotence proved by revert: breaking the merge to append-like produced exactly one failure — the idempotence test — confirming that test alone carries the property.
  • Three further surgical breaks each failed the expected tests and only those: disabling fence awareness (4), dropping preserved separator runs (3), removing body newline normalisation (1). Reverted and green each time.
  • All 13 new tests confirmed by name in runner output. Build exit code asserted before every run — it caught a test registered before its definition, where a stale binary would have reported the old result.
  • Cap enforcement tested by observed failure, asserting the stored ADR is byte-identical afterwards (which also proves rollback).

Known, not touched

cbm_adr_validate_content still requires all six canonical sections. It has zero production callers (tests only), so under "canonical is conventional" it is arguably stale — flagged rather than quietly redefined, since that was outside the decision made here.

DeusData and others added 3 commits August 29, 2026 12:43
manage_adr could only replace: mode='update' overwrites the stored
document in full, so adding one entry costs a re-send of the whole ADR.
That is a data-integrity problem before it is a cost one — the caller has
to reproduce every byte it did not intend to change, so the unchanged
prefix survives only as well as the round-trip that carried it.

mode='set_sections' rewrites only the sections named in `section_updates`
and leaves the rest of the stored document untouched, so the stored copy
stays the authority for everything the caller did not name. Unlike a
whole-document append it is idempotent: a client that loses a response
and retries re-sets the same section to the same body and the document is
byte-identical, where an append would silently duplicate the chunk.

Only the six canonical section names are writable. That is a correctness
constraint rather than a style rule: adr_try_section_header() parses ONLY
canonical headers, so a non-canonical '## FOO' written here would be read
back as body text of the section above it, and a second identical write
would append a duplicate — destroying the idempotence the mode exists for.

Three things the new mode needed that were not there:

- cbm_store_adr_update_sections() now wraps its read-modify-write in
  BEGIN IMMEDIATE. Three writers replace this row wholesale — the indexing
  pipeline, the UI POST /api/adr handler, and mode='update' — so the
  unguarded get/merge/store lost whichever of them committed between the
  read and the UPSERT. mode='update' is a single atomic UPSERT and never
  had that window; a section merge introduces it.
- set_sections joins the write_request classification. A mode missing from
  it takes no per-project mutation lease, resolves the store query-only,
  and never reaches open_adr_store_for_write — its write would be
  attempted through a read-only handle while an index runs.
- The write path reads the legacy <root>/.codebase-memory/adr.md itself.
  The existing migration runs on the read path only because it must not
  block on the lease; without this a section write would merge onto an
  empty document and discard an ADR still present on disk.

CBM_ADR_MAX_LENGTH now applies to an MCP write path: it is enforced inside
cbm_store_adr_update_sections, which mode='update' does not go through. An
empty section body, an unknown section name and a missing section_updates
are all rejected before any store is opened, so a caller that meant to
write never receives a success-shaped read.

Section-level update was chosen over the whole-document append proposed in
PR #1243; the analysis that established the problem is from that PR.

Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
manage_adr mode='set_sections' rebuilt the ADR from cbm_adr_parse_sections()
and cbm_adr_render(). That model is lossy, so the rebuild silently rewrote
text nobody asked to change. Measured on the previous commit:

- Everything before the first recognised heading was dropped.
- A mis-cased '## Purpose' was not a heading, so when it came first its
  whole block was dropped with the preamble — an ordinary typo cost the
  author a section.
- Sections stored out of canonical order were reordered.

mode='update' never had this exposure because it writes the caller's bytes
verbatim; the section merge introduced it.

The merge now locates the target heading's byte span and replaces only that
span, appending a block when the heading is absent. Bytes outside the span
are never reconstructed, so a preamble, a code fence, an unrecognised
heading and the author's section order all survive byte-for-byte. That
property is the acceptance test: every document in
adr_splice_preserves_untouched_bytes is a case the rebuild corrupted.

Arbitrary headings now work, which is what an "add an entry" mode needs.
The canonical six become a convention rather than a privilege: they are
still what ADR_EMPTY_HINT suggests, but they get no ordering priority and
none is required. Under splicing, ordering is whatever the document says,
and any privileged reordering would reintroduce the data loss above.

Names match exactly, including case, so '## Purpose' and '## PURPOSE' are
different sections. Folding them would silently merge two blocks the
author chose to keep apart — the same class of loss this commit removes.

cbm_adr_validate_section_keys() no longer enforces the canonical set. It
now rejects only names that could not round-trip through a '## NAME' line
(empty, '#'-leading, newline-bearing, edge-whitespace, over-long), because
such a name scans back as a different heading or as none, and a second
identical write would append a duplicate instead of being a no-op.

A heading inside a fenced code block is a code sample, not a section. An
unterminated fence hides every heading after it, so a write would append a
duplicate rather than replace: that is refused explicitly, and mode='sections'
reports the same ambiguity instead of answering with a partial list.

manage_adr mode='sections' now uses cbm_adr_scan_headings(), the same
classifier the write path splices with. It used to list any '#'-prefixed
line, so a '## Foo' in prose or inside a fence was reported as a section no
write could target. Two components disagreeing about what a section is was
how a section write came to be able to destroy one.

cbm_adr_parse_sections() is unchanged, so documents parse exactly as before.
TEST(adr_parse_sections_non_canonical) still passes untouched.

Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every ADR fixture in the tree used \n, so nothing exercised a CRLF-stored
document — and CI could not have found this, because the fixtures are LF
whichever platform runs them. A red would only have appeared once a real
user's CRLF ADR met this code.

The exposure is specific to splicing. The old rebuild normalised everything
on the way out, so line-ending confusion was invisible; a byte-span
replacement can miscount a separator run, and byte-identity is the property
the splice design exists to provide.

Two of the three paths were already correct, and now say so in tests:

- Replacing a section preserves CRLF exactly. The walk-back over a trailing
  break run steps across both '\r' and '\n', so it cannot cut mid-pair.
- Heading matching is ending-agnostic: "## PURPOSE\r\n" locates the same as
  "## PURPOSE\n", because the scanner trims '\r' from the name. Trimming and
  matching were different claims and only trimming was pinned.

Appending a new section was wrong. It counted trailing '\n' characters, so a
document ending "...Foo\r\n" was read as having one break but then had a
BARE '\n' appended, and the "## NAME" line it wrote used '\n' too — leaving a
mixed document that neither the author nor the tool asked for. A CRLF file
ending in a blank line also gained a spurious extra break.

Appends now count a "\r\n" pair as one break, and write the separator and
heading line using the document's own ending.

The contract, now that a document can be mixed:

- Bytes that already exist are never rewritten, so each line keeps whatever
  ending it had. This follows from splicing and is what the acceptance
  property already guaranteed.
- Text this code writes itself — the separator and the "## NAME" line —
  follows the document's majority ending, LF on a tie.
- A caller's body is inserted verbatim. Rewriting bytes a caller supplied
  would be the same silent modification this whole change set removes.

Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData
DeusData merged commit 997d087 into main Aug 29, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant