feat(mcp): add manage_adr mode='set_sections' and splice ADR sections instead of rebuilding - #1904
Merged
Conversation
manage_adr could only replace: mode='update' overwrites the stored document in full, so adding one entry costs a re-send of the whole ADR. That is a data-integrity problem before it is a cost one — the caller has to reproduce every byte it did not intend to change, so the unchanged prefix survives only as well as the round-trip that carried it. mode='set_sections' rewrites only the sections named in `section_updates` and leaves the rest of the stored document untouched, so the stored copy stays the authority for everything the caller did not name. Unlike a whole-document append it is idempotent: a client that loses a response and retries re-sets the same section to the same body and the document is byte-identical, where an append would silently duplicate the chunk. Only the six canonical section names are writable. That is a correctness constraint rather than a style rule: adr_try_section_header() parses ONLY canonical headers, so a non-canonical '## FOO' written here would be read back as body text of the section above it, and a second identical write would append a duplicate — destroying the idempotence the mode exists for. Three things the new mode needed that were not there: - cbm_store_adr_update_sections() now wraps its read-modify-write in BEGIN IMMEDIATE. Three writers replace this row wholesale — the indexing pipeline, the UI POST /api/adr handler, and mode='update' — so the unguarded get/merge/store lost whichever of them committed between the read and the UPSERT. mode='update' is a single atomic UPSERT and never had that window; a section merge introduces it. - set_sections joins the write_request classification. A mode missing from it takes no per-project mutation lease, resolves the store query-only, and never reaches open_adr_store_for_write — its write would be attempted through a read-only handle while an index runs. - The write path reads the legacy <root>/.codebase-memory/adr.md itself. The existing migration runs on the read path only because it must not block on the lease; without this a section write would merge onto an empty document and discard an ADR still present on disk. CBM_ADR_MAX_LENGTH now applies to an MCP write path: it is enforced inside cbm_store_adr_update_sections, which mode='update' does not go through. An empty section body, an unknown section name and a missing section_updates are all rejected before any store is opened, so a caller that meant to write never receives a success-shaped read. Section-level update was chosen over the whole-document append proposed in PR #1243; the analysis that established the problem is from that PR. Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
manage_adr mode='set_sections' rebuilt the ADR from cbm_adr_parse_sections() and cbm_adr_render(). That model is lossy, so the rebuild silently rewrote text nobody asked to change. Measured on the previous commit: - Everything before the first recognised heading was dropped. - A mis-cased '## Purpose' was not a heading, so when it came first its whole block was dropped with the preamble — an ordinary typo cost the author a section. - Sections stored out of canonical order were reordered. mode='update' never had this exposure because it writes the caller's bytes verbatim; the section merge introduced it. The merge now locates the target heading's byte span and replaces only that span, appending a block when the heading is absent. Bytes outside the span are never reconstructed, so a preamble, a code fence, an unrecognised heading and the author's section order all survive byte-for-byte. That property is the acceptance test: every document in adr_splice_preserves_untouched_bytes is a case the rebuild corrupted. Arbitrary headings now work, which is what an "add an entry" mode needs. The canonical six become a convention rather than a privilege: they are still what ADR_EMPTY_HINT suggests, but they get no ordering priority and none is required. Under splicing, ordering is whatever the document says, and any privileged reordering would reintroduce the data loss above. Names match exactly, including case, so '## Purpose' and '## PURPOSE' are different sections. Folding them would silently merge two blocks the author chose to keep apart — the same class of loss this commit removes. cbm_adr_validate_section_keys() no longer enforces the canonical set. It now rejects only names that could not round-trip through a '## NAME' line (empty, '#'-leading, newline-bearing, edge-whitespace, over-long), because such a name scans back as a different heading or as none, and a second identical write would append a duplicate instead of being a no-op. A heading inside a fenced code block is a code sample, not a section. An unterminated fence hides every heading after it, so a write would append a duplicate rather than replace: that is refused explicitly, and mode='sections' reports the same ambiguity instead of answering with a partial list. manage_adr mode='sections' now uses cbm_adr_scan_headings(), the same classifier the write path splices with. It used to list any '#'-prefixed line, so a '## Foo' in prose or inside a fence was reported as a section no write could target. Two components disagreeing about what a section is was how a section write came to be able to destroy one. cbm_adr_parse_sections() is unchanged, so documents parse exactly as before. TEST(adr_parse_sections_non_canonical) still passes untouched. Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every ADR fixture in the tree used \n, so nothing exercised a CRLF-stored document — and CI could not have found this, because the fixtures are LF whichever platform runs them. A red would only have appeared once a real user's CRLF ADR met this code. The exposure is specific to splicing. The old rebuild normalised everything on the way out, so line-ending confusion was invisible; a byte-span replacement can miscount a separator run, and byte-identity is the property the splice design exists to provide. Two of the three paths were already correct, and now say so in tests: - Replacing a section preserves CRLF exactly. The walk-back over a trailing break run steps across both '\r' and '\n', so it cannot cut mid-pair. - Heading matching is ending-agnostic: "## PURPOSE\r\n" locates the same as "## PURPOSE\n", because the scanner trims '\r' from the name. Trimming and matching were different claims and only trimming was pinned. Appending a new section was wrong. It counted trailing '\n' characters, so a document ending "...Foo\r\n" was read as having one break but then had a BARE '\n' appended, and the "## NAME" line it wrote used '\n' too — leaving a mixed document that neither the author nor the tool asked for. A CRLF file ending in a blank line also gained a spurious extra break. Appends now count a "\r\n" pair as one break, and write the separator and heading line using the document's own ending. The contract, now that a document can be mixed: - Bytes that already exist are never rewritten, so each line keeps whatever ending it had. This follows from splicing and is what the acceptance property already guaranteed. - Text this code writes itself — the separator and the "## NAME" line — follows the document's majority ending, LF on a tie. - A caller's body is inserted verbatim. Rewriting bytes a caller supplied would be the same silent modification this whole change set removes. Co-authored-by: andis777 <24672074+andis777@users.noreply.github.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
manage_adr mode='set_sections'— a retry-safe way to update named ADR sections — and fixes the document-rewriting it exposed. Distilled from #1243 withCo-authored-by:credit to @andis777, whose analysis and tests this builds on.Why sections rather than append
#1243 proposed whole-document
append. It is non-idempotent on client retry: a lost response means a silently duplicated chunk. Section updates are retry-safe by construction — the same request twice produces the same document.The design that matters: splice, don't re-render
The first implementation merged by parsing the ADR, applying updates to the model, and re-rendering. That silently destroyed content, because
parse → renderis lossy today:## Purpose(wrong case)## CUSTOMin prose```md / ## ExampleAn ADR beginning
## Purposeinstead of## PURPOSE— an ordinary typo — would have lost that entire section on the firstset_sectionscall.mode='update'never had this exposure because it writes the caller's bytes verbatim; the merge introduced it.So the merge no longer rebuilds anything.
cbm_adr_splice_sectionlocates the target heading's byte span and replaces only that span, appending a block when the heading is absent. Nothing outside the span is reconstructed, so untouched bytes are byte-identical by construction rather than by care. Every row in that table is now a regression test asserting the exact expected document.cbm_adr_parse_sectionsis untouched andTEST(adr_parse_sections_non_canonical)passes unchanged — documents parse exactly as they did. That test was the tripwire: wanting to edit it would have meant the design had drifted back toward re-rendering.Arbitrary headings now work
Custom headings are real sections, which is what the original request needed.
## DECISIONSis simply a span to locate or a block to append — no format migration, because stored ADRs are never rewritten wholesale.## Purposeand## PURPOSEare independently targetable in one document. Case-folding would silently merge two blocks a user deliberately kept apart.cbm_adr_validate_section_keysnow rejects only names that cannot round-trip (empty,#-leading, newline-bearing, edge-whitespace, over-long).The fence locator, made refusable rather than destructive
A
##inside a fenced block is a code sample, not a heading — pinned by test. An unterminated fence hides every heading after it, so a write would append a duplicate instead of replacing: that is refused withwrite_errornaming the open fence, leaving the document byte-identical, andmode='sections'reportsunterminated_code_fencerather than answering with a quietly partial list.One scanner, not two
mode='sections'previously used a different classifier from the store parser, so users were told## CUSTOMwas a section while the store treated it as body text.adr_list_sections_from_contentnow calls the samecbm_adr_scan_headingsthe splice locates with, and a test pins that they agree — listing exactly the real headings on a document containing a preamble,# Title,### Suband a fenced## Fenced, then writing one and asserting every unlisted line survives byte-for-byte.Three defects fixed along the way
write_requestclassification (mcp.c) — a new write mode absent from that list runs through a query-only store handle with no mutation lease. Asserted by a mutation-guard probe (begin_count == 1), not by reasoning; rejection paths assertbegin_count == 0so a malformed write cannot block an index.write_requestskips the migration block that populateslegacy(it is gated!write_requestso it cannot block on the lease). The write path therefore reads the legacy file itself under the lease it already holds, rather than merging onto an empty document.BEGIN IMMEDIATEplaced insidecbm_store_adr_update_sections, so it protects all three full-replace writers (MCPupdate, the UI athttp_server.c:939, and the indexing pipeline atpipeline.c:1763), not just this call site.Verification
mcp store_arch store_nodes store_edges complexity httpd pipeline→ 700 passed, 7 skipped, 0 failed.Known, not touched
cbm_adr_validate_contentstill requires all six canonical sections. It has zero production callers (tests only), so under "canonical is conventional" it is arguably stale — flagged rather than quietly redefined, since that was outside the decision made here.