Skip to content

fix: clarify converge assessment of completion claims - #4621

Merged
mnriem merged 4 commits into
github:mainfrom
mnriem:mnriem-converge-intent-clarification
Sep 18, 2026
Merged

mnriem merged 4 commits into
github:mainfrom
mnriem:mnriem-converge-intent-clarification

Conversation

@mnriem

@mnriem mnriem commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds a single six-line paragraph to Step 4 of templates/commands/converge.md; all existing wording is preserved. This contribution targets upstream github/spec-kit:main from mnriem:mnriem-converge-intent-clarification.

The clarification requires assessment of every task regardless of checkbox state or Convergence phase, checks the current result of corrective task chains rather than superseded implementation details, and explicitly covers unmet obligations plus implementation that contradicts, exceeds, or falls outside stated intent. The existing classifications, append-only contract, and workflow remain unchanged.

Evidence: #4603 describes converge as finding only what is not built, while #4576 claims it never rechecks checked tasks. The command's Goal already includes unmet existing tasks, but the inventory and assessment instructions do not explicitly say that completion claims are not evidence. This additive clarification makes that intended responsibility explicit.

Testing

  • Tested locally with uv run specify --help
  • Ran existing tests with uv sync && uv run pytest
  • Tested with a sample project (if applicable)

Targeted validation: .venv/bin/python -m pytest tests/test_command_template_hooks.py tests/test_command_template_py_scripts.py -k converge -q — 5 passed. git diff --check passed. These checks cover template hooks and rendering, not an end-to-end behavioral evaluation of agent output; no sample-project evaluation or full test suite was run.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (describe below)

Authored and submitted on behalf of @mnriem by GitHub Copilot (model: GPT-6 Astra). The agent prepared the change, ran targeted checks, and created the commit; @mnriem directed the scope and reviewed the draft in conversation.

Add one paragraph requiring code-based validation regardless of task checkbox state, accounting for corrective chains and work outside stated intent.

Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 17, 2026 18:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The task inventory terminology is inconsistent and the new prompt contract lacks regression coverage.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Clarifies that /speckit.converge must independently assess all tasks and current behavior rather than trust completion markers.

Changes:

  • Requires assessment regardless of checkbox state or Convergence phase.
  • Adds guidance for corrective task chains and out-of-scope implementation.
File Description
templates/​commands/​converge.md Expands Step 4’s assessment instructions.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread templates/commands/converge.md Outdated
Comment thread templates/commands/converge.md Outdated
@mnriem mnriem added the triage-can-wait Verdict: valid and in-scope but deprioritized; held behind the evidence gate label Sep 17, 2026
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 18, 2026 13:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused documentation change is consistent with existing behavior and has appropriate regression coverage.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 18, 2026 20:02
@mnriem

mnriem commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the latest review clarification in 03cc269f. The regression test is now named and documented as scaffold validation: it verifies that converge.md contains the required task-assessment guidance, and explicitly states that it does not execute an LLM or assert how an LLM will respond.

Validation: .venv/bin/python -m pytest tests/test_converge_template_scaffold.py tests/test_command_template_hooks.py -q (23 passed).

Posted on behalf of @mnriem by GitHub Copilot (model: GPT-5.6 Sol, autonomous); comment fully AI-drafted and changes fully AI-authored.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The regression test omits coverage for the paragraph’s final assessment requirement.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Cover the final assessment requirement

tests/​test_converge_template_scaffold.py:28

The regression tuple does not cover the paragraph's final requirement, so removing the guidance to detect unmet, contradictory, excessive, or out-of-scope implementation would still leave this test green. Add clauses for that sentence so every behavior introduced by this bug fix is protected.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 18, 2026 20:17
@mnriem

mnriem commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the latest review finding in 3f9696f6. The scaffold regression test now also pins the final assessment instruction covering unmet obligations and implementation that contradicts, exceeds, or falls outside stated intent. The test name now reflects that it protects the complete assessment guidance.

Validation: .venv/bin/python -m pytest tests/test_converge_template_scaffold.py tests/test_command_template_hooks.py -q (23 passed).

Posted on behalf of @mnriem by GitHub Copilot (model: GPT-5.6 Sol, autonomous); comment fully AI-drafted and changes fully AI-authored.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused documentation change is consistent with the existing workflow and has appropriate scaffold-level regression coverage.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

@mnriem
mnriem merged commit 68b94af into github:main Sep 18, 2026
15 checks passed
@mnriem
mnriem deleted the mnriem-converge-intent-clarification branch September 18, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-can-wait Verdict: valid and in-scope but deprioritized; held behind the evidence gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants