Skip to content

docs: per-broker restart probes for operator rolling restarts (26.2)#1826

Open
david-yu wants to merge 2 commits into
betafrom
doc-rolling-restart-health-probes
Open

docs: per-broker restart probes for operator rolling restarts (26.2)#1826
david-yu wants to merge 2 commits into
betafrom
doc-rolling-restart-health-probes

Conversation

@david-yu

@david-yu david-yu commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Documents the per-broker pre/post-restart probes that gate operator-driven rolling restarts (redpanda-operator#1547, merged 2026-06-25, ships in operator v26.2.x).

Changes

  1. New section How the Redpanda Operator gates each restart under "What happens during a rolling restart" on manage:kubernetes/k-rolling-restart.adoc:
    • The operator replaces Pods itself, one broker per pass, re-evaluating against a fresh broker view each time
    • Pre-restart probe (GET /v1/broker/pre_restart_probe): the restart is delayed while it would lose acks=1-acknowledged data, make partitions unavailable, or block acks=all produce; replication-factor-1 partitions don't block (no redundancy to preserve — cross-linked to the existing RF-1 guidance in the procedure)
    • Post-restart probe (GET /v1/broker/post_restart_probe): the next restart is held until the recovering broker reports 100% of its in-sync-replica load reclaimed, because Pod readiness can pass while the broker is still replaying partition state
    • Version behavior: probes require Redpanda 25.1+; on older versions the operator falls back to the cluster-wide health check; probes run only while a restart is in progress
  2. What's new: Safer rolling restarts with per-broker health probes entry under Redpanda Operator v26.2.x in get-started:release-notes/operator.adoc.

Verification

Content follows the eng PR's probe contract and was cross-checked against operator main: internal/probes/broker.go (risk-category interpretation, load_reclaimed_pc >= 100 threshold, 404 fallbacks) and internal/lifecycle/client.go (operator-managed StatefulSets use the OnDelete update strategy, so the operator's gated roll loop performs the restarts). The eng PR's EKS E2E (6 brokers, sustained rolls under load) recorded 88 post-restart deferrals and 32 pre-restart blocks with zero acks=1 loss.

Note: this PR inserts a section under the v26.2.x what's-new heading, as do open PRs #1808 and #1825 — whichever merges later needs a trivial conflict resolution (the entries just stack).

Preview pages

🤖 Generated with Claude Code

…restarts

Add 'How the Redpanda Operator gates each restart' to the Kubernetes
rolling-restart page (operator PR #1547): the pre-restart probe blocks a
broker's restart while it would lose acks=1 data, make partitions
unavailable, or stall acks=all produce; the post-restart probe holds the
next restart until the recovering broker reports 100% of its in-sync-replica
load reclaimed, since pod readiness alone doesn't prove catch-up. One broker
per pass, Redpanda 25.1+ for the probe endpoints, cluster-wide health
fallback on older versions. Adds the v26.2.x what's-new entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@david-yu
david-yu requested a review from a team as a code owner July 24, 2026 21:12
@netlify

netlify Bot commented Jul 24, 2026

Copy link
Copy Markdown

Deploy Preview for redpanda-docs-preview ready!

Name Link
🔨 Latest commit 1477576
🔍 Latest deploy log https://app.netlify.com/projects/redpanda-docs-preview/deploys/6a63d59feedd020008ec4295
😎 Deploy Preview https://deploy-preview-1826--redpanda-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bc97b2da-d5ca-472f-9d4c-02ff4ff0e035

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch doc-rolling-restart-health-probes

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

….2.1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@david-yu
david-yu requested a review from JakeSCahill July 25, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant