Take the nightly off GitHub's scheduler - #137
Merged
Merged
Conversation
This is the last stage of a three-stage release train — widgetii/majestic publishes the majestic tarball, OpenIPC/firmware builds the images, this builds from that tree. Each stage was its own GitHub cron sized against a guess at the previous stage's runtime: 17:25, 22:30 and 03:00 UTC. From 2026-08-26 GitHub began delivering scheduled events hours late and independently per repo, and this repository was the worst affected of the three — +10h38m and +11h49m on successive nights, a 03:00 slot sitting in the deepest part of the congestion band. On the 08-28 cycle that made this stage run FIRST, at 14:49, ahead of both the repositories it is supposed to follow. It went green, as it always would: no stage checks that its inputs are newer than the ones it used last time. The delay is not ours and not fixable here: for every late run `created_at == run_started_at` to the second, so all of it sits upstream of run creation, inside GitHub's dispatcher. Moving the cron off the top of the hour and pushing a commit to resync the schedule were both tried on the majestic side; neither changed anything. The ordering moves to the host that runs majestic's self-hosted runners, which dispatches each stage in turn and waits for it to succeed before starting the next. Unlike majestic and firmware this needed no gate change, and the preflight comment now says why: those two skip when nothing has changed and keyed that skip on the event name, which a dispatch would have made unreachable. This one deliberately always builds, so there was no event-name condition to go stale. The clone guard rail keeps its logic and loses half its argument: an inherited workflow can no longer run nightly and unattended on a stranger's Actions bill, but it can still do so on every internal PR.
openipc-ai
requested review from
cronyx,
flyrouter and
viktorxda
as code owners
August 29, 2026 19:35
PR Summary by QodoRemove GitHub Cron from Builder Nightly Workflow
AI Description
Diagram
High-Level Assessment
Files changed (1)
|
Code Review by Qodo🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)
Great, no issues found!Qodo reviewed your code and found no material issues that require reviewTip of the day💡 Did you know, you can group findings by type and pick your Finding display, from Minimal to Full |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Removes the
schedule: '0 3 * * *'trigger. No gate change was needed here — see below.Last stage of a three-stage release train. Companion PRs: widgetii/majestic#496 and OpenIPC/firmware#2334. The ordering moves to the machine that runs majestic's self-hosted runners, which dispatches each stage and waits for it to succeed before starting the next.
Why
From 2026-08-26 GitHub delivered scheduled events hours late, independently per repo. This repository was the worst affected of the three — a 03:00 slot sits in the deepest part of the congestion band:
On the 08-28 cycle that made this stage run first, at 14:49 — ahead of both repositories it is supposed to follow. It went green, as it always would: nothing here checks that its inputs are newer than the ones it used last time.
Not ours and not fixable here: for every late run
created_at == run_started_atto the second, so all the lost time is upstream of run creation, inside GitHub's dispatcher. Moving the cron off the top of the hour and pushing a commit to "resync" the schedule were both tried on the majestic side; neither did anything.Why this one needs no gate change
majestic and firmware both skip when nothing has changed, and both keyed that skip on
event_name == 'schedule'— which a dispatch would have silently made unreachable, turning every night into a full unconditional matrix. This repository deliberately always builds (should_build=true, becausebuilder.shre-clones OpenIPC/firmware at HEAD and the relevant inputs are mostly outside this repo), so there was no event-name condition to go stale. The preflight comment now records that, so the asymmetry with the other two repos is not mistaken for an oversight.Not in this PR
firmware-drift.ymlkeeps itsschedule: '0 5 * * *'. It is not part of the release train — it reads both trees and files an issue, publishes nothing, and late delivery only means a late issue. Flagging it rather than changing it silently.Verification
.github/scripts/ci-matrix.py --self-test— 111 devices, 15 smoke, 39 cases, passes.