ci: compare optimized toolchain cache performance

[fengmk2]

This draft pins urllib's main CI workflow to merged setup-vp commit 6eb3ef76e48ad5aca2d217e4ee277786b3865699 from setup-vp #209. The global CLI, local vite-plus, and vite alias use preview 65f9bc56f1f2d87e138013c5bcde9e7306d8eeb8. The lockfile and registry setting match that preview.

A validated toolchain cache hit now activates the cached native CLI directly. The action skips installer downloads and persistent PATH changes. It still uses the installer if native activation fails. The package-manager binary remains in a separate archive.

The benchmark below measures pre-merge revision 06fe7ecfda756e5871867245052c17baa9b4e974.

The GitHub Actions comparison passed all 57 jobs: three cache warm-ups and six samples per case on each platform. Every job uses the same urllib checkout and tool versions. Each job builds urllib after setup.

The table shows median setup + post time. All three cases ran in this comparison. The baseline has no package-manager binary cache. The previous and latest revisions restore that archive.

Runner	Baseline	Previous PR	Latest	Reduction vs baseline	Reduction vs previous
Linux x64	9.5 s	8 s	6.5 s	31.6%	18.8%
macOS ARM64	20 s	16 s	14.5 s	27.5%	9.4%
Windows x64	18.5 s	17.5 s	11.5 s	37.8%	34.3%

xychart-beta
  title "urllib: median setup + post time"
  x-axis ["Linux base", "Linux latest", "macOS base", "macOS latest", "Windows base", "Windows latest"]
  y-axis "Seconds — lower is better" 0 --> 30
  bar [9.5, 6.5, 20, 14.5, 18.5, 11.5]

Loading

All measured jobs restored the same dependency and Vite+/Node.js archives for their platform. All 36 previous/latest samples also restored the pnpm archive. All 18 latest samples used native activation without installer fallback.

Windows shows the clearest activation saving. The following phase starts after toolchain restoration and ends when Node.js setup starts.

Runner	Previous activation	Latest activation
Linux x64	1.180 s	0.084 s
macOS ARM64	1.298 s	0.052 s
Windows x64	4.587 s	0.136 s

The package-manager archive still adds lookup, transfer, extraction, and executable checks. Cache-transfer and runner variation also affect total time, especially on macOS. These six-sample medians describe this run; they do not guarantee the same percentage on every run.

Revisions, method, raw samples, and cache size

Case	setup-vp revision	Behavior
Baseline	c025a93923d2f9c524652d786386d11065fc3e8f	Shell installer; no manager cache
Previous PR	85cf606c7348256a70a82b783b3cb2ff1e2b703b	Shell installer; manager cache
Latest	06fe7ecfda756e5871867245052c17baa9b4e974	Native activation; manager cache

The benchmark workflow checks out urllib 8d14d55acb8886a2de1102f09abd008c38560c55. All cases use Node.js 24.22.0 and pnpm@12.14.0. The preview version is 0.0.0-commit.65f9bc56f1f2d87e138013c5bcde9e7306d8eeb8.

Each platform starts with fresh, isolated toolchain and dependency paths. One warm-up saves its three archives. All measured jobs set cache-save: false and run vp install on separate hosted runners. Both earlier revisions and the latest revision disable sfw for this preview.

The measurement includes the setup action and its post step. It excludes warm-up, runner startup, checkout, assertions, builds, and artifact uploads. GitHub reports step durations with one-second resolution. The analysis checks matching tool versions, lockfile hashes, pnpm binary hashes, cache keys, and runner images within each platform.

Runner	Baseline samples	Previous samples	Latest samples
Linux x64	9 / 8 / 8 / 10 / 10 / 10 s	11 / 13 / 6 / 8 / 7 / 8 s	7 / 9 / 5 / 6 / 6 / 10 s
macOS ARM64	21 / 16 / 22 / 25 / 16 / 19 s	11 / 13 / 22 / 27 / 15 / 17 s	15 / 14 / 14 / 18 / 9 / 21 s
Windows x64	18 / 19 / 19 / 18 / 19 / 18 s	18 / 15 / 18 / 17 / 16 / 20 s	12 / 11 / 14 / 9 / 17 / 11 s

The compressed archives use this storage:

Runner	Project dependencies	Vite+ / Node.js	pnpm binary
Linux x64	97.4 MiB	89.1 MiB	20.8 MiB
macOS ARM64	178.8 MiB	84.5 MiB	17.9 MiB
Windows x64	82.1 MiB	75.3 MiB	20.3 MiB

The pnpm archives add about 59 MiB across all three platforms. The existing cache-toolchain input controls the manager archive. package-manager-cache-hit reports manager restoration; toolchain-cache-hit reports the Vite+/Node.js archive.

The benchmark workflow remains outside this PR's diff. Its isolated caches do not seed normal PR CI. The normal Node.js matrix and cache-save policy remain in place.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/node-modules/urllib@953

commit: d1268f0

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.77%. Comparing base (db0a16d) to head (d1268f0).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #953   +/-   ##
=======================================
  Coverage   94.77%   94.77%           
=======================================
  Files          10       10           
  Lines         746      746           
  Branches      235      235           
=======================================
  Hits          707      707           
  Misses         36       36           
  Partials        3        3           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.