test: run the regtest stack on an ephemeral VM - #204
Draft
mdozhdev wants to merge 1 commit into
Draft
Conversation
The iOS e2e suite needs the regtest stack and the iOS Simulator on one
machine, because the app reaches Electrum and homegate on 127.0.0.1.
GitHub-hosted macOS runners cannot run Docker — Apple's Virtualization
framework has no nested virtualization for macOS guests — so the suite
runs on a self-hosted Mac, serially, at ~2h per run against ~20m for the
same tests on Linux.
Add tooling to run the stack on a short-lived GCP VM instead, so the Mac
only has to do the one thing only a Mac can do.
regtest-vm-up provisions a VM, opens a firewall scoped to the
runner's egress /32, waits until the stack actually
serves rather than merely listens
startup.sh runs on the VM: installs Docker, unpacks the stack
from instance metadata, starts it, publishes LND's
tls.cert and admin.macaroon on a random path
regtest-vm-down deletes the VM and the rule
regtest-reaper sweeps orphaned firewall rules, which have no TTL
regtest-vm-smoke exercises the above without running the suite
setup-wif.sh one-off Workload Identity Federation setup
The stack travels as a base64 tarball in instance metadata rather than
being cloned, so the VM needs no repository access and no token, and the
stack always matches the checkout the tests run from. Instances carry
--max-run-duration with --instance-termination-action=DELETE, so GCE
removes them even if teardown never runs.
Depends on the configurable-backend-host change: startup.sh sets
LND_EXTERNAL_IP, which does nothing until docker-compose.yml reads it.
Verified on a GitHub-hosted macOS runner: VM provisioned, stack reached,
credentials fetched, and a request from inside a booted Simulator
recorded in the VM's own access log.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mdozhdev
force-pushed
the
test/remote-regtest-backend
branch
from
August 24, 2026 08:26
1a83616 to
7339fc0
Compare
Contributor
Author
|
Rescoped: the four files that are needed regardless of how the stack is hosted have moved to #206, so this PR is now VM tooling only (6 files, +719). Depends on #206. Not stacked because a PR's base has to be a branch in this repo and I only have fork access. Nothing else changed — the tooling is the same as the version verified end to end on a GitHub-hosted macOS runner. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
e2e-tests-localneeds the regtest stack and the iOS Simulator on the same machine, because the app reaches Electrum and homegate on127.0.0.1. GitHub-hosted macOS runners cannot run Docker — Apple's Virtualization framework has no nested virtualization for macOS guests, and GitHub closed the request as not planned in January 2026.So the iOS suite runs on a self-hosted Mac. One machine, one owner, and because it is one machine the suite runs as a single shard: 104–127 min per run, against 12–24 min for the same suite on Android's GitHub-hosted Linux runners.
What this adds
Tooling to run the stack on a short-lived GCP VM, so the Mac only does the one thing only a Mac can do.
regtest-vm-up/32, waits until the stack actually serves rather than merely listensstartup.shtls.certandadmin.macaroonon a random pathregtest-vm-downregtest-reaperregtest-vm-smokesetup-wif.shTwo decisions worth knowing:
--max-run-durationwith--instance-termination-action=DELETE, so GCE removes them even if teardown never runs. The reaper only exists because firewall rules have no equivalent.Behaviour changes
Four, all keeping their current defaults, so existing runs are unaffected:
docker-compose.yml— LND advertisesLND_EXTERNAL_IPinstead of127.0.0.1constants.ts—lndConfighost, ports and credential paths from envlnd.ts—connectToLNDuses the configured hostwdio.conf.ts— forwardsE2E_LOCAL_HOSTinto the app's launch environmentThat is 29 lines of behaviour change; the rest is new, self-contained tooling.
Verified
On a GitHub-hosted macOS runner: VM provisioned, firewall scoped correctly, stack booted, bitcoind mining on regtest, LND reachable, credentials fetched, and a request from inside a booted Simulator recorded in the VM's own access log. Teardown left no orphaned resources.
Not yet verified: a full suite run against a remote stack. That needs the companion workflow in
bitkit-iosand a configured GCP project.Configuration needed before anything runs
Nothing project-specific is committed. A maintainer runs
ci/regtest-vm/setup-wif.shagainst a bitkit CI project and sets what it prints:All new workflows are
workflow_dispatchonly, so merging without that configuration changes nothing and breaks nothing.Related
bitkit-iosadds the workflow that consumes these actions.