Skip to content

feat(server): build, sign, and submit Bulletin preimages in the core#270

Merged
pgherveou merged 154 commits into
mainfrom
bulletin-preimage-in-core
Jul 14, 2026
Merged

feat(server): build, sign, and submit Bulletin preimages in the core#270
pgherveou merged 154 commits into
mainfrom
bulletin-preimage-in-core

Conversation

@pgherveou

@pgherveou pgherveou commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

What

Preimage submission to the Bulletin chain now happens entirely inside the Rust core (truapi-server).
The core builds, signs, and submits the TransactionStorage.store extrinsic itself, routing chain traffic through the host's existing chain.connect JSON-RPC pipe.

Previously the core handed the host a signing capability and the host built and submitted the transaction with PAPI.
That seam is gone. As a result:

  • the wallet-delegated allowance secret is only ever used inside the core: no signer handle and no sign request crosses the host / FFI boundary, and the key arrives from the wallet end-to-end encrypted
  • the host's only remaining preimage job is content lookup (smoldot bitswap, or an IPFS HTTP gateway in RPC-gateway mode);
  • all hosts (web, and later mobile over uniffi) get identical submission behaviour, because the logic lives in the shared core instead of each host.

Ownership: before vs after

The topology is unchanged (the core still reaches the network only through the host).
What changes is ownership.
Before, the secret bytes were core-owned but the capability to use them, and the decision of what gets signed, was host-owned:

BEFORE
         core-owned  (WASM worker)          host-owned  (JS main thread)
  +------------------------------+   ||   +------------------------------+
  | Rust core                    |   ||   | dotli (PAPI)                 |
  |                              |   ||   |                              |
  | owns:                        |   ||   | owns:                        |
  |  - allowance secret (64 B)   |   ||   |  - signer handle             |
  |  - sign() execution          |   ||   |     { publicKey, sign() }    |
  |                              |   ||   |  - tx builder (PAPI)         |
  |                              |   ||   |  - decides WHAT is signed    |
  |                              |   ||   |  - submission + watch        |
  +------------------------------+   ||   +------------------------------+

               core --[ signer handle ]--> host   (a live signing capability)
               core <--[ sign(payload) ]-- host   (one request per tx)
               core --[ signature      ]--> host --> Bulletin

After, everything signing-related sits on the core side; during submission the host just forwards opaque JSON-RPC text:

AFTER
         core-owned  (WASM worker)          host-owned  (JS main thread)
  +------------------------------+   ||   +------------------------------+
  | Rust core                    |   ||   | dotli                        |
  |                              |   ||   |                              |
  | owns:                        |   ||   | owns:                        |
  |  - allowance secret (64 B)   |   ||   |  - permission/confirm modals |
  |  - signer (crate-private,    |   ||   |  - chain.connect JSON-RPC    |
  |     never exported)          |   ||   |     pipe (forwards verbatim) |
  |  - tx builder (subxt)        |   ||   |  - lookupPreimage backend    |
  |  - decides WHAT is signed    |   ||   |     (bitswap / IPFS gateway) |
  |  - submission + watch        |   ||   |                              |
  +------------------------------+   ||   +------------------------------+

               core --[ JSON-RPC request  ]--> host --> Bulletin
               core <--[ JSON-RPC response ]-- host
               Bulletin pipe: opaque JSON-RPC text only,
               no signing capability, no sign requests

Submit flow

 PRODUCT  | CORE (holds keys, builds+signs)    : HOST (modals+pipe) : NET
 ---------+------------------------------------+--------------------+---
 submit   > receive                            :                    :
          | remotePermission                   > prompt user        :
          | confirmUserAction                  > prompt user        :
          | allowanceKey (cached, else SSO     :                    :
          |   via People statement store)      > fwd (E2E encrypted)> wlt
          | slotAccountKey (ciphertext)        < forward            < wlt
          | [[ 64-B key decrypted in core ]]   :                    :
          | chain.connect(bulletin)            > open pipe          :
          | chainHead_v1_follow                > forward            > N
          | Metadata_metadata_at_version *     > forward            > N
          | chainHead_v1_header (mortality)    > forward            > N
          | AccountNonceApi_account_nonce *    > forward            > N
          | [[ build + SIGN store(value) ]]    :                    :
          | TaggedTransactionQueue_            :                    :
          |   validate_transaction * (dry-run) > forward            > N
          | transaction_v1_broadcast           > forward            > N
          | watch: nonce probe * per block,    :                    :
          |   chainHead_v1_body on advance     > forward            > N
          | chainHead_v1_storage(Events)       > forward            > N
       key< return blake2_256(value)           :                    :

 * runtime calls, carried on the wire as chainHead_v1_call

Changed surface

  • host_logic/extrinsic.rs: offline subxt assembler (config-pinned SubstrateConfig, sr25519 signer, metadata / validity / events / header decoders).
  • host_logic/bulletin.rs: store{data} build + sign; the call is resolved by name from the fetched metadata, with a hard check that the data argument really is a byte sequence.
  • runtime/bulletin_rpc.rs: the submit flow above + typed errors.
  • runtime.rs: Preimage::submit ordering with one allowance refresh + retry on rejection; lookup_subscribe cache + integrity check.
  • truapi-platform: PreimageHost keeps only lookupPreimage; BulletinAllowanceKey zeroized on drop; host configs require the Bulletin genesis hash (runtimeConfig.bulletin.genesisHash on the TS side).
  • Host TS (@parity/truapi-host) + dotli submodule: signer bridge removed; dotli keeps content lookup only (smoldot bitswap_v1_get, or IPFS gateway in RPC-gateway mode) and serves Bulletin chain.connect on both its light-client and RPC-gateway backends.

pgherveou added 30 commits June 30, 2026 17:30
Adds the canonical testing module (api/testing.rs) and its v01/v02/versioned
wiring used by the Rust host runtime and generated clients.
New crate defining the host syscall traits (storage, navigation, consent,
permissions, ...) that host runtimes implement. Types are re-exported from
truapi::versioned/v01 rather than redefined.
WASM host runtime that hosts implement: dispatcher, SCALE frames, subscription
streams, chain runtime, host logic (sessions, SSO pairing, permissions,
statement store, dotns) and the wasm bindings. Includes the committed generated
dispatcher/wire-table under src/generated/.
…backs

Extends the rustdoc-JSON code generator to emit the Rust dispatcher and wire
table consumed by truapi-server, plus the TS host-callbacks adapter. Golden
tests pin the emitted shapes.
New WASM-backed host runtime package embedding the Rust core, with web iframe
and Web Worker entry points. Updates the @parity/truapi client (SCALE, sandbox,
transport) and drops the obsolete explorer 0.3.2 codegen snapshot.
Updates CLAUDE.md/README, CI workflows, Makefile, deny.toml, changesets, and
linguist attributes for generated code, and bumps the dotli submodule to the
host integration that consumes the WASM runtime.
Adds the canonical testing module (api/testing.rs) and its v01/v02/versioned
wiring used by the Rust host runtime and generated clients.
New crate defining the host syscall traits (storage, navigation, consent,
permissions, ...) that host runtimes implement. Types are re-exported from
truapi::versioned/v01 rather than redefined.
…backs

Extends the rustdoc-JSON code generator to emit the Rust dispatcher and wire
table consumed by truapi-server, plus the TS host-callbacks adapter. Golden
tests pin the emitted shapes.
@TarikGul TarikGul self-requested a review July 10, 2026 15:41
@pgherveou pgherveou changed the base branch from worktree-issue-96-rust-core-port to main July 13, 2026 09:11
@pgherveou pgherveou requested review from a team July 13, 2026 09:11
…core

# Conflicts:
#	.github/workflows/ci.yml
#	.github/workflows/release.yml
#	.gitignore
#	CLAUDE.md
#	Makefile
#	hosts/dotli
#	js/packages/truapi-host/scripts/build-wasm.mjs
#	js/packages/truapi-host/src/adapter-support.ts
#	js/packages/truapi-host/src/host-callbacks-adapter.test.ts
#	js/packages/truapi-host/src/runtime.ts
#	js/packages/truapi-host/src/web/create-iframe-host.test.ts
#	js/packages/truapi-host/src/web/create-iframe-host.ts
#	js/packages/truapi-host/src/web/create-worker-host-runtime.ts
#	js/packages/truapi-host/src/web/worker-provider.test.ts
#	js/packages/truapi-host/src/worker-protocol.ts
#	js/packages/truapi-host/src/worker-runtime.ts
#	js/packages/truapi/src/sandbox.ts
#	playground/src/lib/auto-test.ts
#	playground/tests/e2e/dotli-diagnosis.ts
#	playground/tests/e2e/dotli/helpers/signer-bot.ts
#	rust/crates/truapi-codegen/src/rust/wasm_bridge.rs
#	rust/crates/truapi-codegen/src/ts/host_callbacks.rs
#	rust/crates/truapi-codegen/tests/golden/host-callbacks-adapter.ts
#	rust/crates/truapi-codegen/tests/golden/host-callbacks.ts
#	rust/crates/truapi-codegen/tests/golden/wasm_bridge.rs
#	rust/crates/truapi-codegen/tests/golden/worker-callbacks.ts
#	rust/crates/truapi-codegen/tests/golden_rust_emit.rs
#	rust/crates/truapi-server/src/runtime.rs
#	rust/crates/truapi-server/src/wasm.rs
#	rust/crates/truapi-server/src/wasm/generated_bridge.rs
Keep physical provider ownership in hosts while moving shared chain behavior and Subxt-backed handling into Rust. Harden Bulletin retries and host lifecycle coverage.
Comment thread rust/crates/truapi-server/src/host_logic/bulletin.rs

@valentinfernandez1 valentinfernandez1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me just some small nits.

Comment thread rust/crates/truapi-server/src/runtime.rs Outdated
Comment thread js/packages/truapi-host/src/runtime.ts Outdated
Comment thread rust/crates/truapi/src/api/preimage.rs Outdated
Comment thread rust/crates/truapi/src/api/preimage.rs Outdated
Comment thread rust/crates/truapi-server/Cargo.toml
Apply propagation waits per block so scheduling delay cannot race the block-limit test. Keep allowance allocation, refresh, and submission within one 60-second deadline.
@pgherveou pgherveou added this pull request to the merge queue Jul 14, 2026
Merged via the queue into main with commit d161d4b Jul 14, 2026
12 checks passed
@pgherveou pgherveou deleted the bulletin-preimage-in-core branch July 14, 2026 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants