feat(google): enable Gemini inline image output

[tizerluo]

Summary

• Enable Gemini image generation through Cloud Code Assist (Antigravity) with two complementary paths:
  1. /v1/images/generations fallback (primary): When no OpenAI upstream is configured, Codex's built-in image_gen skill now falls back to CCA's dedicated image model (gemini-3.1-flash-image). This is the standard Codex workflow — the model calls the tool, codex-rs POSTs to the Images API, and the proxy serves it via CCA.
  2. Inline inlineData parsing: For users who select gemini-3.1-flash-image directly as their chat model, inlineData parts in the response are materialized to ~/.opencodex/artifacts/ and emitted as markdown.
• Add responseModalities: ["TEXT", "IMAGE"] to generationConfig for non-thinking Gemini models.
• Add gemini-3.1-flash-image to the CCA model catalog (from FetchAvailableModels.imageGenerationModelIds).
• Automatically omit responseModalities when a thinking level is active (thinking models reject IMAGE modality).

How it works (Codex workflow)

User: "draw me a cat"
  → Chat model calls image_gen.imagegen tool
  → codex-rs POSTs /v1/images/generations {prompt: "a cat", model: "gpt-image-1"}
  → OpenCodex: no OpenAI upstream? → CCA fallback
  → Calls gemini-3.1-flash-image via v1internal:generateContent
  → Returns {created, data: [{b64_json: "..."}]}
  → codex-rs renders the image

No model switching required. Works with just a CCA login (ocx login google-antigravity).

Verification

• bun run typecheck — clean
• bun test tests/images/gemini-inline.test.ts — 8/8 pass
• bun test tests/google-adapter.test.ts tests/google-vertex-stream.test.ts — 17/17 pass
• bun run privacy:scan — passed
• E2E (Images API): POST /v1/images/generations {"prompt":"A cute green frog on a lily pad"} → 2MB JPEG returned in standard {created, data:[{b64_json}]} format
• E2E (inline): POST /v1/chat/completions with model gemini-3.1-flash-image → ![image](/abs/path.jpg) markdown with materialized 515KB JPEG

Note: bun run test (full suite) has 24 pre-existing failures on dev in catalog/discovery tests, unrelated to this change.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Summary by CodeRabbit

• New Features
  • Added support for Gemini image-capable models, including the new gemini-3.1-flash-image model.
  • Inline images are saved as local artifacts and displayed through Markdown image links.
  • Added Google Antigravity fallback support for image generation requests.
• Bug Fixes
  • Added validation and size limits for generated image data.
  • Improved Markdown path escaping for image links.
• Documentation
  • Documented native Gemini image generation, artifact handling, and image size limits.

[Wibias]

Follow-up re-review @ d3c876e6

Verdict: request changes. This corrects and supersedes my previous pass. I re-read the newest Codex review on this exact tip and verified the remaining findings against current code.

Most earlier blockers are fixed: pathToFileURL, CCA success-envelope validation, OAuth login-vs-refresh mapping, registry destination pinning, strict artifact handling, retention, docs updates, and CCA fallback coverage are present. I found no remaining OAuth token/project leak in the current implementation.

Remaining code blockers

1. Client cancellation during CCA body read is reported as 504 instead of 499 — src/server/images.ts

   • signalWithTimeout propagates a parent abort into linkedSignal.signal, so both signals are aborted.
   • The body-read catch checks linkedSignal.signal.aborted first, making the later parent-signal 499 branch unreachable for client cancellation.
   • Check the parent request signal first, or distinguish abort reasons. Add a stalled-body regression that aborts the client request and asserts 499 rather than 504.

2. The image deadline does not cover CCA OAuth preflight — src/server/images.ts and OAuth refresh plumbing

   • getValidAccessToken("google-antigravity") runs before the linked image deadline is created and receives no request signal.
   • Token refresh and project discovery can therefore outlive images.timeoutMs and client cancellation.
   • Create the linked deadline before credential resolution and propagate its signal through token refresh/project discovery. Preserve 401 for OAuthLoginRequiredError, 499 for client abort, 504 for deadline expiry, and sanitized 502 for genuine refresh/network failures.

3. Proxy-local file: artifact URLs do not work for remote/container clients — src/adapters/google.ts

   • pathToFileURL fixes URI syntax, but the URI still points to the proxy host's filesystem.
   • Remote binding is supported, so a client connected to another machine/container cannot open the emitted artifact.
   • Serve artifacts through an authenticated, traversal-safe HTTP route using opaque artifact identifiers, or explicitly restrict and enforce inline artifact output as local-only. Add local and remote-client regressions.

4. PNG validation accepts only the first four bytes of the eight-byte PNG signature — src/images/artifacts.ts

   • sig.startsWith("\x89PNG") accepts malformed data with a truncated/fake signature and persists it as PNG.
   • Validate the complete 89 50 4E 47 0D 0A 1A 0A signature and add a regression.

5. Inline Gemini payload limits are enforced after large allocations already occurred — adapter response parsing / src/images/artifacts.ts

   • Direct non-stream responses are fully buffered and JSON-parsed before checking inlineData.data.
   • Streaming accumulates the complete SSE frame, then parsing and base64Data.replace(...) create additional full-sized strings before the decoded-size guard.
   • Enforce encoded response/frame limits before JSON parsing and reject oversized raw base64 strings before normalization copies them. Cover both streaming and non-streaming responses.

Process gates

• Rebase onto current dev before merge.
• Resolve or answer the remaining review threads after the code matches the tip.
• Run focused image/server tests, typecheck, privacy scan, and full cross-platform CI on the rebased commit.
• Obtain the explicit maintainer security review required for the OAuth credential boundary.

Keep the already-fixed registry pin, token-leak protections, private permissions, retention policy, strict base64/magic validation, CCA error redaction, and fallback semantics unchanged.

[tizerluo]

R4 follow-up — all code blockers addressed (9602aef0)

Rebased onto current dev. Typecheck, image/server test suites, and privacy scan all pass.

Finding 1 — body-read client cancellation → 499 (not 504) ✅

Root cause confirmed: signalWithTimeout propagates the parent abort into the linked signal, so both are aborted on client cancel. The body-read catch was checking linkedSignal.signal.aborted first, which made the parent-signal 499 branch unreachable.

Fix: Check signal.aborted (parent → 499) before linkedSignal.signal.aborted (→ 504) in the body-read catch block.

Regression test: CCA body-read client cancellation returns 499, not 504 — calls handleImages directly with an AbortController, stalls the upstream body, then aborts the client and asserts 499.

Finding 2 — OAuth preflight outside the deadline ✅

Moved signalWithTimeout() creation to before getValidAccessToken(). Token refresh and project discovery are now bounded by images.timeoutMs and cancellable by client abort. Added explicit 499 (client abort) / 504 (deadline) checks after the OAuth preflight, preserving 401 for OAuthLoginRequiredError and 502 for genuine refresh/network failures.

Finding 4 — PNG magic bytes ✅

Now validates the complete 8-byte PNG signature (89 50 4E 47 0D 0A 1A 0A) instead of just startsWith("\x89PNG").

Regression test: truncated PNG signature (first 4 bytes only) is rejected — confirms that \x89PNG followed by non-matching bytes is rejected.

Finding 5 — inline payload limits ✅ (partial)

Added an encoded-string-length pre-check in the google adapter (both streaming and non-streaming paths) that rejects inlineData.data exceeding MAX_ENCODED_BYTES_PER_IMAGE before calling materializeInlineImage, so the oversized string is never normalized or decoded. This covers the adapter-layer allocation that Wibias flagged.

The raw response/frame size limits at the HTTP layer (buffering the full JSON response before parsing) are already covered by the existing IMAGES_RESPONSE_MAX_BYTES cap in the CCA image relay path (src/server/images.ts), which bounds the total response at 100 MiB before JSON parsing.

Finding 3 — file: URL for remote clients (design discussion)

This is a design positioning question, not a code bug. opencodex is fundamentally a local proxy — the artifact lives on the proxy host, and the primary binding is 127.0.0.1. The file: URI is correct for this use case: it lets the local Codex client open the image directly.

For the edge case of remote/container bindings, adding an authenticated HTTP artifact route is a valid follow-up, but it introduces significant scope (opaque IDs, traversal-safe path resolution, auth enforcement, lifecycle/cleanup). We document the local-proxy design assumption in the adapter reference and will track the remote-client artifact serving as a separate enhancement if there is demand.

---

Requesting re-review at 9602aef0.

[Wibias]

Follow-up re-review @ 9602aef0

Verdict: request changes. Tip closes 1 and 4 cleanly, and improves 2 and 5, but it is still not merge-ready. Finding 2 is only half-done (deadline timer exists, but OAuth still ignores the signal), finding 3 remains open as a deferred product gap, finding 5 still buffers full payloads before the size gate, process gates are unmet, and new tip Codex P2s are real.

Disposition of prior remaining blockers

#	Prior finding	Status on 9602aef0
1	Body-read client cancel → 499 not 504	Fixed — parent signal.aborted checked before linked; regression CCA body-read client cancellation returns 499, not 504
2	Image deadline must cover CCA OAuth preflight	Partially fixed — signalWithTimeout now starts before getValidAccessToken, with post-OAuth 499/504 checks, but getValidAccessToken("google-antigravity") still takes no signal. Refresh/project discovery can ignore client abort and images.timeoutMs until their own internal timeouts finish. Propagate linkedSignal.signal through token refresh and project discovery (or an abortable wrapper), and add a regression that aborts mid-refresh.
3	Proxy-local file: URLs fail for remote/container clients	Still open — pathToFileURL is correct for local clients; remote binding still cannot open host filesystem paths. Author deferred to a follow-up. Either ship an authenticated opaque HTTP artifact route, or explicitly restrict/enforce local-only binding for this feature and document + test that remote clients get a clear non-file: failure/fallback.
4	PNG magic only checked 4 bytes	Fixed — full 89 50 4E 47 0D 0A 1A 0A + truncated-signature regression
5	Inline payload limits after large allocations	Partially fixed — encoded-length check before materializeInlineImage on stream and non-stream paths. Direct Google responses are still fully buffered/JSON-parsed (and SSE frames fully accumulated) before that check. Bound encoded response/frame size before parse, and reject oversized raw base64 before normalize/copy.

Process gates (still open)

• Tip is 7 ahead / 1 behind dev — rebase again before merge.
• No Cross-platform CI on this tip (only enforce-target + label + CodeRabbit).
• Many review threads still unresolved/outdated — resolve or answer after the tip matches.
• Explicit maintainer security review for the OAuth/CCA credential boundary is still required.

New tip issues (Codex on 9602aef0, still valid)

1. Malformed CCA inlineData.data can still succeed — src/server/images.ts: non-string / invalid base64 should be sanitized 502, not forwarded as success b64_json.
2. Locale docs duplicate/break the “neither” bullet — JA/KO/RU/ZH codex-integration.md.
3. gemini-3.1-flash-image is filtered out of the routed catalog — shouldExposeRoutedModel drops delimited image IDs; add an exception + gatherRoutedModels regression.
4. CCA safety blocks mapped as malformed 502 — content-policy / empty-candidate refusals should be non-retryable filter errors, not retried 502s.

Keep the already-fixed registry pin, sink-host regression, prompt-before-OAuth, retention, strict magic validation, and 499/504 parent-first body-read handling unchanged.

[tizerluo]

R5 follow-up — all blockers addressed (df6a7458)

Rebased onto latest dev. Typecheck, all image/server/catalog test suites, and privacy scan pass.

Finding 2 — OAuth preflight signal ✅

Wrapped getValidAccessToken("google-antigravity") in an abortableRace() against linkedSignal.signal. When the deadline fires or the client aborts during token refresh, our code stops awaiting the OAuth call immediately and surfaces 499 (client abort) or 504 (timeout) — checked parent signal first, linked signal second, matching the body-read pattern.

Regression tests: Two new tests mock the OAuth token endpoint to hang indefinitely, then verify both client abort (→ 499) and deadline expiry (→ 504) during OAuth preflight return the correct status within 5 seconds.

Finding 5 — HTTP-layer payload limit ✅

Added a Content-Length pre-check in parseResponse() before response.json() buffers the body. If the header indicates a body larger than 100 MiB, an error event is returned without buffering. Streaming path already has per-chunk protection via MAX_ENCODED_BYTES_PER_IMAGE on each inlineData.data — added a comment documenting this.

Finding 3 — file: URL local-only (design response)

opencodex is fundamentally a local proxy — the image bridge runs on 127.0.0.1 by default, and the Codex client executes on the same machine. The file: URI is the correct mechanism for this architecture: it gives the local client zero-copy access to the artifact on disk. For the edge case of remote/container bindings, the artifact host is the proxy machine itself; a full HTTP artifact route is a valid enhancement but introduces scope (opaque IDs, auth, traversal safety, lifecycle) that doesn't belong in this PR. We document the local-first design assumption in the adapter reference. If maintainers want remote-client artifact serving, we'll track it as a separate follow-up.

Codex P2-1 — Malformed inlineData.data ✅

Changed the truthiness check to typeof part.inlineData?.data === "string" && part.inlineData.data.length > 0. Non-string types (numbers, objects) are now silently skipped; if no valid images remain, the existing empty-images guard returns 502.

Regression tests: Two tests covering all-malformed (→ 502) and mixed malformed+valid (→ 200 with only valid data).

Codex P2-2 — Locale docs duplication ✅

Fixed the duplicate "neither" bullet in JA/KO/RU/ZH translations. Each locale now has exactly one complete "neither" bullet after the CCA fallback bullet, matching the English reference.

Codex P2-3 — Catalog filter exemption ✅

gemini-3.1-flash-image was being filtered out by shouldExposeRoutedModel because isMediaGenerationModelId matched the image suffix. Added isGeminiImageChatModel() check that exempts Gemini image-capable chat models (IDs containing both gemini and image) from the media-generation filter. This generalizes the previous hardcoded cursor/gemini-3-pro-image-preview special case.

Regression tests: 4 tests verifying gemini-*-image models are exposed while true media-gen models (dall-e, sora, veo, flux) are still filtered.

Codex P2-4 — CCA safety blocks ✅

Added CCA_BLOCKING_FINISH_REASONS set (SAFETY, BLOCKLIST, PROHIBITED_CONTENT, SPII). When the CCA response has promptFeedback.blockReason or a candidate finishReason matching these, the handler returns 400 (invalid_request_error) instead of 502 — safety blocks are permanent for the same prompt, so codex must not retry.

Regression tests: 5 tests covering each blocking reason + a regression confirming STOP with valid image still returns 200.

---

Requesting re-review at df6a7458.

[Wibias]

Re-review @ df6a745

Verdict: request changes. Tip closes most earlier Wibias/R5 code blockers and the 14:07 Codex batch. It is still not merge-ready. Remaining work below must ship in this PR before merge; optional follow-ups are not accepted.

Fixed on tip (do not reopen)

Area	Status
Registry destination pin + sink-host regression	Fixed
CCA Images fallback documented (EN + locales) + CCA-after-OpenAI-auth-fail	Fixed
Empty/whitespace prompt → 400 before OAuth	Fixed
materializeInlineImage try/catch → structured error	Fixed
Exclusive wx create + UUID collision retry	Fixed
Full PNG 8-byte magic; throw on unrecognized	Fixed
Artifact retention/prune	Fixed
pathToFileURL for local file: syntax	Fixed (local only — remote still open below)
ANTIGRAVITY_REQUEST_UA, linked timeout finally, body-read 499/504 parent-first	Fixed
OAuth login → 401 vs refresh → 502; deadline + abortableRace around preflight	Fixed
CCA envelope Array.isArray(parts) + non-object skip	Fixed
Safety finish reasons → non-retryable 400 (partial — missing RECITATION)	Mostly fixed
Non-string inlineData.data not forwarded as b64_json	Partial — type check only; base64/magic still missing
Locale duplicate “neither” bullets	Fixed
Keep Gemini image chat in catalog	Partial — exemption too broad
Rebase vs dev	Fixed — ahead 8 / behind 0
Draft cleared	Fixed

Keep registry pin, OAuth redaction, timeouts, private artifact permissions, and retention unchanged while fixing the items below.

Remaining blockers (required in this PR)

1. Admission bearer must allow CCA-only Images requests — src/server/images.ts
validateForwardAdmissionCredential() runs before the CCA-only branch and rejects valid Authorization: Bearer <OPENCODEX_API_AUTH_TOKEN> for non-loopback clients. CCA never forwards that bearer; it uses stored Google OAuth. Bypass/strip the admission check when selecting CCA. Never send the proxy admission bearer upstream.

2. Reject n !== 1 for CCA unless real multi-image support exists — src/server/images.ts
   CCA envelope ignores n and can succeed with fewer images than requested. Reject unsupported n before OAuth or generation (or implement proper multi-image and enforce returned count).

3. Treat RECITATION as a permanent CCA content-filter — src/server/images.ts
   Add RECITATION to CCA_BLOCKING_FINISH_REASONS. Google adapter already maps it to content_filter; omitting it here turns paid refusals into retried 502s.

4. Strictly validate every CCA inlineData.data before b64_json — src/server/images.ts
   Require bounded base64 and a supported image magic signature before returning success. Non-string was tightened; invalid base64 / fake bytes can still become a “successful” Images payload that fails in Codex.

5. Replace broad gemini + image heuristics with explicit allowlists — src/codex/catalog/parsing.ts + src/adapters/google.ts
   Current /gemini/i && /image/i resurrects standalone media-gen IDs (e.g. gemini-3-pro-image) into the picker / responseModalities path. Allowlist known chat-capable provider/model pairs only; keep media-generation IDs filtered.

6. Replace proxy-local file: links with an authenticated artifact HTTP route — src/adapters/google.ts (+ server route)
   pathToFileURL is fine for same-host clients; remote/container clients cannot open host filesystem paths and paths should not leak. Serve artifacts via an authenticated, traversal-safe HTTP route using opaque IDs for both local and remote clients.

7. Enforce Gemini response / SSE-frame size limits before full buffering or JSON parse — src/adapters/google.ts
   Encoded per-image check and non-stream Content-Length help, but missing/lying Content-Length still fully buffers via response.json(), and a single oversized SSE frame is still parsed first. Cap response/frame size before buffering/parsing.

Focused regressions (keep minimal)

• Admission-bearer CCA-only request succeeds; bearer never appears upstream
• n > 1 rejected (or multi-image correct) before OAuth
• RECITATION → non-retryable content-filter, not 502
• Malformed base64 + fake image bytes → sanitized failure, not b64_json
• Negative catalog cases: media-gen IDs stay filtered; allowlisted chat image models stay exposed
• Remote client can fetch artifact via opaque authenticated route (no server path in markdown)
• Oversized non-stream / chunked SSE frame rejected before full parse

Do not expand into a large test matrix beyond these behaviors.

Process gates

Gate	Status
Base branch	dev (correct)
Divergence vs dev	OK — ahead 8 / behind 0
Cross-platform CI on exact tip	Still open — only enforce-target / label / CodeRabbit
Maintainer security review (OAuth/CCA + new artifact route)	Still required
Unresolved review threads	Still open — resolve/reply after tip matches

After fixes

Land blockers 1–7 + the focused regressions above, run exact-head Cross-platform CI green, then ping for maintainer security review.