Skip to content

fix(backends): reject empty user content before sending chat request - #1632

Draft
simonyang08 wants to merge 1 commit into
generative-computing:mainfrom
simonyang08:codex/mellea-1597-empty-user-content-guard
Draft

fix(backends): reject empty user content before sending chat request#1632
simonyang08 wants to merge 1 commit into
generative-computing:mainfrom
simonyang08:codex/mellea-1597-empty-user-content-guard

Conversation

@simonyang08

Copy link
Copy Markdown

Fixes #1597

Root cause

SimpleContext.view_for_generation() returns [] by design — it is a stateless context. But .add() still exists and returns a chainable context, which reads like it records the turn for the next call. A caller who .add()s a user message and then passes an empty or whitespace-only action to generate_from_context ends up sending the model a conversation with no user-role content at all, and nothing errors. This is exactly what happened in #1587's live Ollama telemetry tests, where Granite 4.2 rambled for 2000+ tokens on the resulting empty prompt and stalled CI.

Fix

Add a shared invariant guard in the chat-message assembly path of the three backends that build user-role conversations for chat APIs — OpenAIBackend._generate_from_chat_context_standard, LiteLLMBackend._generate_from_chat_context_standard, and OllamaModelBackend.generate_from_chat_context:

  • if the fully assembled messages list has no user-role message with non-whitespace content, raise ValueError before any HTTP/SDK call is issued;
  • whitespace-only content counts as empty; user messages that carry images, audio, or documents (_docs) still pass, so vision/RAG paths keep working.

ValueError matches the existing repo convention for "call cannot proceed" (cf. context_lengths.py, intrinsic/_util.py). SimpleContext.add/view_for_generation docstrings now state explicitly that recorded turns are never forwarded, so the .add()-then-empty-action trap is documented at the API surface too.

Note on scope: WatsonxAIBackend and LocalHFBackend also assemble user-role conversations and would benefit from the same guard, but are intentionally left untouched to keep this change focused on the backends named in the issue. Happy to mirror the guard in a follow-up if preferred.

Verification

New regression suite test/backends/test_simple_context_guard.py (mocked providers, no network):

  • empty / whitespace-only action raises on OpenAI, LiteLLM, and Ollama backends — verified RED on unpatched main (DID NOT RAISE ValueError), green after the fix;
  • non-empty user content still sends the request on all three backends;
  • a user message with images passes the guard (multimodal path);
  • the guardian.py call pattern (non-empty context + empty assistant action) is not affected.

Focused backend/context suites: 170 passed. test/stdlib/ regression run shows no new failures relative to unpatched main.

DCO: Signed-off-by: simonyang08 <ppt5928@gmail.com>. AI-assisted contribution per CONTRIBUTING.md (Assisted-by: ZCode (GLM) trailer on the commit).

SimpleContext.view_for_generation() returns [] by design (stateless
context), so a caller who chains .add(...) and then passes an empty or
whitespace-only action to generate_from_context used to ship an
empty user-role conversation to the model. Some chat models — Granite
4.2 in particular, see issue generative-computing#1587 — spin on an empty prompt and burn
tokens silently, which is hard to diagnose in CI.

Raise ValueError in three concrete chat assembly paths:
  * OpenAIBackend._generate_from_chat_context_standard
  * LiteLLMBackend._generate_from_chat_context_standard
  * OllamaModelBackend.generate_from_chat_context

The check ignores whitespace-only strings but still accepts user
messages that carry images, audio, or documents (vision / RAG paths
must continue to work). Each guard raises a ValueError before any
HTTP or SDK call is issued.

WatsonxAIBackend (mellea/backends/watsonx.py) and LocalHFBackend
(mellea/backends/huggingface.py via mellea/backends/utils.py:to_chat)
also assemble user-role conversations and would benefit from the same
guard, but they are intentionally left untouched to keep this change
focused on the backends named in the issue. Happy to mirror the same
guard block in those two backends in a follow-up if preferred.

SimpleContext.add's docstring is updated to make the stateless
semantics explicit (recorded turns are never forwarded; the action
argument is the only thing that reaches the model) and to call out
which backends now enforce the invariant.

Closes generative-computing#1597

Assisted-by: ZCode (GLM)
Signed-off-by: simonyang08 <ppt5928@gmail.com>
@github-actions github-actions Bot added the bug Something isn't working label Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SimpleContext.add() content is silently discarded from generation — add a safety net

2 participants