[agoldis]

Owner: @agoldis

Linear: ENG-2181

Adds the ai-queue-worker service, off by default in an ai profile. On-prem webhooks no longer runs AI tasks after the Currents release that contains currents-dev/currents#4874, so an install that uses AI needs this service turned on.

This must ship with, not before, the Currents release that contains currents-dev/currents#4874. The image ai-queue-worker does not exist before then, so docker compose pull fails for the new service on any earlier tag. Merge it into the release that sets DC_CURRENTS_IMAGE_TAG to that tag.

Changes

• templates/compose.currents.yml: service ai-queue-worker, image ${DC_CURRENTS_IMAGE_REPOSITORY}ai-queue-worker:${DC_CURRENTS_IMAGE_TAG}, built like webhooks (env_file: .env, CURRENTS_ENV: onprem, no ports, no command override).
• Off by default. The service is in the ai profile, as traefik is in tls, and starts only with COMPOSE_PROFILES=ai in .env. It needs an LLM endpoint the admin provides. The docs don't yet say what AI features an admin gets, how we use their provider, or what it costs (ENG-2209). AI is also off on-prem until an org admin grants consent.
• The AI indexes. Turning AI on also takes AI_QUEUE_WORKER_ENABLED=true in .env, so the scheduler builds the AI collections' MongoDB indexes, including the TTLs that expire AI sessions. The director builds indexes only with CREATE_MONGO_INDEX, which no compose file sets.
• Redis and MongoDB depends_on, and the RustFS FILE_STORAGE_FORCE_PATH_STYLE, are copied from webhooks in the three data-service templates.
• Compose files regenerated with scripts/sync-templates.sh.
• .env.example and docs/configuration.md: how to turn the worker on (COMPOSE_PROFILES=ai, AI_QUEUE_WORKER_ENABLED=true), and the LLM_* and AI_QUEUE_CONCURRENCY settings. No new required variable.
• docs/container-images.md: ECR policy and both image-mirroring lists. on-prem/README.md: services list.
• on-prem/CHANGELOG.md: Unreleased entries with the upgrade notes below.
• webhooks and the director are unchanged. No compose file sets AI_QUEUE_WORKER_ENABLED.
• No VERSION or image tag change; the release PR does that.

Upgrade notes for on-prem customers

• If you use AI today, it stops at this upgrade until you turn the worker on. Add COMPOSE_PROFILES=ai to .env, and keep AI_QUEUE_WORKER_ENABLED=true and your LLM_PROVIDER_URL / LLM_API_KEY.
• To turn it on, add the currents/on-prem/ai-queue-worker repository to your ECR policy and mirror the image with the others.
• Regenerate your compose file if you use custom templates.
• In webhooks, AI_QUEUE_WORKER_ENABLED=true now only starts the Slack AI answer worker, and only when SLACK_AI_CALLBACK_WORKER_ENABLED is unset.

Checked

• docker compose config --quiet on all three files with the validate workflow's minimal .env: valid. config --services leaves out ai-queue-worker by default and includes it with COMPOSE_PROFILES=ai; with tls,ai it includes traefik too.

• scripts/sync-templates.sh was run after the profile change.

• docker compose -f docker-compose.{full,database,cache}.yml config --quiet with a minimal .env, as the validate workflow does: all three pass.

• scripts/sync-templates.sh: the committed files match the templates.

• Not run: the Podman validation, and the smoke test (it starts only scheduler and api, so it does not start the new service).

Checklist

• PR title is Changelog-friendly: (fix|feat|chore): title [CSR-xxx]
• User-friendly and meaningful description of the changes
• Documentation was updated

🤖 Generated with Claude Code

https://claude.ai/code/session_011baD3izoabUn31Y3WeQHMp

[baz-reviewer]

Baz Summary

Add the optional ai-queue-worker service to run on-prem AI tasks from ai-bmq after webhooks stops processing them. Configure its Docker Compose profiles, dependencies, environment settings, image distribution, and upgrade documentation so administrators can enable AI safely with their own LLM endpoint.

Topics

Topic	Details
AI Deployment Guidance	Explain the upgrade impact and add the worker image to ECR access and mirroring workflows, including guidance that existing AI users must explicitly enable the new service. Modified files (3) docs/configuration.md docs/container-images.md on-prem/CHANGELOG.md Latest Contributors(2) User Commit Date dj@currents.dev docs: say how to turn ... November 12, 2026 agoldis@gmail.com feat: add the ai-queue... November 10, 2026
On-Prem AI Worker	Add the profile-gated ai-queue-worker service to all on-prem Compose variants, configure MongoDB, Redis, and RustFS integration, and document enabling it alongside AI_QUEUE_WORKER_ENABLED and LLM settings. Modified files (9) on-prem/.env.example on-prem/README.md on-prem/docker-compose.cache.yml on-prem/docker-compose.database.yml on-prem/docker-compose.full.yml on-prem/templates/compose.currents.yml on-prem/templates/compose.mongodb.yml on-prem/templates/compose.redis.yml on-prem/templates/compose.rustfs.yml Latest Contributors(2) User Commit Date dj@currents.dev docs: say how to turn ... November 12, 2026 agoldis@gmail.com feat: add the ai-queue... November 10, 2026

---

Merger   ✅ Ready to Merge

MERGE: the new AI queue worker and its enablement/documentation are present, CI ran, and all review discussions are settled.

---

  _{Activate Skill Maintainer to keep your skills up to date}

  _{This PR would have been improved with Baz Planner - Try it now}

[twk3]

Ran this locally on images built from main 6f87424c7 (includes currents-dev/currents#4874), full compose file, with USE_MOCK_AI=true. The worker starts and webhooks no longer takes AI tasks. testSidebarAiAnalysis.spec.ts passed with its GraphQL stub removed: both requests in aiRequests completed, and only ai-queue-worker logged them.

One change requested: put the service behind a Compose profile so it is off by default for now. AI is already off per org until features.aiAnalysis is set, and the worker needs LLM_* settings to do anything, so most installs would run an idle container and have to pull and mirror an image they don't use. With a profile, docker compose pull and up skip it unless COMPOSE_PROFILES includes ai. traefik already does this with tls. Comments inline.

Baz withdrew its approval because this pull request is no longer ready to merge.

[twk3]

Tested docker-compose.full.yml at 82b4068 on Docker Desktop, with the seven release/2026-11-10-003 images built locally. The .env came from scripts/setup.sh --env-only, with the image tag set and the host ports moved.

1. Default .env

• All 10 services started, without ai-queue-worker or traefik.
• The scheduler has no AI_QUEUE_WORKER_ENABLED. Its startup tasks completed, and aiSessions, aiSessionMessages and aiSlackCallbacks weren't created.
• I seeded a run with failed tests, granted the org FAILURE_ANALYSIS consent, and sent POST /v1/ai-requests (errorExplanation). The request stayed pending after 20 s, with its job in bull:ai-bmq:wait.

2. Worker on, following .env.example

I uncommented COMPOSE_PROFILES=ai and AI_QUEUE_WORKER_ENABLED=true, set LLM_PROVIDER_URL, LLM_API_KEY and USE_MOCK_AI=true, and ran docker compose up -d.

• ai-queue-worker started with 0 restarts. The scheduler finished its startup tasks 19 s after up.
• The pending request went to completed with success: true, and the queue emptied. Worker log: "Processing test error analysis", "AI generator completed", "test error analysis completed".
• A new testExplanation request completed in 44 ms.
• The scheduler built the indexes on the three AI collections, including the expiresAt_1 TTL indexes.
• Every app service was recreated, not only the worker, because they all read .env.

3. TLS

With COMPOSE_PROFILES=tls,ai:

• docker compose up -d starts traefik and ai-queue-worker.
• After docker compose down, docker compose --profile tls up -d starts traefik only. That's the case Baz flagged, which 82b4068 fixes in the docs.

4. Turning it off

• With ai removed from COMPOSE_PROFILES, docker compose up -d left ai-queue-worker running with its old settings. --remove-orphans didn't remove it either. docker compose rm -sf ai-queue-worker did.
• dcf7ac6 adds that to docs/configuration.md and .env.example.

Not covered:

• A real LLM provider. USE_MOCK_AI returns a fixed answer without calling one.
• Pulling the images from ECR.
• MongoDB in the last step. After the down, MongoDB couldn't reopen its files in the macOS bind-mounted data/mongodb ("Operation not permitted"), so the services in step 4 ran without a database. I've seen the same on this Docker Desktop VM before, outside this PR. It doesn't change the step 4 result, which is only about which containers run.

Baz withdrew its approval because this pull request is no longer ready to merge.