feat: add Chatterbox as a media-use TTS provider - #3894
Open
sumit-uit wants to merge 1 commit into
Open
Conversation
Chatterbox (Resemble AI, MIT-licensed) is a self-hosted, zero-shot voice-clone TTS server exposing an OpenAI-compatible /audio/speech endpoint (e.g. travisvn/chatterbox-tts-api). Add it as a fourth provider in the media-use audio engine's chain, ranked above ElevenLabs/Kokoro once its local server is reachable (a cloned voice generally beats a generic one when available) but below HeyGen (credential-gated, so an explicit opt-in already signals intent). - lib/tts.mjs: chatterboxBaseUrl()/chatterboxAvailable() (live health check against $CHATTERBOX_BASE_URL, default http://127.0.0.1:4123/v1), a chatterbox branch in resolveVoiceId()/synthesizeOne(), and synthesizeChatterbox() (injectable deps, matching synthesizeHeygen's test pattern). pickProvider() is now async since availability requires a network round-trip, unlike the other providers' env/credential checks. - audio.mjs: await the now-async pickProvider(); update the inline provider-list comment. - tts.md: document the new route, its ranking, and when to reach for it. - tts.test.mjs: 7 new tests covering base-URL resolution, availability on a dead port, provider validation, and synthesizeChatterbox's success/HTTP-error/network-error paths. No word timestamps from Chatterbox itself (same as ElevenLabs/Kokoro) — the caller already chains transcribeWav() for those three providers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/audio/speechendpoint — e.g.travisvn/chatterbox-tts-api) as a fourth provider in the media-use audio engine's TTS chain.$CHATTERBOX_BASE_URL, defaulthttp://127.0.0.1:4123/v1) — a cloned voice generally beats a generic one when the server is reachable — but below HeyGen, whose availability is credential-gated and thus already an explicit opt-in.pickProvider()becomesasyncsince Chatterbox availability needs a network round-trip (health check), unlike the other providers' plain env/credential checks. Updated the one caller inaudio.mjsaccordingly.transcribeWav()chain handles it, no new code needed there.Why
Chatterbox is a common way to get a real cloned voice (e.g. a channel owner's own voice) into a local pipeline without a cloud TTS subscription, and several projects already wire it into custom scripts by hand. This closes that gap in
media-useitself so it Just Works via--provider chatterbox(orauto, once the server is up) instead of every project reinventing the same/audio/speechcall.Test plan
node --test skills/media-use/audio/scripts/lib/tts.test.mjs— 18/18 passing (7 new: base-URL resolution incl. env override,chatterboxAvailableagainst a dead port, provider-name validation, andsynthesizeChatterbox's success / HTTP-error / thrown-network-error paths, all using the same injectable-depspattern as the existingsynthesizeHeygentests).pickProvider(null)→"chatterbox",synthesizeOne()produces a valid WAV,transcribeWav()returns correct word timings, and a fullaudio.mjs --request audio_request.jsonrun synthesized 9 narration lines successfully.heygen-tts.mjs,npx-sync.mjs,tts-local-provider.mjsdon't callpickProvider).🤖 Generated with Claude Code