Add a talking avatar to a voice agent
Give a LiveKit, Pipecat or OpenAI Realtime voice agent a lip-synced face.
What you’ll build
bitHuman turns your voice agent’s reply audio into lip-synced video of an avatar; the agent keeps its own speech recognition, language model and voice. In LiveKit, start a bithuman.AvatarSession beside the AgentSession. In Pipecat, put BitHumanVideoService after the TTS service. With OpenAI Realtime or any other stack, push the reply audio into the Python, Swift or Android SDK.
All three take Essence 2 (a real person) and Expression 2 (any character). You need:
- a voice agent that produces spoken replies, or the LiveKit or Pipecat example on its platform page;
- an API secret for an account on the Creator plan or higher;
- Python 3.10–3.14 for LiveKit, or 3.11–3.14 for Pipecat.
Steps
5 steps
Choose the integration for your stack
Your stack Use The avatar renders LiveKit Agents livekit-plugins-bithumanin the bitHuman cloud ( avatar_id=), or inside your worker (model_path=)Pipecat pipecat-bithumaninside your bot’s process OpenAI Realtime or any stack, in Python the Python SDK, AsyncBithumanon your Mac, Linux or Windows machine Any stack, in an iPhone, iPad, Mac or Android app the Swift package or the Android SDK on the device No voice stack yet a bitHuman agent in the web embed in the bitHuman cloud, or in the browser tab With the web embed, the conversation runs on bitHuman’s servers, even when the avatar renders in the tab.
Expected
One row, and the package it names.
Pick the avatar
Use the
wise-pupsample (Expression 2, agent codeA23WJF0199) orsofia-ramirez(Essence 2,A52DHS2219) while you build, or create your own. A LiveKit cloud avatar takes the agent code. Pipecat and the Python SDK render from an avatar on the machine.python -m bithuman pull(bithuman 2.12.8 or newer:pip install -U bithuman) downloads it and prints its local path. It needsBITHUMAN_API_SECRET, for the samples too:export AVATAR=$(python -m bithuman pull A23WJF0199) # or your own agent codeExpected
An agent code, or the avatar's local path in
$AVATAR.LiveKit: start an avatar session beside the agent session
pip install "livekit-agents[openai,silero]" livekit-plugins-bithuman python-dotenvIn the worker, keep your API secret as
BITHUMAN_MASTER_SECRETand pass the plugin a minted one-hour token, so the secret never reaches the room. Startbithuman.AvatarSession(avatar_id=…)beforesession.start(…, room_options=RoomOptions(audio_output=False)): the avatar publishes the audio. The completeagent.py, with the mint call, is on LiveKit: Run your first avatar.To render inside your worker instead, pass
model_path=the path in$AVATARand install"bithuman[expression-2]"(Voice agent in Python).Expected
python agent.py dev, then join the room from the LiveKit Agents Playground: the avatar appears and answers with its lips in sync.Pipecat: put the avatar after the TTS service
pip install "pipecat-bithuman[expression-2]"Set
BITHUMAN_API_SECRETandBITHUMAN_MODEL_PATH="$AVATAR", turn onvideo_out_enabled=Truein your transport, and addBitHumanVideoService()betweenttsandtransport.output(). The whole bot is on Build a Pipecat bot.Expected
The bot answers with the avatar's video and the avatar's copy of the speech, paired frame by frame.
OpenAI Realtime or any other stack: push the reply audio
In Python,
AsyncBithumantakes each chunk of reply audio with its sample rate and yields frames with the matching audio. Feed it the PCM audio your stack plays, from OpenAI Realtime or a TTS service; the code is on Python: Integrate into your app.Call
flush()when a reply ends andinterrupt()when the user talks over it. In an iPhone, iPad, Mac or Android app, resample the reply to 16 kHz mono first: Resample speech to 16 kHz. To run OpenAI Realtime with no OpenAI key of your own, use the Realtime relay.Expected
Idle frames until the reply arrives, then the lips follow its words, with the audio in step.
All steps done. Next: make it your own.
How it works
The avatar does not listen or think: it renders the speech your agent already produces. With a LiveKit cloud avatar, the audio reaches bitHuman and the avatar renders in the bitHuman cloud, in the US. When it renders in your worker, bot or app, its audio and video stay with you; bitHuman receives a credential check and usage reports. What bills on each path: Pricing.
Make it your own
- Interruptions: stop the avatar mid-sentence when the user talks: Barge-in.
- Your own character: one portrait makes an avatar: Create your own avatar.
- Everything on one computer: a local LiveKit server and the CLI: Voice agent.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| Two voices play in LiveKit | the agent session also publishes audio | set room_ |
The mint call returns 401 | a missing or invalid API secret | check BITHUMAN_ against your API secret |
The Pipecat bot speaks with no video and no ErrorFrame | video out is off in the transport | set video_ in the transport’s params |
| The avatar refuses to start | no API secret, or a plan below Creator | set the secret for an account on the Creator plan or higher |
| The mouth runs slow in an app | the speech is not 16 kHz | resample it before you feed it |
More on LiveKit: Troubleshooting and Pipecat: Troubleshooting.