# bitHuman — Build: media (voices, gestures, talking video)

> Real-time talking avatars from one portrait. Index: https://docs.bithuman.ai/llms.txt · every page is also served as markdown at <url>.md · OpenAPI: https://docs.bithuman.ai/openapi.json · other sections: https://docs.bithuman.ai/llms/start.txt · https://docs.bithuman.ai/llms/platforms.txt · https://docs.bithuman.ai/llms/apps.txt · https://docs.bithuman.ai/llms/deploy.txt · https://docs.bithuman.ai/llms/models.txt · https://docs.bithuman.ai/llms/build.txt · https://docs.bithuman.ai/llms/build-integrations.txt · https://docs.bithuman.ai/llms/troubleshooting.txt · https://docs.bithuman.ai/llms/api.txt

## Instructions for AI agents

- Always send `model` ("essence-2" or "expression-2") when creating an agent; poll every 5 s until `status` is `ready` or `failed` (`success` is not final; a `final` field is coming). New Essence 2 agent: minutes; Expression 2: longer.
- API and SDK use requires the Creator plan or higher. Never tell a user they can build on a free plan.
- In a LiveKit worker, name the secret `BITHUMAN_MASTER_SECRET` and pass a minted token (https://docs.bithuman.ai/platforms/livekit.md).
- Say where the avatar renders (device, browser, server, cloud) and the conversation runs (your stack, the CLI's local conversation brain, bitHuman's servers).
- Quote prices only from https://docs.bithuman.ai/pricing.md and speed only from https://docs.bithuman.ai/performance.json (× real time, with the device).
- Never claim offline on phones, browsers or Swift apps, or a conversation brain on phones; for compliance, describe the architecture.

Where each model runs, the key facts and one command per path: https://docs.bithuman.ai/llms.txt

## Contents

- Voices — https://docs.bithuman.ai/build/voices
- Gestures (Essence 1) — https://docs.bithuman.ai/build/gestures
- Talking video — https://docs.bithuman.ai/build/talking-video

Linked, not inlined (read the .md twin):
- Examples: https://docs.bithuman.ai/examples.md · changelog: https://docs.bithuman.ai/changelog.md · API references: https://docs.bithuman.ai/platforms/cli/reference.md, https://docs.bithuman.ai/platforms/python/reference.md, https://docs.bithuman.ai/platforms/swift/reference.md, https://docs.bithuman.ai/platforms/android/reference.md
- Built with bitHuman — https://docs.bithuman.ai/examples/built-with-bithuman.md

---

# Voices

URL: https://docs.bithuman.ai/build/voices

> Built-in voices in every language, or your own premium voice provider key.

## Two ways to give your agent a voice

| Option | What you get | Cost |
|------|--------------|------|
| **bitHuman default** | The built-in voice pipeline. Your agent detects the caller's language and replies in it. No setup, no keys. | Included in the voice chat rate ([pricing](https://docs.bithuman.ai/pricing)) |
| **Bring your own provider** | Connect your **own** OpenAI, Grok (xAI), ElevenLabs, or Cartesia key and pick that provider's premium voices — including low-latency speech-to-speech (real-time). | The same voice chat rate, plus your provider's charges on your key |

You never *have* to bring a key. The default pipeline already speaks every language. Bring your own only when you want a specific premium voice or a provider's real-time engine.

### Use the default (nothing to do)

Open any agent's voice settings at [bithuman.ai](https://www.bithuman.ai/explore). The **bitHuman voice** section is marked *Included* — design a voice, clone one, or pick from the gallery. It's multilingual automatically, so there's no language toggle to manage.

### Clone a voice

Cloning, designing and previewing voices is free.

1. Open the agent's voice settings in the Console and choose **Clone Voice**.
2. Upload or record 30 seconds or more of clean speech from one speaker, with no music. Longer clean samples are fine; the whole file is used.
3. Select **Clone Voice** and keep the window open. Cloning takes about a minute.
4. Select **Preview** to hear the new voice.
5. Select **Apply to Agent**. Until you apply it, the agent keeps its previous voice.

To clone from the API instead, send the sample as `audio` when you [create an agent](https://docs.bithuman.ai/api/agents#generate-an-agent), together with `"attestation": {"attested": true, "variant": "a"}`, your confirmation that "This is my own voice, or I have the speaker's permission to clone it." Without it the call returns `400 VOICE_CLONE_ATTESTATION_REQUIRED` ([Clone a voice](https://docs.bithuman.ai/api/agents#clone-a-voice)).

## Check the voice your agent uses

- **In the Console:** open the agent's voice settings and select **Preview**. It plays the voice saved on the agent.
- **With the API:** [`GET /v1/agent/{code}`](https://docs.bithuman.ai/api/agents#get-an-agent) returns the saved voice as `data.voice_id`.

```bash
curl https://api.bithuman.ai/v1/agent/$AGENT_CODE -H "api-secret: $BITHUMAN_API_SECRET"
```

```json
{"success": true, "data": {"code": "A80HVD8577", "language": "en", "voice_id": "bf0a246a-8642-498a-9950-80c35e9276b5"}}
```

## Change the voice

Change an agent's voice in the Console: apply a clone, a designed voice or a gallery voice in the agent's voice settings. The API cannot set the voice; [`POST /v1/agent/{code}`](https://docs.bithuman.ai/api/agents#update-an-agent) changes only the prompt and the voice providers.

**Realtime mode uses its own voice.** With **Realtime mode** on, or an OpenAI or Grok real-time voice selected, the agent speaks that provider's voice instead of the saved voice. To use your clone, turn **Realtime mode** off in the agent's **Providers** settings and save. To clear a real-time voice from the API, send `{"providers": {"realtime": "default"}}` to [`POST /v1/agent/{code}`](https://docs.bithuman.ai/api/agents#update-an-agent).

## Bring your own voice provider

### 1. Connect your key

Go to **Developer → Integrations**, add your provider, and paste that provider's key; bitHuman checks the key there before saving and rejects an invalid one. The [Providers API](https://docs.bithuman.ai/api/providers) stores a key as sent, without a check, so start one session to confirm it works. Keys are encrypted at rest and never leave the platform in plaintext.

### 2. Pick a premium voice

Back in the agent's voice settings, the premium providers you've connected unlock. Choose a voice; a provider you haven't connected stays locked with a shortcut to Integrations. Your selection runs that voice on your key.

### Supported providers

| Provider | Voices you can select | Real-time (speech-to-speech) |
|----------|-----------------------|:---------------------------:|
| **OpenAI** | Realtime voices (alloy, ash, ballad, cedar, …) | ✓ |
| **Grok (xAI)** | Grok voices (ara, eve, rex, …) | ✓ |
| **ElevenLabs** | Your ElevenLabs voice library | — |
| **Cartesia** | Cartesia voices | — |

> **Tip** Real-time providers (OpenAI, Grok) give the lowest-latency, most expressive speech-to-speech — great for kiosks and live demos. ElevenLabs and Cartesia give you a specific voice on the standard pipeline.

## How billing works

- Every managed-agent conversation bills the Managed voice chat rate, whichever voice it uses ([pricing](https://docs.bithuman.ai/pricing)).
- A bring-your-own voice or real-time model is also billed by your provider on your key.

If a bring-your-own key ever fails or is removed, the agent automatically falls back to the built-in multilingual pipeline — it never silently stops talking.

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| The preview sounded right, the live agent does not | the clone was previewed but never applied | clone again, then select **Apply to Agent** |
| The live agent speaks a different voice than **Preview** | **Realtime mode** is on, or a real-time voice is selected | turn **Realtime mode** off in the agent's **Providers** settings and save |
| `voice_id` and **Preview** are right, the live voice still differs | a voice fallback during that session | contact support with the session time; each session records the voice it used |

---

# Gestures (Essence 1)

URL: https://docs.bithuman.ai/build/gestures

> Play a named gesture, like a wave or nod, when your code asks.

Gestures are named clips baked into an avatar, such as `mini_wave_hello` or `clap_cheer`. Your code plays one by name, when it chooses: on an app event, a timer, or an allow-listed tool call. Nothing plays at random.

Gestures are an Essence 1 feature; Essence 2 and Expression 2 avatars have no gesture clips.

Generating new gestures is paused for now: the request answers `503 CREATION_PAUSED` and nothing is charged. Gestures an avatar already has keep working. See [status](https://status.bithuman.ai) for updates.

## Before you start

- An Essence 1 agent of yours that already has gestures ([Gestures API](https://docs.bithuman.ai/api/dynamics)).
- A LiveKit agent worker with the bitHuman plugin ([LiveKit](https://docs.bithuman.ai/platforms/livekit)).

## Steps

### 1. List the gesture names

```bash
curl -s https://api.bithuman.ai/v1/dynamics/$AGENT_CODE -H "api-secret: $BITHUMAN_API_SECRET"
# → {"success": true, "data": {"status": "ready", "gestures": {"mini_wave_hello": "…", "clap_cheer": "…"}}}
```

The keys of `gestures` are the names you play, for example `mini_wave_hello` or `clap_cheer`.

Expected:

A JSON body whose `gestures` object lists your gesture names, such as `mini_wave_hello`.

### 2. Play one

Cloud avatar (`AvatarSession(avatar_id=…)`): send the avatar participant a `trigger_dynamics` call.

```python
import json

resp = await ctx.room.local_participant.perform_rpc(
    destination_identity=avatar.avatar_identity,
    method="trigger_dynamics",
    payload=json.dumps({"action": "mini_wave_hello"}),
)
# → {"action": "mini_wave_hello", "animation_triggered": true, "status": "success"}
```

Self-hosted avatar (`AvatarSession(model_path="avatar.imx")`): push a `VideoControl` to the runtime.

```python
from bithuman import VideoControl

await avatar.runtime.push(VideoControl(action="mini_wave_hello"))
```

Expected:

The cloud call returns `animation_triggered: true`, and the avatar plays the gesture.

### 3. Wire it to your events

As an allow-listed tool, so the language model can ask for a gesture only from your set. This tool is for a self-hosted avatar (`AvatarSession(model_path=…)`). For a cloud avatar, put the `perform_rpc` call from step 2 in the tool body instead.

```python
from livekit.agents import function_tool, RunContext

ALLOWED = {"mini_wave_hello", "clap_cheer", "thumbs_up_pulse"}

@function_tool()
async def play_gesture(context: RunContext, gesture: str) -> str:
    if gesture not in ALLOWED:
        return f"Unknown gesture. Options: {sorted(ALLOWED)}"
    await avatar.runtime.push(VideoControl(action=gesture, force_action=True))
    return f"Played {gesture}"
```

Other triggers work the same way: a LiveKit data message, a participant event or a timer calls the same line.

Expected:

Asked for a gesture from your set, the language model calls `play_gesture`, and the avatar plays it.

## Check it worked

The cloud call returns `animation_triggered: true`; the avatar plays the clip, then returns to idle or speech.

## Variations

| `VideoControl` field | Effect |
|---|---|
| `force_action=True` | play even if another gesture is running |
| `stop_on_user_speech=True` | stop the gesture when the user starts talking |
| `stop_on_agent_speech=True` | stop the gesture when the agent starts talking |
| `target_video="<clip>"` | switch the idle loop to another base clip |

`avatar.runtime.interrupt()` stops a gesture at once.

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| `animation_triggered: false` | the name is not one of the avatar's gestures, or it has none | list the names (step 1); generating new gestures is paused for now |
| Nothing plays on a self-hosted avatar | the `.imx` was downloaded before gestures were generated | download the model again after generation |

## Next

- [Gestures API](https://docs.bithuman.ai/api/dynamics) · [LiveKit](https://docs.bithuman.ai/platforms/livekit) · [Python](https://docs.bithuman.ai/platforms/python)

---

# Talking video

URL: https://docs.bithuman.ai/build/talking-video

> Turn one audio file into an MP4 of your avatar speaking it.

## What you'll build

An MP4 of an avatar saying the words in an audio file, with the lips in sync. Nobody watches it live, so it suits greetings, lessons and product videos.

You need:

- an audio file of speech: a recording, or a WAV or MP3 from any text-to-speech tool;
- an [API secret](https://docs.bithuman.ai/start/api-secret) on the Creator plan or higher;
- for the CLI or Python: Linux (x86_64 or arm64) or a Mac with Apple silicon, and `ffmpeg`.

*Capture: The kwame-warm-museum-guide avatar in a talking video, rendered from one audio file by bithuman render on a Linux PC. Captured on a Linux PC, Intel Core i7-13700F (Ubuntu 24.04) · CLI 2.8.2 (bithuman render) · kwame-warm-museum-guide (Essence 2) · 2026-09-27. Rendered on the CPU alone: the PC's graphics card was hidden from the process.* (https://docs.bithuman.ai/examples/talking-video-linux/clip.mp4)

## Steps

### Get the speech

Any audio file `ffmpeg` reads works on your machine; the REST API takes a file at a public URL. This sample is 15 seconds of speech:

```bash
curl -fsSLo speech.wav https://docs.bithuman.ai/samples/speech.wav
```

For a male voice, use [speech-male-neutral.wav](https://docs.bithuman.ai/samples/speech-male-neutral.wav) the same way.

Expected:

`speech.wav` in the current directory.

### Render the video

Pick where it renders. The CLI and Python render on your own machine, and on Linux they need no GPU. The REST API renders in the bitHuman cloud.

```bash tab="CLI"
curl -fsSL https://install.bithuman.ai | sh
export BITHUMAN_API_SECRET="<your API secret>"
bithuman render kwame-warm-museum-guide speech.wav -o kwame.mp4
```

```python tab="Python"
# Needs the bithuman package, ffmpeg on PATH and BITHUMAN_API_SECRET in the environment.
# The avatar file comes from `bithuman pull sofia-ramirez`, or the download URL on /platforms/python.
import bithuman
frames = bithuman.open("sofia-ramirez.imx").render("speech.wav", out_mp4="out.mp4")
print(frames, "frames written")
```

```bash tab="REST"
# A52DHS2219 is the sofia-ramirez sample.
curl -X POST https://api.bithuman.ai/v1/video/generate \
  -H "api-secret: $BITHUMAN_API_SECRET" -H "Content-Type: application/json" \
  -d '{"model": "essence-2", "agent_code": "A52DHS2219", "input": {"type": "audio", "audio_url": "https://docs.bithuman.ai/samples/speech.wav"}, "wait": true}'
```

Expected:

An MP4 as long as the speech: `kwame.mp4`, or `out.mp4` with the number of frames written, on your machine; or a `video_url` in the REST response (`"status": "completed"`). A render that takes longer than the wait returns a `job_id` to [poll](https://docs.bithuman.ai/api/video#get-talking-video-status).

### Check the file

```bash
ffprobe -v error -show_entries stream=width,height -show_entries format=duration -of compact kwame.mp4
```

Expected:

An Essence 2 video is up to 1080×1920 or 1920×1080, following the portrait; an Expression 2 video is 416×720. The duration matches the speech, and the file has one video and one audio track.

## How it works

*Diagram: The engine: speech in, frames out.* Your app pushes 16 kHz mono speech into the bitHuman engine and pulls lip-synced frames out. The same engine sits inside the Swift package, the Android SDK, the Python SDK, the CLI and the web embed.

The same engine renders a live conversation and a file: speech goes in and lip-synced frames come out, then `ffmpeg` writes them with the audio into an MP4. A file rendered on your machine keeps its audio and video there; the CLI and Python sign in online when the render starts. Through the REST API, the render runs in the bitHuman cloud, in the US.

## Make it your own

- **Your avatar:** [create one](https://docs.bithuman.ai/build/create-avatar) from a portrait, then pass its agent code to `bithuman render`, or to the REST call as `agent_code`.
- **Your own words:** record them, or make a WAV or MP3 with any text-to-speech tool, then render that file ([Talking video API](https://docs.bithuman.ai/api/video#audio-input)).
- **A batch:** loop over files with `bithuman render`; every run reuses the downloaded avatar.
- **What it costs:** on your machine, file rendering bills the length of the video it writes at the self-hosted rate; the REST API bills per minute of output, rounded up ([Pricing](https://docs.bithuman.ai/pricing#talking-video--per-minute-of-output)).

## Troubleshooting

| Symptom | Fix |
|---|---|
| `bithuman render` exits with code 77 | No API secret: export `BITHUMAN_API_SECRET`, or run `bithuman login` once. |
| The render is refused before it starts | Set `BITHUMAN_API_SECRET` for an account on the Creator plan or higher, then render again. |
| `402 INSUFFICIENT_BALANCE` from the REST API | Your balance must cover the longest render up front; the difference is refunded when it finishes ([Talking video API](https://docs.bithuman.ai/api/video)). |
| `409 MODEL_NOT_GENERATED` | The agent has no model of that kind yet: check its `supported_models`, or [add the model](https://docs.bithuman.ai/api/agents#add-a-model-to-an-existing-agent). |