Use bitHuman agents in real-time applications with self-hosted deployment, featuring direct model file access and VideoControl-based gesture triggering.
Quick Start
Install dependencies
cd examples/self-hosted
pip install -r requirements.txt
Get credentials
- API Secret: www.bithuman.ai
- Model File: Download your
.imx model from the platform
Configure environment
# bitHuman Configuration
BITHUMAN_API_SECRET=your_api_secret_here
BITHUMAN_MODEL_PATH=/path/to/your/avatar_model.imx
BITHUMAN_AGENT_ID=A31KJC8622 # Optional: for dynamics gestures
# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key_here
# LiveKit Configuration
LIVEKIT_URL=wss://your-livekit-server.com
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret
Basic Self-Hosted Agent
Standard avatar interactions without dynamics:
from livekit.plugins import bithuman
bithuman_avatar = bithuman.AvatarSession(
api_secret=os.getenv("BITHUMAN_API_SECRET"),
model_path=os.getenv("BITHUMAN_MODEL_PATH"),
)
- Direct model file access (
.imx format)
- Native AsyncBithuman runtime integration
- High-performance streaming with VideoGenerator pattern
- Real-time audio/video processing
Self-Hosted Agent with Dynamics
For reactive avatar gestures triggered by user speech keywords.
Step 1: Get Available Gestures
Retrieve available gesture actions for your agent via the Dynamics API.
import requests
agent_id = "A31KJC8622"
url = f"https://api.bithuman.ai/v1/dynamics/{agent_id}"
headers = {"api-secret": "YOUR_API_SECRET"}
response = requests.get(url, headers=headers)
dynamics_data = response.json()
if dynamics_data.get("success"):
gestures_dict = dynamics_data["data"].get("gestures", {})
available_gestures = list(gestures_dict.keys())
print(f"Available gestures: {available_gestures}")
# Example: ["mini_wave_hello", "talk_head_nod_subtle", "laugh_react"]
Gesture actions are user-defined and vary based on your agent’s dynamics generation. Always check the API response to see what’s available.
Step 2: Set Up Keyword-to-Action Mapping
from livekit.agents import AgentSession, JobContext, UserInputTranscribedEvent
from livekit.plugins import bithuman
from bithuman.api import VideoControl
import asyncio
import os
KEYWORD_ACTION_MAP = {
"laugh": "laugh_react",
"laughing": "laugh_react",
"haha": "laugh_react",
"funny": "laugh_react",
"hello": "mini_wave_hello",
"hi": "mini_wave_hello",
}
async def entrypoint(ctx: JobContext):
await ctx.connect()
await ctx.wait_for_participant()
bithuman_avatar = bithuman.AvatarSession(
api_secret=os.getenv("BITHUMAN_API_SECRET"),
model_path=os.getenv("BITHUMAN_MODEL_PATH"),
)
session = AgentSession(...)
await bithuman_avatar.start(session, room=ctx.room)
@session.on("user_input_transcribed")
def on_user_input_transcribed(event: UserInputTranscribedEvent):
if not event.is_final:
return
transcript = event.transcript.lower()
for keyword, action in KEYWORD_ACTION_MAP.items():
if keyword in transcript:
asyncio.create_task(
bithuman_avatar.runtime.push(VideoControl(action=action))
)
break
- Get available gestures from the Dynamics API
- Map keywords to gesture action names
- Listen for user speech via
user_input_transcribed events
- Trigger gestures via
VideoControl(action=action)
Always verify that a gesture action exists in the API response before using it. Non-existent gestures will be silently ignored.
Configuration
| Parameter | Type | Required | Description |
|---|
model_path | string | Yes | Path to the .imx model file |
api_secret | string | Yes | Authentication secret |
api_token | string | No | Optional API token |
agent_id | string | No | Agent ID for fetching dynamics gestures |
Self-Hosted Advantages
- Full Control — Complete control over model files and deployment
- Privacy — Models stay on your infrastructure
- Customization — Modify and extend agent behavior
- Performance — Optimize for your specific hardware
- Offline Capable — Works without internet after initial setup
Common Issues
| Problem | Solution |
|---|
| Model loading errors | Verify model file path and permissions |
| Memory issues | Minimum 4GB RAM, recommended 8GB+ |
| Gesture not triggering | Verify gesture name exists in dynamics API response |
| Connection issues | Verify LiveKit server URL and credentials |
Model Requirements
| Specification | Value |
|---|
| Format | .imx files |
| Minimum RAM | 4GB |
| Recommended RAM | 8GB+ |
| Initialization time | ~20 seconds |
| Frame rate | 25 FPS |
Next Steps
