Android Expression 2

Play with sound (8 seconds, captions)

The app downloads the wise-pup sample avatar once, renders every frame of a speech clip on the phone, then plays the audio with the frames in sync. Tap the screen to replay.

Requirements

Get the code

git clone https://github.com/bithuman-product/bithuman-examples.git
cd bithuman-examples/android/expression2-hello

Set your API secret

Write local.properties next to settings.gradle.kts. The file is git-ignored.

sdk.dir=/path/to/your/Android/sdk
bithuman.apiSecret=<your API secret>

BITHUMAN_API_SECRET in the environment works instead of the second line.

Run it

./gradlew :app:assembleDebug && adb install -r app/build/outputs/apk/debug/app-debug.apk
adb shell am start -n com.example.x2hello/.MainActivity
adb push ../../python/quickstart/speech.wav /storage/emulated/0/Android/data/com.example.x2hello/files/speech.wav && adb shell am start -S -n com.example.x2hello/.MainActivity

The first launch creates the app’s files folder and says that speech.wav is missing; the push fills it and the restart renders. Any 16 kHz mono 16-bit WAV works.

Expected output

adb logcat -s X2HELLO shows the download, the engine starting on the phone’s NPU, then:

rendered 277 frames in … s — playing…
277 frames, 13.87 s — tap to replay

277 frames for 13.87 seconds of audio is 20 frames a second. On the first launch after install, most of the wait is the one-time download (about 160 MB) and create() preparing the model for the accelerator; later launches reuse it, and the frames themselves render in a few seconds.

How it works

MainActivity.kt does four things, all from ai.bithuman:expression2-android:

1. sets your API secret once with Expression2Credential.set(BuildConfig.BITHUMAN_API_SECRET), which covers the download and the session;
2. downloads the avatar with Expression2ModelStore(this).fetch(agentCode) (A23WJF0199 by default);
3. opens it with Expression2Avatar.create(this, model, options) on a background thread;
4. calls feed(pcm) and flushTail(), then pull(frame) until every frame is out: one frame per 50 ms of audio.

The calls, the live-streaming loop and the accelerator are on Android.

The code that matters

The render path of MainActivity.kt, as it is in the repository: set the secret, fetch the avatar, feed the speech, pull frames until the tail is out.

// excerpt: android/expression2-hello/app/src/main/java/com/example/x2hello/MainActivity.kt
Expression2Credential.set(secret)
// …
// Blocks on the network the first time; that is why this is a worker thread.
val model = Expression2ModelStore(this).fetch(agentCode)
// …
val avatar = Expression2Avatar.create(this, model, options)
// …
avatar.use {
    val frame = avatar.newFrameBitmap()   // ARGB_8888, 416 x 720 — allocate once
    avatar.feed(pcm)                      // renders each complete 1.6 s chunk
    avatar.flushTail()                    // the padded tail is the last sentence
    while (true) {
        if (avatar.pull(frame) != null) {
        // …
        }
        if (!avatar.hasPendingTail && avatar.queuedFrames == 0) break
        // A null is "not ready yet". pull() never renders, so asking again at
        // once just burns a core the engine needs — wait, then ask again.
        Thread.sleep(10)

The complete file is on GitHub.

Make it your own

• Your own avatar: create one with the Agents API ("model": "expression-2"), then pass its agent code to fetch; the secret you set with Expression2Credential.set downloads it.
• Live speech: feed microphone audio (16 kHz mono float from AudioRecord with ENCODING_PCM_FLOAT) as it arrives and pull frames at 20 fps; call flushTail() at the end of each reply.
• Ship it: the secret in BuildConfig can be read out of the APK. A real app fetches it from your backend at startup and passes it to Expression2Credential.set.

Troubleshooting

More on Android: Troubleshooting.

Next

• Android example: Essence 2 · Android SDK · Android API reference · source on GitHub