Android

After the one-time model download, the only network traffic is usage reporting. Why on the device.

Before you start

You feed 16 kHz mono speech in and pull picture frames out. Each model is one Gradle dependency, from bitHuman’s Maven repository at maven.bithuman.ai.

• JDK 17, Gradle 8.11 or newer and Android Gradle Plugin 8.7 or newer.
• A paid plan (Creator or higher): usage bills per second while the avatar runs (pricing).
• A physical arm64 phone. Emulators cannot load the engines.
• Frame rate depends on the phone. Essence 2 plays at 25 frames a second, and the phone has to render at least that fast to keep up with live speech. Recent flagship chips render well above that; older ones, such as the Snapdragon 8 Gen 2 in a Galaxy Z Flip5, can fall below real time, more so as the phone warms. The performance page has the measured rates. Test on the phones your app targets.
• Essence 1 is not available on phones: use Essence 2 or Expression 2 on devices (First generation).

Install

Add bitHuman’s Maven repository for the ai.bithuman group, restrict the build to arm64-v8a, and turn on legacy packaging so the engines’ native libraries are extracted to disk. Merge the lines into your existing app module, which keeps its own namespace, compileSdk and Java 17 targets. The API secret reaches your code through BuildConfig.

// settings.gradle.kts
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        exclusiveContent {   // ai.bithuman resolves from bitHuman's repository only
            forRepository { maven { url = uri("https://maven.bithuman.ai") } }
            filter { includeGroup("ai.bithuman") }
        }
    }
}

// app/build.gradle.kts
// bithuman.apiSecret=… in local.properties (git-ignored), or BITHUMAN_API_SECRET in the environment
val bithumanApiSecret: String = run {
    val props = java.util.Properties()   // fully qualified, so these lines can go anywhere in the file
    val f = rootProject.file("local.properties")
    if (f.isFile) f.inputStream().use { props.load(it) }
    props.getProperty("bithuman.apiSecret") ?: System.getenv("BITHUMAN_API_SECRET") ?: ""
}

android {
    buildFeatures { buildConfig = true }
    defaultConfig {
        minSdk = 26   // 29 for Essence 2
        ndk { abiFilters += "arm64-v8a" }
        buildConfigField("String", "BITHUMAN_API_SECRET", "\"$bithumanApiSecret\"")
    }
    packaging { jniLibs { useLegacyPackaging = true } }   // required
}
dependencies {
    implementation("ai.bithuman:expression2-android:0.6.1")
    // or: implementation("ai.bithuman:essence2-android:0.9.5")
}

Put the secret in local.properties as bithuman.apiSecret=… (Android Studio keeps that file out of git), or export BITHUMAN_API_SECRET before you build; the example apps read it the same way. An empty secret makes the session refuse to start.

The build prints Unable to strip the following libraries, packaging them as they are: libLiteRt.so, libQnn…. It is expected when no NDK is installed.

The dependencyResolutionManagement block works unchanged in a Groovy settings.gradle. Only ai.bithuman comes from maven.bithuman.ai; the engines’ own dependencies still come from google() and mavenCentral().

expression2-android brings the Qualcomm accelerator runtime with it (com.qualcomm.qti:qnn-litert-delegate:2.49.0 and com.qualcomm.qti:qnn-runtime:2.49.0). To keep the APK small and render on the CPU instead, exclude it:

implementation("ai.bithuman:expression2-android:0.6.1") {
    exclude(group = "com.qualcomm.qti")
}

Authenticate

Pass your API secret (create one) in code before you download or create an avatar: Expression2Credential.set(secret) for Expression 2, Essence2Credential.set(secret) for Essence 2. That one call covers the download and the session. See Your API secret.

Warning: a buildConfigField compiles the secret into the APK, where anyone with the file can read it. Use it for local builds only.

A shipped app fetches its secret from your backend (What a shipped app holds).

Run your first avatar

Expression 2, with the published wise-pup avatar (agent code A23WJF0199). Call render off the main thread.

import ai.bithuman.expression2.Expression2Avatar
import ai.bithuman.expression2.Expression2Credential
import ai.bithuman.expression2.Expression2ModelStore
import ai.bithuman.expression2.Expression2Options
import android.content.Context
import android.graphics.Bitmap

/** [pcm16k] is 16 kHz mono float32 in [-1, 1]. */
fun render(context: Context, pcm16k: FloatArray, show: (Bitmap) -> Unit) {
    Expression2Credential.set(BuildConfig.BITHUMAN_API_SECRET)        // before fetch() and create()
    val model = Expression2ModelStore(context).fetch("A23WJF0199")     // ~160 MB, first run only

    Expression2Avatar.create(context, model, Expression2Options()).use { avatar ->
        val frame = avatar.newFrameBitmap()   // 416 x 720, allocate once
        avatar.feed(pcm16k)
        avatar.flushTail()                    // end of the utterance
        while (true) {
            if (avatar.pull(frame) != null) { show(frame); continue }
            if (!avatar.hasPendingTail && avatar.queuedFrames == 0) break
            Thread.sleep(10)                  // null means "not ready yet"
        }
    }
}

Expected: show receives 20 frames for each second of audio, and the avatar’s lips follow the speech. The first create() after install prepares the accelerator once and takes noticeably longer than later launches, which reuse it. Create once, at app start, on a background thread.

Essence 2 takes 16-bit little-endian PCM bytes, as a 16 kHz mono WAV stores them, and fills an RGBA ByteBuffer sized from the identity:

import ai.bithuman.essence2.Essence2Avatar
import ai.bithuman.essence2.Essence2Credential
import ai.bithuman.essence2.Essence2ModelStore
import android.content.Context
import java.nio.ByteBuffer

fun render(context: Context, pcm16le: ByteArray, show: (ByteBuffer, Int, Int) -> Unit) {
    Essence2Credential.set(BuildConfig.BITHUMAN_API_SECRET)                   // before fetch() and create()
    val identity = Essence2ModelStore(context).fetch("A52DHS2219")            // sofia-ramirez; 226–281 MB, first run only

    Essence2Avatar.create(identity.dir).use { avatar ->
        val frame = avatar.newFrameBuffer()   // width * height * 4, RGBA
        avatar.feed(pcm16le)
        avatar.endOfAudio()
        var quietMs = 0
        while (quietMs < 5_000) {             // 5 s with no frame at all = finished
            if (avatar.pull(frame)) { show(frame, avatar.width, avatar.height); quietMs = 0; continue }
            Thread.sleep(10)                  // false means "not ready yet"; the first frame can take seconds
            quietMs += 10
        }
        avatar.checkRender()                  // throws Essence2RenderFailed if the engine stopped
        // a refused session throws Essence2MeteringRefused from pull() or idle()
    }
}

Frame size belongs to the identity (portrait 1080×1920, landscape 1920×1080 or 1280×720). Read avatar.width and avatar.height; do not hard-code them.

Play with sound (8 seconds, captions)

Performance