Quick start: iPhone and iPad

Three steps in Xcode: add the package, set your API secret, and paste the avatar code. The avatar renders on the device.

Before you start

• Xcode 26 or newer.
• An iPhone or iPad with Apple silicon on iOS 16 or newer, or the iOS Simulator on an Apple silicon Mac.
• An API secret. API and SDK use needs the Creator plan or higher (pricing).
• A SwiftUI app. To start a new one: File → New → Project → iOS → App, with the SwiftUI interface.

1. Add the package

The package link:

https://gitlab.com/bithuman/sdk/homebrew-bithuman

In Xcode choose File → Add Package Dependencies…, paste the link, keep the rule Up to Next Major Version, and add the Expression2 product to your app target. Xcode names the package homebrew-bithuman; that is expected. In a Package.swift:

.package(url: "https://gitlab.com/bithuman/sdk/homebrew-bithuman", from: "2.23.2")
// your target: .product(name: "Expression2", package: "homebrew-bithuman")

2. Set your API secret

In Xcode choose Product → Scheme → Edit Scheme → Run → Environment Variables and add BITHUMAN_API_SECRET with your secret. The app hands it to the engine before the avatar starts:

Expression2Credential.set(ProcessInfo.processInfo.environment["BITHUMAN_API_SECRET"])

The scheme’s environment is for builds you run from Xcode. A shipped app fetches its key from your server: Keep your API secret off the device.

3. Paste the avatar code

Replace the contents of your app’s @main file with the code below, delete ContentView.swift, pick an iPhone or a Simulator, and run.

// The whole app in one file. Replace the contents of your app's @main file with it,
// and delete ContentView.swift. Uses the Expression2 product of the Swift package.
import SwiftUI
import AVFoundation
import Accelerate
import Expression2

@main
struct QuickStartApp: App {
    init() {
        // Step 2: your API secret, from the scheme's environment variables
        Expression2Credential.set(ProcessInfo.processInfo.environment["BITHUMAN_API_SECRET"])
    }

    var body: some Scene {
        WindowGroup {
            AvatarView(agentCode: "A23WJF0199")   // Step 3: the public sample avatar "Wise Pup"
        }
    }
}

struct AvatarView: View {
    /// A bitHuman agent code. A23WJF0199 is the public sample avatar "Wise Pup".
    let agentCode: String
    @StateObject private var avatar = AvatarModel()

    var body: some View {
        VStack(spacing: 16) {
            FrameView(sink: avatar.sink)
                .aspectRatio(416.0 / 720.0, contentMode: .fit)
                .overlay { if !avatar.hasFrame { ProgressView() } }
            Text(avatar.status).font(.footnote).foregroundStyle(.secondary)
            Button("Speak") { avatar.speak() }
                .buttonStyle(.borderedProminent)
                .disabled(!avatar.ready || avatar.speaking)
        }
        .padding()
        .task { await avatar.start(agentCode: agentCode) }
    }
}

// MARK: - The session: download, start the engine, show frames, play speech

@MainActor
final class AvatarModel: ObservableObject {
    @Published var status = "Downloading the avatar…"
    @Published var hasFrame = false
    @Published var ready = false
    @Published var speaking = false
    let sink = FrameSink()

    private let engine = AvatarEngine()
    private let audio = AVAudioEngine()
    private let player = AVAudioPlayerNode()
    private var reply: AVAudioPCMBuffer?
    private var started = false

    func start(agentCode: String) async {
        guard !started else { return }
        started = true
        do {
            // 1. The files. Each download is cached, so later launches start faster.
            let avatarFile = try await Expression2Download.avatar(agentCode: agentCode)
            status = "Downloading the engine…"
            let engineFile = try await cachedDownload(Files.sharedEngine, as: "mac-arm64-1.0.0.engine")
            let speechFile = try await cachedDownload(Files.speech(agentCode), as: "\(agentCode)-speech16k.wav")

            // 2. Audio out: the reply plays through an AVAudioPlayerNode at 16 kHz mono.
            let (speech, format) = try readSpeech(speechFile)
            reply = speech
            try AVAudioSession.sharedInstance().setCategory(.playback)
            try AVAudioSession.sharedInstance().setActive(true)
            audio.attach(player)
            audio.connect(player, to: audio.mainMixerNode, format: format)
            try audio.start()

            // 3. The engine. The first start prepares it for this device and takes a while.
            status = "Starting the engine…"
            let played = PlayedSeconds(player)
            let frames = try await engine.start(avatar: avatarFile, sharedEngine: engineFile,
                                                staging: try Files.stagingDirectory(),
                                                clock: { played() })
            status = "Ready"
            ready = true

            // 4. Frames: idle motion between replies, speech while a reply plays.
            for await frame in frames {
                if let image = makeCGImage(frame.bgr, frame.width, frame.height) {
                    sink.show(image)
                    if !hasFrame { hasFrame = true }
                }
                if frame.audioTime == 0, let reply {   // the reply's first frame: start its audio
                    player.scheduleBuffer(reply, completionHandler: nil)
                    player.play()
                }
                if frame.endsReply { speaking = false }
            }
        } catch {
            status = "Could not start: \(error)"
        }
    }

    func speak() {
        guard ready, !speaking, let reply, let data = reply.floatChannelData else { return }
        speaking = true
        player.stop()   // the clock reads nil until this reply's audio starts
        let samples = Array(UnsafeBufferPointer(start: data[0], count: Int(reply.frameLength)))
        Task { await engine.say(samples) }
    }
}

/// The engine is not Sendable, so it lives inside an actor. Only Sendable values
/// (URLs, [Float], the frame stream) cross into it.
actor AvatarEngine {
    private var engine: Expression2Engine?

    func start(avatar: URL, sharedEngine: URL, staging: URL,
               clock: @escaping @Sendable () -> Double?) throws -> AsyncStream<Expression2Frame> {
        let e = try Expression2Engine.create(avatarContainer: avatar,
                                             sharedEngineContainer: sharedEngine,
                                             stagingDir: staging)
        engine = e
        return e.frames(audioClock: clock)
    }

    func say(_ samples: [Float]) {
        engine?.feed(samples)   // 16 kHz mono
        engine?.flushTail()     // the end of this reply
    }
}

/// Seconds of the current reply the player has played (nil before it starts).
/// The engine reads it off the main actor, so the player sits in a Sendable box.
final class PlayedSeconds: @unchecked Sendable {
    private let node: AVAudioPlayerNode
    init(_ node: AVAudioPlayerNode) { self.node = node }
    func callAsFunction() -> Double? {
        guard let t = node.lastRenderTime, let p = node.playerTime(forNodeTime: t) else { return nil }
        return Double(p.sampleTime) / p.sampleRate
    }
}

// MARK: - Files

enum Files {
    /// The shared Expression 2 engine file: one for every avatar. The `mac` file is
    /// the right one for iPhone and iPad too.
    static let sharedEngine = URL(string: "https://downloads.bithuman.ai/homebrew-bithuman/expression2-engine-mac-arm64-1.0.0/mac-arm64-1.0.0.engine")!

    /// A 16 kHz mono speech clip from the avatar's own bundle.
    static func speech(_ code: String) -> URL {
        URL(string: "https://api.bithuman.ai/v1/agent/\(code)/model/download?model=expression-2&member=demo_speech_16k.wav")!
    }

    static func cacheDirectory() throws -> URL {
        let dir = try FileManager.default.url(for: .cachesDirectory, in: .userDomainMask,
                                              appropriateFor: nil, create: true)
            .appendingPathComponent("bitHuman-quickstart", isDirectory: true)
        try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
        return dir
    }

    /// Any writable directory the engine unpacks into; keep it between launches.
    static func stagingDirectory() throws -> URL {
        let dir = try FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask,
                                              appropriateFor: nil, create: true)
            .appendingPathComponent("bitHuman-quickstart/staging", isDirectory: true)
        try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
        return dir
    }
}

struct DownloadError: Error, CustomStringConvertible {
    let url: URL, status: Int
    var description: String { "download failed (HTTP \(status)): \(url.host ?? "")\(url.path)" }
}

/// Downloads `url` once into the cache directory and returns the local file.
func cachedDownload(_ url: URL, as name: String) async throws -> URL {
    let file = try Files.cacheDirectory().appendingPathComponent(name)
    if FileManager.default.fileExists(atPath: file.path) { return file }
    let (tmp, response) = try await URLSession.shared.download(from: url)
    let status = (response as? HTTPURLResponse)?.statusCode ?? 0
    guard status == 200 else { throw DownloadError(url: url, status: status) }
    try FileManager.default.moveItem(at: tmp, to: file)
    return file
}

struct SpeechFormatError: Error, CustomStringConvertible {
    let format: AVAudioFormat
    var description: String { "the speech clip must be 16 kHz mono, got \(format)" }
}

/// Reads a 16 kHz mono WAV into one buffer the player can schedule.
func readSpeech(_ url: URL) throws -> (AVAudioPCMBuffer, AVAudioFormat) {
    let file = try AVAudioFile(forReading: url)
    let format = file.processingFormat
    guard format.sampleRate == 16_000, format.channelCount == 1,
          let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(file.length))
    else { throw SpeechFormatError(format: format) }
    try file.read(into: buffer)
    return (buffer, format)
}

// MARK: - Drawing: B, G, R bytes to a CGImage, shown on a CALayer

func makeCGImage(_ bgr: [UInt8], _ w: Int, _ h: Int) -> CGImage? {
    guard w > 0, h > 0, bgr.count >= w * h * 3, let out = malloc(w * h * 4) else { return nil }
    bgr.withUnsafeBufferPointer { src in
        var s = vImage_Buffer(data: UnsafeMutableRawPointer(mutating: src.baseAddress!),
                              height: vImagePixelCount(h), width: vImagePixelCount(w), rowBytes: w * 3)
        var d = vImage_Buffer(data: out, height: vImagePixelCount(h), width: vImagePixelCount(w), rowBytes: w * 4)
        vImageConvert_RGB888toARGB8888(&s, nil, 255, &d, false, vImage_Flags(kvImageNoFlags))  // (255,B,G,R)
        var map: [UInt8] = [3, 2, 1, 0]                                                          // → (R,G,B,255)
        vImagePermuteChannels_ARGB8888(&d, &d, &map, vImage_Flags(kvImageNoFlags))
    }
    guard let provider = CGDataProvider(dataInfo: out, data: out, size: w * h * 4,
                                        releaseData: { info, _, _ in free(info) }) else { free(out); return nil }
    return CGImage(width: w, height: h, bitsPerComponent: 8, bitsPerPixel: 32, bytesPerRow: w * 4,
                   space: CGColorSpaceCreateDeviceRGB(),
                   bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipLast.rawValue),
                   provider: provider, decode: nil, shouldInterpolate: true, intent: .defaultIntent)
}

/// Frames go straight to a CALayer, so SwiftUI does not redraw 20 times a second.
@MainActor
final class FrameSink {
    fileprivate weak var layer: CALayer?
    func show(_ image: CGImage) { layer?.contents = image }
}

struct FrameView: UIViewRepresentable {
    let sink: FrameSink
    func makeUIView(context: Context) -> UIView {
        let view = UIView()
        view.layer.contentsGravity = .resizeAspect   // the whole avatar, never cropped
        view.layer.cornerRadius = 16
        view.layer.masksToBounds = true
        sink.layer = view.layer
        return view
    }
    func updateUIView(_ view: UIView, context: Context) { sink.layer = view.layer }
}

Expected

The status line reads "Downloading the avatar…", then "Downloading the engine…", "Starting the engine…" and "Ready". The avatar moves while idle. Tap Speak and it says a short clip, its lips matched to the audio.

The same code is the sample app, which opens in Xcode and runs once your key is in its Config/Secrets.xcconfig.

What happens on first launch

The view downloads three files into the app’s Caches folder and reuses them on later launches:

• the avatar, with Expression2Download.avatar(agentCode:);
• the shared Expression 2 engine file, one for every avatar;
• a 16 kHz speech clip from the avatar’s own files.

The first start also prepares the engine for the device, so it takes longer than later starts. The download sizes are on Before you submit to the App Store.

Make it your own

• Your avatar: change "A23WJF0199" to your agent’s code.
• Your speech: pass any 16 kHz mono [Float] to AvatarEngine.say(_:), for example your text-to-speech output resampled to 16 kHz (Resample speech to 16 kHz).
• More control: the engine’s whole Swift API is on Build a Swift app and the Swift reference.

Troubleshooting