# Quick start: iPhone and iPad

URL: https://docs.bithuman.ai/platforms/ios/quickstart

> Add the package, set your API secret, paste the avatar code.

Three steps in Xcode: add the package, set your API secret, and paste the avatar code. The avatar renders on the device.

## Before you start

- **Xcode 26 or newer.**
- **An iPhone or iPad with Apple silicon on iOS 16 or newer**, or the iOS Simulator on an Apple silicon Mac.
- **An [API secret](https://docs.bithuman.ai/start/api-secret).** API and SDK use needs the Creator plan or higher ([pricing](https://docs.bithuman.ai/pricing)).
- **A SwiftUI app.** To start a new one: *File → New → Project → iOS → App*, with the SwiftUI interface.

## 1. Add the package

The package link:

```text
https://gitlab.com/bithuman/sdk/homebrew-bithuman
```

In Xcode choose *File → Add Package Dependencies…*, paste the link, keep the rule *Up to Next Major Version*, and add the `Expression2` product to your app target. Xcode names the package `homebrew-bithuman`; that is expected. In a `Package.swift`:

```swift
.package(url: "https://gitlab.com/bithuman/sdk/homebrew-bithuman", from: "2.23.2")
// your target: .product(name: "Expression2", package: "homebrew-bithuman")
```

## 2. Set your API secret

In Xcode choose *Product → Scheme → Edit Scheme → Run → Environment Variables* and add `BITHUMAN_API_SECRET` with your secret. The app hands it to the engine before the avatar starts:

```swift
Expression2Credential.set(ProcessInfo.processInfo.environment["BITHUMAN_API_SECRET"])
```

The scheme's environment is for builds you run from Xcode. A shipped app fetches its key from your server: [Keep your API secret off the device](https://docs.bithuman.ai/start/secret-off-device).

## 3. Paste the avatar code

Replace the contents of your app's `@main` file with the code below, delete `ContentView.swift`, pick an iPhone or a Simulator, and run.

```swift
// The whole app in one file. Replace the contents of your app's @main file with it,
// and delete ContentView.swift. Uses the Expression2 product of the Swift package.
import SwiftUI
import AVFoundation
import Accelerate
import Expression2

@main
struct QuickStartApp: App {
    init() {
        // Step 2: your API secret, from the scheme's environment variables
        Expression2Credential.set(ProcessInfo.processInfo.environment["BITHUMAN_API_SECRET"])
    }

    var body: some Scene {
        WindowGroup {
            AvatarView(agentCode: "A23WJF0199")   // Step 3: the public sample avatar "Wise Pup"
        }
    }
}

struct AvatarView: View {
    /// A bitHuman agent code. A23WJF0199 is the public sample avatar "Wise Pup".
    let agentCode: String
    @StateObject private var avatar = AvatarModel()

    var body: some View {
        VStack(spacing: 16) {
            FrameView(sink: avatar.sink)
                .aspectRatio(416.0 / 720.0, contentMode: .fit)
                .overlay { if !avatar.hasFrame { ProgressView() } }
            Text(avatar.status).font(.footnote).foregroundStyle(.secondary)
            Button("Speak") { avatar.speak() }
                .buttonStyle(.borderedProminent)
                .disabled(!avatar.ready || avatar.speaking)
        }
        .padding()
        .task { await avatar.start(agentCode: agentCode) }
    }
}

// MARK: - The session: download, start the engine, show frames, play speech

@MainActor
final class AvatarModel: ObservableObject {
    @Published var status = "Downloading the avatar…"
    @Published var hasFrame = false
    @Published var ready = false
    @Published var speaking = false
    let sink = FrameSink()

    private let engine = AvatarEngine()
    private let audio = AVAudioEngine()
    private let player = AVAudioPlayerNode()
    private var reply: AVAudioPCMBuffer?
    private var started = false

    func start(agentCode: String) async {
        guard !started else { return }
        started = true
        do {
            // 1. The files. Each download is cached, so later launches start faster.
            let avatarFile = try await Expression2Download.avatar(agentCode: agentCode)
            status = "Downloading the engine…"
            let engineFile = try await cachedDownload(Files.sharedEngine, as: "mac-arm64-1.0.0.engine")
            let speechFile = try await cachedDownload(Files.speech(agentCode), as: "\(agentCode)-speech16k.wav")

            // 2. Audio out: the reply plays through an AVAudioPlayerNode at 16 kHz mono.
            let (speech, format) = try readSpeech(speechFile)
            reply = speech
            try AVAudioSession.sharedInstance().setCategory(.playback)
            try AVAudioSession.sharedInstance().setActive(true)
            audio.attach(player)
            audio.connect(player, to: audio.mainMixerNode, format: format)
            try audio.start()

            // 3. The engine. The first start prepares it for this device and takes a while.
            status = "Starting the engine…"
            let played = PlayedSeconds(player)
            let frames = try await engine.start(avatar: avatarFile, sharedEngine: engineFile,
                                                staging: try Files.stagingDirectory(),
                                                clock: { played() })
            status = "Ready"
            ready = true

            // 4. Frames: idle motion between replies, speech while a reply plays.
            for await frame in frames {
                if let image = makeCGImage(frame.bgr, frame.width, frame.height) {
                    sink.show(image)
                    if !hasFrame { hasFrame = true }
                }
                if frame.audioTime == 0, let reply {   // the reply's first frame: start its audio
                    player.scheduleBuffer(reply, completionHandler: nil)
                    player.play()
                }
                if frame.endsReply { speaking = false }
            }
        } catch {
            status = "Could not start: \(error)"
        }
    }

    func speak() {
        guard ready, !speaking, let reply, let data = reply.floatChannelData else { return }
        speaking = true
        player.stop()   // the clock reads nil until this reply's audio starts
        let samples = Array(UnsafeBufferPointer(start: data[0], count: Int(reply.frameLength)))
        Task { await engine.say(samples) }
    }
}

/// The engine is not Sendable, so it lives inside an actor. Only Sendable values
/// (URLs, [Float], the frame stream) cross into it.
actor AvatarEngine {
    private var engine: Expression2Engine?

    func start(avatar: URL, sharedEngine: URL, staging: URL,
               clock: @escaping @Sendable () -> Double?) throws -> AsyncStream<Expression2Frame> {
        let e = try Expression2Engine.create(avatarContainer: avatar,
                                             sharedEngineContainer: sharedEngine,
                                             stagingDir: staging)
        engine = e
        return e.frames(audioClock: clock)
    }

    func say(_ samples: [Float]) {
        engine?.feed(samples)   // 16 kHz mono
        engine?.flushTail()     // the end of this reply
    }
}

/// Seconds of the current reply the player has played (nil before it starts).
/// The engine reads it off the main actor, so the player sits in a Sendable box.
final class PlayedSeconds: @unchecked Sendable {
    private let node: AVAudioPlayerNode
    init(_ node: AVAudioPlayerNode) { self.node = node }
    func callAsFunction() -> Double? {
        guard let t = node.lastRenderTime, let p = node.playerTime(forNodeTime: t) else { return nil }
        return Double(p.sampleTime) / p.sampleRate
    }
}

// MARK: - Files

enum Files {
    /// The shared Expression 2 engine file: one for every avatar. The `mac` file is
    /// the right one for iPhone and iPad too.
    static let sharedEngine = URL(string: "https://downloads.bithuman.ai/homebrew-bithuman/expression2-engine-mac-arm64-1.0.0/mac-arm64-1.0.0.engine")!

    /// A 16 kHz mono speech clip from the avatar's own bundle.
    static func speech(_ code: String) -> URL {
        URL(string: "https://api.bithuman.ai/v1/agent/\(code)/model/download?model=expression-2&member=demo_speech_16k.wav")!
    }

    static func cacheDirectory() throws -> URL {
        let dir = try FileManager.default.url(for: .cachesDirectory, in: .userDomainMask,
                                              appropriateFor: nil, create: true)
            .appendingPathComponent("bitHuman-quickstart", isDirectory: true)
        try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
        return dir
    }

    /// Any writable directory the engine unpacks into; keep it between launches.
    static func stagingDirectory() throws -> URL {
        let dir = try FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask,
                                              appropriateFor: nil, create: true)
            .appendingPathComponent("bitHuman-quickstart/staging", isDirectory: true)
        try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
        return dir
    }
}

struct DownloadError: Error, CustomStringConvertible {
    let url: URL, status: Int
    var description: String { "download failed (HTTP \(status)): \(url.host ?? "")\(url.path)" }
}

/// Downloads `url` once into the cache directory and returns the local file.
func cachedDownload(_ url: URL, as name: String) async throws -> URL {
    let file = try Files.cacheDirectory().appendingPathComponent(name)
    if FileManager.default.fileExists(atPath: file.path) { return file }
    let (tmp, response) = try await URLSession.shared.download(from: url)
    let status = (response as? HTTPURLResponse)?.statusCode ?? 0
    guard status == 200 else { throw DownloadError(url: url, status: status) }
    try FileManager.default.moveItem(at: tmp, to: file)
    return file
}

struct SpeechFormatError: Error, CustomStringConvertible {
    let format: AVAudioFormat
    var description: String { "the speech clip must be 16 kHz mono, got \(format)" }
}

/// Reads a 16 kHz mono WAV into one buffer the player can schedule.
func readSpeech(_ url: URL) throws -> (AVAudioPCMBuffer, AVAudioFormat) {
    let file = try AVAudioFile(forReading: url)
    let format = file.processingFormat
    guard format.sampleRate == 16_000, format.channelCount == 1,
          let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(file.length))
    else { throw SpeechFormatError(format: format) }
    try file.read(into: buffer)
    return (buffer, format)
}

// MARK: - Drawing: B, G, R bytes to a CGImage, shown on a CALayer

func makeCGImage(_ bgr: [UInt8], _ w: Int, _ h: Int) -> CGImage? {
    guard w > 0, h > 0, bgr.count >= w * h * 3, let out = malloc(w * h * 4) else { return nil }
    bgr.withUnsafeBufferPointer { src in
        var s = vImage_Buffer(data: UnsafeMutableRawPointer(mutating: src.baseAddress!),
                              height: vImagePixelCount(h), width: vImagePixelCount(w), rowBytes: w * 3)
        var d = vImage_Buffer(data: out, height: vImagePixelCount(h), width: vImagePixelCount(w), rowBytes: w * 4)
        vImageConvert_RGB888toARGB8888(&s, nil, 255, &d, false, vImage_Flags(kvImageNoFlags))  // (255,B,G,R)
        var map: [UInt8] = [3, 2, 1, 0]                                                          // → (R,G,B,255)
        vImagePermuteChannels_ARGB8888(&d, &d, &map, vImage_Flags(kvImageNoFlags))
    }
    guard let provider = CGDataProvider(dataInfo: out, data: out, size: w * h * 4,
                                        releaseData: { info, _, _ in free(info) }) else { free(out); return nil }
    return CGImage(width: w, height: h, bitsPerComponent: 8, bitsPerPixel: 32, bytesPerRow: w * 4,
                   space: CGColorSpaceCreateDeviceRGB(),
                   bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipLast.rawValue),
                   provider: provider, decode: nil, shouldInterpolate: true, intent: .defaultIntent)
}

/// Frames go straight to a CALayer, so SwiftUI does not redraw 20 times a second.
@MainActor
final class FrameSink {
    fileprivate weak var layer: CALayer?
    func show(_ image: CGImage) { layer?.contents = image }
}

struct FrameView: UIViewRepresentable {
    let sink: FrameSink
    func makeUIView(context: Context) -> UIView {
        let view = UIView()
        view.layer.contentsGravity = .resizeAspect   // the whole avatar, never cropped
        view.layer.cornerRadius = 16
        view.layer.masksToBounds = true
        sink.layer = view.layer
        return view
    }
    func updateUIView(_ view: UIView, context: Context) { sink.layer = view.layer }
}
```

Expected:

The status line reads "Downloading the avatar…", then "Downloading the engine…", "Starting the engine…" and "Ready". The avatar moves while idle. Tap **Speak** and it says a short clip, its lips matched to the audio.

The same code is the [sample app](https://gitlab.com/bithuman/sdk/bithuman-examples/-/tree/main/swift/ios-quickstart), which opens in Xcode and runs once your key is in its `Config/Secrets.xcconfig`.

## What happens on first launch

The view downloads three files into the app's Caches folder and reuses them on later launches:

- the avatar, with `Expression2Download.avatar(agentCode:)`;
- the shared Expression 2 engine file, one for every avatar;
- a 16 kHz speech clip from the avatar's own files.

The first start also prepares the engine for the device, so it takes longer than later starts. The download sizes are on [Before you submit to the App Store](https://docs.bithuman.ai/platforms/ios/app-store).

## Make it your own

- **Your avatar:** change `"A23WJF0199"` to your agent's code.
- **Your speech:** pass any 16 kHz mono `[Float]` to `AvatarEngine.say(_:)`, for example your text-to-speech output resampled to 16 kHz ([Resample speech to 16 kHz](https://docs.bithuman.ai/build/companion-app#resample-speech-to-16-khz)).
- **More control:** the engine's whole Swift API is on [Build a Swift app](https://docs.bithuman.ai/platforms/swift/app) and the [Swift reference](https://docs.bithuman.ai/platforms/swift/reference).

## Troubleshooting

| What you see | What to do |
|---|---|
| "Could not start: …" with a refusal about the API secret | check `BITHUMAN_API_SECRET` in the scheme and your plan |
| "Could not start: download failed (HTTP …)" | check the connection; for your own agent, check the code and that your key's account owns it |
| "Unable to find module dependency" on a Simulator build | the package's Simulator slices are arm64 only: set `EXCLUDED_ARCHS[sdk=iphonesimulator*]` to `x86_64` in your target's build settings |