# iOS Essence 2

URL: https://docs.bithuman.ai/examples/ios-essence-2

> A complete SwiftUI app that renders a photoreal Essence 2 avatar at full resolution on an iPhone or iPad, on the device: four files, one setup script.

A SwiftUI app that opens an Essence 2 avatar, shows its idle motion, and speaks a line with the lips in sync, all rendered on the phone at the avatar's own resolution (up to 1080p). **Speak** plays the line again.

`Sources/App.swift` uses `Essence2Kit` ([Apple](https://docs.bithuman.ai/platforms/ios)): `Essence2Engine.create(identity:resourcesDirectory:)` opens the bundled avatar, and `frames(following:)` paces the picture to the audio player.

## Requirements

| You need | Notes |
|---|---|
| A Mac with Xcode 26 or newer, and an Apple Developer team | a device build is a signed build |
| A physical iPhone, or an M-series iPad, on iOS 26 | the Simulator cannot run the engine; no Apple entitlement is needed |
| Swift package **2.18.0** or newer, `Essence2Kit` product | the project depends on it; a fresh clone resolves the newest release |
| An [API secret](https://docs.bithuman.ai/start/api-secret) | the engine bills session time, talking or idle |
| About 430 MB free on the phone and 380 MB on the Mac | the avatar and the engine resources ride in the app bundle |

## Get the code

```bash
git clone https://github.com/bithuman-product/bithuman-examples.git
cd bithuman-examples/swift/ios-essence2
./setup.sh
```

`setup.sh` downloads the default avatar into `Sources/Model/` and the engine resources into `Sources/EngineResources/` (both git-ignored), checks both, and makes a speech clip. `./setup.sh <AGENT_CODE>` fetches another avatar.

## Set your API secret

In Xcode: **Product → Scheme → Edit Scheme → Run → Environment Variables**, add `BITHUMAN_API_SECRET`. The app reads it at launch and passes it to `Essence2Credential.set(_:)`.

## Run it

```bash
open IOSEssence2.xcodeproj
```

Pick your team under **Signing & Capabilities**, choose your iPhone as the run destination, and press **Run**.

## Expected output

The avatar appears in its idle motion, then says the bundled line once. The console prints:

```text
[ios-essence2] engine ready: 1080x1920, ready in <n> s
```

The first launch unpacks the avatar and prepares the engine, so it is slower than later launches.

## How it works

1. **Credential first:** `Essence2Credential.set(_:)` is called before the engine is created.
2. **Open the avatar:** `Essence2Engine.create(identity:resourcesDirectory:)` opens the bundled `.imx` with the engine's runtime files from `Sources/EngineResources`.
3. **One draw loop:** `frames(following: player)` hands out 25 frames a second for the life of the app: idle motion between replies, and a reply's frames as the player plays their audio.
4. **Speak:** `feed(_:)` the reply's 16 kHz samples, then `flushTail()`; the draw loop starts the reply's audio with its first speech frame, so the lips stay on the voice.
5. **Stream, don't collect:** one 1080×1920 frame is 6.2 MB, so the app draws each frame as it arrives.

`Sources/App.swift` is the whole app. The full API is on [Apple](https://docs.bithuman.ai/platforms/ios#integrate-into-your-app) and [Apple API reference](https://docs.bithuman.ai/platforms/swift/reference).

## The code that matters

`Sources/App.swift` sets the secret, opens the avatar, and draws the frames `frames(following:)` hands out, starting the reply's audio with its first speech frame:

```swift
// excerpt: swift/ios-essence2/Sources/App.swift
Essence2Credential.set(ProcessInfo.processInfo.environment["BITHUMAN_API_SECRET"])
// …
let e = try await Essence2Engine.create(identity: imx, resourcesDirectory: Payload.resources)
```

```swift
// excerpt: swift/ios-essence2/Sources/App.swift
/// 5a. The draw loop, for the life of the app. `frames(following:)` paces itself
/// (25 per second): idle motion between replies, and a reply's frames as the player
/// plays their audio. Start the reply's audio with its first speech frame.
private func startLoop(_ e: Essence2Engine) {
    loop?.cancel()
    let frames = e.frames(following: player)
    loop = Task { [weak self] in
        for await f in frames {
            guard let self else { return }
            if f.audioTime == 0, let r = self.reply {
                self.player.stop()
                self.player.scheduleBuffer(r)
                self.player.play()
            }
            if let cg = makeCGImage(bgr: f.bgr, f.width, f.height) {
                self.sink.show(cg)
                self.hasFrame = true
            }
    // …
    e.feed(samples)
    e.flushTail()
```

The complete file is [on GitHub](https://github.com/bithuman-product/bithuman-examples/blob/main/swift/ios-essence2/Sources/App.swift).

## Make it your own

- **Another sample avatar:** `./setup.sh <AGENT_CODE>` with any of these; the app reads the frame size back from the engine.

  | Avatar | Agent code | Frame |
  |---|---|---|
  | warm-clear-professional-presenter (default) | `A21SKT4314` | 1080×1920 |
  | sofia-ramirez | `A52DHS2219` | 1080×1920 |
  | kwame-warm-museum-guide | `A62SJB3901` | 1080×1920 |
  | afro-latina-astrophysics-mentor | `A23KSG5258` | 1920×1080 |
  | calm-product-specialist-advisor | `A24EKJ8433` | 1280×720 |

- **Your own avatar:** create one with the [Agents API](https://docs.bithuman.ai/api/agents) (`"model": "essence-2"`), then run `BITHUMAN_API_SECRET=… ./setup.sh <AGENT_CODE>`.
- **Ship it:** fetch the secret from your backend or the Keychain at launch and pass it to `Essence2Credential.set(_:)`; never put it in the app bundle.

## Troubleshooting

| Symptom | Fix |
|---|---|
| `create` throws `Essence2KitError.meteringRefused(reason:)` | no API secret, or the service rejected it (`reason` says which): set `BITHUMAN_API_SECRET` in the Run scheme |
| `create` throws `.identityUnreadable` | re-run `./setup.sh`; it checks the download |
| *"the shared audio front end is missing"*, or the engine never becomes ready | add `Sources/EngineResources` as a **group**, not a folder reference |
| The avatar moves but never speaks | resolve Swift package **2.18.0** or newer (*File → Packages → Update to Latest Package Versions*) |
| The link fails naming a newer minimum OS | set Minimum Deployments to **iOS 26.0** |
| `no such module 'Essence2Kit'` | attach the `Essence2Kit` product to the app target |
| `ld` warns *"built for newer 'iOS' version (26.0)"* once per object | expected; the build is good |
| It builds for the Simulator and crashes there | run on a physical device |

More on [Apple: Troubleshooting](https://docs.bithuman.ai/platforms/ios#troubleshooting).

## Next

- [iOS example: Expression 2](https://docs.bithuman.ai/examples/ios-expression-2) · [Android example: Essence 2](https://docs.bithuman.ai/examples/android-essence-2) · [iOS & iPadOS](https://docs.bithuman.ai/platforms/ios) · [source on GitHub](https://github.com/bithuman-product/bithuman-examples/tree/main/swift/ios-essence2)