A private chief of staff for your tasks, projects and notes. Nothing leaves the phone.

Evaluation

Parsed from the EVALJSON lines of the iOS test job in CI run 37745540170, with the "before" column from run 37721075662 (the v0.3 branch before any router change). All corpora are synthetic and written for the eval. Nothing below is typed by hand. These numbers describe the deterministic brain and the tool trimmer only; no Foundation Models or MLX accuracy is claimed.

held-out never used to tune rules, misses never itemized · dev rules may be tuned on it · gated must stay at 100% or CI fails.

Deterministic router

Every utterance runs through the deterministic brain's real entry point against a freshly seeded in-memory store. Tool accuracy: the right tool (for compound requests, the exact sequence). End-to-end: right tool, every call succeeded, and the resulting state is right. The v0.3 held-out row is the honest generalization number.

Tool trimming for the Foundation Models brain (recall@k)

Share of cases whose expected tool (every step, for compound requests) is inside the set ToolSelector would offer the on-device model, out of 24 tools. This measures the trim, not the model. The selector reuses the deterministic grammar, so canonical rows are not independent of it.

Ask your notes (retrieval and citations)

Synthetic fixture of 22 notes and 61 questions, each with one relevant note and an answer span. recall@k: the relevant note is among the notes of the top k passages. Citation faithfulness: every [n] in the extractive answer follows a sentence found in a retrieved passage of note n. Span attribution: when the answer contains the span, its [n] points at a passage that contains the span.

Retrieval eval, hybrid: skipped (asset unavailable).

Routing plus tool execution latency on the CI simulator (in-memory store): p50 6.71 ms, p95 15.68 ms. Latency varies between runner instances; accuracy is deterministic.

In 60 seconds

• Product bet: the people who most need an AI assistant for their work (consultants under NDA, founders, leads juggling workstreams) are often the ones who cannot put that work into a cloud chatbot. iOS 26's on-device model makes a useful assistant possible without that trade.
• What it does: "create a project Launch, add 3 tasks for Friday and link it to brand voice" runs as a visible five-step plan; "what do my notes say about the venue?" answers with [1] citations to the source notes; "remind me to take out the bins every other Thursday" creates a recurring task; "undo that" reverts the whole last request. Each step is a tool call against real structured data, not a chat reply.
• How it stays reliable: a three-tier brain. Apple Foundation Models with native tool calling and guided-generation plans, an optional MLX build running Qwen3-4B, and a deterministic planner and router that always work, so the product never shows "AI unavailable". Every mutation is journaled with its inverse, so every action is undoable.
• How it is measured: three synthetic evals run on every CI build; the headline figures are in Numbers and the misses in Where it fails. No Foundation Models or MLX accuracy is claimed yet.
• Product docs: docs/PRODUCT.md has the problem, users, jobs to be done, metrics, local-versus-cloud trade-offs, decisions and roadmap.

Demo: a short device recording (plan card, Undo, cited answer) will go here as docs/img/demo.gif. Screenshots are not yet in the repository either; see Screenshots.

Architecture

flowchart LR
    U["User utterance<br/>or Siri / Shortcuts"] --> O["AgentOrchestrator<br/>one journal group per turn"]
    O --> S{"Which brain?"}
    S -->|"Apple Intelligence"| FM["Foundation Models<br/>native Tool calling +<br/>@Generable plans"]
    S -->|"MLX build flag"| MLX["MLX Qwen3-4B<br/>JSON tool calls"]
    S -->|"otherwise"| DP["Deterministic router<br/>lexicon + NLTagger lemmas<br/>+ slot frames"]
    FM -.->|"generation fails"| DP
    FM --> PL["Planner<br/>CompoundPlanner / PlanExecutor<br/>$N references, postconditions"]
    MLX --> TP["ToolCallParser<br/>one or many calls"] --> PL
    DP --> PL
    PL --> TB["ToolBox<br/>24 tools"]
    FM --> TB
    TB --> J["Action journal<br/>inverse per mutation"]
    J --> UN["UndoEngine<br/>atomic per plan"]
    TB --> RAG["NotesRAG<br/>chunks, BM25 + NLEmbedding,<br/>cited answers"]
    TB --> INT["Integrations (opt-in)<br/>EventKit Calendar read,<br/>Reminders export"]
    TB --> DS["DataServices<br/>single write path"]
    UN --> DS
    DS --> DB[("SwiftData<br/>on device")]
    DS --> W["App Group snapshot"] --> WG["Today widget"]
    DS --> V["Today / Projects /<br/>Knowledge spaces"]

Every brain emits the same event stream (tool started, tool finished, text snapshot, done); plans use it too, as a plan card that updates in place plus numbered step cards, so the UI never knows which brain answered. If a model reports itself available but fails to generate, the orchestrator swaps to the deterministic router and answers the same prompt. Agent, views, Siri intents and the widget feed all go through one service layer. Details: docs/ARCHITECTURE.md.

Why on-device

The design keeps the model's job narrow and puts the data work in deterministic, tested code. That is what makes a ~3B on-device model sufficient for this product.

Build and run

Requirements: a Mac with Xcode 26 (iOS 26 SDK) and XcodeGen. The Xcode project is generated from project.yml and is not committed.

brew install xcodegen
git clone https://github.com/seanmcrae/pocketbrains-app.git
cd pocketbrains-app
xcodegen generate
open PocketBrains.xcodeproj

Run on an iOS 26 simulator or device. On hardware with Apple Intelligence the Foundation Models brain engages automatically; elsewhere, including the simulator, the deterministic router answers. Settings, Intelligence lets you force the deterministic brain. Calendar and Reminders stay off until you enable them in Settings, Integrations. The Today widget builds unsigned in CI; to run it on a device, sign both the app and PocketBrainsWidget with your team and keep the App Group group.com.pocketbrains.shared on both (declared in project.yml). To build the optional MLX backend, follow the comments in project.yml (adds the mlx-swift-examples package and the POCKETBRAINS_MLX flag; the model is a ~2.3 GB download on first use).

Tests, exactly as CI runs them:

xcodebuild test -project PocketBrains.xcodeproj -scheme PocketBrains \
  -destination 'platform=iOS Simulator,name=iPhone 17 Pro' \
  -parallel-testing-enabled NO CODE_SIGNING_ALLOWED=NO

Limitations

• Not released. No App Store build, no users, no telemetry. The app has been run on the iOS 26 simulator; the Foundation Models path, including guided-generation planning, has not been verified on device in this repository (see docs/VERIFICATION.md).
• The deterministic floor is rigid off-grammar: 39.6% end to end on the frozen v0.3 held-out paraphrases (see Where it fails).
• Tool budget on the ~3B model. By default Foundation Models sees 22 tools (24 with both integrations on). Per-request trimming exists but ships off, and a trimmed session does not carry multi-turn context.
• Integrations are untested on device. EventKit is covered by mapping tests behind protocols; the permission flows, Siri phrases and the widget need a device or simulator run by hand. The widget needs the App Group capability when signed.
• MLX is opt-in and not compiled in CI. Its tool-call parser, including multi-call plans, is tested.
• English only, fixed type sizes (Dynamic Type mapping is pending, #8), and the VoiceOver pass is incomplete.
• Semantic search and passage embeddings are brute-force cosine over a personal-scale corpus and are disabled under the test host, so tests cover the keyword path.
• Open engineering items are tracked in docs/QA_REPORT.md.

More

• Architecture: the agent loop, planner, journal and undo, cited retrieval, router tiers, tool trimming.
• Product brief: problem, users, scope, metrics, trade-offs, roadmap.
• Verification checklist: what CI covers, what needs a device.