Why AI Should Never Leave Your Phone

I built a journaling app around one constraint I refused to compromise on: the AI reads your most private writing, so it must never leave the phone. No server sees it. No API call transmits it. Not mine, not anyone's.

That sentence is easy to say and genuinely hard to build. Most "AI journaling" apps today are a thin notes UI in front of a cloud model — you type, your entry gets shipped to an API somewhere, a response comes back. That's a reasonable architecture for almost any other product. It's a bad one for a diary.

The problem with cloud AI and private writing

A journal is the one place people write things they wouldn't say out loud. Fears they haven't named yet. Things they're avoiding. Who they're becoming and aren't sure they chose it. That's exactly the content you don't want sitting in a request log on a server you don't control, subject to a privacy policy that can change, a breach that can happen, or a subpoena that can arrive.

"We encrypt it in transit and at rest" doesn't solve this. The AI still has to read the plaintext to generate a useful response — encryption protects the pipe, not the destination. If a server is doing the thinking, a server is reading your journal.

So the only way to make the privacy guarantee real, not just a policy promise, was to make the server disappear entirely.

Getting a useful model to run on an iPhone

That meant shipping the model itself inside the app: Gemma 3, 1B parameters, running on-device via CoreML / llama.cpp, doing inference on the phone's Neural Engine. No network call, no account, works in airplane mode.

A 1B model is small. It's not going to out-argue GPT-4 on an open-ended prompt. Getting it to produce something a person actually wants to read — a daily reflection prompt that feels specific instead of generic, a mood read that isn't a coin flip, a monthly report that finds a real pattern instead of restating the month — took most of the actual engineering effort. Three things mattered more than model size:

• Context selection, not context dumping. Feeding a small model your entire journal history produces mush. MirrorNotes caps what it sends per generation (max 10,000 characters, truncating oldest entries first) and is deliberate about which entries are relevant to which insight — recent entries for a daily nudge, the full week for a digest, mood-tagged entries for a mood alert.
• Narrow prompts, not open-ended ones. A small model asked to "reflect on this journal" wanders. A small model asked one specific, scoped question — every system prompt in the app is narrow on purpose — stays coherent.
• Caching as a design constraint, not an optimization. Nothing regenerates within 24 hours. Partly for battery and thermal reasons on-device inference isn't free. Partly because re-running the model on a whim produces inconsistent output that undermines trust in the insight itself.

What this costs

I'd rather say this plainly than let a marketing page imply otherwise: on-device AI is a real trade-off, not a free upgrade.

• A 1B local model is not as capable as a frontier cloud model. It will occasionally miss a pattern a bigger model would catch.
• Inference uses battery and can be slower on older devices — there's no server to absorb that cost for the user.
• It only works on hardware capable of running the model well, which is part of why this is iOS-first rather than cross-platform from day one.

I think that trade is worth it for a diary specifically. I wouldn't necessarily make the same call for a product where the input isn't inherently sensitive.

The rest of the privacy design follows from the same rule

Once "the AI must run locally" is fixed, a lot of other decisions fall out of it rather than needing separate justification:

• No account required. There's no server-side user record to breach because there's no server-side anything.
• Sync is just iCloud — CloudKit, under the user's own Apple ID, encrypted, invisible to us. Not a custom sync service we'd have to secure and could get wrong.
• The free tier keeps everything, forever. Unlimited entries, full history, full-text search, iCloud backup — never gated, because punishing free users for writing more would be a strange thing to do to someone journaling. Only the AI layer is paid.

If you want to build something similar

The app itself is open source (AGPL-3.0): github.com/lokii49/mirror. And separately, I pulled the non-personal parts of the journaling logic — daily prompts, mood-vocabulary detection, a weekly review framework, CBT-style reframing questions — into a small MCP server anyone can run locally with any MCP-compatible AI assistant, no MirrorNotes account needed: mirrornotes.org/mcp.html.

If you're building anything that reads sensitive user input and thinking about whether it has to be a cloud model — it usually doesn't have to be. It's just more work.