I wanted a voice notes app that was actually smart — one that could transcribe, summarise, extract tasks, and organise everything automatically. But I also wanted it to be private. No cloud. No API calls. Every byte of data stays on the phone.
Haven AI (internal codename: noto) runs a quantised Qwen 2.5 0.5B model and OpenAI Whisper entirely on-device using llama.rn and whisper.rn. You record a voice note, and the app transcribes it, generates a summary and key points, extracts tasks with natural language dates, classifies it into a folder, and builds an embedding for semantic search — all without an internet connection.
How it works
The pipeline starts the moment you stop recording. Whisper transcribes the audio on-device. The transcript is then fed to the local LLM, which returns structured JSON: a summary, key points, and an array of tasks with optional due dates and reminder times. A custom natural language date parser converts spoken references like “tomorrow at 7pm” or “next Monday morning” into ISO timestamps. Tasks get created, reminders get scheduled via expo-notifications, and the note is embedded for semantic search.
The LLM also classifies each note into existing folders. If confidence is above 0.7, it auto-assigns silently. Below that, it shows a suggestion banner. If no folder matches, it suggests creating a new one. Folder summaries regenerate on a 5-second debounce whenever their contents change.
On-device AI
The model is Qwen 2.5 0.5B Instruct, quantised to Q4_K_M — about 400MB. It downloads from HuggingFace on first launch with a progress modal. The context is lazy-initialised (2–5 seconds on first load, reused after) and runs with full GPU acceleration via llama.rn’s native C++ inference. Temperature is kept at 0.3 for deterministic, structured output.
Embeddings are generated from the same model and stored in AsyncStorage as float arrays with content hashes for change detection. The AI chat feature computes cosine similarity against all stored note embeddings, retrieves the top-K relevant notes, and generates answers with source citations. It falls back to keyword matching if the model isn’t available.
Architecture
The app uses Expo Router with file-based routing — five visible tabs (Home, Notes, Voice, Calendar, Chat) plus hidden routes for Tasks and Profile. The voice tab has a custom elevated circular orange button in the centre of the tab bar.
State management is custom hooks over AsyncStorage — no Redux, no Zustand. Each entity (notes, tasks, recordings, folders) has its own hook with CRUD operations and persistence. IDs are generated with a timestamp-plus-random scheme. The whole design system uses the Geist font family with a warm paper-and-ink colour palette and a 4pt spacing scale.
There’s also an iOS widget via @bacons/apple-targets that syncs note count and the latest note title through App Group shared storage. Notifications support recording controls (pause/resume/stop as action buttons) and task reminders with Done and Snooze actions that work without opening the app.
Technical decisions
Running a 0.5B parameter model on a phone is a compromise. It’s not GPT-4 — the summaries are functional, not eloquent. But the tradeoff is worth it: zero latency to a server, zero privacy concerns, and it works on a plane. The model is locked in memory with use_mlock and gets 99 GPU layers for maximum inference speed.
The date parser is custom — no external dependencies. It handles relative dates, weekday names, time-of-day words (morning maps to 9:00, evening to 18:00), clock times, and ambiguous hours (1–6 without AM/PM default to PM). Anything it can’t parse gracefully falls through without crashing the task creation flow.
React Native New Architecture and React Compiler are both enabled. Typed routes via Expo Router catch routing bugs at build time. The app requires a dev build — it can’t run in Expo Go because of the native llama.rn and whisper.rn dependencies.
What I learned
On-device inference is real and usable, but model selection matters enormously. Anything above 1B parameters is too slow for real-time-feeling UX on current phones. At 0.5B with Q4 quantisation, you get structured extraction that’s fast enough to feel automatic. The JSON parsing needs regex fallbacks — small models aren’t perfectly reliable at structured output.
Privacy as a feature is compelling but hard to market. “Your data never leaves your phone” is a great line, but users don’t viscerally feel the difference until you show them airplane mode working flawlessly. The best argument for on-device AI isn’t privacy — it’s that the app works everywhere, always, with no loading spinners.