A production-grade AI platform for Android that brings local .litertlm model inference on Google's LiteRT-LM runtime, CPU and GPU acceleration, cloud provider integration, persistent memory, and hands-free voice interaction into one unified application.
Run powerful language models directly on your device.
Zero cloud dependency. Zero data leaves your phone โ unless you choose otherwise.
Features ยท AI Agent ยท Architecture ยท Getting Started ยท Voice ยท Cloud ยท Memory ยท Docs ยท Contributing
introduction.mp4
Or watch it on YouTube
Most mobile AI apps either route everything through the cloud or ship as a lightweight demo. AndroLLM is a complete, production-quality product:
| Typical Mobile AI Apps | AndroLLM | |
|---|---|---|
| Local LLMs | None or limited experiments | LiteRT-LM runtime + .litertlm containers |
| GPU Acceleration | Rarely available | OpenCL GPU delegate with automatic CPU fallback |
| Multi-turn Chat | None or re-prefill every turn | KV-cache persistence, diff-based continuation |
| Cloud Providers | One proprietary backend | Any LiteLLM-compatible endpoint |
| Persistent Memory | None | Vector embeddings + hybrid retrieval |
| Voice Assistant | Cloud-dependent | Fully offline: wake word โ ASR โ LLM โ TTS |
| Your Data | Sent to provider servers | Stays on-device by default |
|
Run
|
Hardware-accelerated inference on supported devices via the OpenCL-based LiteRT GPU delegate, with CPU (XNNPACK) always available.
|
|
Connect to any OpenAI-compatible API through a unified LiteLLM proxy layer.
|
Remember facts across conversations โ locally or via cloud embeddings.
|
|
Hands-free interaction entirely on-device. Say "Hey Andro" and chat naturally.
|
"The Parchment Ledger" design system โ warm, editorial, calm.
|
|
Understand, plan, and execute multi-step tasks through a capability-based tool system.
|
Connect external MCP servers or drive third-party apps directly.
|
|
ChatGPT-style, conversation-scoped file attachments for cloud models.
|
Every capability runs on-device by default โ nothing leaves the phone unless you opt in.
|
flowchart TB
USER(["๐ค User"])
subgraph PRESENTATION["Presentation Layer"]
COMPOSE["๐ Jetpack Compose UI<br/>Material 3 ยท Adaptive Nav"]
VIEWMODELS["๐ ViewModels<br/>StateFlow ยท Combine"]
end
subgraph CHAT["Chat Layer"]
STREAM["โก Streaming Engine<br/>Token flow ยท Markdown ยท Memory context"]
end
subgraph ROUTING["Model Router"]
ROUTER["๐ง InferenceRouter<br/>Local โ Cloud selection"]
end
subgraph LOCAL["Local Runtime"]
LITERT["โก LiteRT-LM<br/>Google runtime ยท pure Kotlin"]
CONTAINER["๐ฆ .litertlm Models<br/>Metadata validation + budget"]
GPU["๐ฎ GPU Delegate<br/>OpenCL ยท CPU fallback"]
MEMORY["๐ง Persistent Memory<br/>Embeddings + Retrieval"]
end
subgraph CLOUD["Cloud Gateway"]
LITELLM["โ๏ธ LiteLLM Client<br/>Retrofit + OkHttp SSE"]
PROVIDERS["๐ Provider Manager<br/>Health monitor + KeyCipher"]
end
subgraph VOICE["Voice Pipeline"]
KWS["๐ค Wake Word<br/>sherpa-onnx KWS"]
ASR["๐ฃ๏ธ ASR<br/>Streaming recognizer"]
TTS["๐ TTS<br/>Piper VITS-LJS"]
VAD["๐ VAD<br/>Barge-in detection"]
end
subgraph PERSISTENCE["Data Layer"]
ROOM["๐พ Room Database<br/>4 entities ยท WAL mode"]
DS["๐ DataStore<br/>Preferences"]
KEYSTORE["๐ Android Keystore<br/>AES-256/GCM encryption"]
end
subgraph AUTH["Authentication"]
FIREBASE["โ๏ธ Firebase Auth<br/>Google + GitHub OAuth"]
end
USER --> COMPOSE
COMPOSE --> VIEWMODELS
VIEWMODELS --> STREAM
STREAM --> ROUTER
ROUTER --> LOCAL
ROUTER --> CLOUD
LOCAL --> LITERT
LITERT --> CONTAINER
LITERT --> GPU
STREAM --> MEMORY
CLOUD --> LITELLM
LITELLM --> PROVIDERS
USER --- VOICE
VOICE --> KWS
KWS --> ASR
ASR --> ROUTER
ROUTER --> TTS
TTS --> VAD
VAD --> KWS
MEMORY --> ROOM
VIEWMODELS --> DS
PROVIDERS --> KEYSTORE
VIEWMODELS --> AUTH
AUTH --> FIREBASE
| Layer | Technology | Purpose |
|---|---|---|
| Language | Kotlin 2.1.20 | Primary development language |
| UI Framework | Jetpack Compose 1.7.2 + Material 3 | Declarative, modern Android UI |
| Architecture | MVVM + Clean Architecture + Repository Pattern | Separation of concerns |
| DI Container | Hilt (Dagger) 2.57.1 | Compile-time dependency injection |
| Navigation | Navigation Compose 2.8.4 | Type-safe screen routing |
| Async | Kotlin Coroutines 1.8.0 + Flow | Structured concurrency |
| Database | Room 2.8.4 (WAL mode, v5 schema) | Local SQL persistence |
| Preferences | DataStore Preferences 1.1.1 | Reactive key-value storage |
| Inference Engine | LiteRT-LM 0.16.0 (com.google.ai.edge.litertlm) |
Local .litertlm model execution |
| GPU Backend | OpenCL-based LiteRT GPU delegate (+ XNNPACK CPU) | Hardware-accelerated inference |
| Voice Stack | sherpa-onnx 1.13.4 (ONNX Runtime Mobile) | ASR, TTS, KWS, VAD |
| Networking | Ktor 3.0.3 + Retrofit + OkHttp 4.12.0 | HTTP client for downloads & APIs |
| Auth | Firebase Auth 34.12.0 | Google Sign-In + GitHub OAuth |
| Secrets | Android Keystore (AES-256/GCM) | API key encryption |
| Logging | Timber 5.0.1 | Structured logging |
| Testing | JUnit 4 ยท mockk ยท Turbine ยท Espresso | 51 test classes across 19 modules |
| Code Quality | Spotless ยท Detekt | Formatting + static analysis |
| Requirement | Minimum | Recommended |
|---|---|---|
| Android Studio | Hedgehog (2023.1.1) | Latest stable |
| JDK | 17 | 17 (auto-managed by Gradle) |
| Android SDK | API 36 | API 36 |
| LiteRT-LM | AARs from Maven Central (no NDK/CMake needed) | Latest |
# Clone the repository
git clone https://github.com/ShadowSafin/AndroLLM.git
cd AndroLLM
# Build debug APK (pure Kotlin โ no NDK, no CMake, no Vulkan SDK)
./gradlew assembleDebug
# Install on connected device
adb install app/build/outputs/apk/debug/app-debug.apk๐ Full Building Guide ยท Development Workflow
- Install the APK on an arm64-v8a device (4 GB+ RAM recommended)
- Sign in with Google or GitHub (optional โ guest mode works fully)
- Browse models in the Catalog tab โ filters by your device's RAM
- Download and load a model (Qwen3-0.6B mixed int4, ~475 MB, is a great starting point)
- Start chatting โ messages stream in real-time with markdown rendering
- Enable voice in Settings โ Voice Assistant and say "Hey Andro"
- Unlock the agent โ enable Tool Calling in Settings โ Automation and try a multi-step request
Complete offline voice pipeline โ no internet required:
[Microphone @ 16kHz]
โ
โผ
Wake Word Detection
"Hey Andro" / "Okay Andro"
(sherpa-onnx KWS, ~3 MB)
โ
โผ
Streaming ASR
English speech โ text
(zipformer-en-20M int8, ~8 MB)
โ
โผ
Command Router โโโโบ 12 local commands
โ
โผ
LLM Generation โโโโบ LiteRT-LM (.litertlm) or Cloud API
โ
โผ
TTS Playback
Piper VITS-LJS @ 22050Hz
(~114 MB, lazy-loaded)
โ
โผ
Barge-in via VAD โโโบ Interrupt and re-listen
Supported voice commands: mute, unmute, stop speaking, new chat, open settings, open models, switch theme, delete conversation, summarize chat, enable/disable offline mode and voice.
- Spoken confirmations โ for high-risk actions the assistant asks aloud ("send the SMS to Mom?") and listens for yes/no
- Multi-step spoken tasks โ voice runs the same tool workflow as chat
- Smart TTS โ LLM output is normalized before speaking: numbers, dates, currencies, units, math, emoji, URLs, phones, and out-of-lexicon words ("LLM" โ "el el em") are all pronounced correctly; each stage is configurable in Settings โ Text Normalization
๐ Voice Architecture ยท Text Normalization
Beyond chat, AndroLLM is a full on-device AI agent. Enable it in Settings โ Automation โ Tool Calling, then ask for multi-step tasks:
"Check today's weather and text Mom if it will rain" "Search GitHub for LiteLLM and summarize the latest release" "Turn on Bluetooth, connect my earbuds, then play my workout playlist"
Your request
โ
โผ
PLANNER โโโ local LiteRT-LM (JSON-compat planner) โโ or โโ cloud (native tool calls)
โ picks tools + arguments
โผ
EXECUTOR โโ permission gate โ confirmation gate โ timeout (20 s)
โ (the only place tool code runs)
โผ
TOOLS โโ 45+ built-ins ยท accessibility ยท MCP remote tools
โ
โผ
RESULTS feed back โ re-plan (up to 6 rounds) โ grounded final answer
| Capability | Highlights |
|---|---|
| Tool catalog | Weather, search, SMS/calls/email (confirmed), calendar, alarms, reminders, clipboard, notes, files, calculator, unit & currency converters, translation, GitHub, media, PDF/Markdown export, QR, and more โ full catalog |
| Workflow engine | Multi-round planโexecuteโre-plan, IF/ELSE & loops via variable_set/variable_get, live device context (battery, time, clipboard, app) injected every round โ details |
| Confirmations | High-risk actions ask first โ chat card and spoken voice question; modes: High-risk / Always / Never โ details |
| MCP servers | Import tools from any MCP (Streamable HTTP) server; they become first-class mcp_<server>_<tool> capabilities โ details |
| UI automation | Drive any app via the accessibility service: tap, type, scroll, swipe, pinch, multi-step tasks โ details |
| Transparency | Every call is trace-logged (Developer โ Tool Debug); tools never run silently; retry-once on transient failures |
Add any LiteLLM-compatible endpoint from Settings โ Cloud Providers:
| Provider | Status | Notes |
|---|---|---|
| Google Gemini | โ Supported | Via LiteLLM proxy |
| Anthropic Claude | โ Supported | Via LiteLLM proxy |
| OpenAI GPT | โ Supported | Native OpenAI API |
| xAI Grok | โ Supported | OpenAI-compatible endpoint |
| Meta Llama | โ Supported | Via self-hosted LiteLLM |
| Mistral | โ Supported | OpenAI-compatible API |
| Custom LiteLLM | โ Supported | Any OpenAI-compatible router |
API keys are encrypted with AES-256/GCM via Android Keystore โ they never touch shared preferences or plaintext storage.
๐ Cloud Provider Architecture
Memories extracted from conversations are stored locally and injected into future contexts:
Conversation exchange
โ
โโโโถ Extract facts & preferences
โ (JSON schema: category, content, importance, tags)
โ
โโโโถ Embed content
โ โโโ Cloud path: LiteLLM embeddings API
โ โโโ Local path: LiteRT embedding engine (CompiledModel API)
โ
โโโโถ Store in SQLite + CosineVectorIndex
โ
โโโโถ Future conversation:
Hybrid search (vector + keyword)
โ Inject into system prompt
Works fully offline. Falls back to keyword/recency sorting when embeddings are unavailable.
๐ Memory Architecture
Format: .litertlm (LiteRT-LM engine file format) โ the primary and only runnable local format. The catalog ships 7 curated models across Qwen, Gemma, and DeepSeek families (architectures: gemma3, gemma4, gemma-embedding, qwen2, qwen3) from the litert-community repos on Hugging Face and ModelScope.
| Model | Quantization | Size | Use Case |
|---|---|---|---|
| Qwen3-0.6B Mixed Int4 | Mixed int4 | ~475 MB | Recommended daily driver ยท 2โ4 GB RAM |
| Gemma 3 1B IT Q4 | Q4 | ~560 MB | Small + capable ยท 2โ4 GB RAM |
| Qwen2.5-1.5B Q8 | Q8 | ~1.3 GB | Best quality on 4 GB+ devices |
Context length is detected from container metadata at load time; the tool advertisement is budgeted to the real window so small models never overflow. Legacy GGUF files can be inspected (metadata) in the import flow but are not runnable โ the app has no llama.cpp runtime.
๐ Model Support Guide
| Aspect | Implementation |
|---|---|
| Local inference | Runs entirely on-device; zero network transmission |
| API key storage | AES-256/GCM encrypted in Android Keystore |
| Database | Room in app-private sandbox (/data/data/io.androllm.app/) |
| Network | HTTPS-only; cleartext disabled (usesCleartextTraffic=false) |
| Permissions | Minimal set; requested lazily (not at launch) |
| Voice audio | Processed entirely on-device; no server transmission |
| Analytics | None. Zero telemetry or crash reporting services. |
| Guest mode | Full functionality without any authentication |
See PRIVACY.md and SECURITY.md for full details.
| Document | Description |
|---|---|
| ARCHITECTURE.md | Complete system architecture with diagrams |
| BUILDING.md | Environment setup and build instructions |
| CONTRIBUTING.md | Development guidelines and code style |
| TESTING.md | Test strategy, frameworks, and conventions |
| DEVELOPMENT.md | IDE setup, debugging, profiling guide |
| TROUBLESHOOTING.md | Common issues and solutions |
| FAQ.md | Frequently asked questions |
| PERFORMANCE.md | Token speed, RAM, Vulkan, battery guidance |
| MODEL_SUPPORT.md | .litertlm format, families, quantizations, RAM guidance |
| ROADMAP.md | Completed, planned, and future features |
| CHANGELOG.md | Version history |
| RELEASE_PROCESS.md | Build, sign, and publish procedures |
| PROJECT_STRUCTURE.md | 34-module dependency graph |
documentation/
โโโ getting-started/first-run.md # Installation and first-time setup
โโโ ai/ # AI engine internals
โ โโโ litert-lm.md # LiteRT-LM runtime, compat layer, lifecycle
โ โโโ model-formats.md # .litertlm containers, catalog sources, GGUF inspection
โ โโโ acceleration.md # CPU (XNNPACK) vs GPU (OpenCL delegate), fallback
โโโ voice/voice-assistant.md # Full voice pipeline: KWS โ ASR โ TTS
โโโ voice/text-normalization.md # TTS text normalization + OOV spelling
โโโ agent/ # AI agent platform
โ โโโ agent-platform.md # Planning, executor safety gates, chat/voice
โ โโโ tools.md # Complete built-in tool catalog
โ โโโ workflow-engine.md # Multi-step execution, variables, confirmations
โ โโโ mcp.md # MCP server integration
โ โโโ accessibility-automation.md # UI automation, gestures, planners
โโโ cloud/cloud-providers.md # LiteLLM client, streaming, security
โโโ memory/memory-architecture.md # Embeddings, retrieval, vector index
โโโ ui/ # UI architecture
โ โโโ ui-architecture.md # Design system, components, theming
โ โโโ chat-architecture.md # Streaming, markdown, state management
โโโ backend/ # Data layer
โ โโโ database.md # Room schema v5, migrations, DAOs
โ โโโ networking.md # Ktor + Retrofit stacks
โ โโโ firebase-auth.md # Google + GitHub auth flow
โโโ security/security-architecture.md # 4-layer security model
โโโ development/error-handling.md # Result/UiState patterns, recovery
โโโ android/permissions.md # All declared permissions reference
โโโ INDEX.md # Complete documentation index
34 Gradle modules organized into three tiers:
AndroLLM/
โโโ app/ # Entry point, navigation host, auth
โโโ core/ # 17 shared library modules
โ โโโ common/ Base types (Result, UiState, BaseViewModel)
โ โโโ ui/ Compose theme, design system, shared components
โ โโโ database/ Room DB (4 entities, version 5, WAL mode)
โ โโโ datastore/ Preferences DataStore
โ โโโ navigation/ Route constants and extensions
โ โโโ models/ Domain models + catalog engine (137 architectures)
โ โโโ network/ Ktor client + HuggingFace API
โ โโโ cloud/ LiteLLM client + provider manager + KeyCipher
โ โโโ utils/ Permissions, storage, connectivity helpers
โ โโโ telemetry/ Performance metrics storage
โ โโโ memory/ Vector memory system with embeddings
โ โโโ voice/ sherpa-onnx ASR/TTS/KWS/VAD engines
โ โโโ tools/ AI agent: planner, executor, registry, workflow, traces
โ โโโ mcp/ MCP client: connection manager + remote tool adapter
โ โโโ accessibility/ UI automation service, gestures, QR scanning
โ โโโ runtime/ Runtime registry: tools, voice, automation registration
โ โโโ permissions/ Central permission/access manager + feature map
โโโ engine/ # LiteRT-LM inference engine (pure Kotlin)
โ โโโ api/ InferenceEngine, EngineRepository, DefaultEngineRepository
โ โโโ core/ LiteRtLmEngine โ session lifecycle, streaming
โ โโโ compat/ ModelFamily, registry, templates, special tokens, metadata
โ โโโ models/ EngineModelInfo, GenerationConfig, MemoryStats, backends
โ โโโ embedding/ LiteRtEmbeddingEngine (CompiledModel API)
โ โโโ utils/ MemoryEstimator, ThreadManager, CoherenceChecker, LiteRtValidator
โโโ feature/ # 12 independent feature modules
โ โโโ home/ Home screen, recent chats, quick actions
โ โโโ chat/ Chat UI, streaming, markdown, drawer
โ โโโ models/ Model catalog browser, downloader, benchmark
โ โโโ voice/ Foreground service, overlay UI, state machine
โ โโโ settings/ App settings, voice config, memory controls
โ โโโ splash/ Animated splash screen
โ โโโ onboarding/ First-run onboarding flow
โ โโโ setup/ First-launch permission & access setup, Permissions & Access
โ โโโ profile/ User profile with Firebase sync
โ โโโ prompts/ Prompt library
โ โโโ developer/ Developer tools and diagnostics
โ โโโ cloud/ Cloud provider management and model browsing
โโโ documentation/ # Internal documentation module
โโโ gradle/libs.versions.toml # Centralized version catalog
Feature modules depend only on core:* modules โ never on each other.
| Status | Feature |
|---|---|
| โ | LiteRT-LM 0.16.0 runtime โ pure Kotlin, no native code |
| โ | .litertlm model catalog (7 curated models, 5 architectures) |
| โ | CPU (XNNPACK) + OpenCL GPU delegate with automatic fallback |
| โ | Cloud provider abstraction via LiteLLM |
| โ | Persistent memory with vector embeddings |
| โ | Offline voice assistant (wake word โ ASR โ TTS) |
| โ | Firebase Auth (Google + GitHub) |
| โ | AI agent platform (47 tools, workflow engine, confirmations) |
| โ | MCP server integration (Streamable HTTP) |
| โ | Accessibility UI automation (gestures, multi-step app tasks) |
| โ | Voice confirmations + TTS text normalization |
| โ | NPU backend support |
| ๐ง | Multi-language ASR (Chinese, Japanese, Korean) |
| ๐ง | CI/CD pipeline |
| ๐ฎ | Multi-modal vision models |
See ROADMAP.md for the full list.
We welcome contributions of all kinds โ bug reports, documentation fixes, new features.
๐ Contributing Guide
This project is licensed under the Apache License 2.0. See LICENSE for details. Third-party component licenses are listed in LICENSES.md.
- LiteRT-LM โ On-device LLM inference runtime (Google AI Edge)
- LiteRT โ On-device ML runtime (CompiledModel API for embeddings)
- sherpa-onnx โ Offline voice processing
- Jetpack Compose โ Modern Android UI
- Material 3 โ Design system
- Firebase Auth โ Authentication
- LiteLLM โ Cloud provider abstraction
- Android Keystore โ Secure key storage
