Skip to content

Latest commit

ย 

History

269 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

AndroLLM

Private AI. Native Android. Your Models. Your Choice.

A production-grade AI platform for Android that brings local .litertlm model inference on Google's LiteRT-LM runtime, CPU and GPU acceleration, cloud provider integration, persistent memory, and hands-free voice interaction into one unified application.


AndroLLM Logo

Run powerful language models directly on your device.
Zero cloud dependency. Zero data leaves your phone โ€” unless you choose otherwise.

Features ยท AI Agent ยท Architecture ยท Getting Started ยท Voice ยท Cloud ยท Memory ยท Docs ยท Contributing

Android Kotlin Jetpack Compose Material 3 License

minSdk targetSdk Kotlin Version LiteRT-LM Hilt Room


๐ŸŽฌ Introduction

introduction.mp4

Or watch it on YouTube


โœจ What Makes AndroLLM Different

Most mobile AI apps either route everything through the cloud or ship as a lightweight demo. AndroLLM is a complete, production-quality product:

Typical Mobile AI Apps AndroLLM
Local LLMs None or limited experiments LiteRT-LM runtime + .litertlm containers
GPU Acceleration Rarely available OpenCL GPU delegate with automatic CPU fallback
Multi-turn Chat None or re-prefill every turn KV-cache persistence, diff-based continuation
Cloud Providers One proprietary backend Any LiteLLM-compatible endpoint
Persistent Memory None Vector embeddings + hybrid retrieval
Voice Assistant Cloud-dependent Fully offline: wake word โ†’ ASR โ†’ LLM โ†’ TTS
Your Data Sent to provider servers Stays on-device by default

๐Ÿง  Features

โšก Local Inference Engine

Run .litertlm language models entirely on-device through Google's LiteRT-LM runtime โ€” a pure-Kotlin engine with zero native code. No internet required after model download.

  • LiteRT-LM 0.16.0 runtime (com.google.ai.edge.litertlm)
  • Container validation and memory estimation before load (LiteRtValidator, MemoryEstimator)
  • Per-family chat templates, special tokens, and stop sequences from container metadata
  • Streaming token output with stop-sequence tracking
  • Multi-turn conversations via KV-cache persistence
  • Tool-advertisement budgeted to the real context window (no more context overflow)

๐ŸŽฎ CPU + GPU Acceleration

Hardware-accelerated inference on supported devices via the OpenCL-based LiteRT GPU delegate, with CPU (XNNPACK) always available.

  • CPU backend: XNNPACK, works on every arm64 device
  • GPU backend: OpenCL-based LiteRT GPU delegate
  • Automatic GPU โ†’ CPU fallback with corruption recovery (NaN/INF logits, device-lost)
  • Real-time diagnostics: gpuFree, gpuTotal, recoveryCount, active backend
  • NPU acceleration available on supported devices (Qualcomm Hexagon, MediaTek NeuroPilot, Google Tensor)
  • Interpreter warmup after model load for faster first-token latency
  • Prefix cache reuses prompt prefixes across turns, avoiding re-tokenization
  • Buffer pooling eliminates per-token allocations during streaming and JNI transfer
  • Maximum safe core allocation (2โ€“12 threads) with CPU topology detection
  • Device-class-adaptive threading (5-tier: lowโ†’high)
  • Performance profiles for different device classes
  • Crash hardening with backend auto-disable and crash telemetry

โ˜๏ธ Cloud Provider Integration

Connect to any OpenAI-compatible API through a unified LiteLLM proxy layer.

  • Google Gemini ยท Anthropic Claude ยท OpenAI GPT ยท xAI Grok ยท Meta Llama
  • Encrypted API keys via Android Keystore (AES-256/GCM)
  • SSE streaming with exponential backoff retry (IOException, 408, 429, 5xx)
  • Automatic health monitoring and provider failover
  • Model discovery via /v1/models endpoint
  • Custom providers with per-model overrides

๐Ÿงฉ Persistent Memory

Remember facts across conversations โ€” locally or via cloud embeddings.

  • SQLite-backed storage with in-memory vector index
  • Hybrid search: cosine similarity + keyword matching
  • Automatic extraction of preferences, facts, projects, opinions
  • Model-independent: memories work with any loaded model
  • Background indexing via WorkManager
  • Full privacy: data stays on device unless cloud embedding is enabled

๐ŸŽ™๏ธ Offline Voice Assistant

Hands-free interaction entirely on-device. Say "Hey Andro" and chat naturally.

  • Wake word: sherpa-onnx KWS zipformer2 (~3 MB)
  • Speech recognition: sherpa-onnx streaming ASR en-20M (~8 MB)
  • Text-to-speech: Piper VITS-LJSpeech (~114 MB, lazy-loaded)
  • Energy-based VAD for barge-in detection
  • 12 local voice commands (mute, settings, new chat, etc.)
  • Foreground service with system overlay and persistent notification

๐Ÿ’Ž Premium UI / UX

"The Parchment Ledger" design system โ€” warm, editorial, calm.

  • Jetpack Compose with Material 3
  • Adaptive navigation: bottom bar (phone) โ†’ rail (tablet/foldable)
  • Real-time markdown rendering with syntax-highlighted code blocks
  • Streaming text at ~60 fps with stable item callbacks
  • Light & dark themes with terracotta accent (#D97757)
  • Conversation drawer, model parameter sheet, search overlay

๐Ÿค– AI Agent Platform

Understand, plan, and execute multi-step tasks through a capability-based tool system.

  • 47+ built-in tools: weather, web search, SMS, calls, email, calendar, alarms, notes, calculator, converters, PDF/Markdown export, GitHub, QR & more
  • Multi-round plan โ†’ execute โ†’ re-plan workflow engine with variables & conditionals
  • Safety gates: per-tool permission toggles + high-risk confirmations (chat card & spoken voice)
  • Contact-name resolution for messaging ("text Mom") and multipart SMS
  • Effectively unlimited answer length โ€” generation runs until the model finishes

๐Ÿงฉ MCP & UI Automation

Connect external MCP servers or drive third-party apps directly.

  • MCP (Streamable HTTP) server import โ€” remote tools become mcp_<server>_<tool>
  • Accessibility engine: read screens, tap, type, scroll, drag, swipe, pinch
  • Multi-step app tasks (ui_run) with LLM or heuristic step planning
  • QR scanning, screenshot, share, and media control tools
  • Strict confirmations for anything that sends, pays, books or deletes

๐Ÿ“Ž Chat Attachments (Cloud)

ChatGPT-style, conversation-scoped file attachments for cloud models.

  • Attach PDF, DOCX, PPTX, XLSX, TXT, Markdown, CSV, JSON, HTML, images & screenshots
  • Files are parsed/OCR'd on-device; only extracted content rides with the prompt
  • Conversation-scoped: nothing is indexed, no searchable library, and the temporary cache is removed when the conversation is deleted
  • Paperclip picker: Files / Images / Camera / Gallery, with processing chips and attachment cards in chat history
  • Cloud-only โ€” local models hide the feature entirely (no button, no settings)

๐Ÿ”’ Local-First Guarantee

Every capability runs on-device by default โ€” nothing leaves the phone unless you opt in.

  • LLM inference: LiteRT-LM, zero cloud dependency
  • Voice: wake word โ†’ ASR โ†’ TTS, fully offline
  • Memory: vector index in local SQLite
  • MCP / cloud: strictly opt-in per provider
  • Zero analytics, zero telemetry, zero crash reporters

๐Ÿ—๏ธ Architecture

flowchart TB
    USER(["๐Ÿ‘ค User"])

    subgraph PRESENTATION["Presentation Layer"]
        COMPOSE["๐Ÿ’Ž Jetpack Compose UI<br/>Material 3 ยท Adaptive Nav"]
        VIEWMODELS["๐Ÿ“ ViewModels<br/>StateFlow ยท Combine"]
    end

    subgraph CHAT["Chat Layer"]
        STREAM["โšก Streaming Engine<br/>Token flow ยท Markdown ยท Memory context"]
    end

    subgraph ROUTING["Model Router"]
        ROUTER["๐Ÿง  InferenceRouter<br/>Local โ†” Cloud selection"]
    end

    subgraph LOCAL["Local Runtime"]
        LITERT["โšก LiteRT-LM<br/>Google runtime ยท pure Kotlin"]
        CONTAINER["๐Ÿ“ฆ .litertlm Models<br/>Metadata validation + budget"]
        GPU["๐ŸŽฎ GPU Delegate<br/>OpenCL ยท CPU fallback"]
        MEMORY["๐Ÿง  Persistent Memory<br/>Embeddings + Retrieval"]
    end

    subgraph CLOUD["Cloud Gateway"]
        LITELLM["โ˜๏ธ LiteLLM Client<br/>Retrofit + OkHttp SSE"]
        PROVIDERS["๐Ÿ”— Provider Manager<br/>Health monitor + KeyCipher"]
    end

    subgraph VOICE["Voice Pipeline"]
        KWS["๐ŸŽค Wake Word<br/>sherpa-onnx KWS"]
        ASR["๐Ÿ—ฃ๏ธ ASR<br/>Streaming recognizer"]
        TTS["๐Ÿ”Š TTS<br/>Piper VITS-LJS"]
        VAD["๐Ÿ“Š VAD<br/>Barge-in detection"]
    end

    subgraph PERSISTENCE["Data Layer"]
        ROOM["๐Ÿ’พ Room Database<br/>4 entities ยท WAL mode"]
        DS["๐Ÿ“ DataStore<br/>Preferences"]
        KEYSTORE["๐Ÿ” Android Keystore<br/>AES-256/GCM encryption"]
    end

    subgraph AUTH["Authentication"]
        FIREBASE["โ˜๏ธ Firebase Auth<br/>Google + GitHub OAuth"]
    end

    USER --> COMPOSE
    COMPOSE --> VIEWMODELS
    VIEWMODELS --> STREAM
    STREAM --> ROUTER
    ROUTER --> LOCAL
    ROUTER --> CLOUD
    LOCAL --> LITERT
    LITERT --> CONTAINER
    LITERT --> GPU
    STREAM --> MEMORY
    CLOUD --> LITELLM
    LITELLM --> PROVIDERS
    USER --- VOICE
    VOICE --> KWS
    KWS --> ASR
    ASR --> ROUTER
    ROUTER --> TTS
    TTS --> VAD
    VAD --> KWS
    MEMORY --> ROOM
    VIEWMODELS --> DS
    PROVIDERS --> KEYSTORE
    VIEWMODELS --> AUTH
    AUTH --> FIREBASE
Loading

๐Ÿ“ฆ Technology Stack

Core Technologies

Networking & Native

Layer Technology Purpose
Language Kotlin 2.1.20 Primary development language
UI Framework Jetpack Compose 1.7.2 + Material 3 Declarative, modern Android UI
Architecture MVVM + Clean Architecture + Repository Pattern Separation of concerns
DI Container Hilt (Dagger) 2.57.1 Compile-time dependency injection
Navigation Navigation Compose 2.8.4 Type-safe screen routing
Async Kotlin Coroutines 1.8.0 + Flow Structured concurrency
Database Room 2.8.4 (WAL mode, v5 schema) Local SQL persistence
Preferences DataStore Preferences 1.1.1 Reactive key-value storage
Inference Engine LiteRT-LM 0.16.0 (com.google.ai.edge.litertlm) Local .litertlm model execution
GPU Backend OpenCL-based LiteRT GPU delegate (+ XNNPACK CPU) Hardware-accelerated inference
Voice Stack sherpa-onnx 1.13.4 (ONNX Runtime Mobile) ASR, TTS, KWS, VAD
Networking Ktor 3.0.3 + Retrofit + OkHttp 4.12.0 HTTP client for downloads & APIs
Auth Firebase Auth 34.12.0 Google Sign-In + GitHub OAuth
Secrets Android Keystore (AES-256/GCM) API key encryption
Logging Timber 5.0.1 Structured logging
Testing JUnit 4 ยท mockk ยท Turbine ยท Espresso 51 test classes across 19 modules
Code Quality Spotless ยท Detekt Formatting + static analysis

๐Ÿ”ง Getting Started

Prerequisites

Requirement Minimum Recommended
Android Studio Hedgehog (2023.1.1) Latest stable
JDK 17 17 (auto-managed by Gradle)
Android SDK API 36 API 36
LiteRT-LM AARs from Maven Central (no NDK/CMake needed) Latest

Build & Install

# Clone the repository
git clone https://github.com/ShadowSafin/AndroLLM.git
cd AndroLLM

# Build debug APK (pure Kotlin โ€” no NDK, no CMake, no Vulkan SDK)
./gradlew assembleDebug

# Install on connected device
adb install app/build/outputs/apk/debug/app-debug.apk

๐Ÿ“– Full Building Guide ยท Development Workflow

First Run

  1. Install the APK on an arm64-v8a device (4 GB+ RAM recommended)
  2. Sign in with Google or GitHub (optional โ€” guest mode works fully)
  3. Browse models in the Catalog tab โ€” filters by your device's RAM
  4. Download and load a model (Qwen3-0.6B mixed int4, ~475 MB, is a great starting point)
  5. Start chatting โ€” messages stream in real-time with markdown rendering
  6. Enable voice in Settings โ†’ Voice Assistant and say "Hey Andro"
  7. Unlock the agent โ€” enable Tool Calling in Settings โ†’ Automation and try a multi-step request

๐ŸŽ™๏ธ Voice Assistant

Complete offline voice pipeline โ€” no internet required:

[Microphone @ 16kHz]
       โ”‚
       โ–ผ
  Wake Word Detection
  "Hey Andro" / "Okay Andro"
  (sherpa-onnx KWS, ~3 MB)
       โ”‚
       โ–ผ
  Streaming ASR
  English speech โ†’ text
  (zipformer-en-20M int8, ~8 MB)
       โ”‚
       โ–ผ
  Command Router โ”€โ”€โ”€โ–บ 12 local commands
       โ”‚
       โ–ผ
  LLM Generation โ”€โ”€โ”€โ–บ LiteRT-LM (.litertlm) or Cloud API
       โ”‚
       โ–ผ
  TTS Playback
  Piper VITS-LJS @ 22050Hz
  (~114 MB, lazy-loaded)
       โ”‚
       โ–ผ
  Barge-in via VAD โ”€โ”€โ–บ Interrupt and re-listen

Supported voice commands: mute, unmute, stop speaking, new chat, open settings, open models, switch theme, delete conversation, summarize chat, enable/disable offline mode and voice.

Voice + Agent

  • Spoken confirmations โ€” for high-risk actions the assistant asks aloud ("send the SMS to Mom?") and listens for yes/no
  • Multi-step spoken tasks โ€” voice runs the same tool workflow as chat
  • Smart TTS โ€” LLM output is normalized before speaking: numbers, dates, currencies, units, math, emoji, URLs, phones, and out-of-lexicon words ("LLM" โ†’ "el el em") are all pronounced correctly; each stage is configurable in Settings โ†’ Text Normalization

๐Ÿ“– Voice Architecture ยท Text Normalization


๐Ÿค– AI Agent

Beyond chat, AndroLLM is a full on-device AI agent. Enable it in Settings โ†’ Automation โ†’ Tool Calling, then ask for multi-step tasks:

"Check today's weather and text Mom if it will rain" "Search GitHub for LiteLLM and summarize the latest release" "Turn on Bluetooth, connect my earbuds, then play my workout playlist"

How it works

Your request
     โ”‚
     โ–ผ
  PLANNER โ”€โ”€โ”€ local LiteRT-LM (JSON-compat planner) โ”€โ”€ or โ”€โ”€ cloud (native tool calls)
     โ”‚  picks tools + arguments
     โ–ผ
  EXECUTOR โ”€โ”€ permission gate โ†’ confirmation gate โ†’ timeout (20 s)
     โ”‚  (the only place tool code runs)
     โ–ผ
  TOOLS โ”€โ”€ 45+ built-ins ยท accessibility ยท MCP remote tools
     โ”‚
     โ–ผ
  RESULTS feed back โ†’ re-plan (up to 6 rounds) โ†’ grounded final answer

Capabilities

Capability Highlights
Tool catalog Weather, search, SMS/calls/email (confirmed), calendar, alarms, reminders, clipboard, notes, files, calculator, unit & currency converters, translation, GitHub, media, PDF/Markdown export, QR, and more โ€” full catalog
Workflow engine Multi-round planโ†’executeโ†’re-plan, IF/ELSE & loops via variable_set/variable_get, live device context (battery, time, clipboard, app) injected every round โ€” details
Confirmations High-risk actions ask first โ€” chat card and spoken voice question; modes: High-risk / Always / Never โ€” details
MCP servers Import tools from any MCP (Streamable HTTP) server; they become first-class mcp_<server>_<tool> capabilities โ€” details
UI automation Drive any app via the accessibility service: tap, type, scroll, swipe, pinch, multi-step tasks โ€” details
Transparency Every call is trace-logged (Developer โ†’ Tool Debug); tools never run silently; retry-once on transient failures

โ˜๏ธ Cloud Providers

Add any LiteLLM-compatible endpoint from Settings โ†’ Cloud Providers:

Provider Status Notes
Google Gemini โœ… Supported Via LiteLLM proxy
Anthropic Claude โœ… Supported Via LiteLLM proxy
OpenAI GPT โœ… Supported Native OpenAI API
xAI Grok โœ… Supported OpenAI-compatible endpoint
Meta Llama โœ… Supported Via self-hosted LiteLLM
Mistral โœ… Supported OpenAI-compatible API
Custom LiteLLM โœ… Supported Any OpenAI-compatible router

API keys are encrypted with AES-256/GCM via Android Keystore โ€” they never touch shared preferences or plaintext storage.

๐Ÿ“– Cloud Provider Architecture


๐Ÿง  Memory System

Memories extracted from conversations are stored locally and injected into future contexts:

Conversation exchange
        โ”‚
        โ”œโ”€โ”€โ–ถ Extract facts & preferences
        โ”‚       (JSON schema: category, content, importance, tags)
        โ”‚
        โ”œโ”€โ”€โ–ถ Embed content
        โ”‚       โ”œโ”€โ”€ Cloud path: LiteLLM embeddings API
        โ”‚       โ””โ”€โ”€ Local path: LiteRT embedding engine (CompiledModel API)
        โ”‚
        โ”œโ”€โ”€โ–ถ Store in SQLite + CosineVectorIndex
        โ”‚
        โ””โ”€โ”€โ–ถ Future conversation:
                Hybrid search (vector + keyword)
                โ†’ Inject into system prompt

Works fully offline. Falls back to keyword/recency sorting when embeddings are unavailable.

๐Ÿ“– Memory Architecture


๐Ÿ“Š Model Support

Format: .litertlm (LiteRT-LM engine file format) โ€” the primary and only runnable local format. The catalog ships 7 curated models across Qwen, Gemma, and DeepSeek families (architectures: gemma3, gemma4, gemma-embedding, qwen2, qwen3) from the litert-community repos on Hugging Face and ModelScope.

Model Quantization Size Use Case
Qwen3-0.6B Mixed Int4 Mixed int4 ~475 MB Recommended daily driver ยท 2โ€“4 GB RAM
Gemma 3 1B IT Q4 Q4 ~560 MB Small + capable ยท 2โ€“4 GB RAM
Qwen2.5-1.5B Q8 Q8 ~1.3 GB Best quality on 4 GB+ devices

Context length is detected from container metadata at load time; the tool advertisement is budgeted to the real window so small models never overflow. Legacy GGUF files can be inspected (metadata) in the import flow but are not runnable โ€” the app has no llama.cpp runtime.

๐Ÿ“– Model Support Guide


๐Ÿ” Privacy & Security

Aspect Implementation
Local inference Runs entirely on-device; zero network transmission
API key storage AES-256/GCM encrypted in Android Keystore
Database Room in app-private sandbox (/data/data/io.androllm.app/)
Network HTTPS-only; cleartext disabled (usesCleartextTraffic=false)
Permissions Minimal set; requested lazily (not at launch)
Voice audio Processed entirely on-device; no server transmission
Analytics None. Zero telemetry or crash reporting services.
Guest mode Full functionality without any authentication

See PRIVACY.md and SECURITY.md for full details.


๐Ÿ“š Documentation

Document Description
ARCHITECTURE.md Complete system architecture with diagrams
BUILDING.md Environment setup and build instructions
CONTRIBUTING.md Development guidelines and code style
TESTING.md Test strategy, frameworks, and conventions
DEVELOPMENT.md IDE setup, debugging, profiling guide
TROUBLESHOOTING.md Common issues and solutions
FAQ.md Frequently asked questions
PERFORMANCE.md Token speed, RAM, Vulkan, battery guidance
MODEL_SUPPORT.md .litertlm format, families, quantizations, RAM guidance
ROADMAP.md Completed, planned, and future features
CHANGELOG.md Version history
RELEASE_PROCESS.md Build, sign, and publish procedures
PROJECT_STRUCTURE.md 34-module dependency graph

Deep-Dive Documentation (documentation/)

documentation/
โ”œโ”€โ”€ getting-started/first-run.md        # Installation and first-time setup
โ”œโ”€โ”€ ai/                                 # AI engine internals
โ”‚   โ”œโ”€โ”€ litert-lm.md                    # LiteRT-LM runtime, compat layer, lifecycle
โ”‚   โ”œโ”€โ”€ model-formats.md                # .litertlm containers, catalog sources, GGUF inspection
โ”‚   โ””โ”€โ”€ acceleration.md                 # CPU (XNNPACK) vs GPU (OpenCL delegate), fallback
โ”œโ”€โ”€ voice/voice-assistant.md            # Full voice pipeline: KWS โ†’ ASR โ†’ TTS
โ”œโ”€โ”€ voice/text-normalization.md         # TTS text normalization + OOV spelling
โ”œโ”€โ”€ agent/                              # AI agent platform
โ”‚   โ”œโ”€โ”€ agent-platform.md               # Planning, executor safety gates, chat/voice
โ”‚   โ”œโ”€โ”€ tools.md                        # Complete built-in tool catalog
โ”‚   โ”œโ”€โ”€ workflow-engine.md              # Multi-step execution, variables, confirmations
โ”‚   โ”œโ”€โ”€ mcp.md                          # MCP server integration
โ”‚   โ””โ”€โ”€ accessibility-automation.md     # UI automation, gestures, planners
โ”œโ”€โ”€ cloud/cloud-providers.md            # LiteLLM client, streaming, security
โ”œโ”€โ”€ memory/memory-architecture.md       # Embeddings, retrieval, vector index
โ”œโ”€โ”€ ui/                                 # UI architecture
โ”‚   โ”œโ”€โ”€ ui-architecture.md              # Design system, components, theming
โ”‚   โ””โ”€โ”€ chat-architecture.md            # Streaming, markdown, state management
โ”œโ”€โ”€ backend/                            # Data layer
โ”‚   โ”œโ”€โ”€ database.md                     # Room schema v5, migrations, DAOs
โ”‚   โ”œโ”€โ”€ networking.md                   # Ktor + Retrofit stacks
โ”‚   โ””โ”€โ”€ firebase-auth.md                # Google + GitHub auth flow
โ”œโ”€โ”€ security/security-architecture.md   # 4-layer security model
โ”œโ”€โ”€ development/error-handling.md       # Result/UiState patterns, recovery
โ”œโ”€โ”€ android/permissions.md              # All declared permissions reference
โ””โ”€โ”€ INDEX.md                            # Complete documentation index

๐Ÿ› ๏ธ Project Structure

34 Gradle modules organized into three tiers:

AndroLLM/
โ”œโ”€โ”€ app/                          # Entry point, navigation host, auth
โ”œโ”€โ”€ core/                         # 17 shared library modules
โ”‚   โ”œโ”€โ”€ common/      Base types (Result, UiState, BaseViewModel)
โ”‚   โ”œโ”€โ”€ ui/          Compose theme, design system, shared components
โ”‚   โ”œโ”€โ”€ database/    Room DB (4 entities, version 5, WAL mode)
โ”‚   โ”œโ”€โ”€ datastore/   Preferences DataStore
โ”‚   โ”œโ”€โ”€ navigation/  Route constants and extensions
โ”‚   โ”œโ”€โ”€ models/      Domain models + catalog engine (137 architectures)
โ”‚   โ”œโ”€โ”€ network/     Ktor client + HuggingFace API
โ”‚   โ”œโ”€โ”€ cloud/       LiteLLM client + provider manager + KeyCipher
โ”‚   โ”œโ”€โ”€ utils/       Permissions, storage, connectivity helpers
โ”‚   โ”œโ”€โ”€ telemetry/   Performance metrics storage
โ”‚   โ”œโ”€โ”€ memory/      Vector memory system with embeddings
โ”‚   โ”œโ”€โ”€ voice/       sherpa-onnx ASR/TTS/KWS/VAD engines
โ”‚   โ”œโ”€โ”€ tools/       AI agent: planner, executor, registry, workflow, traces
โ”‚   โ”œโ”€โ”€ mcp/         MCP client: connection manager + remote tool adapter
โ”‚   โ”œโ”€โ”€ accessibility/ UI automation service, gestures, QR scanning
โ”‚   โ”œโ”€โ”€ runtime/     Runtime registry: tools, voice, automation registration
โ”‚   โ”œโ”€โ”€ permissions/ Central permission/access manager + feature map
โ”œโ”€โ”€ engine/                       # LiteRT-LM inference engine (pure Kotlin)
โ”‚   โ”œโ”€โ”€ api/            InferenceEngine, EngineRepository, DefaultEngineRepository
โ”‚   โ”œโ”€โ”€ core/           LiteRtLmEngine โ€” session lifecycle, streaming
โ”‚   โ”œโ”€โ”€ compat/         ModelFamily, registry, templates, special tokens, metadata
โ”‚   โ”œโ”€โ”€ models/         EngineModelInfo, GenerationConfig, MemoryStats, backends
โ”‚   โ”œโ”€โ”€ embedding/      LiteRtEmbeddingEngine (CompiledModel API)
โ”‚   โ””โ”€โ”€ utils/          MemoryEstimator, ThreadManager, CoherenceChecker, LiteRtValidator
โ”œโ”€โ”€ feature/                      # 12 independent feature modules
โ”‚   โ”œโ”€โ”€ home/    Home screen, recent chats, quick actions
โ”‚   โ”œโ”€โ”€ chat/    Chat UI, streaming, markdown, drawer
โ”‚   โ”œโ”€โ”€ models/  Model catalog browser, downloader, benchmark
โ”‚   โ”œโ”€โ”€ voice/   Foreground service, overlay UI, state machine
โ”‚   โ”œโ”€โ”€ settings/ App settings, voice config, memory controls
โ”‚   โ”œโ”€โ”€ splash/  Animated splash screen
โ”‚   โ”œโ”€โ”€ onboarding/ First-run onboarding flow
โ”‚   โ”œโ”€โ”€ setup/    First-launch permission & access setup, Permissions & Access
โ”‚   โ”œโ”€โ”€ profile/ User profile with Firebase sync
โ”‚   โ”œโ”€โ”€ prompts/ Prompt library
โ”‚   โ”œโ”€โ”€ developer/ Developer tools and diagnostics
โ”‚   โ””โ”€โ”€ cloud/   Cloud provider management and model browsing
โ”œโ”€โ”€ documentation/               # Internal documentation module
โ””โ”€โ”€ gradle/libs.versions.toml     # Centralized version catalog

Feature modules depend only on core:* modules โ€” never on each other.


๐Ÿšง Roadmap Highlights

Status Feature
โœ… LiteRT-LM 0.16.0 runtime โ€” pure Kotlin, no native code
โœ… .litertlm model catalog (7 curated models, 5 architectures)
โœ… CPU (XNNPACK) + OpenCL GPU delegate with automatic fallback
โœ… Cloud provider abstraction via LiteLLM
โœ… Persistent memory with vector embeddings
โœ… Offline voice assistant (wake word โ†’ ASR โ†’ TTS)
โœ… Firebase Auth (Google + GitHub)
โœ… AI agent platform (47 tools, workflow engine, confirmations)
โœ… MCP server integration (Streamable HTTP)
โœ… Accessibility UI automation (gestures, multi-step app tasks)
โœ… Voice confirmations + TTS text normalization
โœ… NPU backend support
๐Ÿšง Multi-language ASR (Chinese, Japanese, Korean)
๐Ÿšง CI/CD pipeline
๐Ÿ”ฎ Multi-modal vision models

See ROADMAP.md for the full list.


๐Ÿค Contributing

We welcome contributions of all kinds โ€” bug reports, documentation fixes, new features.

๐Ÿ“– Contributing Guide


๐Ÿ“„ License

This project is licensed under the Apache License 2.0. See LICENSE for details. Third-party component licenses are listed in LICENSES.md.


๐Ÿ”— References

About

Open-source Android AI using LiteRT-LM with hardware acceleration, cloud providers, memory, and voice.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

145 stars

Watchers

1 watching

Forks

Releases

Used by

Contributors

Languages