File size: 2,245 Bytes
e9314d1 98c5d51 fd1dcd6 26c3373 fd1dcd6 e9314d1 98c5d51 fd1dcd6 26c3373 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 26c3373 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 e948663 fd1dcd6 c693234 fd1dcd6 e948663 fd1dcd6 98c5d51 fd1dcd6 e948663 fd1dcd6 c693234 fd1dcd6 c693234 fd1dcd6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
language:
- en
license: llama3.2
tags:
- mobile
- edge-ai
- quantized
- gguf
- on-device
- small-language-model
pipeline_tag: text-generation
---
# Llama 3.2 1B Instruct - Q4 Mobile (GGUF)
**Meta's Llama 3.2 1B Instruct**, quantized to INT4 GGUF format for mobile deployment by Dispatch AI.
| Property | Value |
|----------|-------|
| **Base** | meta-llama/Llama-3.2-1B-Instruct |
| **Parameters** | 1.23 billion |
| **Quantization** | Q4_K_M (4-bit k-means) |
| **Size** | ~767 MB |
| **Format** | GGUF (llama.cpp) |
| **License** | Llama 3.2 Community |
## Why This Model?
Mobile-optimized for deployment on Android phones (Snapdragon 865+), laptops, IoT devices, and any hardware with 4GB+ RAM. No GPU required.
## Performance on Samsung S20 FE (Snapdragon 865)
| Metric | This Version | Original FP16 |
|--------|------------|---------------|
| Size | 767 MB | ~2.5 GB |
| Speed | ~28 tok/s CPU | ~8 tok/s |
| Memory | ~1.2 GB | ~3.8 GB |
| Quality | ~95% of original | 100% baseline |
## Use Cases
- Chatbots & conversational AI on mobile devices
- Instruction following in resource-constrained environments
- Content summarization, text classification, RAG pipelines
- Educational apps, tutoring systems
## Quick Start
```bash
# Install llama.cpp
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && cmake -B build -DLLAMA_NATIVE=ON && cmake --build build --config Release
# Download this model
huggingface-cli download dispatchAI/Llama-3.2-1B-Instruct-Q4-mobile ggml-model-Q4_K_M.gguf --local-dir ./models
# Run inference immediately
./build/bin/main -m ./models/ggml-model-Q4_K_M.gguf -p "Hello" -n 256 -t 4
```
## Hardware Requirements
| Requirement | Minimum | Recommended |
|-------------|---------|-------------|
| RAM | 4 GB | 6 GB+ |
| Storage | 1 GB free | 2 GB+ |
| CPU | 4-core ARM64/x86_64 | 8-core Snapdragon 865+ |
| GPU | Not required | Any (faster) |
## Limitations
- ~5% quality degradation vs FP16 on complex reasoning tasks
- Not suitable for high-precision numerical computation
- Context window follows base model (~128K tokens)
## About Dispatch AI
Re-engineering LLMs for mobile and edge deployment.
[HuggingFace](https://huggingface.co/dispatchAI) - 40+ models, 13K+ downloads
|