A model runs on your device if its file fits comfortably in memory alongside everything else, which in practice means a 4-bit model of about 0.5 GB on a 3 GB phone, about 1.3 GB on a 6 GB phone, and about 2.7 GB on 8 GB with room to spare. Bigger is better at reasoning and worse at everything else: speed, battery and heat. Pick the largest one that still answers fast enough that you will actually use it.
The three numbers
Parameters (0.8B, 2B, 4B) are the size of the model's learned weights. More parameters generally means better reasoning, wider world knowledge and better handling of languages outside English. This is the number that most predicts answer quality.
Quantization (Q4_K_M, Q8 and so on) is how many bits each weight is stored in. Full precision is 16 bits; Q4 is roughly four. Dropping to four bits cuts the file to about a quarter, with a quality loss that is small for chat and noticeable for math and code. Almost everything you will run on a phone is Q4, and that is the right default.
File size in bytes is the number that decides whether it runs at all, because the whole file has to be held in memory while it works. Take the file size, add room for the conversation context, and compare with the memory your device has free rather than the memory it has in total.
The speed you should expect
Measured numbers beat estimates, so here are ours, taken on our own hardware rather than from a benchmark table.
| Model | Size | 2018 Android flagship | iPhone 13 Pro | Desktop browser |
|---|---|---|---|---|
| Instant, 0.8B | 0.53 GB | 12-15 tok/s | ~36 tok/s | 33 tok/s |
| Fast, 2B | 1.28 GB | 5-6 tok/s | not measured | not measured |
| Sharp, 4B | 2.74 GB | below 1 tok/s | not measured | not measured |
| Sharp (Phi), 3.8B | 2.49 GB | 2-3 tok/s | not measured | not measured |
Anything above about 10 tokens a second reads at a comfortable pace. Below 5, you feel every word. The 4B model on a 2018 phone is the clearest case on that table: it technically installs and it is not worth installing, which is why Inborn marks it as too slow on the card rather than letting you find out after the download.
Time to first token matters as much as throughput. A long document pushed through a small model can take many seconds before the first word appears, because the model has to read the whole thing before it writes anything.
What each size is actually good at
0.8B is for quick answers, rewrites and summaries, and it is the size that starts instantly and barely touches the battery. It is weak at code, math and long documents, and outside a handful of languages it gets shaky after a couple of sentences.
2B is the everyday assistant: writing, general questions, translation into the major languages, and questions about a document. This is the right default on any phone with 6 GB of memory.
4B is for analysis, code and long documents. It is slower, it warms the device, and on a phone you feel it. Worth it when the answer matters more than the wait.
Beyond size there is training. A model tuned for reasoning and math, such as Phi-4-mini, beats a same-size general model on a logic puzzle and loses to it on translation. There is no single best model at any size, only a best model for what you are about to ask.
How to choose in practice
Start with the smallest model, use it for a week, and notice where it fails you. If it fails on languages or long documents, go up a size. If it fails on math and code, change family rather than size. If it never fails, you have saved yourself several gigabytes and a warmer phone.
Inborn shows the expected speed range for each model on your specific device before you install it, and tells you when the model that just answered would be beaten by one you have not installed. That is a better guide than any benchmark table, because the only benchmark that matters is the one running on the hardware in your hand.
NextWhy the AI should run on your device · How to prove an AI app sends nothing