Alibaba's MNN team at Taobao has released and open-sourced MNN-Pocket-0.3B, a language model of roughly 300 million parameters built for a specific and often ignored deployment target: the inside of a large mobile app. The model, its training pipeline and multiple deployment variants are published on GitHub and Hugging Face, with Chinese and English as the design-focus languages.
The team framed the release around four self-imposed design goals for what it calls an app-native large model. Decoding throughput should exceed 200 tokens per second on specified devices, in reference to the pace of fast cloud services. Peak memory should stay below 300MB inside a model budget of roughly 500MB, leaving room for context growth and integration overhead on phones that share 8-16GB with the system and other apps. A single typical inference should cost no more than one ten-thousandth of a reference phone's battery, so 100 runs stay under 1%. And on specified business tasks, the model should match the output quality of an 8B-class model.
On the team's own measurements, the shipped model clears those bars. Running fully offline through the MNN engine on an iPhone 18 Pro, it decodes at an average of about 350 tokens per second. On an iPad Pro with the M5 chip over the Metal backend, the team measured 8,721 tokens per second of prefill and 466 tokens per second of decoding, with peak memory rising 228MB over baseline. For comparison, Qwen3-0.6B in the same setup occupies 340MB and adds 427MB — meaning MNN-Pocket's package is about 49% smaller and its memory footprint about 47% lower. On the CPU backend of a Snapdragon 8 Gen 5 phone it decodes at 251 tokens per second, and on a OnePlus 13 the team measured about 0.3% of battery per 100 runs. These are all vendor self-tests and have not been independently verified.
The architecture choices are as interesting as the numbers. After sweeping dense configurations from about 50 million to 1.3 billion parameters, the team found roughly 300 million to be the largest size that satisfied both the memory and speed constraints. The final model uses 28 layers with grouped-query attention; going to 36 layers did not raise average benchmark scores but cut decoding speed from 355 to 323 tokens per second. The vocabulary was trimmed from about 151,000 tokens in the Qwen3 tokenizer to 64,000, cutting embedding and output-layer costs at the price of prioritizing Chinese and English tokenization efficiency.
On the team's evaluation setup, the model scores 73.20 on English IFEval and 74.59 on a Chinese instruction-following set, and averages 50.00 across 11 Chinese and English benchmarks covering knowledge, reasoning, math, code and instruction following — ahead of same-size peers and of larger comparisons including Qwen3-0.6B, with MGSM-zh at 51.20 and HumanEval at 53.66. The model card itself cautions that, like other sub-billion models, it is not recommended for advanced math or long-horizon code generation.
The release ships the full ladder: a base model for continued pretraining, an SFT checkpoint, the instruction-tuned model, a GGUF build, an INT4 MNN deployment build, a decision-head variant, and a DFlash2 speculative-decoding draft model the team says can double decoding throughput on certain tasks. The strategic read is that China's super-apps are starting to treat on-device models as infrastructure: a model that costs 173MB to ship and a fraction of a percent of battery to run can be embedded in hundreds of millions of sessions a day without touching the cloud — and without the latency, cost or privacy trade-offs that cloud inference carries.
Comments (0)
Log in to join the discussion
Log InNo comments yet