Bonsai 27B

The first 27B-class multimodal large model that runs locally on a smartphone, compressed to 3.9GB from Alibaba Qwen3.6 27B via extreme quantization

large language model LLM

https://prismml.com/

Visit Website

In-depth Report

On July 14, 2026, PrismML, an AI startup with a background at Caltech, released Bonsai 27B, the world’s first 27B-level multi-modal large model that can run locally on a smartphone. Based on the Alibaba Qwen3.6 27B base, the model is compressed to 3.9GB (1-bit version) and 5.9GB (three-value version) through extreme low-bit quantization. While retaining about 90-95% of the benchmark performance, the 27 billion parameter-level reasoning capability is available for free on consumer hardware for the first time under the Apache 2.0 open source protocol.
PrismML was co-founded by Babak Hassibi, professor of electrical engineering and computational mathematics at the California Institute of Technology (Caltech), and the core team also includes Sahin Lale, Omead Pooladzandi, and Reza Sadri, all from Caltech. The company's core technology originates from the school's many years of mathematical theoretical research, focusing on neural network compression. Professor Hassibi participated in proposing the neural network pruning method Optimal Brain Surgeon as early as the 1990s, and has profound experience in model simplification. PrismML raised a total of US$16.25 million in SAFE and seed rounds, led by Khosla Ventures, a well-known Silicon Valley venture capital firm, with participation from Cerberus Capital and the California Institute of Technology. Google and Samsung also provided computing power and industry support. Investor Vinod Khosla described this technology as "a mathematical breakthrough, not another small model" and believed that the future of AI is not defined by the size of the data center, but by the density of intelligence per unit energy consumption and cost. The company ended stealth mode in March 2026 and released its first product, the 1-bit Bonsai 8B, followed by the Bonsai Image 4B in May. The Bonsai 27B is the flagship model of the Bonsai series, marking the advancement of its compression route to higher levels of capability. Apple is reportedly in talks with PrismML for early technology evaluation.
Bonsai 27B is not a basic model trained from scratch, but a heavily quantized version based on Alibaba Qwen3.6 27B. It is extremely compressed while retaining the full feature set: 262K token ultra-long context windows, multi-modal visual understanding (4-bit HQQ Vision Tower), structured tool calls, computer-operated agent loops, and speculative decoding to accelerate inference. The model provides two quantification variants: the Ternary version uses three-valued weights {-1, 0, +1} plus FP16 group scaling, with an effective bit of 1.71 and a volume of 5.9GB. It is oriented to notebook scenarios and pursues quality first; the 1-bit version uses binary weights {-1, +1}, effective bits 1.125 and a volume of 3.9GB. It is oriented to mobile phone scenarios and pursues volume first. The key difference between the two is this: Choosing the 1-bit version means accepting a more significant loss of precision in Agentic tool calls, visual understanding, and complex mathematical reasoning. In terms of mobile phone adaptation, the actual available single-application memory of the 12GB memory iPhone 17 Pro is about 6GB. The 1-bit version of 3.9GB plus KV cache and activation value can just fit into this budget. Official measured data shows that the 1-bit version on the iPhone 17 Pro Max is about 11 tokens/s, and about 672 tokens consume 1% of the battery. On the desktop, the 1-bit version can reach 163 token/s on the RTX 5090, and the ternary version can reach 134 token/s; on the M5 Max Mac, the 1-bit version can reach 87 token/s, and the ternary version can reach 58 token/s. Actual feedback from independent developers (based on RTX 3090 running the Q4_K_M GGUF version) shows that the speed of this magnitude on consumer-grade graphics cards is satisfactory: about 28 token/s in the 4K context, and still 19 token/s in the 16K context. Structured JSON extraction and single-round RAG performance are "stable and without hallucinations", but multi-step reasoning (more than 3-step tool call chain) has obvious shortcomings. Actual testing in the Chinese community also confirms this: Bonsai 27B can run on an old 8GB graphics card (only 3.8GB is required), with good speed and accuracy, but thinking mode must be turned on to ensure output quality.
Bonsai 27B is completely open source under the Apache 2.0 protocol. The weights are free to download from Hugging Face, and commercial use is allowed without authorization. PrismML also provides a limited-time free developer preview API (through Together.ai) to facilitate developers to do prototype verification before pulling weights. This business strategy is similar to Apple's early developer ecosystem strategy: through open source, the model can be penetrated into more products and ecosystems, and an open attitude can be used to establish industry standards and developer stickiness. The core value of the device-side model is that inference is free and data does not leave the device, which is very attractive for privacy-sensitive fields and offline scenarios.
Real feedback from the community shows that developers’ overall evaluation of Bonsai 27B is positive, especially that it “does what other models cannot do” – allowing 27B-class models to run on mobile phones. A developer who tested on RTX 3090 gave a detailed analysis of the advantages and disadvantages: the classification pipeline, structured extraction, and single-round RAG scenarios are "fully production-ready", and the license is Apache 2.0, which means "it can be deployed without legal review, which is much more important than a few points on the benchmark." But he also pointed out that in complex agent loop scenarios, due to the sparsity of MoE, the model is prone to losing clues in more than 3 steps of reasoning chains and repeating the previous step instead of continuing to advance. A detailed test article on the Nuggets Chinese community also pointed out that the Agentic tool call dimension has the largest drop (17.5% drop in 1-bit version), and the usability drop in actual testing may be more significant than the numbers. Toutiao’s test based on RTX 2070 8GB gives an intuitive “IQ ranking”: Gemma-4-26B-A4B > Qwen3.6-35B-A3B > Bonsai-27B > Qwythos-9B. The core evaluation is “extremely simple deployment, good speed, honest and no nonsense”, but the thinking mode must be turned on.
Industry media’s coverage of Bonsai 27B focuses on the “milestone” level. The mainstream view is that the real significance of this model does not lie in the single benchmark score, but in what it "normalizes": a 27B-level multi-modal model that lives on consumer-grade hardware with a loose license and is available offline. CoinDesk pointed out from the perspective of crypto finance that end-side AI eliminates the encryption industry’s pain point of trusting cloud infrastructure: user data does not have to leave the device, which is a meaningful privacy upgrade for crypto wallets, DeFi interfaces and trading tools. Hacker News community discussions are more pragmatic: acknowledging technological breakthroughs while repeatedly reminding that the shortcomings of extreme quantification in real engineering scenarios cannot be ignored. Independent analysis blog Sean Kim gave the most direct judgment: "Running on a mobile phone" and "matching a full-precision model" are not the same proposition. A reasonable model is a hybrid deployment: the local device-side model handles lightweight, privacy-sensitive tasks, and complex reasoning is left to the cloud. From the competitive landscape, Apple, Google, and Qualcomm are all promoting end-side AI, but the mainstream remains at 3-7B parameter scale models. Bonsai 27B directly increases the parameter scale by nearly 4 times, opening up the competitive dimension of "intelligent density". The intelligence density indicator (capability value per GB) proposed by PrismML shows that 1-bit Bonsai 27B is 0.53/GB, which is more than 10 times the full precision baseline.
The main controversy surrounding Bonsai 27B centers on the extent to which extreme quantification erodes Agentic capabilities. The official benchmark shows that the 1-bit version has dropped by 17.5% in the tool call dimension, but community testing believes that the actual drop may be more significant. Some developers pointed out that the model has a tendency to "drop the chain" after the third layer of inference, which is enough to make the entire process unusable in severe agentic scenarios. Another point of discussion is that "Bonsai 27B is not a new model, but a package". Critics believe that PrismML did not train its own basic model, but did quantitative packaging based on Qwen3.6. Although the compression technology itself is groundbreaking, it remains to be seen how high the technical barriers are and whether other teams can quickly reproduce similar effects. Questions about visual ability cannot be ignored: the 1-bit version of MMMU Pro/OCRBench scored only 59.6, a significant drop from the baseline of 72.6, indicating that extreme quantization has a greater impact on visual understanding than text tasks.
Bonsai 27B is suitable for the following developers: mobile application developers who need offline local AI capabilities; privacy-sensitive scenarios (medical, legal, financial document processing); researchers who need to run 27B-level models on consumer-grade graphics cards (8-12GB VRAM); and text processing pipelines that do not require high tool call accuracy. It is not recommended to be used in the following scenarios: production systems that require stable multi-step Agentic capabilities, high-precision visual understanding pipelines, and complex mathematical reasoning tasks. The most reasonable deployment strategy at present is a hybrid architecture: Bonsai 27B (Ternary version) serves as the main force on the local client side and is used for tasks with privacy-sensitive and low-latency requirements; the cloud large model handles complex reasoning and scenarios requiring high accuracy. This stack will only be truly feasible in 2026 - because the local end finally has a strong enough model.
Bonsai 27B is an iconic node in the development of device-side AI: it allows "27B-level models to be installed on mobile phones" from being theoretically discussable to actually being downloadable and runnable. The accuracy retention on structured workpieces such as mathematics and programming is acceptable, but the shortcomings on agentic and visual tasks are also obvious enough. As the open source client-side model of Apache 2.0, it provides developers with a new stack level - not a comprehensive replacement, but a new option that did not exist before. As of July 2026, it is the biggest step on the road to device-side AI, but not the last.

User Reviews

Richard112

—

终于有个 27B 模型能在手机上跑了，下载完试了下，3.8GB 确实小得离谱。日常写写文案、做做总结完全够用，但别指望它像云端模型那样聪明，开 thinking 模式才稳，这个要注意。

SMartinez_66

—

部署是真的很简单，LM Studio 下搜一下就能用，门槛低到离谱，这点做得比很多开源模型好。我 8G 老卡也能跑，虽然必须开 thinking 模式。

TMorgan_20248

—

用 RTX 3090 跑了一下，Q4_K_M 版本 16GB 单卡就能跑，28 tok/s 的生成速度对于日常聊天完全够了。做结构化 JSON 提取贼稳，没有幻觉，这点好评。

BettyLopez007

—

我在 iPhone 17 Pro 上试了 1-bit 版本，速度确实只有 11 tok/s 左右，但考虑到是本地跑 27B 的模型，这个表现已经超出预期了。电池消耗还行，不会明显发热。

BAndersonK

—

实测发现它做工具调用的时候不太行，多步 agent 流程到了第三步就开始掉链子，格式也飘了。当聊天模型用可以，做自动化 agent 还差点意思。

月光_17

—

Apache 2.0 开源协议是真香，部署不需要法务审查，直接下载就能用。这一点比 benchmark 上的分数重要多了，商用省心。

Jesse_Foster_66

—

说它是 DeepSeek 时刻有点过了，毕竟它只是 Qwen 3.6 的量化打包，没有训练自己的基础模型。但压缩技术本身确实有两把刷子。

GEbel

—

用 M5 Max 跑 ternary 版本，58 tok/s 的速度已经能当日常主力了。262K 上下文窗口是真的大，加载一整本技术书进去做问答，记忆没丢。

StephanieFoster369

—

和 Gemma-4-26B 对比了一下，智商确实不如人家，但 Bonsai 的代价最小，3.8GB 就能跑，生成速度还快一倍，就看你在意什么了。

Ryan_RussellQ

—

苹果在和 PrismML 谈评估我觉得是好事，如果未来 iPhone 能本地跑 27B 的模型，Siri 应该能支棱起来吧。现在手机上的端侧模型才 3B，差距太大了。

云烟_6

—

弱智吧测试翻车了哈哈，问它洗车店离 30 米是开车去还是走路去，它真在那分析油耗。不过这种脑筋急转弯本来也不是大模型的强项。

Billy.Green

—

运行在手机上和匹配全精度模型不是一回事，这个清醒的认识很重要。用之前先想清楚你的场景是哪一类，需要 agent 能力就选 ternary 5.9GB 版。

海浪_21

—

最让我惊讶的是它的智慧密度概念。54GB 压到 3.9GB，保留 90% 的能力，这在数学和编程任务上确实做到了。视觉能力掉得有点多，MMMU 才 59.6，看图还是差点。

Mason.TorresSr

—

说实话我对这个模型最大的期待是离线翻译和文档处理，出差坐飞机也能用。试了下把一份 30 页的 PDF 丢进去做总结，效果还行，就是加载慢了点。

Michelle_RussellQ

—

在 Together.ai 上试了 preview API，不用下载就能测，这点很友好。回复质量的确定位在「够用」级别，和 GPT 4.5 肯定比不了，但看在免费的份上，很香了。

Jonathan196

—

Hacker News 上讨论挺热烈的，大家最担心的还是极端量化对 agentic 能力的侵蚀。数学掉 2% 无所谓，但 tool calling 掉 17.5% 在真实场景里影响太大了。

Karen.Rodriguez007

—

我看到的观点是混合部署才合理，本地模型处理隐私敏感和轻量任务，复杂推理留到云端。Bonsai 让本地这端终于有足够强的模型了，不用所有事都上云。

Jennifer.Kelly_99

—

视觉塔需要额外下载 900MB，而且不开 thinking 模式图片识别不准。但日常看图读文档识别验证码还是挺快的，0.5 秒就出结果。

Lydia.MendozaX

—

对于搞本地 AI 的开发者来说，这是今年夏天最有意思的发布。它把手机端侧的门槛从 3-8B 的小模型推进到了 27B 级别，质的飞跃。

Arthur90

—

最大的亮点是真正离线可用，数据不出设备。对于处理敏感信息的场景，比如律师看合同、医生看病例，这个价值比 benchmark 分数重要得多。

Matthew.MurphyK

—

用 Locally AI 这个 App 直接就能在 iPhone 上跑，不需要折腾环境配置。不过下载模型最好用 WiFi，13GB 的数据量用流量扛不住。

KathleenRoberts_Plus1

—

刚下载就翻车了，没有 Hugging Face 的 token 认证的话 27B 的仓库下载不了，要先申请访问权限。还好小的模型可以直接下，建议先试试 8B 版本。

GregoryNelson_66

—

好奇它和常规 4-bit 量化的本质区别，看了技术文档才发现是贯穿到所有层的低比特设计，不留高精度逃生舱。这也是它能压到 3.9GB 的原因。

DPrice007

—

从 54GB 到 3.9GB，14 倍的压缩比，还能保持 90% 的智力保留，不管怎么说都是技术奇迹。微软苹果谷歌都该紧张一下了。

NWrightIII

—

17.5% 的工具调用精度损失在多步场景里会被指数放大，每一步 95% 成功率，跑 10 步就剩 60% 了。生产环境用的话还是得配云端兜底。

Bonsai 27B

In-depth Report

User Reviews

Similar Products