TL;DR
- What was announced: Prism ML Inc. launched Bonsai 2 27B, a second-generation ultra-compact multimodal AI model designed to run on personal computers and select high-end mobile devices.
- How it works: By applying ternary compression (+1, 0, and -1 weights), PrismML shrank a 56GB base model down to 5.9GB while preserving roughly 98.2% of its performance.
- Hardware execution: The model achieves 143 tokens per second on an Nvidia GeForce GTX 5090 without quantization and 46.8 tokens per second on Apple's M5 Max chip.
- Availability: The model weights have been released under an open-source Apache 2.0 license.
Prism ML Inc. has officially launched Bonsai 2 27B, marking the second generation of the company's ultra-compact multimodal generative artificial intelligence. Engineered specifically to bridge the gap between large-scale intelligence and consumer-grade computing constraints, the system is designed to run entirely on personal computers and select high-end mobile devices without relying on external cloud data centers.
Understanding Ternary Compression: Shrinking AI Without Sacrificing Smarts
Bonsai 2 27B is built upon the Qwen3.8 27B base architecture. In its standard, uncompressed 16-bit configuration, Qwen3.8 has a storage and memory footprint of approximately 56 gigabytes. While conventional model compression typically relies on quantization techniques, standard quantization often leads to noticeable degradation in accuracy, knowledge retention, and tool-handling capabilities. Even under standard minimal quantization, Qwen3.8 requires a memory footprint of roughly 9.4 gigabytes.
To overcome these trade-offs, PrismML utilized a ternary system, which relies on three distinct values. In neural networks, parameters or "weights" function like numerical dials that dictate how the model analyzes input tokens and calculates outputs. Whereas traditional full-size models store these parameters using 16 bits per value, PrismML's compression simplifies parameter representations into ternary states: +1, 0, and -1.
This mathematical simplification allows the model to compress down to approximately 5.9 gigabytes. According to company disclosures, Bonsai 2 retains roughly 98.2% of the underlying model's original capabilities, enabling it to deliver high-tier intelligence within an exceptionally small memory envelope.
Benchmark Evaluations: Competitive Agentic and Coding Performance
Benchmark evaluations released with the announcement show that Bonsai 2 27B performs in close proximity to the uncompressed Qwen3.8 baseline across core evaluation suites:
- Agentic and Tool Calling: Bonsai 2 scored 77.6, finishing within three points of Qwen3.8's 79.8.
- Coding Capabilities: Across an aggregate suite comprising HumanEval+, LiveCodeBench v6, MBPP+, and BigCodeBench, Bonsai 2 registered an aggregate score of 81.6, compared to 82.2 for Qwen3.8.
- Knowledge and Reasoning: Across an aggregate benchmark of MMLU-Redux, GPQA Diamond, and AA-LCR, Bonsai 2 achieved a score of 82.7, slightly outperforming Qwen3.8's 81.3.
Hardware Speeds and Energy Efficiency
Bonsai 2 is engineered to run locally across major consumer chip architectures using low-bit kernels. On desktop PC hardware, the model operates on Nvidia graphics processing units via CUDA, reaching 143 tokens per second on an Nvidia GeForce GTX 5090 without requiring secondary quantization. For Apple devices, including Mac computers, iPhones, and iPads, the model runs through Apple's MLX machine learning framework, reaching 46.8 tokens per second on an Apple M5 Max chip.
PrismML also highlighted power metrics, reporting that Bonsai 2 consumes 0.714 megawatt-hours per token. According to the company, this rate makes the model 40% more energy-efficient than typical 8-billion-parameter (8B) models executing at full 16-bit uncompressed precision.
Why Local On-Device AI Matters for Privacy and Cost
Executing artificial intelligence workloads locally on user devices addresses several core limitations inherent to cloud-hosted APIs. Sending prompts across the internet introduces network latency and poses potential privacy risks when proprietary or regulated data is shared with third-party providers. By keeping inference on-device, organizations and individuals can safeguard sensitive records, meet stringent regulatory compliance requirements, and strengthen overall operational security.
From an architecture perspective, running compact on-device models allows enterprises and everyday users to implement hybrid AI strategies. Routine tasks—such as text translation, summarization, and query organization—can be completed on local hardware at near-zero incremental operational cost. Meanwhile, resource-intensive, long-horizon comprehension and deep analytical tasks can still be escalated to cloud-based clusters when necessary.
Licensing and Open Availability
PrismML has released the model weights for Bonsai 2 27B publicly on Hugging Face. The release is published under the open-source Apache 2.0 license, allowing developers, researchers, and enterprises to examine, deploy, and build applications upon the lightweight model.
Frequently Asked Questions
- What is Bonsai 2 27B?
- Bonsai 2 27B is an ultra-compact multimodal generative AI model developed by Prism ML Inc., based on the Qwen3.8 27B architecture and scaled down to operate on consumer PCs and mobile hardware.
- How does ternary compression work in Bonsai 2?
- Rather than storing model weights in standard 16-bit formats, PrismML simplifies parameter representations into three discrete values (+1, 0, and -1). This slashes the model size from 56GB to 5.9GB while maintaining roughly 98.2% of its capabilities.
- Which devices and platforms can run Bonsai 2 27B?
- The model runs on Nvidia GPUs via CUDA (reaching 143 tokens per second on a GeForce GTX 5090) and on Apple devices including Macs, iPhones, and iPads via MLX (reaching 46.8 tokens per second on an M5 Max chip).
- Under what license is Bonsai 2 27B distributed?
- PrismML has published the model weights on Hugging Face under the permissive Apache 2.0 open-source license.
0 Comments