From TechCrunch · Julie Bort · · 2m
PrismML's tiny AI models could reshape how we use AI
A Caltech startup is compressing powerful AI models down to smartphone size without losing performance.
In brief
A Caltech startup is compressing powerful AI models down to smartphone size without losing performance. PrismML has figured out how to shrink advanced AI models to a tenth their original size while keeping 98% of their performance, making it possible to run them locally on your device instead of in the cloud…
Startup shrinks AI models to fit phones
PrismML released Bonsai 2, compressing Alibaba's Qwen model from 27 billion parameters down to 5.9 GB. That's a 9 to 10 times reduction, small enough for PCs and high-end smartphones.
Performance stays nearly identical
Bonsai 2 matches 98% of the original model's benchmark scores, up from 95% in the first version released in March. The original has been downloaded over 11 million times.
“Bonsai 2 matches 98% of Qwen's aggregate benchmark scores.”
How the compression actually works
PrismML uses ternary weights, reducing each weight from 16 bits to just three values: +1, -1, or 0. This dramatically shrinks the model's size while preserving its reasoning abilities.
Team includes Caltech researchers
PrismML was founded by Caltech researchers and is led by Babak Hassibi, a Caltech professor and compression expert. Ion Stoica, co-founder of Databricks and director of Berkeley's Sky Computing Lab, advises the company.
Bigger models compress even better
Hassibi says larger models are easier to compress without losing intelligence. PrismML plans to release several-hundred-billion-parameter models in the coming months.
“As model size grows, there is more room to be able to compress them without losing the intelligence.”
Local AI means privacy and speed
Running models on your device means no cloud uploads, lower latency, and free operation since it uses hardware you already own. This shifts AI from cloud-dependent to truly personal.
“You are going to have intelligence at your fingertips, and it's going to be free because it's going to run on the device you already bought.”
Backed by top venture investors
PrismML raised a 22.25 million dollar seed round from Khosla Ventures, Cerberus Capital, and Caltech. The company is rumored to be in talks with Apple.
Not alone in compression race
Other companies like Multiverse Computing are also working on LLM compression. But Hassibi says PrismML's approach is unique because it loses virtually no performance compared to originals.
The bottom line
PrismML has figured out how to shrink advanced AI models to a tenth their original size while keeping 98% of their performance, making it possible to run them locally on your device instead of in the cloud.
