What ternary models actually are

Ternary large language models store every weight as just one of three values: negative one, zero, or positive one. This extreme simplification makes models tiny and fast to run.

The old storage math

The theoretical minimum is 1.585 bits per weight. Current packing methods round this up to 1.625 bits per weight by treating all three symbols as equally common.

The discovery that changed everything

Researchers measured 29 real ternary models and found zeros make up to 51.5 percent of all weights. The old method assumed equal distribution, wasting space.

“zeros account for up to 51.5% of all weights”

Introducing BITCOS

The new method uses a presence bitmap to mark where zeros are, then stores only the non-zero weights and their signs. It costs two minus z bits per weight, where z is the zero density.

Real-world results

BITCOS beats five-trit packing in 26 of 29 models tested, reaching 1.485 bits per weight on the sparsest model. That breaks through the 1.58-bit barrier.

“reaches 1.485 bits per weight on the sparsest of them”

Speed gains across hardware

Matrix multiplication runs up to 1.28 times faster on CPUs and 1.27 times faster on GPUs. Full inference throughput improves by up to 1.18 times on CPUs and 1.27 times on GPUs.

Why this matters

Smaller, faster models mean AI can run on cheaper hardware and use less power. This makes advanced language models accessible to more people and devices.