Researchers measured symbol distribution in 29 ternary LLM models. Zeros accounted for up to 51.5% of all weights. They introduced BITCOS, a layout using a presence bitmap and a sign vector. BITCOS costs 2 minus zero density bits per weight. It beat five-trit packing in 26 of 29 tested models. It reached 1.485 bits per weight on the sparsest model. Optimized unpacking sequences were presented for AVX-512, AVX2 and Intel Xe2 GPUs. Matrix-vector multiplication kernels ran up to 1.28× faster. End-to-end inference showed up to 1.18× CPU and 1.27× GPU decode throughput gains.

• 51.5% of weights are zero
• 1.485 bits per weight on sparsest model
• 1.28× faster ternary matrix-vector multiplication
• 1.18× CPU decode throughput gain