Extropic introduced Z1T, a family of transformer-like models built for its Z1 probabilistic chip. The company reports over 100 times better energy efficiency than graphics processors. Z1 uses probabilistic bits with fixed, sparse wiring baked into the silicon. Z1T reshapes standard transformer parts into sparse operations that fit that wiring. Extropic splits a decoding pass between Z1 chips and companion FPGA processors. Its energy and speed figures are estimates for that FPGA setup, not measured production runs. The sparser models need about ten times more training compute to match GPT-2 quality. Extropic published a scaling law that adds connectivity as a new variable. It released open weights on Hugging Face and open training recipes on GitHub. Extropic calls this a preliminary study meant to inform future chip designs.
What changed
Extropic's public work covered probabilistic graphical models and diffusion-like models, not transformers.
What it unlocks
Training and testing sparse transformer-style models shaped for Extropic's probabilistic chip, using released weights and recipes.
- over 100x energy efficiency vs GPUs
- 269,568 pbits per chip
- 16 couplings per pbit
- ~9.5×10^19 FLOPs to match GPT-2-small
What you need to act on it
- Z1 hardware for actual energy gains
- FPGA or other co-processor for non-Z1 operations
Sources