Kyutai replaced the one-step flow matching objective in Pocket TTS with drifting. This is the first autoregressive model and the first speech model trained with drifting. It matches the LSD baseline on every metric at the same inference cost. The drifting objective avoids Jacobian-vector products and most tricks from the drifting paper. The kernel temperature must be learned rather than fixed. Drifting moves iteration into training and uses a plain squared error loss. Pocket TTS generates audio as continuous latent vectors every 80 ms. The new model is available in Pocket TTS as `english_drifting_26-09`. It is a 6-layer model distilled from a 24-layer teacher with guidance baked in. On LibriSpeech test-clean it matches the default English model in WER and UTMOS.
What changed
Pocket TTS used Lagrangian self-distillation for one-step flow matching with Jacobian-vector products in the loss.
What it unlocks
Training a one-step autoregressive speech model without Jacobian-vector products or iterative sampling.
- 0.90% WER
- 4.37 UTMOS
- 6-layer drifting model
- 24-layer drifting teacher
What you need to act on it
- a continuous audio language model with latent vectors
- a transformer backbone and sampler head
- a learned kernel temperature
Sources