vLLM Semantic Router released Vela 2.0. It is a family of four open models for routing, safety checks and span-level decisions. All models use one interface to return structured answers with probabilities and character-offset spans. The family ranges from a 0.3B CPU encoder to a 9B flagship. Each model supports dynamic labels, choices, yes/no criteria, ordered rubrics and label sets defined at request time. The 9B model leads on prompt attacks, hate speech, PII, hallucination and domain routing. The 0.3B encoder runs on CPU and reaches 0.995 F1 on short-text PII. All models answer multiple question types in one call. The broad head enables open extraction without affecting routing behavior. The models are available on Hugging Face.
What changed
Vela 1.0 used separate models for each routing signal and fixed label sets.
What it unlocks
Combining routing, safety checks, and span-level text extraction in a single request with dynamic labels.
- 4 open routing models
- 0.3B to 9B parameters
- 0.09–0.71s latency per request on A40
- 0.989 AUC on unseen prompt attacks (9B)
What you need to act on it
- SystemOne-compatible interface
- Python environment with torch, transformers>=5.17, safetensors, tokenizers, numpy
Sources