Netflix's AI Platform Model Runtime and Inference teams published a detailed account of the in-house LLM serving stack they run inside their existing production environment rather than a separate ML silo. Large models are served by the Model Scoring Service (MSS), which sits on NVIDIA Triton Inference Server with a Java control plane for deployment, versioning, autoscaling and multi-region rollout; the platform migrated from TensorRT-LLM to vLLM as its paved-path engine by summer 2025, citing custom-architecture loading without a compilation step, decoding extensibility hooks and debuggability. An OpenAI-compatible HTTP frontend runs alongside gRPC, reusing NVIDIA's Triton OpenAI frontend, which the team patched after finding response_format was silently dropped before reaching vLLM's guided decoding.