GitHub shipped a context-aware secret classifier for its scanning service. It uses a two-stage pipeline with regex and a fine-tuned ModernBERT model. The model reads candidate strings with surrounding code and text. It improves owner-weighted acceptance by 11.5%. It cuts median post-push scan time by 86%. It reduces serving cost by over 98% versus the prior LLM pipeline. The classifier now also runs during push protection. It targets hard negatives like test fixtures and documentation placeholders. GitHub trained it on real, synthetic and regex-derived examples. The model is exported in ONNX with 8-bit quantization for fast inference.
What changed
Secret scanning used an LLM-based pipeline for post-push detection only.
What it unlocks
Running generic secret detection during push protection without slowing commits.
- 11.5% higher owner-weighted acceptance
- 86% lower median scan execution time
- >98% lower serving cost
Sources