Noesion Radarnoesion.ai
Regulation2026-10-27

OpenAI will honor a shared robots.txt signal for AI training preferences. The signal lets website operators set one rule for all participating providers. It replaces the need to maintain separate rules for each crawler. OpenAI will keep its existing crawler controls in place. The change draws on work by the IETF AI Preferences working group. OpenAI aims to implement it in the coming months. It encourages other AI providers to adopt the signal.

What changed

Website operators had to set separate rules for each AI provider's crawler.

What it unlocks

Website operators can now set one AI training preference for all participating providers.

What you need to act on it

  • implementation in coming months

Sources

Send this to someone who needs it

Shares the story and its sources. Nothing about you.

What does this mean for your job?

This is the story as everyone gets it. Once a week we send you the version written for your role — what changed, why it matters for the work you actually do, and one thing to try. Free while we tune it.