Anthropic filed an updated S-1 warning it may soon reach recursive self-improvement. RSI is when AI trains itself so well it triggers rapid capability growth. Anthropic says it has already missed past capability leaps during development. Its internal estimates now indicate it may imminently meet its own RSI threshold. No current model has crossed that threshold yet. Anthropic pledges to keep humans in the loop for model development. It is considering policy updates to maintain human control over frontier models.
Sources