Later tokens cost up to seven times more to generate than earlier ones. Frontier open weight models show this cost gap. Frequent compaction lets models keep short contexts. Short contexts lose little to no accuracy on long-horizon and long-input tasks. Models also keep performance on long-horizon software engineering tasks with one action per context. The paper introduces semantic state space models based on these findings. The work questions the need for long-context language models.

• 7x cost disparity for later tokens
• 31 pages
• 8 figures