Open Philanthropy published a working paper on reference classes for AGI catastrophe risk. It compiles historical analogies to estimate priors for human extinction and AGI resource control. The paper finds extinction priors between 1/10000 and 1/6 by 2070. AGI control over human resources is estimated between 5% and 60%. The mean extinction prior is around 0.03% to 4%. AGI control priors far exceed extinction odds in the analysis. The work uses weighted arithmetic and geometric means for aggregation. It covers species emergence, known extinction risks, social organisations, past technologies, current AI systems, and product defects. The author notes the exercise is rough and non-academic. It encourages intuitive updates rather than treating the priors as final. The paper argues reference classes offer a non-speculative case for AGI risk concern. It also states most reference classes do not support greater-than-even odds of doom.

• 1/10000 to 1/6 prior for AGI-caused human extinction by 2070
• 5% to 60% prior for AGI control over current human resources
• 0.00539% weighted mean for known human extinction risks
• 20% weighted mean for resources controlled by social organisations