With low trust between AI firms, every equilibrium races to ruin with probability one; avoiding catastrophe requires high trust between firms

With low trust, every equilibrium races to ruin: the disaster arrives with probability one. With intermediate trust, immediate stopping and racing to ruin are both equilibria. With high trust, in every equilibrium, the probability that two rational firms race forever vanishes quadratically in the prior odds ratio of rationality

https://importai.substack.com/p/import-ai-468-23-rsi-ideas-posttrainbench