Labs Chase Self-Improving AI. The Results Remain Modest. The Ambition Does Not.

Major AI labs including OpenAI, xAI, and Microsoft are pursuing what researchers call recursive self-improvement, where AI systems contribute to designing their own successor models. Researchers like OpenAI co-founder Andrej Karpathy have experimented with this concept for years, producing only minor improvements rather than significant creative leaps, according to Thickstun. However, Aguirre indicates that current AI companies are now considerably closer to achieving those larger leaps. The labs describe their approach as automated but supervised, though the prospect intensifies concerns about superintelligence exceeding human control.

The mental model here is the feedback loop. Recursive self-improvement is not magic. It is a system whose output becomes input for its own next iteration. The principle governs everything from compound interest to viral epidemics. When improvement compounds, early gains look trivial and late gains look explosive. The labs have been stuck in the trivial phase. Aguirre's suggestion is that they are approaching the inflection point where the curve steepens. Whether supervised control survives that steepening is the open question nobody can answer.

OpenAI, xAI, and Microsoft have unveiled competing visions for automated AI development. Andrej Karpathy, OpenAI co-founder, has conducted prior experiments in this area. Thickstun and Aguirre provided assessments of progress, with Aguirre indicating proximity to larger breakthroughs.