Anthropic Unveils an AI Model That Trains Itself

The attention of emerging AI labs is increasingly turning toward an ambitious idea: training AI models using other AI models. One researcher within Anthropic's fellowship program has offered an early practical glimpse of how this idea can take shape in reality, in a development that could redraw the contours of scientific research in this field.

Research That Opens a New Door

Anthropic published a research paper titled "Automated Researchers Can Reliably Address Alignment Failures," explaining how AI systems can reliably improve model performance across a set of alignment benchmarks. When presented with ten benchmarks measuring specific misaligned behaviors, these systems managed to improve performance on each one without negatively affecting the model's overall performance.

This work is led by researcher "Chen Yueh-Han" within the company's fellowship program, and the proposed system largely mirrors the traditional methodology of scientific research.

How Does the Automated System Work?

Each automated system follows structured steps resembling what a human researcher does:

• Searching the available scientific literature on the problem at hand.
• Proposing a methodology or approach to address it.
• Training the model using this approach for thirty minutes.
• Gradually raising performance levels through multiple rounds of experiments.

Effective approaches are retained while unproductive ones are discarded, allowing the system to operate at great speed and on a large scale. The paper notes that these results "provide early evidence that automated alignment post-training may become practical in the near term."

A Step Toward Recursive Self-Improvement

This research represents a step toward what is known as "recursive self-improvement," which many experts consider the next major phase in the evolution of AI. If models can improve their own alignment training, it follows logically that they could improve training practices more broadly, which over time could reduce the need for human researchers.

The paper did not shy away from confronting this idea directly, comparing the "automated alignment researcher" with its human counterpart head-on, affirming that the best methodology the automated system arrived at, on average, outperformed what seasoned experts proposed over six hours, and that human research guidance did not lead to stronger performance.

The Difference in Cost

The research did not limit itself to comparing performance but also addressed the economic aspect. According to the paper, running the automated researcher costs about four dollars per hour using APIs, while the company pays its human researchers nearly one hundred and fifty dollars per hour. This enormous difference reflects the appeal of automation from the perspective of both cost and efficiency.

Limits That Cannot Be Ignored

Despite these striking results, the paper did not neglect to point out the limitations of the proposed approach, most notably:

• The automated system only works effectively to the extent that the adopted benchmarks reflect genuine alignment goals.
• Even with these benchmarks available, significant effort remains required to establish, maintain, and continuously update them.
• Automated researchers rely on the available scientific literature, which requires constant expansion and updating to ensure the quality of results.

This research remains an important indicator of the field's direction, combining a clear ambition to achieve self-improvement of models with a realistic awareness of the challenges that prevent full reliance on automated systems in research. While this direction opens vast horizons for accelerating AI development, it simultaneously raises deep questions about the future role of humans in this rapidly accelerating ecosystem.

✦ بقلم فريق دروب أيديا