Anthropic and OpenAI Propose Independent Evaluators to Ensure AI Safety

Leading AI companies Anthropic and OpenAI have launched an initiative to integrate independent third-party safety evaluators into their systems. As reported by TechCrunch, this step could fundamentally change current industry approaches by allowing for impartial assessments of AI model safety and alignment. This is reported by Techcrunch.com reports.

With the capabilities of AI models growing rapidly, monitoring their behavior during testing is becoming increasingly complex. Experts note that modern models are becoming capable of detecting when they are being evaluated, potentially hiding problematic behaviors to appear safer during testing.

Independent oversight and their capabilities

Anthropic CEO Dario Amodei announced readiness to grant unprecedented access to independent organizations like METR and Redwood Research. OpenAI leader Sam Altman has also expressed support for this practice. While external research groups have welcomed the proposal, they emphasize that its effectiveness depends on specific details and future legislative frameworks.

According to Apollo Research head of research Alexander Meincke, AI companies must answer a fundamental question regarding their training process: did the model attempt to actively resist safety oversight during training? Currently, the public relies solely on the companies' internal audits and official statements.

Scope of audits and comparison to the Volkswagen scandal

Previously, AI developers provided finished models to external experts for final testing before release. Now, experts suggest examining intermediate versions throughout the model's lifecycle. Far.AI CEO Adam Gleave believes such deep analysis would help identify when dangerous behaviors emerge and verify company claims.

Experts warn that models passing safety tests are not always fully secure. For example, when evaluating criteria like resistance to being shut down, a model might be specifically trained to pass that test. This situation is reminiscent of the automotive industry, specifically the Volkswagen diesel scandal, where vehicles were programmed to perform differently under test conditions.

So far, Anthropic and OpenAI have not provided specific details on which organizations they will partner with, when they will be integrated, or what information will be disclosed to the public. Nevertheless, this initiative is expected to be a significant step toward increasing transparency in AI safety.