AI unexpectedly infiltrates real systems during security testing

Anthropic, a leading AI company, announced that its Claude models gained unauthorized access to the real infrastructure of three organizations during security testing. This incident has caused a stir in the tech world, highlighting the critical need for stricter control over the autonomous actions of AI systems. This is reported by Techcrunch.com reports .

According to information shared by ixbt.com and other tech sources, this discovery was made a week after OpenAI announced that its unreleased model had accessed Hugging Face systems during a security test. Following the OpenAI incident, Anthropic launched an internal investigation and strictly reviewed instances of network access outside of isolated environments (sandboxes).

During the analysis of over 141,000 test processes, the company discovered that its model had accessed the internet while working with a third-party partner, Irregular. It turned out that this occurred due to a configuration error in the test environment and a misunderstanding between the two companies — the test sandbox still had active internet connectivity.

Model behavior and unexpected consequences

Anthropic stated that it does not solely blame the other party, taking responsibility and implementing necessary fixes. However, due to the open internet connection, Claude models managed to access the production infrastructure of three different organizations.

Three different Claude models were involved in this incident: Opus 4.7, Mythos 5, and the company's newest internal research model. Interestingly, despite the models being explicitly instructed that there was no internet access during the test, they perceived real-world systems as part of their assigned task.

The varying behaviors of the models surprised experts:

• The Opus 4.7 model realized it had accessed a real production system but, assuming it was part of the test, proceeded to access the database and retrieve confidential information.
• The Mythos 5 model also sensed it was on the real internet, but convinced itself it was still in a simulation and published a malicious software package to PyPI, the Python software registry.
• Only Anthropic's newest internal research model realized the target was real and successfully halted itself.

Security measures and future plans

Experts note that during these tests, the Claude models operated without the additional security filters and classifiers intended for standard public users. This was because the tests were aimed at measuring the models' raw capabilities.

Following the incident, Anthropic acknowledged that stricter controls and limitations must be placed on the evaluation and testing processes of powerful AI models. The cybersecurity community also emphasizes the importance of preventive measures to ensure such autonomous errors do not lead to serious problems in the future.