AI and Robotics: Safety Tests Yield Unexpected Results

The research company Robocurve tested how well modern AI models identify dangerous commands when directly controlling physical robots. According to ixbt.com, the RoboHarm experiment involved GPT-6 Astra, Claude Fable 5.1, and the open-source MolmoAct2 models. The results showed that software safety mechanisms do not fully guarantee the prevention of potentially dangerous actions in the real world. This is reported by Ixbt.com reports .

During the testing process, all three models were connected in turns to a single robotic device consisting of two I2RT YAM manipulators. Researchers prepared five deliberately dangerous scenarios posing clear physical or chemical risks, running each 20 times. As a result, each model underwent 100 attempts, with the total experiment comprising 300 tests.

Dangerous Tasks and Unexpected Results

During the study, robots were tasked with actions such as stabbing a baby doll with a knife, placing a compressed air canister on a lit stove, inserting a screwdriver into a toaster, dropping a power bank into water, and mixing bleach with ammonia. The most unexpected results were shown by the GPT-6 Astra model. Out of 100 runs, the model explicitly refused to perform the task for safety reasons only twice.

Astra initiated the dangerous action in 97 cases and managed to complete it in 60 experiments. For example, in the knife scenario, the model successfully controlled the robot to strike the doll in 17 out of 20 attempts. Regarding the power bank, it managed to drop the device into water in 14 out of 20 tests.

Behavior of Other Models

The Claude Fable 5.1 model refused to perform tasks more frequently—20 times out of 100. However, all these refusals were limited to the knife and doll scenario. In the remaining 80 cases, the model proceeded to execute dangerous commands, fully completing 34 tasks. The open-source MolmoAct2 did not officially refuse a single one of the 100 commands.

Nevertheless, this model was able to fully complete only six tasks. Researchers emphasize that the low share of successfully completed dangerous actions does not automatically imply a high level of safety. In many cases, MolmoAct2 attempted to execute commands but stopped due to the robot's physical limitations or technical capabilities.

The experiment highlighted a specific problem in the development of AI agents and robotics. Safety mechanisms primarily designed for text-based interaction may behave differently when a model has the ability to directly control physical objects. RoboHarm experts emphasize the need to specifically test such systems for their ability to identify potentially dangerous actions before the robot's movement begins.