Can AI create robots? A new test is being trialed

Scientists from Harvard University and the Massachusetts Institute of Technology have introduced an open-source benchmark called RLE-Bench, designed to evaluate the ability of AI agents to perform engineering tasks. According to ixbt.com, this platform is intended to test the potential of modern neural networks not only to learn control algorithms but also to design entire robotics systems, marking a significant step in the field's progress. This is reported by Ixbt.com reports.

While many existing tests currently focus primarily on training ready-made control algorithms, RLE-Bench covers the entire cycle. It simultaneously evaluates software, data processing from sensors, hardware components, and interface design.

Complex tasks in four key areas

The new test suite includes 48 tasks across four key areas. The AI agent is required to have a deep understanding of how software code interacts with sensors, motors, and mechanical assemblies. It must also develop perception algorithms, analyze sensor data, and eventually design robotic elements itself.

All tasks are performed in a specialized simulation environment. The AI agent independently writes code, executes the solution, performs deep analysis of the results, and makes corrections when necessary. The tasks are designed taking into account real engineering constraints, such as weight, torque, balance, and structural geometry.

Real engineering constraints and the testing process

In practice, the design process is associated with serious challenges. For instance, in one task, the AI must design a mobile platform capable of supporting multiple manipulators that retrieve items from shelves of varying heights. While a design might look perfect in a digital environment, it could be unstable and tip over in real life.

For this reason, researchers led by Harvard professor Na Li and MIT associate professor Bo Dai view this version as an initial step. There are plans to further expand the RLE-Bench 2.0 suite based on real engineering problems faced by roboticists.

Experts note that the new test evaluates the engineering reasoning of AI rather than the performance quality of finished robots. Therefore, it cannot be directly compared to physical machine competitions like the 'World Humanoid Robot Games' held in China: in the first case, the mental capabilities of AI are tested, while in the second, the performance of finished hardware in physical conditions is examined.