What happens when you take a “frontier” (read: general-purpose) AI model like Claude or GPT and ask it to get behind the wheel and drive a car? Sure, some self-driving systems utilize artificial intelligence in one sense or another, but they’re specifically built for the task. Researchers wanted to see how well your typical AI agent handles the task of driving, via Comma vision hardware and OpenPilot. The results surely won’t leave anyone racing to replace themselves in the driver’s seat anytime soon, but they are eye-opening.

 

Aditya Ramabadran, Simon Mahns, and Tobias Gessler created DrivingBench: A simple test for Claude, GPT, and Grok models to navigate a short point-to-point course around a parking lot with a few curves, outlined by small cones. Each agent was issued the same command, with the same guidelines and directions to share what they’re seeing and doing after every step, as well as full control of a Toyota Corolla. They regularly stop along the way, because the commands are being sent away to data farms over a wireless hotspot, and all of this happens within one continuous chat. They have three attempts to get it right.