Waymo, a leading AV company, recognizes the challenge of creating and training intelligent agents for it’s autonomous vehicle (AV) program. Simulators are used by companies to train their self-driving systems and teach them how to react to various elements such as pedestrians, cyclists, traffic signals, and other vehicles.
However, to create an advanced AV system, these simulated agents need to behave and react realistically to the AVs and to each other.
Recently, Waymo introduces advanced simulators called Waymax. This simulator is designed to provide an environment for training intelligent agents, complete with prebuilt sim agents and a wealth of Waymo perception data.
Traditional simulators:
Traditional simulators often have predefined agents, but their actual behavior may differ. Waymo’s simulator pairs with a large dataset gathered from its vehicles observing real-world behaviors. By learning from these observations, Waymo aims to develop robust and scalable AV systems. This approach is referred to as a stronger imitative component.
The Waymax simulator is described as “lightweight,” allowing researchers to iterate quickly. It focuses more on complex behaviors among multiple road users rather than on detailed visual representations of agents and roads. Instead, the simulator provides a simplified environment that facilitates the study of agent behavior.
Simulated Agents Challenge:
Although Waymo cannot view the work created by researchers using Waymax, the company benefits from sharing its tools and data. It regularly hosts challenges, such as the “Simulated Agents” challenge in 2022. Here, researchers train agents within the simulator to behave realistically. To enhance this challenge, Waymo partnered with Google Research to develop a more suitable environment that allows continuous monitoring and adjustments to agent behavior.
As Waymo introduces advanced simulators, their ultimate goal is to rerun the challenge in the future using the Waymax simulator to gauge the progress of the AV industry and compare its technology with others. Additionally, Waymo aims to influence academic and research discussions by providing the Waymo Open Dataset and simulators like Waymax. These challenges also help attract talent and attention to the field of AV and robotics research.
Furthermore, Waymax has the potential to unlock improvements in reinforcement learning, which can result in emergent behavior within AV systems. Reinforcement learning is a machine learning technique where agents learn to make decisions by interacting with the environment and receiving rewards or penalties. By simulating scenarios, agents can learn behaviors that humans might not naturally exhibit. For example, AVs can exhibit different types of lane changes or drive consistently if they recognize each other as AVs. This emergent behavior can contribute to safer autonomous driving.