One Thousand and One Hours: Self-driving Motion Prediction Dataset

John Houston, Guido Zuidhof, Luca Bergamini, Yawei Ye, Long Chen, Ashesh Jain, Sammy Omari, Vladimir Iglovikov, Peter Ondruska

CoRL 2020


Dataset for self-driving prediction and planning

This dataset includes the logs of movement of cars, cyclists, pedestrians, and other traffic agents encountered by our autonomous fleet. These logs come from processing raw lidar, camera, and radar data through our team’s perception systems and are ideal for training motion prediction models. The dataset includes:

What is included

The dataset consists of 170,000 scenes capturing the environment around the autonomous vehicle. Each scene encodes the state of the vehicle’s surroundings at a given point in time.


hours of traffic agent movement.


miles of data from 23 vehicles.


semantic map annotations

ML prediction baseline

ML planning baseline



title={One thousand and one hours: Self-driving motion prediction dataset},

author={Houston, John and Zuidhof, Guido and Bergamini, Luca and Ye, Yawei and Chen, Long and Jain, Ashesh and Omari, Sammy and Iglovikov, Vladimir and Ondruska, Peter},

journal={arXiv preprint arXiv:2006.14480},