OATS Dataset

The OATS Dataset is captured from a moving vehicle on highly interactive traffic scenes in San Francisco Bay Area.


- OATS dataset uses a novel representation called Ordered Atomic Activity for interactive scenario understanding. The representation decomposes each scenario into a set of ordered atomic activities, where each activity consists of an action and the corresponding actors involved and the order denotes the temporal development of the scenario. This design also helps in identifying important interactive relationships, such as yielding. The action is a high-level semantic motion pattern that is grounded in the surrounding road topology, which we decompose into zones and corners with unique IDs.

-The OATS dataset comprises of 1026 video clips (∼ 20s) captured at intersections in San Francisco Bay Area. Each clip is labeled with the proposed language, resulting in 59 activity categories and 6512 annotated activity instances with a resolution of 1920×1200 and a frame rate of 10 fps.


Dataset Statistics

Dataset Format



This dataset corresponds to the paper, "Ordered Atomic Activity for Fine-grained Interactive Traffic Scenario Understanding", as it appears in the proceedings of International Conference on Computer Vision (ICCV) 2023". In the current release, the data is available for researchers from universities. 

Use this link to make the download request.


This work has been accepted at ICCV 2023. Please cite this work if you use this dataset.