Honda Research Institute-Advice Dataset


The Honda Research Institute-Advice Dataset (HAD) is driving data with natural language advices. The driving videos are gathered from HDD dataset, a large-scale naturalistic driving dataset collected in San Francisco Bay Area. HAD consists of following features:

  • 30 hours of driving video data with natural language advices
  • Videos with can-bus signal data
  • Advices consist of (1) goal-oriented advice (top-down signal) – to influence the vehicle in a avigation task and (2) stimulus-driven advice (bottom-up signal) – conveys some visual stimuli that the user expects their attention to be actively looked by the vehicle controller


This dataset corresponds to the paper, "Grounding Human-to-Vehicle Advice for Self-driving Vehicles", as it appears in the proceedings of CVPR 2019. In the current release, the dataset is only being made available to researchers in universities. The process to obtain the dataset is as follows:

If you are affiliated with a university, please send email to  using your official university email account.  

A "Data Sharing Agreement" will be sent to you and must be signed by a university representative (typically a university professor, not a student).

After execution of this agreement, a link and password to download the dataset will be provided.


Please cite the following paper if you find the dataset useful in your work:

        author = {Jinkyu Kim and Teruhisa Misu and Yi-Ting Chen and Ashish Tawari and John Canny},
        title = {Grounding Human-To-Vehicle Advice for Self-Driving Vehicles},
        booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
        year = {2019}