The H3D is a large scale full-surround 3D multi-object detection and tracking dataset. It is gathered from HDD dataset, a large scale naturalistic driving dataset collected in San Francisco Bay Area. H3D consists of following features:

  • Full 360 degree LiDAR dataset (dense pointcloud from Velodyne-64)
  • 160 crowded and highly interactive traffic scenes
  • 1,071,302 3D bounding box labels
  • 8 common classes of traffic participants (Manually annotated every 2Hz and linearly
  • propagated for 10 Hz data)
  • Benchmarked on state-of-the art algorithms for 3D only detection and tracking algorithms.


Data Format


scenario_xxx: sequence of sensor data
	----CAN_* (Decoded CAN DATA):
	|		|
	|		---CAN_steer_yyy.csv: steer_angle,steer_speed
	|		|
	|		---CAN_vel_yyy.csv: speed
	----gps_*(GPS+IMU DATA):
	|		|
	|		---gps_yyy.csv: Long_Rel,Lat_Rel,In_Height,Tilt_Roll,Tilt_Pitch,Tilt_Yaw,Vel_x,Vel_y,Vel_z,Std_Dev_x,Std_Dev_y,Std_Dev_z,Std_Dev_roll,Std_Dev_pitch,Std_Dev_yaw,Std_Dev_vel_x,Std_Dev_vel_y,Std_Dev_vel_z,Abs_Lat,Abs_Long
	----labels_*(labels) [c: center, l: length]:
	|		|						
	|		|
	|		---labels_3d1_yyy.txt: Full 360 deg pointcloud (label, trackerID, state[static/dynamic], c_x, c_y, c_z, l_x, l_y, l_z, yaw) 3D bounding box labeled in velodyne frame
	----pointcloud* (pointcloud):
			---pointcloud1_yyy.ply: Full 360 deg pointcloud (surfel format, fields: xyz, radius->intensity, confidence->ring_number, curvature->encoder_angle)


This dataset corresponds to the paper, "The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes", as it appears in ICRA 2019. In the current release, the data is available for universities in the US. To obtain the dataset, please follow the following instructions.

Important: The following instructions must be followed exactly:

  • Please use your official university email to ychen@honda-ri.com to request the agreement.
  • You will receive a link and password to the data in your official email address after the review.

Please cite the following paper if you find the dataset useful in your work:

    author = {Abhishek Patil and Srikanth Malla and Haiming Gang and Yi-Ting Chen},
    title = {The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes},  
    booktitle = {International Conference on Robotics and Automation},
    year = {2019}