The idea of creating a 3D Head-up Display (3D-HUD) to create an augmented reality windshield started with the need for an automobile to communicate with its driver using a modality other than audio and haptics. Audio beeps and alarms, like haptic vibrations, lack specificity on what the source of a problem could be. Speech interfaces, while being very informative and symbolic in nature, do not have the bandwidth to commuicate rapidly changing data, especially in a fast, moving vehicle. Driving is also a highly spatio-temporal task, and the visual modality could communicate these concepts and phenomena in an unambiguous and direct manner. We wanted to create entire new categories of driving experiences and solutions to address emerging challenges for drivers. In particular, we sought answers to the following questions. How can an autonomous vehicle communicate its intentions to its passengers? How can we improve situation awareness for drivers to help prevent accidents in a stress-free manner? How can we safely enrich a driver’s access to new sources of location-based information while driving?