New developments in advanced driver assistance systems (ADAS) can help drivers deal with risky driving maneuvers, preventing potential hazard scenarios. A key challenge in these systems is to determine when to intervene. While there are situations where the needs for intervention or feedback is clear (e.g., lane departure), it is often difficult to determine scenarios that deviate from normal driving conditions. These scenarios can appear due to errors by the drivers, presence of pedestrian or bicycles, or maneuvers from other vehicles. We formulate this problem as a driving anomaly detection, where the goal is to automatically identify cases that require intervention. Towards addressing this challenging but important goal, we propose a multimodal system that considers (1) physiological signals from the driver, and (2) vehicle information obtained from the controller area network (CAN) bus sensor. The system relies on conditional generative adversarial networks (GAN) where the models are constrained by the signals previously observed. The difference of the scores in the discriminator between the predicted and actual signals is used as a metric for detecting driving anomalies. We collected and annotated a novel dataset for driving anomaly detection tasks, which is used to validate our proposed models. We present the analysis of the results, and perceptual evaluations which demonstrate the discriminative power of this unsupervised approach for detecting driving anomalies

Driving anomaly detection is an important problem in advanced driver assistance systems (ADAS). The ability to immediately detect potentially hazardous scenarios will prevent accidents by allowing enough time to react. Toward this goal, our previous work proposed an unsupervised driving anomaly detection system using conditional generative adversarial network (GAN), which was built with physiological data and features extracted from the controller area network-Bus (CAN-Bus). The approach generates predictions for the upcoming driving recordings, constrained by the previously observed signals. These predictions were contrasted with actual physiological and CAN-Bus signals by subtracting the corresponding activation outputs from the discriminator. Instead, this study proposes to use a triplet-loss function to contrast the predicted and actual signals. The triplet-loss function creates an unsupervised framework that rewards predictions closer to the actual signals, and penalizes predictions deviating from the expected signals. This approach maximizes the discriminative power of feature embeddings to detect anomalies, leading to measurable improvements over the results observed by our previous approach. The study is implemented and evaluated with recordings from the driving anomaly dataset (DAD), which includes 250 hours of naturalistic data manually annotated with driving events. Objective and subjective metrics validate the benefits of using the proposed triplet-loss function for driving anomaly detection.

As a driver prepares to complete a maneuver, his/her internal cognitive state triggers physiological responses that are manifested, for example, in changes in heart rate (HR), breath rate (BR), and electrodermal activity (EDA). This process opens opportunities to understand driving events by observing the physiological data of the driver. In particular, this work studies the relation between driver maneuvers and physiological signals during naturalistic driving recordings. It presents both feature and discriminant analysis to investigate how physiological data can signal driver's responses for planning, preparation, and execution of driving maneuvers. We study recordings with extreme values in the physiological data (high and low values in HR, BR, and EDA). The analysis indicates that most of these events are associated with driving events. We evaluate the values obtained from physiological signals as the driver complete specific maneuvers. We observe deviations from typical physiological responses during normal driving recordings that are statistically significant. These results are validated with binary classification problems, where the task is to recognize between a driving maneuver and a normal driving condition (e.g., left turn versus normal). The average F1-score of these classifiers is 72.8%, demonstrating the discriminative power of features extracted from physiological signals.

Corner cases are the main bottlenecks when applying Artificial Intelligence (AI) systems to safety-critical applications. An AI system should be intelligent enough to detect such situations so that system developers can prepare for subsequent planning. In this paper, we propose semi-supervised anomaly detection considering the imbalance of normal situations. In particular, driving data consists of multiple positive/normal situations (eg, right turn, going straight), some of which (eg, U-turn) could be as rare as anomalous situations. Existing machine learning based anomaly detection approaches do not fare sufficiently well when applied to such imbalanced data. In this paper, we present a novel multi-task learning based approach that leverages domain-knowledge (maneuver labels) for anomaly detection in driving data. We evaluate the proposed approach both quantitatively and qualitatively on 150 hours of real-world driving data and show improved performance over baseline approaches.