Driving anomaly detection is an important problem in advanced driver assistance systems (ADAS). The ability to immediately detect potentially hazardous scenarios will prevent accidents by allowing enough time to react. Toward this goal, our previous work proposed an unsupervised driving anomaly detection system using conditional generative adversarial network (GAN), which was built with physiological data and features extracted from the controller area network-Bus (CAN-Bus). The approach generates predictions for the upcoming driving recordings, constrained by the previously observed signals. These predictions were contrasted with actual physiological and CAN-Bus signals by subtracting the corresponding activation outputs from the discriminator. Instead, this study proposes to use a triplet-loss function to contrast the predicted and actual signals. The triplet-loss function creates an unsupervised framework that rewards predictions closer to the actual signals, and penalizes predictions deviating from the expected signals. This approach maximizes the discriminative power of feature embeddings to detect anomalies, leading to measurable improvements over the results observed by our previous approach. The study is implemented and evaluated with recordings from the driving anomaly dataset (DAD), which includes 250 hours of naturalistic data manually annotated with driving events. Objective and subjective metrics validate the benefits of using the proposed triplet-loss function for driving anomaly detection.