Overview:
CAUS (Context-Aware Usability Scale) is a framework for evaluating the usability of multimodal interfaces while accounting for the context in which interactions take place. This project explores how touch, voice, and gesture interactions perform under changing driving conditions through a multimodal automotive dashboard prototype. The three modalities were evaluated across low- and high-traffic conditions using task completion time, error frequency, and CAUS scores to examine how context influences both interaction performance and perceived usability.
Problem Statement
Drivers increasingly interact with in-vehicle systems through multiple modalities, including touch, voice, and gesture, but each modality introduces different demands on visual attention, cognitive workload, and driving behaviour. Existing automotive HCI research has examined these modalities individually and in combination, with studies showing that modality choice can influence distraction, workload, and interaction performance. As in-vehicle interfaces become increasingly multimodal, there remains a need to understand how usability changes across different driving contexts and interaction modalities. CAUS investigates this through a multimodal automotive interface incorporating touch, voice, and gesture, examining how contextual factors such as low- and high-traffic driving environments shape interaction and usability.
Role:
UX Researcher & Interaction Designer
Team:
x1 UX Researcher & Interaction Designer
Tools:
Figma, ProtoPie Genie
Methods:
Prototyping, Usability Testing, Task Analysis, Quantitative Measures
Solution
To explore multimodal interaction in automotive environments, I designed a dashboard interface integrating touch, voice, and gesture-based interactions. The prototype was developed in two iterations, first to validate the interaction structure and task flow with the supervising professor, and subsequently to conduct participant testing across low- and high-traffic driving contexts. The evaluation examined how users interacted with each modality across navigation, media, and calendar tasks, building on automotive HCI research examining the effects of multimodal interaction on usability, visual demand, and driving performance.
Interactive automotive dashboard prototype integrating
touch, voice, and gesture for context-aware usability evaluation.
Methodology
We followed a two-stage HCI research methodology combining prototype development with a pilot user evaluation. The multimodal automotive interface was first designed and structurally validated with the supervising professor, followed by participant testing in real-world campus driving conditions. Four participants completed predefined touch, voice, and gesture tasks under low- and high-traffic conditions, after which they completed a 20-item CAUS questionnaire using a 5-point Likert scale. The resulting subjective usability data were collected across both driving contexts to examine how interaction modality and environmental context influenced perceived usability.
Testing Setup: Prototype was displayed on an iPad
attached to the dashboard of a Honda Civic
Data Collected
Three measures were collected to evaluate both objective and subjective usability: task completion time, error frequency, and CAUS score. Together, these measures captured how efficiently and accurately participants completed tasks, as well as how usable they perceived each interaction modality to be across different driving contexts.
Task completion time: Measured manually with a stopwatch from the home screen until the participant reached the required task page, capturing interaction efficiency.
Error frequency: Recorded the number of times participants clicked an incorrect interface element and performed an unintended action while completing a task, capturing interaction accuracy.
CAUS score: Collected through the 20-item CAUS questionnaire to capture participants’ perceived usability across the different modalities and traffic conditions.
Procedure
Participants first received an overview of the study and the three interaction modalities: touch, voice, and gesture. Each participant then drove around campus under low-traffic and high-traffic conditions while completing predefined tasks using the prototype. Immediately after each driving condition, participants completed 20 CAUS items using a 5-point Likert scale, capturing their perceived usability and interaction experience within that specific driving context. This within-participant procedure allowed responses to be compared across driving conditions and interaction modalities.
Gesture Modality: Participant exploring the gesture to
swipe through Calender views
Touch Modality: Navigating through the music options to perform exploration task
Voice Modality: Touchless modality to explore naivgation tasks and music playback
User Tasks
Participants completed predefined tasks across three interaction modalities, informed by prior research on representative in-vehicle interactions. Touch tasks focused on navigation and media control, voice tasks on calendar and navigation requests, and gesture tasks on media playback and calendar navigation. These tasks were selected to reflect realistic secondary in-vehicle activities while enabling comparison of usability across interaction modalities and traffic conditions (Chang et al., 2017; Zhong et al., 2024).
Key Findings
Voice remained the most resilient modality across traffic conditions:
Voice consistently produced the fastest task completion times, averaging 8.98 seconds in low traffic and 9.45 seconds in high traffic. Participants also reported the highest CAUS ratings for voice across both conditions. While touch and voice were perceived similarly in low traffic, voice became the clearly preferred modality as traffic demands increased, suggesting that voice may be better suited to maintaining interaction efficiency when visual attention is more constrained.
Traffic conditions disproportionately affected touch and gesture:
Moving from low to high traffic made touch and gesture interactions increasingly challenging. Touch, which was the slowest modality in low traffic at 14.56 seconds, became less intuitive as traffic demands increased and produced additional errors. Gesture was affected even more strongly, reaching an average completion time of 22.78 seconds in high traffic and producing the highest error frequency. This suggests that increased environmental demand can amplify the challenges of visually and manually controlled interaction modalities.
Familiarity influenced the gap between perceived and measured performance:
Gesture produced the highest number of errors, particularly in high traffic, largely because participants were unfamiliar with the interaction technique. However, participants still perceived gesture as relatively easy to use once they became familiar with it. This creates an important distinction between initial performance and perceived usability: a modality may generate more errors during early interaction while still being perceived positively after users learn its interaction pattern.
Task characteristics influenced the suitability of gesture interaction:
Voice was consistently the fastest modality across tasks, while touch was perceived as the most intuitive. Gesture required the most time overall, but participants found it easier to use for calendar navigation than for media control. This suggests that the effectiveness of a modality is not determined by the modality alone, but also by how well its interaction mechanism maps to the task.
Subjective and objective measures revealed complementary insights:
The combination of CAUS, completion time, and error frequency provided a more complete picture than any single measure. Voice demonstrated strong performance across all three measures, while gesture revealed a more complex pattern: higher errors and longer completion times were accompanied by positive perceptions once participants became familiar with the interaction. This highlights the importance of evaluating automotive multimodal interfaces through both behavioural performance and perceived usability.
Future Inferences
Existing research/technology
CAUS Implementation
Adaptive Multimodal In-Vehicle Information Systems (AMiVIS): Existing research has proposed systems that dynamically select interaction modalities based on driving situation and driver workload.
CAUS could contribute context-specific usability data for determining when touch, voice, or gesture may be more appropriate under changing traffic conditions.
2026 CHI: Adaptive In-Car Touchscreen Interfaces: Hwang et al. developed a framework for adapting in-car touchscreen interfaces to driver behaviour, perception, and cognitive state, including changes to interface layout based on cognitive demand.
CAUS could extend this direction beyond touchscreen adaptation by examining how the choice of interaction modality itself could adapt to driving context.
List of citations
Akshay Deshmukh, A. M., & Pfleging, B. (2025). Towards context-aware usability assessments in vehicles: Developing the CAUS scale for multimodal interfaces. In Adjunct proceedings of the 17th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 239–244). Association for Computing Machinery. https://doi.org/10.1145/3744335.3758511
Zhang, T., Liu, X., Zeng, W., Tao, D., Li, G., & Qu, X. (2023). Input modality matters: A comparison of touch, speech, and gesture based in-vehicle interaction. Applied Ergonomics, 108, 103958. https://doi.org/10.1016/j.apergo.2022.103958
Ma, J., Zuo, Y., Gong, Z., & Meng, Y. (2025). Design methodology and evaluation of multimodal interaction for enhancing driving safety and experience in secondary tasks of IVIS. Displays, 87, 102991. https://doi.org/10.1016/j.displa.2025.102991
Graichen, L., & Graichen, M. (2024). Do users desire gestures for in-vehicle interaction? Towards the subjective assessment of gestures in a high-fidelity driving simulator. Computers in Human Behavior, 156, 108189. https://doi.org/10.1016/j.chb.2024.108189
Qu, Q.-X., Zhang, L.-X., Huang, Y.-Y., Zhang, X.-T., Guo, F., Duffy, V. G., & Man, S. S. (2026). Effect of multimodal HMI on mental workload and driving safety in smart cockpits. Accident Analysis & Prevention, 237, 108739. https://doi.org/10.1016/j.aap.2026.108739
Hilbert, J., Sevcenko, N., & Ebel, P. (2026). How do drivers interact with in-vehicle information systems in everyday driving? Evidence from large-scale naturalistic data. Transportation Research Part F: Traffic Psychology and Behaviour, 121, 103669. https://doi.org/10.1016/j.trf.2026.103669
Chang, C.-C., Boyle, L. N., Lee, J. D., & Jenness, J. (2017). Using tactile detection response tasks to assess in-vehicle voice control interactions. Transportation Research Part F: Traffic Psychology and Behaviour, 51, 38–46. https://doi.org/10.1016/j.trf.2017.06.008
Zhong, Q., et al. (2024). Assessing driver distraction from in-vehicle information system: An on-road study exploring the effects of input modalities and secondary task types. Scientific Reports, 14. https://doi.org/10.1038/s41598-024-71226-4