Autonomous Vehicle Human-Machine Interface Design for Visually Impaired Users

    • Focus: Accessibility in autonomous vehicle HMIs

    • Method: Between-subjects experimental usability study

    • Participants: 24 participants (3 user conditions)

    • Objective: Evaluate how multimodal feedback affects trust and navigation confidence

  • How does multimodal (audio-visual) feedback influence trust, navigation confidence, and usability for visually impaired autonomous rideshare passengers?

    • UX Researcher

      • Designed experimental protocol

      • Conducted literature review

      • Ran usability testing sessions

      • Analyzed qualitative and quantitative data

      • Synthesized research insights

Peer-reviewed and published in the Proceedings of the Human Factors and Ergonomics Society Annual Meeting (HFES)

DOI: 10.1177/10711813251357895

Autonomous rideshare interfaces rely heavily on visual interaction, creating accessibility barriers for visually impaired users. These barriers limit users’ ability to confidently navigate rideshare experiences and ultimately reduce the independence that autonomous transportation could provide.

Problem:

Research Question:

How does multimodal feedback influence trust, usability, and navigation confidence for visually impaired riders in autonomous rideshare systems?

  • Many visually impaired users express skepticism due to past experiences with technology that failed to account for their needs. While users were excited about the independence autonomous vehicles could provide, they worried that AV systems may not be designed inclusively and could overlook accessibility requirements.

  • Visually impaired riders often struggle to understand where they are, what the vehicle is doing, and what will happen next during a trip.

    Users want:

    • Continuous route progress updates

    • Environmental cues and landmarks

    • Notifications when approaching destinations

  • A common challenge for visually impaired riders is confirming they are entering the correct vehicle and arriving at the correct destination.

  • Participants expressed concerns about what would happen during unexpected events, such as system failures or crashes. Since visually impaired users cannot take manual control of the vehicle, they worry about:

    • Accessing emergency assistance

    • Understanding why the vehicle is making certain decisions

    • Knowing when the system encounters problems

  • Many existing autonomous vehicle interfaces rely heavily on visual displays and alerts, which are ineffective for visually impaired users.

    Research shows that:

    • Visual warnings can be easily missed

    • Visual interfaces do not support accessibility

    • Visual information alone cannot provide sufficient feedback

Key Pain Points from literature review

Design Opportunity:

  • Provide audio feedback to reduce reliance on vision

  • Improve route transparency and trip awareness

  • Enable independent vehicle and destination confirmation

  • Offer clear emergency communication pathways

Hypothesis

H1

Visually impaired participants using multimodal feedback will report higher confidence in navigation and route awareness compared to participants without audio feedback.

H2

Visually impaired participants using multimodal feedback will demonstrate usability outcomes comparable to non-visually impaired participants.

H3

Removing audio feedback will reduce trust and situational awareness during the rideshare experience.

Prototype

  • 1. Interface Set Up

  • 2. Rider Verification

  • 3. Destination Confirmation

  • 4. Default Screen

  • 5. Event: Traffic Light

  • 6. Event: Obstacle Detection & Avoidance

Testing

Between-subjects design was selected to prevent learning effects between interface conditions. 24 participants divided into three groups:

  1. non-visually impaired (NVI)

  2. visually impaired with audio feedback (VI),

  3. visually impaired without audio feedback (VIX).

1

Test Design

Participants completed a 3–4 minute simulated rideshare experience using our prototype HMI

  • NVI: experienced the system with no visual distortion and audio cues

  • VI: experienced the system with visual distortion and audio cues

  • VIX: experienced the system with visual distortion and no audio cues

Each were asked to complete the same set of tasks:

  1. Confirm the correct rideshare vehicle

  2. Monitor route progression during the ride

  3. Exit the vehicle and orient themselves

2

Testing

3

Measuring functionality

After the testing, participants received a 5 point likert-scale survey to measure confidence, satisfaction, and usability across key moments of the ride.

FIndings

H1 Supported

audio feedback significantly improved confidence in ride identification, route navigation, and exit direction. H2 Supported

H2 Supported

visually impaired participants using multimodal feedback performed similarly to non-visually impaired participants.

H3 Supported

participants without audio feedback demonstrated consistently lower confidence across tasks.

In Conclusion:

This research demonstrates that multimodal interfaces can significantly improve accessibility and independence for visually impaired riders. Specifically, auditory feedback plays a critical role in building trust and navigation confidence.

Inclusive interface design will be essential to ensuring autonomous transportation benefits all users.