Eye Vergence & Gaze Sensing

Head-up displays and transparent displays put information between the driver and the road. That creates a new problem: the system cannot tell whether the driver is reading the display or looking at the traffic behind it. Without that distinction a HUD is either always on and distracting, or always minimal and useless.

Measure eye vergence, the angle between the two eyes, to know the depth the driver is focusing on. Combined with gaze direction, the interface knows whether attention is on the display or on the road, and adapts. Information appears where the driver is looking and disappears when they look through it. Drivers interact with eye contact, gestures, or simply by focusing on an area of interest, with no deliberate input.

I designed and built the system: the projection-based transparent display, the interaction methods and the vergence detection, using a Tobii eye tracker and early Tobii SDK capabilities. The prototype demonstrates gaze-triggered navigation guidance, real-time display of critical information, and safety applications.

A Tobii eye tracker provides gaze direction for each eye. From the two gaze vectors the system estimates vergence and therefore focal depth, classifying attention as on-display or beyond it. The display itself is back-projected glass with a visibility-enhancing coating; we tested transparent LCD panels from Samsung and LG, but back projection gave the best visibility. Later collaborations explored windshield projection with custom projectors and fluorescent coatings for bright environments.

The project culminated in a working proof of concept and a UX demonstration film, and seeded a family of granted patents: eye vergence detection on a display, detecting visual inattention and daydreaming from eye convergence, directed alerts such as honking at a specific vehicle or pedestrian, in-cabin projection that places information where the driver is looking, and gaze-enabled navigation. It also led into stress and cognitive load analysis and AR/VR/XR applications.