Davide Di Censo

See-Through Dashboard

Robotic stereo cameras and view-dependent 3D rendering that let the driver look down through the dashboard at the road directly ahead.

Company
HARMAN International
Year
2014 — 2015
Role
Project lead · built the proof-of-concept prototypes: actuated cameras, view-dependent rendering, stereoscopic 3D
Team
Future Experience Team
Location
Palo Alto, CA

Technology

  • Robotically actuated stereo camera pair
  • Head tracking
  • View-dependent rendering
  • Stereoscopic 3D (zSpace)
  • Pan-tilt camera control
See-through dashboard prototype: the instrument cluster shows the road below the hood with view-dependent rendering

01The opportunity

The dashboard hides a wedge of road right in front of the car: the low, close area that matters most when parking, creeping in traffic or pulling up to a curb. Backup cameras had solved this for the rear. The front, where the driver's own eyes are, was still blind.

02The idea

Extend the driver's vertical field of view downward through the instrument cluster. A stereo camera pair on the bumper feeds a 3D display in the cluster; view-dependent rendering makes the road appear at its real distance rather than on the display surface, so the cockpit feels transparent.

03My role

As project lead I built the proof-of-concept prototypes: the mechanically actuated cameras, the view-dependent rendering and the stereoscopic 3D rendering on a zSpace display. The work produced two granted patents.

04How it worked

A stereo camera pair sits on a robotically actuated rail with pan-tilt on each camera. High-resolution head tracking drives the cameras to match the driver's eye positions, and the zSpace 3D screen renders the two feeds view-dependently so the depth stays in sync with the eyes. In the proof of concept the robotic rig demonstrates the synchronization; a production system would use a solid-state camera array on the bumper with no moving parts, and glasses-free auto-stereoscopic displays.

05Prototype / product

Fig. 01PrototypeThe fully functional prototype. The stereo pair on the actuated rail follows the user's head; the zSpace display renders the feeds in 3D in sync with the eyes. The 3D effect itself cannot be captured on video.
Fig. 02ConceptThe driver's view extends downward through the cluster. A 3D display, view-dependent rendering and stereoscopic cameras make the road appear at its actual distance.
Patent illustration comparing robotic cameras and a solid-state camera array
Fig. 03ResearchTwo implementations from the patent. The proof of concept uses robotic cameras to demonstrate synchronization with head movement; production would use a solid-state bumper array.
Fig. 04PrototypeAn early moment in prototyping: the motors programmed to jump between positions to test speed and clearance between the cameras. Rapid prototyping is made of small successes like this one.
The actuated stereo camera rig
Fig. 05PrototypeThe actuated stereo rig.
See-through dashboard rendering, first view
Fig. 06PrototypeView-dependent rendering on the cluster.
See-through dashboard rendering, second view
Fig. 07PrototypeThe road appears below the hood line.

06Outcome

A working proof of concept and two granted patents, a virtual see-through instrument cluster with live video and a virtual 3D instrument cluster with 3D navigation, which contributed to the advancement of automotive visualization technologies.

  • Working prototype
  • Two granted patents

07Patents, press, links

Patents

  • US 9756319 B2Virtual see-through instrument cluster with live video2017
  • US 10062204 B2Virtual three-dimensional instrument cluster with three-dimensional navigation system2018

Links

Related work

Next project · PRJ-004Bare-Hand Gesture Interactive Sound PanoramaGrab a sound, move it, throw it to another seat. Gesture control of spatial audio in the car and on headphones, with no display required.