For years, a skunkworks team at Microsoft has been quietly exploring augmenting reality with virtual holograms. The fruits of their efforts were unveiled during today's Windows 10 showcase, and the unexpected demo was arguably the highlight of the event.
Dubbed Windows Holographic, the scheme looks like a form of augmented reality. It relies on HoloLens, which Kinect alum and Microsoft Fellow Alex Kipman described as a "fully untethered holographic computer." The head-mounted unit projects holographs—basically virtual 3D objects—onto see-through lenses through which users can view the rest of the world. A surround-sound system provides the appropriate audio cues, while an array of sensors monitors the environment and the user's actions.
Sensor input is processed by a proprietary "holographic processor" of Microsoft's own design. This chip is used to map the surrounding area, determine where the user is looking, interpret gestures, and monitor voice input. Everything is handled by the HoloLens hardware, with no need for external cameras or auxiliary processing.
An external machine isn't required because the HoloLens has PC hardware built in. A "high-end CPU and GPU" are integrated into the headset, though Kipman didn't reveal more specifics on that front. It seems unlikely that Microsoft has squeezed potent desktop hardware—or even a discrete GPU—into the head-mounted chassis. My money's on an ultrabook-class Intel processor with integrated graphics, but I wouldn't be surprised by custom silicon derived from the Xbone's SoC.
The HoloLens headset will be available in the same time frame as Windows 10. Microsoft has developed its own Holo Studio software for creating virtual holograms, and it's working with developers on third-party apps. Minecraft integration is on the menu, of course, and so is support for exporting virtual objects to 3D printers. Microsoft has also been collaborating with NASA's Jet Propulsion Laboratory, which plans to use HoloLens to explore a virtual version of Mars based on images taken from the Martian surface. Pretty slick.