Headset to Make Augmented Reality Less of a Gimmick

A novel optical technique could overlay virtual imagery on the real world through a compact pair of glasses.

By Simon Parkin

Andrew Maimone thinks augmented reality hasn’t been much more than a gimmick so far.

Maimone, a PhD student at the University of North Carolina at Chapel Hill, is developing a new kind of head-worn display that could make augmented reality —whereby digital objects or pieces of information are overlaid on the real world via a screen—significantly more immersive.

While it’s possible to use a smartphone or tablet to, for example, conjure a virtual character and place it onto a real world table viewed on a smartphone’s screen, this just “isn’t very compelling” says Maimone. “The experience doesn’t occur in one’s own vision,” he says. “It acts as a little more than a small window into the virtual place.”

Conventional augmented reality glasses use lenses, beam splitters, waveguides, reflectors, and other optics to relay an image to the eye, and place the image at a distance where the eye can focus on it. These components add bulk, however, and the resulting glasses usually have a limited field of view.

Together with three other researchers from the University of North Carolina and two from Nvidia Research, Maimone has been working on an entirely new kind of augmented reality device that is light and compact, and offers a wide field of view.

Maimone’s device, called a Pinlight Display, does not use conventional optical components. It replaces these with an array of bright dots dubbed pinlights. “A transparent display panel is placed between the pinlights and the eye to modulate the light and form the perceived image,” says Maimone. “Since the light rays that hit each display pixel come from the same direction, they appear in focus without the use of lenses.”

In this configuration, small fragments of the image are flipped and superimposed, so the team has compensated for this by performing some image manipulation in software.

“One could think of Pinlight Displays as exploiting how the eye sees an image that is out of focus, in order to form an image that is in focus,” says Maimone. “The resulting hardware configuration is very simple—there are no reflective, refractive, or diffractive elements—so we do not run into the trade-off between form factor and field of view that has been encountered in past glasses designs.”

The benefits of the approach over previous devices are significant. While state-of-the-art commercial augmented reality glasses have a field of view of 40° or less, early Pinlight prototypes have demonstrated fields of view of 100° or more. It’s an impressive breakthrough, as evidenced by this explanatory video, which shows the difference that a wide field of view makes when viewing, say, a holographic-style spaceship from Star Wars.

Maimone argues that the potential uses for the technology are wide-ranging. “I’d love to be able to navigate a city by following some virtual bread crumbs laid down on the sidewalk,” he says. “I’d love to have a virtual lunch with my wife every day as if she’s seated across the table. I’d love to see the name of a new acquaintance floating next to them when we meet. I’d love to have all of things happen effortlessly in my glasses, and when they do, I think we’ll start to see computer graphics more as integral part of our visual system, rather than something that exists only on external screens.”

There may be other potential benefits to the team’s approach. “Since part of the image formation process takes place in software, we can adjust parameters such as eye separation and focus dynamically,” says Maimone. “[Therefore] we can imagine incorporating the pinlights into the corrective lenses or ordinary glasses, creating a display that looks like ordinary glasses with the addition of an LCD panel.”

Problems remain for the team, which recently showed off the technology at the Siggraph 2014 conference in Vancouver. The prototype suffers from low resolution and image quality, far below the level of existing commercial augmented reality glasses. Additionally they must successfully implement tracking, networking, low latency rendering, various and other features.

“The next step is to improve these factors,” says Maimone. And despite his skepticism about the current state or augmented reality, he believes that with the right research and engineering, the technology could be “transformed into something practical for everyday use.”