Interface advances are set to blur the line between the real world and the digital one. While new interfaces such as AR glasses, hand-free headsets and even smart contact lenses are being developed, the question is being raised whether screens, which are so predominant now, are set to disappear.
It is nearly 10 years since the iPad first hit the market. Along with the iPhone, it was among the first devices to popularize portable touchscreen interfaces. But since then, not much has changed in computer interface design, and the majority of computer interaction still revolves around the QWERTY keyboard, first devised in the 1870s.
But all this is gradually changing. Introduced in October 2019, Google’s Pixel 4 phone allows users to control the device with gestures via on-board sensors. Meanwhile, developments in augmented reality, glasses, lenses and interactive projection are also advancing. Research by Goldman Sachs has found that the global market for augmented and virtual reality is expected to grow by over 50% every year, reaching $95 billion by 2025.
The Future is in Augmented Reality
AR places an information layer on top of the real world the user sees, usually via a smartphone, but headsets, glasses and even contact lenses are in development to support the interface.
Rather than a shift to a new mode of computer-human interactions, developers need to accept the idea that the future is multi-model: that users will switch between keyboards, touchscreens, voice, gesture and AR, according to Tuong Nguyen, senior principal analyst at Gartner:
“If you think back to when the computing era started, someone had to figure out how humans were going to interact with the digital realm and they ended up with the mouse and keyboard paradigm. Then we moved on to smartphones and tablets, and the keyboard doesn’t quite work in that realm, so someone figured touch interfaces. The next era we’re moving into is the marrying of physical and digital in augmented reality.”
The tricky part is AR is not a single technology, in and of itself. It puts together different technologies in a certain way that creates an experience that you will know as AR. Nguyen continues:
“It will be a mix of interfaces that we currently have together with the next-generation human-machine interface. So, we will not just be typing on keyboards, but using motion, gesture, and voice. It will be multimodal.”
Microsoft’s HoloLens and Facebook’s Oculus headsets, along with others, are popularizing AR for gaming and other consumer applications. An elementary form of AR already hit the mass-market with Niantic’s Pokemon Go smash-hit mobile game.
In this video interview we did in 2018, Nancy Li, from the AI perception department at Microsoft explains how Hololens is driving digital transformation in companies. She also believes that in the future, we may all wear smart glasses before moving to smart contact lenses.
“There are technologies patterns out there that are already in that phase, looking at things like that.”
Smart Contact Lenses
Smart contact lenses with built-in AR displays able to provide information without the need for any screens are indeed under development. Japanese contact lens manufacturer Menicon Co has teamed up with US-based Mojo Vision to develop what they call “invisible computing.”
In an interview with our sister publication MedicalExpo e-magazine*, Steve Sinclair, SVP of Product and Marketing at Mojo Vision explained:
“To lead us into this invisible computing future, we’re building Mojo Lens, the world’s first true smart contact lens with a built-in display that can provide eyes-up, hands-free content right when you need it without having to look down at other screens or devices. And when you don’t need it, Mojo Lens just disappears so you always look like yourself. At the heart of Mojo Lens is the smallest and densest dynamic display ever built. Our 14KPPI display is a microLED array about the size of a grain of sand that projects text, photos and video content onto your retina.”
The company says the contact lens has precision motion sensors built in that detect eye movement to stabilize the AR content as the eyes move and to enable the unique eye-controlled user interface. An embedded image sensor is used to detect ambient light levels in order to automatically adjust the display’s brightness.
The company says Mojo Lens will be powered by small, thin-film biosafe batteries built into the lens. The battery life will depend on the user experience, but the goal is to let people wear and use the lens throughout the day, just like normal contact lenses.
However, to date, Mojo Vision is unable to give a launch date or potential price for lense.
The State of Industrial Augmented Reality
Industry is also keen to reap the benefits of AR. According to research from industrial software supplier PTC, 24% of enterprises experimenting with AR have done so with the aid of “new” devices including headsets allowing hands-free work. B-netween 62% and 55% of respondents are making experiences available on smartphones or tablets. The 2019 study found 68% of organizations feel confident that they will move their AR initiatives to production within 12 months.
While some headsets allow users to view augmented reality, others let them interact with it, for example, AR hardware company Magic Leap has shipped a headset supporting cameras that allow users limited interaction with the virtual object they see in front of them. Meanwhile, projected interactive displays can turn almost any surface into a touchscreen. Mixed with object recognition, they allow the system to add smart features to real-world objects, from the user’s perspective.
Mark Rolston is the founder and chief creative officer of Argodesign. He came up with a technology, Interactive Light, that allows users to interact with virtual content without necessarily having a computer in front of him or her or without wearing a headset. For him, such a system will be able to “lock” information to real objects for when it is needed.
“If I’m working on a jet engine, I can set some information to float right next to a specific component, and it will stay right there, even as I turn my head, and maybe go to the bathroom, but the instructions are still over there attached to the object.”
Watch Argodesign video
Others are looking to let computer interaction slide into the background. French firm Woodoo has developed a treatment for natural woods which removes the lignin and refills the material with a specially developed polymer, which also fills the gaps between wood fibres. In addition to making the material extremely strong and fire-resistant, it can also be translucent and turned into a touch display. The effect is a display hidden as a wooden panel or tabletop, which comes to life when the user requires, according to Olivier Grange, senior vice president marketing communications. The company is already working with Daimler and Airbus to integrate displays into cars and aircraft.
The technology industry has long fantasized about giving users the immersive, ever-present and discrete user interfaces seen in films like Tom Cruise’s Minority Report. Over the next five years, we’ll see if this will become reality.
*The interview with Steve Sinclair, SVP of Product and Marketing at Mojo Vision was conducted by Abigail Saltmarsh for our sister publication MedicalExpo e-magazine.