Friends and family can count the months they haven’t seen each other due to the ongoing global pandemic. While video calls are useful, flat video lacks warmth. That’s where Google’s new Project Starline comes in—a magical-looking window that brings your far-flung relatives into the room in lifelike 3D.
Let’s start with the bad news: Google’s new magic window isn’t available for purchase. This was dubbed “Project” Starline for a reason. Even if Google tried to sell it today, it would most certainly be priced so extraordinarily high that the average person couldn’t buy it in the first place.
But the entire concept is incredible, if not downright magical, to use a word that is often misused. At first glance, Project Starline resembles a modern vanity, but without the mirror. When you switch it on, though, you’ll be greeted by a life-size 3D image of a person who looks so lifelike you’ll want to reach out and touch them.
A headset may not be required in the future of immersive video video. It creates a “magic window” that can beam a life-sized, 3D image of the person you’re conversing with into a customized display that makes it appear as if they’re sitting in front of you using a mix of specialist hardware and computer vision technologies.
It’s Zoom meets Augmented Reality, and the magic is made possible by plenty of cameras. A network of cameras on each end captures images of a person from various angles and then combines the data. Even if the person you’re conversing with isn’t in the room, spatial audio and proper camera arrangement allow you to make “eye contact” with them.
“To make this experience possible, we are applying research in computer vision, machine learning, spatial audio and real-time compression,” Google’s Clay Bavor wrote in a blog post. “We’ve also developed a breakthrough light field display system that creates a sense of volume and depth that can be experienced without the need for additional glasses or headsets.”
Light field is a term used to describe the qualities and direction of light, with AR firm Magic Leap being the most well-known example to date.
Google had to invent new ways to compress the massive amounts of data required for Project Starline to function, and it’s unclear whether this will hold up over long distances.
For the time being, all of the system’s tests have taken place within Google’s premises. It’s also not a flawless system just yet. If you look closely at the videos, you’ll notice that there are flaws in the hair, the chin line, and even the skin colors.
Also, it appears that Starline needs a fairly complex setup. Images shared by Google resemble a photo booth, complete with multiple lights, sensors, and a large display.
For now, the setup is only available at a few Google offices, and Google claims to have spent thousands of hours testing the technology among its staff. The business didn’t say what its future ambitions for Starline are or how much the setup will cost, but it did say it’ll “be doing trial installations with enterprise partners later this year.”
Google says it hopes to make Starline “more affordable and accessible” in the future, and that it may be integrated into Google’s other communications products.