Why Facebook’s plan to give virtual assistants bodies is both awesome and terrifying

Facebook recently showed off the progress its AI research team has made in the realm of household robotics. The dream is to take the virtual assistant out of the speaker and put it into an autonomous body capable of traversing your house. If you’re thinking “Rosie the Robot” from The Jetsons, you’re not far off.

Just think smaller.

Per a Facebook AI blog post:

To accomplish a task like checking to see whether you locked the front door or retrieving a cell phone that’s ringing in an upstairs bedroom, AI assistants of the future must learn to plan their route, navigate effectively, look around their physical environment, listen to what’s happening around them, and build memories of the 3D space.

Facebook’s created a new system called SoundSpaces that gives robots the ability to interpret sounds. Current virtual assistants merely listen for wake words and then use natural language processing to interpret verbal commands as triggers.

While the microphone array technology behind some of these assistants is impressive, they’re merely designed to pick up voices in noisy environments. In order for a robot to retrieve a phone ringing in another room it has to be able to “hear,” the sound and determine where, relative to it’s own current position, it’s coming from.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

According to the blog post, SoundSpaces allows developers to “train AI agents in 3D environments with highly realistic acoustics. This opens up an array of new embodied AI tasks, such as navigating to a sound-emitting target, learning from echolocation, or exploring with multimodal sensors.”

Facebook’s other big reveal was its Semantic MapNet. This one’s a bit more complex, but in essence it gives robots the ability to navigate their environment using both computer vision and the aforementioned SoundStage system while also recognizing “feature-based” semantic correlations. In other words it can remember where the couch is with greater precision than it could by simply examining pixels of images generated by its other sensors.

The robotics market has gone through a bit of upheaval as of late. A few years back it seemed like everyone was working on a gadget you could squeeze a smart assistant into that would follow you around. These novelties mostly ended up being convoluted amalgams of random virtual assistant abilities crammed into toy robot bodies. While they can be fun for a moments, they’re ultimately not useful. Many of those startups are no longer in business. 

So far the idea of a robot maid, like “Rosie,” is science fiction. That’s why we have to think smaller. It sounds like what Facebook’s trying to create is a useful robot that won’t run the risk of accidentally killing you if it bumps into you on the stairs. And that means it’ll be tiny.

There are only so many ways a robot can navigate a home environment. And unless these things are only going to be sold to rich people who live esoterically in modified aircraft hangars, the smart money’s on a small gadget that’ll be capable of carrying a kilogram or two, walking up stairs, and potentially climbing the walls like a spider right before dropping on your face to record your dreams while you sleep – that last part is pure conjecture.

If the idea of Facebook putting a robot in your house capable of exploring, mapping, and recording everything that happens sounds scary to you… well, it should be. Facebook is an AI company that makes its money selling advertising. Anything it can get in your home is going to logically work toward that end.

But, it’s also awesome. If we could put aside the privacy concerns for a moment (which is a lot like saying “If we can ignore the impending nuclear explosion for a second…”) the idea of smart assistants that can actually assist is huge.

It might be cool to imagine having a giant robot bodyguard that can cook, clean, kick in doors, and subdue a burglar. But the reality is that level of safe autonomy could be decades away. Instead, think of it like Alexa or Google Assistant, only you could give it commands such as “grab me a beer,” or “go give the baby their pacifier.” Instead of pulling out your phone to start a Zoom call, you could simply whistle for your robot and it’d mount itself on a tripod and connect you. If you got up to pace while you’re talking it’d rotate to follow you.

They might even be capable of passing the butter.

These, of course, are just a few potential abilities a small robot-embodied virtual assistant could provide. In reality there’s no end to the possible utility a robust, well-engineered robot with Facebook’s new AI features could bring. Of course, any company attempting to tie us at the hip to what, essentially, is the equivalent of an iPhone that can walk and carry things is going to have to gain the public trust in a major way.

It’s one thing to know that Alexa and Siri can hear everything going on in your house. But your Echo Dot or iPhone can’t wander into the bathroom while you’re doing your business and then report back to advertising HQ. Who knows what kind of data these things will gather and how the companies making them will use it.

Still, how cool would it be to have a little robot sidekick?

So you like our media brand Neural? You should join our Neural event track at TNW2020, where you’ll hear how artificial intelligence is transforming industries and businesses.