The metaverse is in its early days, but many are already concerned as they anticipate the content moderation problems that have bedeviled traditional social media increasing exponentially in virtual worlds. The confluence of realistic immersive environments, the anonymity of avatars and potential for deepfakes is enough to give anyone pause. Throw in machine learning that will make today’s ad targeting seem primitive and it’s an even more volatile mix. Experts agree, the very qualities that make the metaverse appealing — false facades and hyperreality — make it potentially more dangerous than the digital platforms of today.

Bloomberg shared a funny but chilling example of a bot run amok in a demo by startup Sensorium. Attendees innocuously chatted with virtual personas, learning about their tastes in wine and music. But, asked what he thought of vaccines, a middle-aged bot named David “began spewing health misinformation.”

“Vaccines, he claimed in one demo, are sometimes more dangerous than the diseases they try to prevent,” Bloomberg writes, adding that assurances were offered that filters would be added to limit what David could say about sensitive topics, “but the moment illustrated how easy it might be for people to encounter offensive or misleading content in the metaverse — and how difficult it will be to control it.”

This summer, documents provided to Congress and the media by Facebook whistleblower Frances Haugen showed how algorithms spread harmful information. While Facebook changed the name of its parent company to Meta Platforms, it hasn’t deflected the fire.

“Meta still allows purveyors of dangerous misinformation to thrive on its existing apps,” NewsGuard managing director Alex Cadier said on Bloomberg. “If the company hasn’t been able to effectively tackle misinformation on more simple platforms like Facebook and Instagram, it seems unlikely they’ll be able to do so in the much more complex metaverse.”

Meta Platforms isn’t the only company building out the metaverse. Microsoft and Apple were also mentioned in the Bloomberg piece. Game platform Roblox supports both HTC Vive and Meta Quest VR headsets. And even in Meta’s own “Horizon Worlds” virtual environment, a variety of creators and companies are creating sub-domains. Because the technology is just revving its engines on a runway experts say will take at least five, maybe 10 years to clear, there is plenty of time to plan and put safeguards in place, observers say.

“In the past, the speed at which new technologies arrived sometimes left policy makers and regulators playing catch-up. It doesn’t have to be the case this time around because we have years before the metaverse we envision is fully realized,” Meta VP global affairs Nick Clegg is quoted as saying in the Bloomberg report, which goes on to say the company “plans to work with human rights groups and government experts to responsibly develop the virtual world.”

The extent to which the metaverse will be able to ensure safety will be affected by things like whether digital environments are designed as wide-open raves or small invitation-only groups. It will also depend, at least in part, on “how companies train their AI systems to moderate,” Bloomberg suggests, explaining “AI can be trained to detect and take down hate speech and misinformation, and systems can also inadvertently amplify it.”

Related:
The Head of Facebook’s Metaverse Project Says Users Are to Blame for Its Misinformation Problem, Not Facebook, Inc., 12/15/21
Why Frances Haugen Is ‘Super Scared’ About Facebook’s Metaverse, TIME, 12/16/21