Google Develops AI That Can Detect Hateful Internet Speech

Google technology incubator Jigsaw has released software designed to help Web publishers moderate the unruly comments on their sites. The software is called Perspective and it is available free of charge to publishers that apply for access. Jigsaw used machine learning to help train Perspective to identify toxic comments. Each comment is assigned a score, so that human moderators or even readers can filter out responses that score above a certain toxicity level. Perspective is part of Jigsaw’s Conversation AI initiative. The team wants to help foster more civil discourse and eradicate Internet trolls. 

Currently, publishers have limited, and often time-consuming options, for fighting disrespectful speech, reports Wired. Some look at user-generated upvotes and downvotes. Some publishers rely on human moderators or eliminate comment sections altogether.


Conversation AI product manager CJ Adams said, “This gives them a new option: Take a bunch of collective intelligence — that will keep getting better over time — about what toxic comments people have said would make them leave, and use that information to help your community’s discussions.”

Jigsaw’s new software will get better over time because it relies on machine learning. It has already evaluated hundreds of thousands of comments, some of which came from The New York Times website and Wikipedia. As it evaluates more comments, the software will become more accurate because users can report corrections.

Perspective is available as an API, so that publishers can easily integrate it into their websites. Publications like The New York Times, The Guardian, and The Economist are all exploring how they can use the software on their sites.

The Jigsaw team believes that its software is not a solution to the Internet trolling problem. They argue that it is not a form of censorship because it should only be used as a tool to flag comments, not make editorial decisions about whether to publish them. The team hopes that Perspective makes the Internet a place where people feel more comfortable engaging with those who disagree with them.