Fighting Online Harassment with New Tools Powered by AI
The company’s Jigsaw division is on a mission to protect the Internet’s most vulnerable. Trouble is, doing good is a lot harder than “Don’t be evil.” The post Inside Google’s Internet Justice League and Its AI-Powered War on Trolls.
Called Conversation AI, Jigsaw’s new software is aimed at blocking vitriolic or harmful statements online. The software uses machine learning to automatically catch abusive language, giving it an “attack score” of 100 (with 100 being extremely harmful and 0 being not at all harmful). The technology will first be tested out in the comments sections within The New York Times, and Wikipedia also plans on using it, though the company hasn’t said how, according to Wired.
The technology will eventually be open-source, so websites or social media platforms could use it to catch abuse before it even hits its intended target. According to Wired, Conversation AI can “automatically flag insults, scold harassers, or even auto-delete toxic language.”
It’s not clear how accurate the technology is quite yet . There are some distinct flaws in the software in tests, but Google told Wired it has a 92% certainty and a 10% false-positive rate, and that it will continue to improve over time.
