![facebook red zeplin facebook red zeplin](https://cdn.shopify.com/s/files/1/1301/1643/products/sugar-cosmetics-matte-lipstick-matte-attack-transferproof-lipstick-02-red-zeppelin-17258676617375@2x.jpg)
That could help address the gap Kumar highlights between the numbers of deployed machine-learning algorithms and the workforce of people knowledgeable about their potential vulnerabilities. Biggio says improving the security of AI systems in use will require similar tools, potentially building on attacks he and others have demonstrated in academic research.
FACEBOOK RED ZEPLIN HOW TO
That “stolen” copy can either be put to work directly or used to discover flaws that allow attackers to manipulate the original, paid service.īattista Biggio, a professor at the University of Cagliari who has been publishing studies on how to trick machine-learning systems for more than a decade, says the tech industry needs to start automating AI security checks.Ĭompanies use batteries of preprogrammed tests to check for bugs in conventional software before it is deployed. It discusses threats such as “model stealing,” where an attacker sends repeated queries to an AI service and uses the responses to build a copy that behaves similarly. Last fall Microsoft released documentation on AI security developed in partnership with Harvard that the company uses internally to guide its security teams. “Phishing and malware on the box is still their main thing.” “The bulk of security analysts are still wrapping their head around machine learning,” he says. He contributed to a paper published in March that found 22 of 25 companies queried did not secure their AI systems at all. But Ram Shankar Siva Kumar, who works on AI security at Microsoft, says they should still worry about people messing with their AI models. Most companies using AI in their business don’t have to worry as Facebook does about being accused of skewing a presidential election. “We’re trying to think very broadly about the pressing problems in the upcoming elections,” he says. The results show that preventing AI trickery isn’t easy.Ĭanton’s team is now examining the robustness of Facebook's misinformation detectors and political ad classifiers. The red team’s weightiest project aims to better understand deepfakes, imagery generated using AI that looks like it was captured with a camera. One project is examining the circulation of posts offering goods banned on the social network, such as recreational drugs.
![facebook red zeplin facebook red zeplin](https://cdn1.vectorstock.com/i/1000x1000/41/20/cartoon-smiling-red-zeppelin-mascot-vector-40394120.jpg)
It also began working with another research team inside the company that has built a simulated version of Facebook called WW that can be used as a virtual playground to safely study bad behavior. In the past year, Canton’s team has probed Facebook’s moderation systems. “That inspired me that this should be my day job.” A second discovered the attack used in early 2019 to spread porn on Instagram, but it wasn’t considered an immediate priority to fix at the time. One team at the contest showed that using different languages within a post could befuddle Facebook’s automated hate-speech filters. Some teams found weaknesses that Canton says convinced him the company needed to make its AI systems more robust. In 2018, Canton organized a “risk-a-thon” in which people from across Facebook spent three days competing to find the most striking way to trip up those systems. He was proud of his team’s work on AI systems to detect banned content such as child pornography and violence, but he began to wonder how robust they really were. The process doesn’t perfectly recreate the original, but it allows the porn classifier to do its work without getting tripped up.įacebook’s AI red team is led by Cristian Canton, a computer-vision expert who joined the company in 2017 and ran a group that works on image moderation filters. His team eventually tamed the problem of AI-evading nudity by adding another machine-learning system that checks for patterns such as grids on photos and tries to edit them out by emulating nearby pixels.
![facebook red zeplin facebook red zeplin](https://66.media.tumblr.com/fcb48a86bbdf7ac2ca909586e3e7b1eb/tumblr_ovcmpnZQBA1snb6qwo1_1280.jpg)
Users “started adapting by going with different patterns,” says Manohar Paluri, who leads work on computer vision at Facebook. That meant more work for Facebook's human content reviewers.įacebook’s AI engineers responded by training their system to recognize banned images with such patterns, but the fix was short-lived. But some users found they could sneak past Instagram’s filters by overlaying patterns such as grids or dots on rule-breaking displays of skin.
![facebook red zeplin facebook red zeplin](https://cdn.dribbble.com/users/1162077/screenshots/3097468/airship.png)
In February 2019, some Instagram users began editing their photos with a different audience in mind: Facebook’s automated porn filters.įacebook depends heavily on moderation powered by artificial intelligence, and it says the tech is particularly good at spotting explicit content. Instagram encourages its billion or so users to add filters to their photos to make them more shareable.