Facebook ‘revealed names of counter-terrorism moderators to suspected extremists’

A Facebook moderator says he was forced to go into hiding after a security glitch made his personal profile available to suspected terrorists.

The man, an Iraqi-born Irish citizen in his 20s, was working on Facebook’s counter-terrorism team in Dublin, Ireland. As an Arabic speaker, he was part of a 40-strong team responsible for monitoring and removing content that breached the social media giant’s terms of service.

Speaking to the Guardian, the unnamed man said: “[When] you have people like that knowing your family name you know that people get butchered for that.

“The punishment from IS [Islamic State, formerly ISIS/ISIL] for working in counter-terrorism is beheading.”

The lapse was discovered in November last year, when a bug in the software was found to put the personal information of moderators in the activity log of the Facebook groups targeted for breaching the terms of service. In groups whose administrators had been removed, the remaining admins were free to view the moderators’ details.

The man, who came to Ireland as an asylum seeker, left the country for Eastern Europe after learning that his profile had been viewed by an Egypt-based group he’d removed over its support of Hamas and Islamic State.

“It was getting too dangerous to stay in Dublin,” he said.

The Guardian reports that more than 1,000 moderators across 22 different departments have potentially had their personal profiles viewed by posters removed from the network. Moderators reportedly first suspected the breach when they began to receive friend requests from the people associated with groups they had removed.

A group of six moderators in the Dublin office were classed as ‘high-risk’ after Facebook first detected the problem, with accounts tied to IS and Hezbollah known to have viewed their profiles.

Facebook’s head of global investigations, Craig D’Souza, sought to alleviate concerns by telling the group there was “a good chance” suspected terrorists would not identify them as moderators.

A spokesman for the social media giant confirmed the leaks to RT.com, saying: “Our investigation found that only a small fraction of the names were likely viewed, and we never had evidence of any threat to the people impacted or their families as a result of this matter.”

“We care deeply about keeping everyone who works for Facebook safe. As soon as we learned about the issue, we fixed it and began a thorough investigation to learn as much as possible about what happened.”

Removing ‘terrorist’ content

Meanwhile, Facebook has announced details of its plans to crackdown on terrorist-related content using AI technology, saying: "We want to find terrorist content immediately, before people in our community have seen it."

Using a technique known as ‘image-matching,’ the company’s system matches stills from known extremist content to new uploads, stopping it from being uploaded in the first place.

The company says it is also using algorithms to detect ‘clusters’ of profiles relating to support for terrorism and whether an account is friends with a high number of accounts disabled for supporting terrorism.

“This work is never finished because it is adversarial, and the terrorists are continuously evolving their methods too,” Monika Bickert, director of global policy management, and Brian Fishman, counterterrorism policy manager, said in a statement.