Inside Meta's Law Enforcement Outreach Team, Where AI Flags the Threats and Humans Decide Whether to Call the Police

A New Yorker feature published Friday by Rachel Monroe goes inside a part of Meta most users never see: a small law enforcement outreach team that sits between the company's platforms and the police. The group, now about a decade old, runs on a division of labor that is becoming a template for applied AI in high-stakes moderation — automated systems do the first pass at flagging, and human reviewers decide what, if anything, gets referred to authorities.

The method rests on what FBI profiler Mary Ellen O'Toole calls "leakage": the finding, from an FBI study of active-shooter cases between 2000 and 2013, that nearly nine in ten shooters under 18 had revealed their intentions to someone beforehand. In 2018, a former agent identified in the piece as Vacher, who had worked with the FBI's Quantico behavioral analysis unit, assembled specialists in campus shootings — including former journalists and detectives — to map how attackers leave those traces online.

The pipeline is straightforward to describe and uncomfortable to contemplate. AI tools scan content and flag signals of possible danger; human reviewers then examine the user's wider online behavior — posts, shares, interactions, and in some cases the prompts they typed into AI chatbots. When reviewers see what the team calls substantial steps, such as target lists or manifestos, Meta passes information to law enforcement. The company says its automated review flagged content for referral "around 800 times worldwide" last year.

The feature documents cases where the pipeline worked as intended. In one, a New York City high school student posted a threat written over a photo of homework on Instagram; Meta's flag reached the FBI and then the NYPD, and a search of the student's backpack turned up a loaded handgun — with the arrest coming about two hours after the post. In another, a February 2025 tip relayed by the FBI's Houston field office to the Spring Branch school district, which serves 31,000 students, led to two teenage girls being intercepted before an alleged planned attack; both later pleaded guilty.

The threat landscape the team tracks has a name: the True Crime Community, an online fandom that glorifies mass shooters and increasingly hides behind coded language — nicknames for perpetrators' weapons, childhood photos in place of infamous faces. The Institute for Strategic Dialogue links the subculture to at least nine attacks and sixteen disrupted plots over the past two years, and says its symbols evolve faster than moderation tooling can catch them.

The sharpest contrast in the piece is with OpenAI. In a case previously reported by Mother Jones, the company's automated system flagged a Canadian teenager's chats about gun violence last June, but after review OpenAI closed the account without alerting police. The teenager continued on new accounts, and in February attacked his family and opened fire at a secondary school in British Columbia, leaving six people dead including himself. The province and victims' families are suing OpenAI, whose chief executive Sam Altman has said publicly: "I am deeply sorry that we did not alert law enforcement."

Experts quoted in the piece stress that the same machinery cuts both ways. Katherine Keneally, senior director of US threat analysis and prevention at the Institute for Strategic Dialogue, notes that platforms are simultaneously referring individual cases to police and, through their recommendation algorithms, helping the radicalizing content travel — she names TikTok as the fastest-spreading mainstream carrier. Over-reporting risks pulling troubled teenagers into the criminal justice system for expressing dark thoughts; under-reporting, as the Canadian case shows, can be lethal. Meta's team leader frames the operation as "a very small part" of a societal problem that requires schools, parents, mental health professionals and police alike. The article's own verdict is more uneasy: the platforms are now both part of the solution and part of the problem, and AI has made that contradiction operational at scale.