How trolls, lawsuits caused ‘trust and safety winter’ before election

Nina Jankowicz’s dream job has turned into a nightmare.

For the past 10 years, she’s been a disinformation researcher, studying and analyzing the spread of Russian propaganda and internet conspiracy theories. In 2022, she was appointed to the White House’s Disinformation Governance Board, which was created to help the Department of Homeland Security fend off online threats.  

Now, Jankowicz’s life is filled with government inquiries, lawsuits and a barrage of harassment, all the result of an extreme level of hostility directed at people whose mission is to safeguard the internet, particularly ahead of presidential elections.

Jankowicz, the mother of a toddler, says her anxiety has run so high, in part due to death threats, that she recently had a dream that a stranger broke into her house with a gun. She threw a punch in the dream that, in reality, grazed her bedside baby monitor. Jankowicz said she tries to stay out of public view and no longer publicizes when she’s going to events.

“I don’t want somebody who wishes harm to show up,” Jankowicz said. “I have had to change how I move through the world.”

In prior election cycles, researchers like Jankowicz were heralded by lawmakers and company executives for their work exposing Russian propaganda campaigns, Covid conspiracies and false voter fraud accusations. But 2024 has been different, marred by the potential threat of litigation by powerful people like X owner Elon Musk as well congressional investigations conducted by far-right politicians, and an ever-increasing number of online trolls. 

Alex Abdo, litigation director of the Knight First Amendment Institute at Columbia University, said the constant attacks and legal expenses have “unfortunately become an occupational hazard” for these researchers. Abdo, whose institute has filed amicus briefs in several lawsuits targeting researchers, said the “chill in the community is palpable.” 

Jankowicz is one of more than two dozen researchers who spoke to CNBC about the changing environment of late and the safety concerns they now face for themselves and their families. Many declined to be named to protect their privacy and avoid further public scrutiny. 

Whether they agreed to be named or not, the researchers all spoke of a more treacherous landscape this election season than in the past. The researchers said that conspiracy theories claiming that internet platforms try to silence conservative voices began during Trump’s first campaign for president nearly a decade ago and have steadily increased since then.

The chilling effect is of particular concern because online misinformation is more prevalent than ever and, particularly with the rise of artificial intelligence, often even more difficult to recognize, according to the observations of some researchers. It’s the internet equivalent of taking cops off the streets just as robberies and break-ins are surging.  

Jeff Hancock, faculty director of the Stanford Internet Observatory, said we’re in a “trust and safety winter.” He’s experienced it firsthand. 

After the SIO’s work looking into misinformation and disinformation during the 2020 election, the institute was sued three times in 2023 by conservative groups, who alleged that the organization’s researchers colluded with the federal government to censor speech. Stanford spent millions of dollars to defend its staff and students fighting the lawsuits. 

During that time, SIO downsized significantly.

“Many people have lost their jobs or worse and especially that’s the case for our staff and researchers,” said Hancock, during the keynote of his organization’s third annual Trust and Safety Research Conference in September. “Those attacks take their toll.”

SIO didn’t respond to CNBC’s inquiry about the reason for the job cuts. 

Google last month laid off several employees, including a director, in its trust and safety research unit just days before some of them were scheduled to speak at or attend the Stanford event, according to sources close to the layoffs who asked not to be named. In March, the search giant laid off a handful of employees on its trust and safety team as part of broader staff cuts across the company.

Google didn’t specify the reason for the cuts, telling CNBC in a statement that, “As we take on more responsibilities, particularly around new products, we make changes to teams and roles according to business needs.” The company said it’s continuing to grow its trust and safety team. 

Jankowicz said she began to feel the hostility two years ago after her appointment to the Biden administration’s Disinformation Governance Board. 

She and her colleagues say they faced repeated attacks from conservative media and Republican lawmakers, who alleged that the group limited free speech. After just four months in operation, the board was shuttered. 

In an August 2022 statement announcing the termination of the board, DHS didn’t provide a specific reason for the move, saying only that it was following the recommendation of the Homeland Security Advisory Council. 

Jankowicz was then subpoenaed as a part of an investigation by a subcommittee of the House Judiciary Committee intended to discover whether the federal government was colluding with researchers to “censor” Americans and conservative viewpoints on social media.

“I’m the face of that,” Jankowicz said. “It’s hard to deal with.”

Since being subpoenaed, Jankowicz said she’s also had to deal with a “cyberstalker,” who repeatedly posted about her and her child on social media site X, resulting in the need to obtain a protective order. Jankowicz has spent more than $80,000 in legal bills on top of the constant fear that online harassment will lead to real-world dangers.

On notorious online forum 4chan, Jankowicz’s face grazed the cover of a munitions handbook, a manual teaching others how to build their own guns. Another person used AI software and a photo of Jankowicz’s face to create deep-fake pornography, essentially putting her likeness onto explicit videos. 

“I have been recognized on the street before,” said Jankowicz, who wrote about her experience in a 2023 story in The Atlantic with the headline, “I Shouldn’t Have to Accept Being in Deepfake Porn.”

One researcher, who spoke on condition of anonymity due to safety concerns, said she’s experienced more online harassment since Musk’s late 2022 takeover of Twitter, now known as X. 

In a direct message that was shared with CNBC, a user of X threatened the researcher, saying they knew her home address and suggested the researcher plan where she, her partner and their “little one will live.” 

Within a week of receiving the message, the researcher and her family relocated. 

Misinformation researchers say they are getting no help from X. Rather, Musk’s company has launched several lawsuits against researchers and organizations for calling out X for failing to mitigate hate speech and false information. 

In November, X filed a suit against Media Matters after the nonprofit media watchdog published a report showing that hateful content on the platform appeared next to ads from companies including Apple, IBM and Disney. Those companies paused their ad campaigns following the Media Matters report, which X’s attorneys described as “intentionally deceptive.” 

Then there’s House Judiciary Chairman Jim Jordan, R-Ohio, who continues investigating alleged collusion between large advertisers and the nonprofit Global Alliance for Responsible Media (GARM), which was created in 2019 in part to help brands avoid having their promotions show up alongside content they deem harmful. In August, the World Federation of Advertisers said it was suspending GARM’s operations after X sued the group, alleging it organized an illegal ad boycott. 

GARM said at the time that the allegations “caused a distraction and significantly drained its resources and finances.”

Abdo of the Knight First Amendment Institute said billionaires like Musk can use those types of lawsuits to tie up researchers and nonprofits until they go bankrupt.

Representatives from X and the House Judiciary Committee didn’t respond to requests for comment.

Musk said at the time that the change was needed because the “free API is being abused badly right now by bot scammers & opinion manipulators.” 

Kate Starbird, an associate professor at the University of Washington who studies misinformation on social media, said researchers relied on Twitter because “it was free, it was easy to get, and we would use it as a proxy for other places.”

“Maybe 90% of our effort was focused on just Twitter data because we had so much of it,” said Starbird, who was subpoenaed for a House Judiciary congressional hearing in 2023 related to her disinformation studies. 

A more stringent policy will take effect on Nov. 15, shortly after the election, when X says that under its new terms of service, users risk a $15,000 penalty for accessing over 1 million posts in a day.

“One effect of X Corp.’s new terms of service will be to stifle that research when we need it most,” Abdo said in a statement. 

It’s not just X. 

In August, Meta shut down a tool called CrowdTangle, used to track misinformation and popular topics on its social networks. It was replaced with the Meta Content Library, which the company says provides “comprehensive access to the full public content archive from Facebook and Instagram.”

Researchers told CNBC that the change represented a significant downgrade. A Meta spokesperson said that the company’s new research-focused tool is more comprehensive than CrowdTangle and is better suited for election monitoring.

In addition to Meta, other apps like TikTok and Google-owned YouTube provide scant data access, researchers said, limiting how much content they can analyze. They say their work now often consists of manually tracking videos, comments and hashtags.

“We only know as much as our classifiers can find and only know as much as is accessible to us,” said Rachele Gilman, director of intelligence for The Global Disinformation Index. 

In some cases, companies are even making it easier for falsehoods to spread. 

For example, YouTube said in June of last year it would stop removing false claims about 2020 election fraud. And ahead of the 2022 U.S. midterm elections, Meta introduced a new policy allowing political ads to question the legitimacy of past elections. 

YouTube works with hundreds of academic researchers from around the world today through its YouTube Researcher Program, which allows access to its global data API “with as much quota as needed per project,” a company spokeswoman told CNBC in a statement. She added that increasing access to new areas of data for researchers isn’t always straightforward due to privacy risks.

A TikTok spokesperson said the company offers qualifying researchers in the U.S. and the EU free access to various, regularly updated tools to study its service. The spokesperson added that TikTok actively engages researchers for feedback.

Fresh in everyone’s mind is Jan. 6, 2021, when rioters stormed the U.S. Capitol while Congress was certifying the results, an event that was organized in part on Facebook. Harbath, who was previously a public policy director at Facebook, said the certification process could again be messy. 

“There’s this period of time where we might not know the winner, so companies are thinking about ‘what do we do with content?'” Harbath said. “Do we label, do we take down, do we reduce the reach?” 

Despite their many challenges, researchers have scored some legal victories in their efforts to keep their work alive.

In March, a California federal judge dismissed a lawsuit by X against the nonprofit Center for Countering Digital Hate, ruling that the litigation was an attempt to silence X’s critics.

Three months later, a ruling by the Supreme Court allowed the White House to urge social media companies to remove misinformation from their platform.

Jankowicz, for her part, has refused to give up. 

Earlier this year, she founded the American Sunlight Project, which says its mission is “to ensure that citizens have access to trustworthy sources to inform the choices they make in their daily lives.” Jankowicz told CNBC that she wants to offer support to those in the field who have faced threats and other challenges.

“The uniting factor is that people are scared about publishing the sort of research that they were actively publishing around 2020,” Jankowicz said. “They don’t want to deal with threats, they certainly don’t want to deal with legal threats and they’re worried about their positions.”

Watch: OpenAI warns of AI misinformation ahead of election