Skip to content
Spread & networksPlatforms, policy & society videoinfluence campaignsrussian interference

Beyond "Bots and Trolls" — Understanding Disinformation as Collaborative Work

Stanford Seminar · 2019
Open on YouTube

In brief

Kate Starbird presents a nuanced framework for understanding disinformation not as false posts or fake accounts, but as coordinated campaigns that infiltrate and cultivate online communities. Drawing on three case studies—Russian interference in the 2016 US election via the Internet Research Agency (IRA), the anti-white-helmets campaign during the Syrian civil war, and conspiracy theories about crisis events—she argues that disinformation's power lies in its recruitment of unwitting crowds of sincere activists and believers who amplify false narratives without realizing they're part of a larger manipulation strategy. The key insight: solutions focused on detecting bots and paid trolls miss the majority of the problem.

Key claims and cases

Misinformation vs. disinformation

Disinformation is intentionally false information deployed by influence agents to manipulate information spaces and achieve strategic political objectives. This differs from misinformation, which is false but not necessarily deliberately spread. Starbird borrows Ladislav Bittman's historical definition (Bittman was a Soviet intelligence defector who studied disinformation tactics).

Case 1: 2016 U.S. Election — Russian interference via the Internet Research Agency (IRA)

In October 2017, Starbird's lab published a paper on framing contests in Black Lives Matter discourse on Twitter—comparing pro-BLM and anti-BLM activist accounts. Weeks later, Twitter released its list of IRA accounts, and Starbird's team discovered that several IRA troll accounts appeared in their own dataset.

Key findings:

  • IRA accounts operated on both sides of the Black Lives Matter conversation, amplifying division
  • These accounts created "multi-dimensional online personas" impersonating Black Lives Matter activists on the left and conservative activists on the right
  • IRA trolls cultivated authentic-seeming audiences, often using content indistinguishable from grassroots activism
  • Nearly 85% of IRA retweets came from ~300 accounts running political personas; they were most effective not at spreading fake news but at infiltrating and shaping genuine political communities
  • Total activity: ~3,000 IRA accounts sending ~3 million tweets in English over multiple years
  • Success came after they established political personas in late 2015; before that, they toiled in obscurity with few retweets

Strategic insight: The campaign was orchestrated—centrally coordinated inside a single building in St. Petersburg—and designed to sow division and shape political discourse toward desired ends (pro-Trump messaging on the right, anti-Clinton messaging on the left).

Case 2: White Helmets campaign during the Syrian civil war

The Syrian Civil Defense (White Helmets) is a volunteer rescue organization that documents atrocities and chemical weapons attacks by Assad's government and Russian allies. In 2016, they were featured in an Oscar-winning documentary, sparking a coordinated counter-campaign.

Key findings:

  • Anti-white-helmets activists dominate the discourse; the red (anti-) cluster vastly outweighs the blue (pro-) cluster in retweet networks
  • Most dominant accounts are self-described journalists and activists—not bots; Starbird found no automated accounts contributing meaningfully to the campaign
  • These activists are sincere believers in their narratives; many identify as anti-imperialist, anti-war, and are part of overlapping online communities (pro-Palestinian, anti-US-military-intervention)
  • Russian government media (RT, Sputnik) amplifies these voices, provides platforms for aligned journalists, and hosts content on "gray propaganda" sites that repeat the same narratives
  • The campaign is cultivated rather than orchestrated: Russian state media provides a megaphone and strategic platform for organic activists whose views align with state interests
  • Key narratives attacked: portraying white helmets as propaganda constructs, terrorists, or crisis actors; claiming funding from Western governments makes them inauthentic

Strategic insight: This is a more mature stage of a disinformation campaign than the IRA case—it's less about impersonation and more about strategic amplification of existing activist communities whose indigenous narratives happen to serve state objectives.

Case 3: Conspiracy theories and crisis events

Starbird's earliest case study examined rumors spreading after the Boston Marathon bombing (2013). She observed a recurring pattern: each crisis event (Boston bombing, Sandy Hook, etc.) spawned conspiracy theories claiming it was a "false flag," a "hoax staged by crisis actors," or orchestrated by hidden perpetrators and the CIA.

Key findings:

  • These conspiracy narratives formed an ecosystem supported by websites like InfoWars, RT, Sputnik, various activist blogs, and "gray propaganda" sites
  • The ecosystem contained anti-government, anti-media, anti-science, and anti-Semitic content, unified by populist and conspiratorial ideologies
  • State-controlled media (Russian, Iranian) were present in the ecosystem, amplifying conspiratorial narratives when useful
  • Communities developed a shared epistemology: new events are interpreted through old conspiratorial frameworks; for each new crisis, they generate new theories that fit the existing meta-narrative (powerful people manipulating events, mainstream media lying)
  • This conspiracy thinking later mainstreamed into QAnon and influenced political leaders and media figures
  • This is organic conspiracy theorizing with strategic amplification, not centrally orchestrated; communities self-generate narratives; state actors amplify when aligned with interests

Analysis: From orchestrated to cultivated to organic

Starbird identifies a spectrum:

  1. Orchestrated: Centrally controlled (IRA, St. Petersburg building), using fake personas, designed from the top down to infiltrate communities
  2. Cultivated: Strategic amplification (RT, Sputnik, gray propaganda) of existing organic activism that aligns with state objectives; activists are sincere, not paid
  3. Organic with strategic amplification: Communities develop conspiracy theories independently; state actors and aligned media amplify when useful; communities don't need priming

How disinformation works: Key mechanisms

Layering truth with lies

Disinformation is not simply false information. Effective disinformation layers truth with lies, building on a plausible or true core and omitting details or adding new ones to shape a specific narrative. Example: white helmets receive Western funding (true) → therefore they're propaganda constructs and crisis actors (strategic inference, not necessarily true).

Recruitment of unwitting crowds

The central thesis: disinformation is collaborative work. Influence agents don't just spread content; they infiltrate communities, model behaviors, and recruit unwitting crowds of sincere activists who amplify narratives and generate new propaganda without realizing they're part of a larger campaign. Bittman called these "unwitting agents."

Multi-channel, multi-dimensional campaigns

Disinformation isn't a single post or account but a campaign. The 2016 IRA operation included Twitter, Facebook, Instagram, Tumblr, SoundCloud, web comments, email lists, and stolen data releases (hacked emails), all coordinated toward the same political objectives.

Targeting vulnerability through epistemology

Communities embedded in conspiracy-minded ecosystems develop a particular way of knowing (epistemology) that makes them vulnerable to new disinformation. Once a conspiracy framework is in place, new information gets filtered through it, and new evidence is retroactively fit to old theories.

Implications for detection and platforms

Why "bots and trolls" framing fails

Most research and platform policy focus on detecting automated accounts and inauthentic behavior. But Starbird's research shows:

  • In the White Helmets case, there were no bots and few paid trolls; the dominant amplifiers were sincere activists
  • Identifying disinformation requires understanding the campaign, not just individual posts or accounts
  • A person doesn't need to be aware they're part of a disinformation campaign to be a participant in it
  • Platforms cannot silence sincere activists without also silencing legitimate political speech

Why we're all vulnerable

Researchers studying these disinformation campaigns—including Starbird's own team—were affected by the narratives. When content aligns with existing beliefs (e.g., left-leaning researchers seeing anti-imperialist narratives in the white helmets case), it's hard to recognize as manipulation, even with full knowledge that you're studying a disinformation campaign.

Consequences: Erosion of shared reality

  • Diminished trust in information: Disinformation reduces our ability to know whom or what to trust
  • Loss of agency: When we don't know what's true, we lose our ability to act decisively
  • Democratic erosion: Disinformation tears at the social fabric, amplifying existing divisions; when divisions become too extreme, democratic deliberation breaks down; people may turn to authoritarian leadership

Platform and policy recommendations

Starbird argues that platforms should:

  1. Understand disinformation as a campaign with intent. Focus on the strategic intent to mislead for political purposes, not just on the truthiness of individual posts
  2. Trace disinformation through intermediaries. Action against a campaign should follow disinformation downstream, even through unwitting crowds far removed from the original influence agents
  3. Acknowledge platform values. Neutrality is a fiction; platforms embed values through design. Starbird encourages platforms to identify and own their values (e.g., supporting democratic discourse) and anchor themselves to them
  4. Avoid false balance. Attempts to address disinformation will be attacked as "biased" or "not neutral" by actors benefiting from manipulation. This is not a reason to abstain

Notes

This talk is essential for reframing the disinformation problem away from "bots vs. humans" and toward systems-level understanding. Starbird's three case studies form a useful taxonomy: orchestrated campaigns (IRA) vs. cultivated campaigns (White Helmets) vs. organic movements with strategic amplification (conspiracy theories). This framework helps explain why one-dimensional solutions (detecting bots, labeling fake accounts) fail.

The emphasis on unwitting crowds is the talk's most important contribution. Research and policy have focused heavily on inauthentic actors (the orange accounts in Starbird's graphs); but the amplification comes from the blue and pink accounts—real people, real activists, real believers. Silencing or removing the orange accounts without addressing how the blue and pink are targeted and incorporated into the campaign leaves the underlying vulnerability intact.

The personal anecdotes—Starbird's team struggling to recognize anti-white-helmets content as propaganda because it aligned with their left-leaning, anti-imperialist views—underscore that this is not a problem of ignorant people falling for obvious lies. It's a problem of epistemology and vulnerability. Smart, well-informed researchers can be disoriented by well-crafted disinformation. This insight should humble technologists and policy makers who think better algorithms or fact-checks will solve the problem.

Finally, the call to "chip away" at the problem incrementally—technology design, education, journalism, policy—rather than looking for a single solution reflects maturity about the scope of the challenge. Disinformation is a socio-technical problem, not a technology problem.

How it connects

Similar papers

Discussed in GUIDE by

Summarized from the full text. Added 2026-11-10 from a submission by reza. Every statement on this page traces back to the source.