Skip to content

Disinformation and Platform Governance: Moderation, Alt-Tech, and Radicalization

Speaker: Kate Starboard, Center for an Informed Public, University of Washington
Host: Alexander Heffner, The Open Mind (PBS)
Year: 2021
Video ID: bUWfp3SPPBY
Program: The Open Mind, Channel 13 (PBS)

TL;DR

Kate Starboard, co-principal investigator at the University of Washington's Center for an Informed Public, discusses the state of disinformation in the aftermath of platform removals (Trump, QAnon), focusing on how moderation actions redirect harmful activity to alternative ("alt-tech") platforms rather than eliminating it. She emphasizes that disinformation campaigns are cross-platform efforts using different platforms in complementary ways, that algorithms and community norms shape platform culture and radicalization pathways, and that effective de-radicalization likely requires offline intervention (family conversations, community relationships) rather than platform design alone. The conversation explores the regulatory vacuum around platform moderation, the distinction between legally-required vs. ethically-required platform action, and the tension between holding large platforms accountable and enabling new platforms to emerge without entrenching existing power.

Key claims

Platform moderation redirects rather than eliminates disinformation: When platforms remove accounts (e.g., Trump, QAnon networks), users and organizers migrate to alternative platforms like Gab, Parler, and Clubhouse. The disinformation infrastructure doesn't disappear—it relocates. This creates a bifurcated information ecosystem where mainstream platforms and alt-tech platforms function as complementary spaces for coordinated campaigns.

Disinformation is inherently cross-platform: People organize and plan in niche, private spaces (alt-tech platforms, private forums) and then amplify content on mainstream platforms to reach larger audiences. Understanding disinformation requires analyzing activity across this network, not on any single platform.

Initial communities shape platform culture: Early adopters and their norms establish the baseline culture of new platforms. Clubhouse, for example, hasn't yet delineated boundaries around misinformation and extremism, unlike more established platforms. Platform trajectory depends on both designer choice and community dynamics.

YouTube is a "cesspool for disinformation": Because it hosts video content, YouTube serves as both a primary source of disinformation and a resource for campaigns that cite and share videos across other platforms. The scale of content makes moderation at scale extremely difficult.

Economic models enable disinformation: Platforms profit from engagement regardless of content veracity. Scaling moderation would require economic models that prioritize accuracy over engagement—a costly shift that might impede emergent platforms but impose competitive costs on large ones.

Regulatory frameworks do not exist: The U.S. has no legal requirement for platforms to moderate misinformation or disinformation. Twitter and Facebook's recent moderation efforts are discretionary, not mandated. Designing policy that addresses disinformation while respecting freedom of speech remains unresolved.

Algorithms and radicalization: The mechanisms by which algorithms amplify content and users become radicalized are understood in broad terms, but effective de-radicalization methods remain elusive. Platforms are designed to isolate users in filter bubbles, which enables radicalization but offers no pathways to de-radicalization.

De-radicalization is offline work: Meaningful de-radicalization likely happens through personal relationships—family conversations, community engagement, trusted intermediaries—not through algorithm changes. The challenge is keeping communication pathways open while cutting off pathways that drive people toward extremism.

Pandemic amplified vulnerability: COVID-19 created circumstances of high uncertainty, anxiety, and social isolation that made people susceptible to misinformation. The influx of new online users during lockdowns created opportunities for radicalization networks to recruit and expand.

Mechanisms discussed

Platform moderation dynamics: - Removing prominent accounts creates a "success" narrative but displaces activity to alt-tech platforms - Moderation scale: at-scale moderation is economically expensive and requires automated detection tools that don't yet exist at the necessary fidelity - Ethical vs. legal obligations: platforms face ethical pressure to moderate but lack legal mandate to do so in the U.S. - Platform power: large platforms have global influence over discourse, but regulatory frameworks haven't caught up

Radicalization pathways: - High-uncertainty crises (pandemic, election) create susceptibility to conspiratorial narratives - Conspiracy theories provide simple explanations for complex, frightening situations - Algorithms promote engagement, which often means promoting extreme content - Community norms on platforms either constrain or enable extremist rhetoric - Coordinated campaigns plan in private spaces and amplify on public platforms

Information ecosystems: - Mainstream platforms, cable news, and alt-tech platforms are interconnected rather than isolated - Disinformation spreads across this ecosystem via cross-referencing, video sharing, and coordinated amplification - Different platforms serve different functions in coordinated campaigns

De-radicalization challenges: - Platforms are poor venues for de-radicalization (they don't create shared reality) - Personal relationships and offline community work are more effective - Building common ground requires prolonged, patient engagement—not algorithm tweaks

Connections

Notes

This interview is valuable for its emphasis on cross-platform dynamics and the paradox of moderation: removing harmful content from mainstream platforms doesn't eliminate the behavior—it displaces it. Starboard's observation that platforms are profit-driven and that scaling moderation is economically difficult challenges techno-optimistic narratives about "better algorithms." She makes clear that a platform's choice to moderate is a choice, not an inevitability, and the costs of that choice fall differently on large, established platforms vs. emergent ones.

The distinction between "misinformation" (false, unintentional) and "disinformation" (false, deliberate) is crucial for policy design. Misinformation requires rapid, credible correction; disinformation requires detection, attribution, and deterrence. Conflating them leads to ineffective interventions.

Starboard's emphasis on personal relationships and community work as the basis for de-radicalization is grounded in observation rather than speculative theory. She notes that journalists doing on-the-ground reporting have learned more about radicalization mechanics than academic research has yet captured—a valuable epistemic humility.

The pandemic context is important: COVID created a moment of high uncertainty and isolation when misinformation about vaccines, public health, and government intent was particularly resonant. The influx of new online users during lockdowns created a recruitment environment for radicalization networks.

Finally, Starboard and the host highlight the regulatory vacuum: it's unclear what should be required of platforms, how to balance platform liability with free speech, and whether regulation would entrench the power of large platforms or enable competition. These remain open questions.