Adversarial Attacks on Misinformation Detection

Adversarial attacks on misinformation detection systems exploit vulnerabilities in automated models to force misclassifications. These attacks are particularly relevant in the context of fake news and rumor detection, where bad actors may strategically craft content to evade detection.

Attack patterns

Message injection attacks: Attackers inject crafted messages into propagation structures (e.g., message propagation trees) to alter the signals the model uses for classification. Recent work shows that Large Language Models can generate semantically coherent malicious messages that fool detection systems.

Evasion techniques: Adversarial perturbations at the text, graph, or user level can cause detection models to flip their predictions.

Key papers

• SINcon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor: Addresses LLM-generated malicious message injection attacks on MPT-based rumor detection; proposes contrastive learning defense to balance node influence and improve robustness.

Related topics

• Adversarial machine learning: broader field of adversarial robustness.
• Robustness in misinformation detection: defensive perspectives.
• Large Language Models: sources of sophisticated attack content.