Key findings
- Content-based classification using LLMs consistently demonstrates strong accuracy across various influence campaigns, even more recent operations.
- Metadata-based approaches also achieve promising results, sometimes outperforming content-based methods.
- LLM-based methods show better generalization than traditional linguistic-cue baselines, which degrade significantly on recent, more sophisticated campaigns.
- The paper uses Llama 2 with zero-shot and few-shot prompting via Microsoft Guidance, plus fine-tuning with Llama Factory.
- Evaluated against established datasets: Twitter Information Operations, Russian Twitter Trolls, and campaigns from Egypt, Ecuador, and Venezuela.
Methodology
The authors propose four classification models based on:
- Content analysis — using LLM features extracted from tweets
- User interaction — analyzing user engagement patterns
- User centrality — leveraging network structure and graph properties
- User metadata — demographics, account properties, and auxiliary signals
Both tweet-level (binary: in-campaign or organic) and user-level classification (drivers of campaigns vs. organic users) tasks are addressed.
Notes
This is a methodological talk presenting work likely submitted to The Web Conference. The emphasis on cross-campaign generalization is notable — many detection systems degrade when tested on new operations, but these LLM-based approaches maintain performance across geographically diverse campaigns. The use of public influence-operation datasets (Twitter's official disclosures, the Russian troll datasets) makes this reproducible and grounded in real-world threat models.