Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection

Beizhe Hu; Qiang Sheng; Juan Cao; Yuhui Shi; Yang Li; Danding Wang; Peng Qi

In brief

Large language models like GPT-3.5 perform poorly at fake news detection in zero- and few-shot settings, underperforming fine-tuned BERT. However, LLMs excel at generating multi-perspective rationales (textual descriptions, commonsense, factuality judgments) that small language models can leverage. The paper proposes ARG (Adaptive Rationale Guidance), a network that selectively incorporates LLM-generated rationales to improve BERT-based detection, outperforming both LLM-only and SLM-only baselines on Weibo and GossipCop datasets.

Contributions

Detailed investigation: Empirical evaluation of GPT-3.5's effectiveness at fake news detection across four prompting strategies (zero-shot, zero-shot CoT, few-shot CoT with perspective, few-shot CoT) on both Chinese and English datasets.
Architectural innovation: ARG network that enables adaptive collaboration between small and large language models, where SLMs exploit perspective-specific rationales from LLMs for improved veracity judgment.
Practical distillation: ARG-D, a rationale-free variant derived via knowledge distillation, maintains most gains while eliminating dependence on LLM availability for inference (cost-sensitive scenarios).
Public resource: Release of perspective-based rationale collection from GPT-3.5 for fake news detection in Chinese and English to facilitate further research.

Method

Detection architecture: The paper uses BERT as the base small language model, pre-trained models fine-tuned on Weibo21 (Chinese) and GossipCop (English) datasets with temporal splits to avoid data leakage.

LLM prompting: Four approaches tested on GPT-3.5-turbo:

Zero-shot prompting — task description + news item, no examples
Zero-shot chain-of-thought — adds reasoning instruction ("Let's think step by step")
Few-shot CoT prompting — four in-context examples (2 real, 2 fake) with reasoning steps
Few-shot perspective-specific prompting — examples annotated with textual description, commonsense, or factuality perspective

ARG network (Representation + Collaboration + Prediction):

Representation: BERT encoders for both news and rationales; parallel processing via two separate BERT models.
News-Rationale Collaboration: Dual cross-attention mechanism between news X and rationales from three perspectives (r_t, r_c, r_f). Cross-attention outputs feed into LLM judgment predictors for each perspective. Rationale usefulness evaluated via an MLP to weight contribution.
Prediction: Final judgment aggregates news vector x, perspective-weighted rationale vectors (f^x_→c, f^x_→t, f^x_→f) with learned weights for final classification.

ARG-D (distilled version): Initialized with ARG-trained parameters, uses mean-squared-error distillation loss on rationale-free scenarios to transfer knowledge when LLM access unavailable.

Results

LLM-only performance (Table 2):

GPT-3.5-turbo with best prompting approach: macroF1 0.725 (Chinese), 0.702 (English).
Fine-tuned BERT baseline (G2-SLM-only): macroF1 0.765 (Chinese), 0.765 (English).
LLM underperforms by ~5% on both languages, demonstrating the gap between LLM versatility and task-specific fine-tuning.

ARG performance (Table 5):

ARG outperforms all baselines: macroF1 0.784 (Chinese, +2.4%), 0.784 (English, +3.2%).
ARG-D maintains improvement with single perspective input (macroF1 0.778 Chinese, 0.778 English).
Ablation studies show both LLM Judgment Predictor and Rationale Usefulness Evaluator components are critical; removing either degrades performance significantly.

Rationale analysis (Table 3, perspective-specific):

Textual description rationales: 65% accuracy on Chinese, 71% on English (LLM perspective-based analysis often outperforms baseline BERT directly).
Commonsense rationales: 71% (Chinese), 60% (English).
Factuality analysis: 17% (Chinese), 24% (English) — poorest performance, attributed to hallucination in LLM reasoning.
Key finding: LLM generates human-like multi-perspective rationales, but factuality reasoning is unreliable; combining perspectives mitigates this weakness.

Cost-analysis (Figure 5): By routing only 23% of data through full ARG and using cheaper ARG-D for remainder, detection maintains F1 of 0.784 while reducing inference cost.

Notes

Strengths: Rigorous empirical design with clear research questions, bilingual evaluation, and practical cost-aware distillation. The distinction between LLM-as-detector (fails) and LLM-as-advisor (succeeds) is conceptually clear and actionable. ARG-D addresses real deployment constraints.

Weaknesses: 1) The paper only tests GPT-3.5-turbo; other LLMs (Claude, Ernie Bot) and closed-source variants might behave differently. 2) Factuality perspective produces worse performance (17–24%), not well understood; authors attribute to hallucination but don't deeply diagnose. 3) Multi-perspective judgment oracle (Table 4) assumes correct model per sample, unrealistic; real deployment must predict perspective usefulness blindly. 4) No analysis of what makes certain rationales helpful vs. misleading for SLMs.

Follow-up directions: Testing with frontier LLMs (GPT-4, Claude 3); interpretability analysis of which rationale features SLM actually exploits; human evaluation of LLM rationale quality; extension to multimodal fake news with vision-language rationales.