Skip to content
Synergizing LLMs with Global Label Propagation for Multimodal Fake News Detection

Synergizing LLMs with Global Label Propagation for Multimodal Fake News Detection

Authors: Shuguo Hu, Jun Hu, Huaiwen Zhang

Venue: arXiv preprint, 2025 — arxiv:2506.00488

TL;DR

This paper proposes Global Label Propagation Network with LLM-based Pseudo Labeling (GLPN-LLM), a framework for multimodal fake news detection that integrates LLM-generated pseudo labels with label propagation via graph neural networks. A mask-based global label propagation mechanism prevents label leakage during training while enabling effective information propagation across multimodal samples. Experimental results on Twitter, PHEME, and Weibo datasets demonstrate superior performance over state-of-the-art baselines.

Contributions

  • GLPN-LLM Framework: Proposes a novel framework that synergizes LLM capabilities with label propagation techniques, leveraging LLM-generated pseudo labels to enhance multimodal fake news detection.

  • Mask-Based Global Label Propagation: Introduces a Global Random Mask (GRM) mechanism that prevents label leakage during training by masking label information of randomly selected nodes, ensuring nodes do not learn to rely on self-propagated labels.

  • Mixed-Initiative Labeling: Presents an approach that combines LLM-generated pseudo labels with structured prompting, using contextual information to enhance the reliability of pseudo label generation.

  • Multimodal Feature Extraction: Uses CLIP for unified feature extraction from both text and images, constructing a cross-modal graph with multiple similarity metrics (text-to-text, image-to-text, image-to-image, text-to-image similarity).

  • Experimental Validation: Demonstrates consistent improvements across three benchmark datasets (Twitter, PHEME, Weibo), with results showing the effectiveness of integrating LLM capabilities via label propagation.

Method

Multimodal Feature Extraction: The framework employs CLIP (Contrastive Language-Image Pre-training) to extract high-dimensional feature vectors for both visual and textual modalities. For each news item, a unified feature vector \(x_i = t_i \oplus v_i\) is created by concatenating text embedding \(t_i\) and visual embedding \(v_i\), where \(\oplus\) denotes concatenation.

Cross-Modal Graph Construction: A graph is constructed where each node represents a news item with features derived from CLIP. Edges are established based on multiple similarity metrics: (1) concatenated-feature similarity (cosine similarity between concatenated embeddings), (2) image-to-text similarity, (3) text-to-image similarity, (4) image-to-image similarity, and (5) text-to-text similarity. An edge is created between nodes \(i\) and \(j\) if their similarity exceeds a predefined threshold \(\theta = 0.95\).

Label Integration Module: Ground-truth labels and LLM-generated pseudo labels are integrated into node features. For node \(i\), the augmented feature vector is computed as \(x_i' = x_i \oplus y_i'\), where:

\[y_i' = \begin{cases} y_i & \text{if node } i \text{ is truly labeled,}\\ \hat{y}_i & \text{if unlabeled node } i \text{ is high-confidence,}\\ 0 & \text{otherwise.} \end{cases}\]

Global Random Mask (GRM): During training, label information is selectively masked to prevent label leakage. Given a mask ratio \(\rho\) (e.g., \(\rho = 0.3\)), a proportion \(\rho \times N\) of nodes are randomly selected, and their label embeddings are replaced with zero vectors:

\[y_i'' = y_i' \cdot m_i\]

where \(m_i\) is a binary mask scalar defined as:

\[m_i = \begin{cases} 0 & \text{if node } i \text{ is selected for masking,}\\ 1 & \text{otherwise.} \end{cases}\]

This prevents nodes from learning to propagate their own labels back to themselves during training.

Graph Convolutional Network (GCN) Inference: The augmented feature vectors \(x_i'\) are fed into a GCN, which propagates label and content information across the graph to predict labels for each node. Cross-entropy loss is used during training.

Mixed-Initiative Labeling Process: The framework constructs structured prompts incorporating both context (claim text, specific task requirements) and cleaned content to guide the LLM in generating reliable pseudo labels. Prompts follow the format: [CLS] <prompt> [SEP] <cleaned Twitter text>, enabling the LLM to perform detection and generate confidence scores.

Results

Overall Performance (Table 1): GLPN-LLM (CLIP backbone) achieves state-of-the-art results across all three datasets:

  • Twitter: 88.83% accuracy, 84.02% precision, 92.68% recall, 90.93 F1
  • PHEME: 86.47% accuracy, 89.24% precision, 92.13% recall, 90.66 F1
  • Weibo: 86.74% accuracy, 89.83% precision, 93.27% recall, 91.52 F1

GLPN-LLM consistently outperforms strong baselines including FCN-LP (CLIP), which integrates label propagation without LLM-generated pseudo labels, demonstrating the effectiveness of synergizing LLMs with label propagation.

Ablation Study (Table 2): Isolates the contribution of individual components:

  • GLPN (without LLM): Achieves 85.47% accuracy on Twitter; incorporates basic label propagation
  • GLPN + LLM (naive integration): Achieves 85.37% accuracy; shows that naive LLM integration does not improve performance
  • GLPN-LLM (full framework): Achieves 88.83% accuracy, demonstrating that the mask-based propagation mechanism is critical for effective integration

Mask Rate Analysis (Figure 3): A mask rate of 0.5 yields optimal performance, with the model achieving 86.80% accuracy, 86.3% precision, and 91.8% recall. The results indicate that balancing label availability and masking for label leakage prevention is crucial for effective propagation.

Pseudo Label Quantity Analysis (Figure 3): Incorporating pseudo labels from the top 5% confidence scores yields the highest performance improvements. Including pseudo labels beyond the 5% threshold introduces noise into the label propagation process, potentially degrading performance.

Connections

Notes

Strengths: The paper addresses a genuine limitation of LLM-generated pseudo labels—their individual unreliability—through a novel mask-based global label propagation mechanism that leverages graph structure to enhance collective reliability. The experimental validation across three diverse datasets (Twitter, PHEME, Weibo) with consistent improvements demonstrates the generalizability of the approach. The ablation studies clearly isolate the contribution of each component, showing that naive LLM integration is insufficient and the masking mechanism is critical. The mask rate and pseudo-label quantity analyses provide practical insights for practitioners.

Weaknesses: The paper's reliance on CLIP for multimodal feature extraction may limit applicability to domains where CLIP's zero-shot capabilities are weak. The threshold \(\theta = 0.95\) for graph edge construction is not well justified and may not generalize across different datasets or modalities. The framework assumes that connected nodes in the constructed graph have aligned labels, which may not hold in adversarial or manipulated content scenarios. The paper lacks discussion of computational complexity and scalability to larger graphs. Limited analysis of failure cases or scenarios where the mask-based mechanism might hinder performance.