Propasafe-Hybrid: A Text-Based Hybrid Propaganda Detection Tool

Authors: Thomas Kimmeth, Avijit Roy, Vivek Sharma

Affiliation: John Jay College of Criminal Justice, CUNY

Venue: arXiv cs.CL, October 2026 — arXiv:2610.33464v1

TL;DR

Propasafe-Hybrid is a sentence-level propaganda detection tool that combines an offline BERT classifier with LLM-based technique identification. The hybrid architecture reduces API token usage (filtering 80–87% of sentences locally) while delivering interpretable technique labels and explanations at the point of consumption. The tool extends the privacy-focused Propasafe framework with model-supported explainability, targeting real-world deployment in browser extensions and news readers.

Contributions

• Hybrid detection pipeline: Offline BERT classifier serves as a cost-aware pre-filter, forwarding only high-confidence sentences to LLM APIs for technique classification and explanation.
• Explainability module: Structured prompts paired with propaganda technique taxonomy (17 closed labels) enable LLM to return both technique labels and concise rationales.
• Token efficiency: Pre-filtering reduces API calls by ~80–87%, cutting inference cost while maintaining visual feedback from BERT confidence scores.
• User-facing design: Color-coded severity highlighting (yellow for 0.50–0.75 confidence, red for > 0.75) with inline technique explanations help non-expert readers understand not just what is propagandistic but why.
• Extensible architecture: Modular design reuses existing components (article extraction, BERT classification) from the original Propasafe tool, enabling straightforward addition of new explainability backends.

Method

Pipeline stages:

1. Article Extractor: Fetches raw HTML via requests; applies four-stage cascade (DOM pre-cleaning with lxml, text extraction via trafilatura/readability-lxml/newspaper3k in priority order); normalizes sentence boundaries with regex and NLTK Punkt tokenizer. Output: sentence list (capped at 400).

2. Offline Classification Module: Fine-tuned BERT model (trained on Propaganda Techniques Corpus) classifies each sentence as propagandistic (probability ≥ 0.50) or not. Reports F1 ≈ 0.71 on PTC dataset. Binary output is authoritative for visual highlighting.

3. Cost-Aware Pre-Filtering: Only sentences with BERT probability ≥ 0.50 are forwarded to the LLM stage. Threshold set at 0.50 (natural decision boundary) to maximize recall; borderline cases surface as llm_status=empty in output.

4. Explainability Module: Each forwarded sentence paired with structured prompt including propaganda technique definitions (17 closed labels from PTC taxonomy). Prompt sent to OpenAI API (default: gpt-5.2-2025-12-112, configurable). LLM returns predicted technique(s) and reasoning. Output validated against closed taxonomy; out-of-taxonomy labels silently dropped.

5. Conflict Resolution: BERT classification is authoritative. LLM disagreement does not suppress visual feedback; highlighted sentences remain visible even if LLM returns no recognized technique. llm_status field tracks four states: not_run, ok, empty, error.

Verbosity modes (low, medium, verbose) control word limits on evidence/explanation fields, allowing cost-latency trade-offs.

Results

• Pre-filtering efficiency: ~80–87% of sentences filtered by BERT, reducing LLM forwarding from ~40 sentences (typical mid-length article) to ~5–8 sentences.
• User interface: Glance view shows percentage of flagged sentences, total sentence count, and breakdown by severity (safe/mild/severe). Highlighted sentences annotated with LLM technique labels and rationales.
• Representative output: System successfully identifies Loaded Language, Obfuscation/Intentional Vagueness, and other techniques with human-readable explanations.

Connections

• Extends Propasafe (Sharma et al. 2025a) with LLM-backed explanations while preserving privacy-first offline architecture.
• Builds on fine-grained propaganda annotation and the SemEval-2020 Task 11 standardization.
• Shares the ecosystem of propaganda detection tools with document-level approaches (Proppy) and browser-extension-based systems (Apollolytics).
• Related to work on Propaganda detection, Explainability in misinformation detection, and practical deployment of misinformation detection systems.

Notes

Strengths: - Practical hybrid design balances cost and interpretability — a genuine trade-off for deployed tools, not just an academic exercise. - Clear architectural separation between local BERT (authoritative for visual feedback) and remote LLM (enriches with explanations) reduces coupling. - Solid engineering: handles API failures gracefully, supports multiple verbosity modes, Docker packaging for deployment. - Grounded in established datasets (PTC) and evaluation protocols (SemEval-2020 Task 11).

Limitations acknowledged: - Single-provider dependency (OpenAI) constrains cost/latency tuning and creates vulnerability to model version changes. - Current default hybrid mode limits user control over privacy-reliability trade-offs; future work proposes offline/hybrid/online mode selector. - No user study comparing effectiveness vs. alternatives like Apollolytics; evaluation limited to architecture and design rationale.

Open questions: - How do readers with varying media literacy interpret the color-coded severity cues and LLM explanations? (HCI evaluation pending.) - Does BERT's ~0.71 F1 on PTC reflect real-world propagandistic content in diverse news sources, or does distribution shift degrade performance? - Closed 17-technique taxonomy may under-represent emerging manipulation strategies; dynamic taxonomy updates not discussed.