CoVer: Conflict-Aware Claim Verification

Authors: Shuning Zhang, Dai Shi, Bohao Chu, Hui Wang, Yuwei Chuai, Yifan Wang, Jingruo Chen, Simin Li, Xin Yi, Hewu Li

Venue: arXiv cs.CL, 2026 — arXiv:2609.00508

TL;DR

Social media fact-checking frequently encounters conflicting evidence where different sources support or refute a claim. This paper proposes CoVer, a framework that explicitly resolves evidence-level conflicts (contradictory stances from different sources) and aggregation-level conflicts (sub-claims within complex claims yielding different verdicts). The method uses evidence schema normalization, factual consensus scoring, and support verification. The ContraNote dataset provides 88K instances of real conflicting evidence from X's Community Notes.

Contributions

• CoVer framework: A three-stage pipeline (evidence schema normalization, factual consensus, support verification) that prioritizes evidence based on quality signals and grounds verdicts in selected supporting evidence rather than aggregating all evidence equally.
• ContraNote dataset: A large-scale dataset with 33,686 conflicting instances (evidence-level) and 54,474 prioritization instances (aggregation-level) derived from X's Community Notes system.
• Empirical validation: Achieves 86.0% accuracy on ContraNote Conflict and 88.5% on Prioritization, outperforming SOTA baselines including CONFACT (86.0% / 88.5%), FIRE (64.5%), and FactCheckGPT (63.5%).

Method

CoVer addresses two types of conflicts in claim verification:

Evidence-level conflict: Retrieved documents present contradictory stances on a single fact. For example, a news story about COVID-19 vaccine safety may be interpreted as "vindicating" the vaccine by journal epidemiologists but as "implicating" it by anti-vaccine commenters. The system must determine which source is authoritative.

Aggregation-level conflict: A complex claim decomposes into sub-claims (e.g., causal and observational components) that yield conflicting verdicts. Refuting one does not necessarily refute the whole.

Stage 1: Evidence schema normalization. Each evidence item e_i contains textual content and structured metadata (source URL, community helpfulness ratings). The module normalizes evidence into a canonical representation z_i with standardized cues: directness (whether the evidence directly addresses the claim), attribute alignment (entity and temporal consistency), schema reliability (source authority via metadata), and informativeness (avoiding repetitive or speculative content).

Stage 2: Factual consensus. The system jointly predicts a normalized evidence item's stance (s_i ∈ {support, refute, irrelevant}) and four fine-grained quality components via a single LLM call with structured constraints, outputting a deterministic quality score q_i. These scores are aggregated across evidence items to compute supporting and refuting strength, with the label chosen by majority-vote comparison of cumulative strengths. Ties are broken conservatively as "Refute" to avoid unfounded positive claims.

Stage 3: Support verification. When the factual consensus verdict is "Supported," an independent verifier checks whether selected supporting evidence directly entails the claim or merely discusses related content, rejecting support if the evidence is vague, quotes are out-of-context, or factual statements are contradicted by stronger evidence.

Results

Evaluated on multiple datasets representing different conflict levels:

• ContraNote Conflict: 86.0% Acc., 64.5 balanced Acc.
• ContraNote Prioritization: 88.5% Acc., 89.2 balanced Acc., 88.4% F1 on CONFACT-HumC
• CONFACT-ModC: 81.3% Acc. (88.5% without conflict; improvement driven by conflict resolution)
• ECON: 77.5% Acc. (baseline ECON 74.0%)
• FEVER: 93.4% Acc. (Confact 87.0%; FactCheckGPT 81.5%)

Ablation studies confirm each module's contribution: removing evidence schema normalization drops performance from 77.5% to 45.5% on ECON; removing factual consensus drops ContraNote Conflict from 86.0% to 69.0%; removing support verification degrades performance across all datasets.

Connections

• Extends evidence-ranking and conflict modeling beyond Evidence Retrieval approaches.
• Builds on LLM-based fact-checking frameworks using structured LLM outputs with constraints.
• Applies Natural Language Inference to evidence adjudication and quality scoring.
• Cited data: Community notes as a source of real-world conflicting evidence and human judgments.

Notes

Strengths: The paper addresses a genuine and understudied challenge in real-world fact-checking: evidence conflicts are common on social media, yet most prior work assumes consistent evidence. The ContraNote dataset is well-curated from actual Community Notes (with inter-annotator agreement of 0.82), providing a realistic benchmark. The three-stage pipeline is modular and interpretable, with clear validation of each component's contribution.

Weaknesses: The work is limited to textual claims and English-language evidence; efficacy on multimodal misinformation (deepfakes, manipulated images) and specialized domains (medical, legal) remains unclear. The framework treats Community Notes consensus as ground truth, which may reflect majority bias rather than factual accuracy, though the authors acknowledge this limitation. Computational cost analysis is included (5.8s per claim, $0.0059 per request) but efficiency on very large evidence pools is not explored.

Follow-up: Future work should explore conflict resolution in other languages and multimodal contexts, validate whether CoVer generalizes to non-social-media claims, and investigate whether the factual consensus module's reliance on LLM-generated quality signals can be improved with external knowledge bases or expert rankings.