Media Bias/Fact Check (MBFC)¶

Media Bias/Fact Check (MBFC) is a comprehensive manual taxonomy of news media outlets annotated for factuality and political bias. Rather than annotating individual articles or claims, MBFC assigns outlet-level labels to enable media profiling and source reliability assessment.

Structure¶

Outlets: 1,066+ news media outlets (as of the papers using MBFC in this wiki)

Factuality labels (3-point scale): - High: Outlets that publish factual information with minor errors; corrections issued when needed - Mixed: Mix of factual and false content; may have corrections but inconsistent - Low: Outlets publishing satire, pseudo-science, conspiracy theories, or significant false content; may lack corrections

Political bias labels (7-point scale): - Extreme-left: Far-left ideological positioning; often conspiratorial or highly partisan - Left: Left-leaning; pro-Democratic (US context) - Center-left: Slightly left of center; generally factual with minor left-leaning editorial bias - Center: Centrist; minimal political bias; balanced coverage - Center-right: Slightly right of center; generally factual with minor right-leaning bias - Right: Right-leaning; pro-Republican (US context) - Extreme-right: Far-right ideological positioning; often conspiratorial or highly partisan

Supplementary labels: - Media type (news, opinion, mixed, satire) - Country of origin - Bias category (gatekeeping bias, editorial bias, political bias, scientific bias)

Annotation methodology¶

MBFC uses volunteers trained on a detailed methodology to ensure objectivity. Each outlet is evaluated based on: - Published articles and editorial content - News selection and coverage patterns (gatekeeping) - Tone and framing of reporting - Track record of corrections or retractions - Transparency about ownership and funding

Annotations are updated iteratively; readers can provide feedback, and annotations are adjusted after review if large discrepancies arise.

Usage in this wiki¶

Papers using MBFC for outlet-level profiling: - Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media: Uses 949 outlets (excluding 117 satire/pseudo-science outlets) for ordinal regression on factuality and bias - What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context: 1,066 outlets, predicting bias and factuality via text, social media, and Wikipedia features - A Survey on Predicting the Factuality and the Bias of News Media: Survey reviewing source profiling work using MBFC and similar datasets

Caveats¶

Annotation scope: Ratings are global—a single outlet factuality/bias label. Outlets may vary in accuracy across topics or time periods.

Temporal dynamics: Ratings change over time as outlets evolve; the dataset does not capture how outlets' reliability shifts.

Geographic bias: Majority coverage of US/Western media; limited coverage of non-English or non-Western outlets.

Subjectivity: Despite methodology, political bias assessment has inherent subjectivity and can vary by annotator perspective.

Label distribution imbalance: More "high" factuality outlets than "low"; extreme-left and extreme-right are minority classes.

FakeNewsNet: Article-level labels from PolitiFact and GossipCop fact-checkers; rich social context
NELA-GT-2018, NELA-GT-2019, NELA-GT-2020, NELA-GT-2022: News outlet-level features (NELA toolkit) and labels; non-proprietary
AllSides: Similar outlet-level bias ratings (3-point: left/center/right scale)

Access¶

MBFC website: https://mediabiasfactcheck.com/ Downloadable datasets (with research permission): Limited programmatic access; most research uses manual collection or indirect access via published papers.