Skip to content

Multi-class classification

Multi-class classification extends binary classification to settings where each example belongs to one of C > 2 classes. Unlike binary classification (true/false, positive/negative), multi-class problems require distinguishing among three or more distinct categories.

Motivation in fake news detection

Early fake news detection systems treated the problem as binary: real vs. fake. However, real-world news credibility is often more nuanced. The LIAR dataset, for example, uses a six-way classification scheme: pants-fire, false, barely-true, half-true, mostly-true, true. This finer granularity better reflects the spectrum of statement accuracy and helps researchers and practitioners understand how false a claim is.

Advantages of multi-class over binary: - Captures nuanced distinctions in credibility that binary classification flattens - Provides interpretability: a prediction of "half-true" is more informative than "false" - Reflects real editorial and fact-checking workflows (PolitiFact, FactCheck.org, and others use ordered scales) - Enables ordinal relationships: "barely-true" is closer to "half-true" than to "pants-fire"

Challenges

Label imbalance: Real-world multi-class datasets often have imbalanced class distributions. LIAR has roughly equal representation across classes, but other datasets may not. Imbalance makes naive accuracy misleading and requires careful metric selection (macro F1, macro precision/recall).

Ordinal structure: Fake news credibility labels have natural ordering (false < barely-true < mostly-true < true). Some errors are worse than others: predicting "pants-fire" when the truth is "mostly-true" is worse than predicting "barely-true." Ordinal regression and semantic distance metrics can exploit this structure.

Increased model complexity: Predicting C classes instead of 2 typically requires more model capacity and training data. Methods must learn to distinguish among more decision boundaries.

Approaches

Traditional methods: Multi-class logistic regression, decision trees, and SVMs with one-vs-rest or one-vs-one decomposition.

Deep learning: Neural networks naturally support multi-class prediction via softmax over C output nodes. Cross-entropy loss generalizes to C classes directly.

Ordinal-aware methods: Ordinal classification methods that encode label ordering. DHGAT uses a custom loss function combining cross-entropy (for classification accuracy) with semantic distance (to penalize ordinal violations).

Key papers

A Decision-Based Heterogeneous Graph Attention Network for Multi-Class Fake addresses six-class fake news classification on LIAR, introducing a semantic distance loss term that reflects the ordinal nature of credibility labels. Demonstrates that respecting label ordering improves accuracy by penalizing mistakes more heavily when they span large ordinal gaps.