Contributions
- Proposes a conceptual framework distinguishing satirical cues from other forms of deception, focusing on how satire uses rhetorical devices and linguistic patterns
- Introduces a manually annotated dataset of 360 news articles across 4 domains (civics, science, business, "soft" news) and 12 topics, balanced between satirical and legitimate sources
- Develops five linguistic feature classes for satire detection: Absurdity (unexpected named entities), Humor (semantic distance in punchline detection), Grammar (phrase complexity), Negative Affect (negative emotion words), and Punctuation patterns
- Demonstrates that combined linguistic features (particularly Absurdity) substantially improve over baseline tf-idf approach, with best model achieving 90% precision, 84% recall, 87% F-score
Method
The paper employs a supervised machine learning approach using scikit-learn SVM with the following components:
Dataset: 360 articles collected from 2 satirical news sites (The Onion and The Beaverton) and 2 legitimate news sites (Toronto Star and New York Times), stratified across 4 domains and 12 topics. Data split: 75% training (270 articles), 25% testing (90 articles), with 10-fold cross-validation.
Features: The pipeline extracts five feature classes:
- Absurdity (Abs): Measures unexpected introduction of new, named entities (people, places, locations) in final sentences using Natural Language Toolkit Named Entity Recognition; calculated as intersection of entities between final sentence and remainder of article
- Humor (Hum): Based on semantic distance between lead and final sentences using WordNet taxonomy depth and least common subsumer; hypothesis that humor arises from semantic discontinuity
- Grammar (Gram): Normalized term frequencies from Linguistic Inquiry and Word Count (LIWC) 2015 dictionary capturing parts of speech and linguistic categories
- Negative Affect (Neg): Presence of negative emotion words and punctuation from LIWC 2015
- Punctuation (Pun): Frequency of punctuation marks (periods, commas, colons, semi-colons, question marks, exclamation points)
Classification: Uses scikit-learn SVM with linear kernel, tf-idf weighting for text preprocessing, and hyperparameter tuning via grid search.
Results
Feature ablation experiments show incremental improvement when features are added cumulatively:
- Base (tf-idf only): 78% precision, 87% recall, 82% F-score
- Base + Abs: 85% precision, 89% recall, 87% F-score
- Base + Hum: 80% precision, 87% recall, 83% F-score
- Base + Gram: 93% precision, 82% recall, 87% F-score
- Base + Neg: 81% precision, 84% recall, 83% F-score
- Base + Pun: 93% precision, 82% recall, 87% F-score
- All features combined: 88% precision, 82% recall, 87% F-score
The best-performing combination (Base + Gram + Pun + Abs) achieves 90% precision, 84% recall, and 87% F-score. Absurdity showed the highest individual recall contribution, while punctuation and grammar jointly contributed to precision gains.
Notes
This is an early systematic study of satire as a detection problem. The paper's strength lies in explicitly grounding satire in linguistic theory (incongruity theory of humor, semantic validity imbalances) and demonstrating that rhetorical features can be operationalized. The dataset is relatively small (360 articles) and limited to specific sources and domains; generalization to new sources and broader domains remains unclear. The Absurdity feature is intuitive but sensitive to domain-specific entity distributions. The paper frames satire as unambiguously deceptive, but acknowledges the complexity that satire can be misunderstood by readers with different backgrounds or cultural contexts. This work laid groundwork for later studies on humor and irony in social media misinformation.