Contributions
- Establishes first dataset of rumors on Sina Weibo with official, high-quality labels (κ=0.95) from the platform's rumor-busting service
- Identifies significant differences between Sina Weibo and Twitter for rumor analysis: linguistic features vary (Chinese vs. English), trending patterns differ, and Weibo has an official verification service
- Proposes two novel platform-specific features not previously studied: (1) client program type (web-based vs. mobile-based posting), (2) event location (domestic vs. foreign)
- Validates statistical independence between these features and rumor truthfulness using χ² tests
- Demonstrates that incorporating new features improves classification accuracy from ~72% to ~78% across all feature subsets
Method
Data collection: Keywords from Sina Weibo's official rumor-busting account were used to retrieve microblogs via Twitter Monitor's keyword-based query interface. Collection period: March 1, 2010 to February 2, 2012. Total dataset: 5,144 microblogs manually annotated by two annotators to eliminate off-topic messages and label orientation (consistent with rumor vs. inconsistent with rumor).
Feature extraction: 19 features organized into five categories:
- Content-based (4): multimedia presence, sentiment, URLs, time span since registration
- Account-based (8): account verification, gender, avatar type, follower/friend counts, number of posted microblogs, user name type, registration location
- Propagation-based (3): retweet status, comment count, retweet count
- Client-based (1, novel): web-client vs. mobile-client
- Location-based (1, novel): event location (domestic China vs. abroad)
Classification: SVM with RBF kernel (γ=0.313, optimized via 10-fold cross-validation).
Results
Effect of established features: Content-based features alone achieve 72.6% accuracy but are weak at detecting false information (F-score 0.25). Account-based features achieve 72.6% accuracy and perform better at false-information detection (F-score 0.40). Propagation-based features alone perform worst (72.3%).
New features significance: χ² independence tests reject the null hypothesis for both client program (χ²=32.06 >> 3.84 threshold) and event location (χ²=735.27 >> 3.84 threshold), confirming statistical dependence with rumor truthfulness.
Performance improvement: Adding client program and event location features together to all existing features increases accuracy by 4.7–6.3 percentage points:
- Content-based + both new features: 78.0%
- Account-based + both new features: 77.4%
- Propagation-based + both new features: 78.7%
Key finding: 71.8% of false rumors were posted via web-based clients; 56.1% of false rumors concerned foreign events (vs. 82.3% of true rumors about domestic events).
Notes
This is the first peer-reviewed study of rumor detection on Sina Weibo despite the platform's scale (300+ million users, 8× Twitter's size at the time). The paper highlights critical differences in how rumors propagate across culturally and structurally distinct platforms: linguistic features don't transfer, client patterns differ, and the presence of an official verification service changes the verification dynamics.
The two proposed features are simple but effective: they capture user and event characteristics that correlate strongly with rumor truthfulness. The work foreshadows later findings on how posting clients and geographic scope can be signals of misinformation.
Dataset quality is notable: rather than crowdsourced annotations (which introduce noise), this work leverages Sina Weibo's official rumor-busting account as ground truth, yielding high inter-rater agreement (κ=0.95) and ensuring labels reflect real-world verification outcomes.