Skip to content
VietPrism: A Large-Scale Vietnamese Speech and Deepfake Corpus with Diverse Dialects and Code-Switching

VietPrism: A Large-Scale Vietnamese Speech and Deepfake Corpus with Diverse Dialects and Code-Switching

Authors: Minh Hoang, Thai Le
Venue: arXiv cs.CL, 2026 — arxiv:2609.30005

TL;DR

VIETPRISM is a large-scale Vietnamese speech dataset combining 993.4 hours of bona fide utterances from 1,262 speakers with 3.1K hours of synthetically generated speech from four deepfake systems. The dataset uniquely provides transcripts, dialect annotations, speaker identities, and natural Vietnamese–English code-switching coverage. Zero-shot evaluation of five multilingual deepfake detectors reveals significant robustness issues: detector performance degrades substantially with increased speaker similarity, and varies wildly across generator–detector pairings, with performance disparities especially pronounced on minority dialects.

Contributions

  • Large-scale multilingual deepfake corpus: 993.4 hours of bona fide speech from 8,388 real-world videos; 3.1K hours of speaker-matched synthetic speech generated by OmniVoice, Higgs Audio v3, VoxCPM2, and MiniMax.
  • Comprehensive linguistic coverage: First large-scale open Vietnamese dataset combining transcripts, speaker identities, five dialect groups (North, South, Central, Southwest, Overseas Vietnamese), and naturally occurring Vietnamese–English code-switching (~47% of corpus by duration).
  • Controlled evaluation setup: Speaker-matched and transcript-matched bona fide–spoof pairs enable isolated assessment of deepfake detector robustness across controlled variables.
  • Benchmark of multilingual detectors: Zero-shot evaluation of DF-Arena (DFA-1B, DFA-500M) and AntiDeepfake (ADF) checkpoints across four synthesizers, dialect groups, and speaker-similarity ranges.

Method

Data collection pipeline: 1. Video discovery: LLM-assisted search of publicly accessible YouTube videos, stratified by dialect and code-switching characteristics; three-hour cap per channel. 2. Speech processing: Audio extraction, resampling to 16 kHz mono, peak and loudness normalization (−14 LUFS), denoising with ClearerVoice-Studio. 3. Transcript alignment: LLM transcription (Gemini 2.5 Pro) combined with traditional Montreal Forced Aligner (MFA); manual correction of disagreements >2 seconds. 4. Segment filtering: LLM-based filtering (Qwen3-Omni-30B-A3B) to exclude multi-speaker, singing, and low-quality segments. 5. Speaker-conditioned synthesis: Manual speaker clustering and verification; 9–9.5 second reference extraction for deepfake synthesis using four systems (OmniVoice, Higgs Audio v3, VoxCPM2, MiniMax); volume, pitch, and speed variation for diversity.

Deepfake detector evaluation: - Five pre-trained multilingual detectors (DF-Arena 1B/500M, AntiDeepfake XLS-R-2B/W2V2-L/MMS-300M) evaluated in zero-shot setting. - Metrics: Equal Error Rate (EER) computed per detector–generator pair. - Stratified evaluation: speaker-similarity ranges, dialect groups, code-switching vs. monolingual.

Results

Dataset scale and diversity: - 1,262 verified speakers; 403,941 bona fide utterances; 1,290,883 total spoof segments. - Dialect distribution: South (46.7%), North (50.0%), Central (2.0%), Southwest (0.7%), Overseas (0.6%). - 52.9% monolingual Vietnamese; 47.1% code-switched (avg. 2.83 English words per code-switched segment).

Detector robustness: - DFA-1B: 13.14% EER on MiniMax, 16.60% on OmniVoice, but 34.33% on VoxCPM2 (mean 17.79%). - ADF-XLS-R-2B: 13.7% mean EER, but exhibits near-chance performance on MiniMax (range 0.01–0.00% for several generators). - Similarity-stratified degradation: DFA-1B EER rises from 16.3% ([0.5, 0.6) similarity) to 33.6% ([0.9, 1.0]), indicating brittleness to speaker similarity. - Dialect-stratified disparities: DFA-1B performance ranges from 16.6% (South) to 28.7% (Southwest); ADF-family models more stable across dialects. - Code-switching penalty: MiniMax deepfakes show resistance to detection under code-switching; DFA-500M EER increases up to +7.1 pp on code-switched utterances.

Connections

Notes

This work fills a critical gap in multilingual deepfake research: prior datasets heavily skew toward English, and multilingual resources often omit speaker metadata, dialect diversity, or natural code-switching. The zero-shot evaluation reveals that state-of-the-art detectors trained on multilingual data do not generalize robustly to Vietnamese—a pattern likely extending to other low-resource languages. The speaker-similarity stratification is particularly valuable: it exposes a failure mode where high-fidelity synthesis (producing spoofs with high speaker similarity) systematically degrades detector performance, a phenomenon that had not been quantified this thoroughly in prior work. The code-switching results suggest that proprietary commercial synthesis systems (MiniMax) achieve acoustic smoothing during language switches that open-source systems lack, which is both a finding about synthesis quality and a concern for detection robustness. The work is methodologically sound and the dataset will be a useful resource for the community; the primary limitation is scale relative to English deepfake corpora, which is understandable for a single-language focused effort.