When building Summarize — the 9th tool in our Craft suite — we needed to answer one question: which engine actually works well for both English and Vietnamese?
Instead of trusting paper benchmarks (ROUGE scores), we tested with 4 real news articles: 2 English (AI regulation, fusion energy) and 2 Vietnamese (VN economy, VN semiconductors). Below are the real-world results — strengths and areas for improvement for each engine.
5 engines tested
| Engine | Type | Size | Cost |
|---|---|---|---|
| DeepSeek V4 Flash | API (AI) | 0 (cloud) | $0.27/1K calls |
| DistilBART-6-6 | ONNX local | 284 MB | Free |
| ViT5-base | ONNX local | 900 MB | Free |
| mT5-small | ONNX local | 300 MB | Free |
| TextRank | Pure JS | 0 | Free |
Speed
| Engine | English avg | Vietnamese avg | Notes |
|---|---|---|---|
| TextRank | 1ms | 1ms | Extractive, no AI |
| mT5-small | 166ms | 155ms | Fast, needs fine-tuning |
| ViT5-base | — | 470ms | Vietnamese only |
| DistilBART-6-6 | 870ms | — | English only |
| DeepSeek V4 Flash | 17.6s | 7.2s | API, network-bound |
Quality — actual outputs
Vietnamese: Vietnam Economy 2026
DeepSeek V4 Flash ✅ BEST
• GDP nửa đầu 2026 tăng 7,2%, vượt mục tiêu.
• Xuất khẩu đạt 210 tỷ USD; điện tử, bán dẫn đóng góp 45 tỷ USD (+28%).
• FDI đạt 18,5 tỷ USD; Samsung, Intel, NVIDIA mở rộng đầu tư.
• Lạm phát 4,5%; BĐS giảm 10-15%; VND mất 3%.
• Mục tiêu cả năm: tăng trưởng 7,5-8%.
ViT5-base ⚠️ CONTENT BEYOND SOURCE
0-7, Cục trưởng Cục Phát triển Nam Bộ ( Bộ Công thương ) Nguyễn Văn Đức cho biết, tăng trưởng kinh tế có những diễn biến tích cực...
⚠️ "Cục trưởng Nguyễn Văn Đức" does not appear in the source text — the model generated content beyond the original material. This is a known challenge for smaller models processing Vietnamese, and VietAI continues to improve their ViT5 family.
mT5-small ⚠️ NEEDS FINE-TUNING
<extra_id_0> lớn. /.. /... /......mT5-small is a base model not yet fine-tuned for summarization — it needs specialized training to unlock its potential for this task.
English: AI Regulation 2026
DeepSeek V4 Flash ✅ BEST
• EU AI Act bans unacceptable-risk AI; high-risk needs conformity assessments; fines up to €35M / 7% turnover
• Compliance costs $50-100M per major tech company; startup consolidation wave
• Transatlantic divide: US voluntary, China own framework
• OECD proposes mutual recognition; early-stage negotiations
DistilBART-6-6 ✅ GOOD
The European Union's Artificial Intelligence Act took full effect in February 2026. The regulation establishes a risk-based framework that classifies AI applications into four tiers. Companies face fines of up to 35 million euros or 7 percent of global annual turnover.
mT5-small ⚠️ NEEDS FINE-TUNING
<extra_id_0> to the Artificial Intelligence Act. Artificial Intelligence.com/.Quality rankings
| Engine | English | Vietnamese | Verdict |
|---|---|---|---|
| DeepSeek V4 Flash | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | Best for both languages |
| DistilBART-6-6 | ⭐⭐⭐⭐ | — (EN only) | Good English offline option |
| TextRank | ⭐⭐⭐ | ⭐⭐⭐ | Instant fallback, any language |
| ViT5-base | — (VI only) | ⭐⭐ | Has potential, needs improvement |
| mT5-small | ⭐ | ⭐ | Needs specialized fine-tuning |
Cost
DeepSeek V4 Flash costs only ~$10/year at 100 summaries/day. Local engines are free but significantly lower quality.
Conclusion
Auto engine chain: DeepSeek → DistilBART → TextRank
- DeepSeek V4 Flash — default when online. Best quality for both EN+VI.
- DistilBART-6-6 — offline fallback for English, ~1 second.
- TextRank — instant fallback for any language when both unavailable.
ViT5-base and mT5-small can still be selected manually — they have potential and will improve with specialized fine-tuning in the future.