# 蚂蚁 inclusionAI 开源 Ming-3.0-tts 语音合成模型
> 原标题:inclusionAI/Ming-3.0-tts
- 来源:蚂蚁 inclusionAI:GitHub 新仓库
- 发布时间:2026-10-19T07:03:25.000Z
- AIHOT:https://aihot.news/items/uvgayscfmc9ujjak94ymsnxnd
- 原文:https://github.com/inclusionAI/Ming-3.0-tts
## 摘要
蚂蚁 inclusionAI 在 GitHub 发布 Ming-3.0-tts,包含 0.5b dense 和基于 Ling V3 的 tiny(7.9B 总参数、1.3B 激活)两个语音生成模型,MIT 协议开源,权重可在 Hugging Face 和 ModelScope 下载。
## 推荐理由
官方仓库给出两版模型的榜单名次、完整任务列表和部署细节,读者可据此评估其在多语言与副语言控制上的实际可用性。
## 正文 · 中文译文
支持自由形式与副语言控制的多语言语音合成。
🤗 [Hugging Face](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)| 🤖 [ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) | 📑 [博客与演示](https://nomorecoding.github.io/Ming-3.0-tts/) | 💬 [DingTalk(钉钉)](https://qr.dingtalk.com/action/joingroup?code=v1,k1,O7MiZqrOrB2c7PnUZVBNvmDh/6tghNLMEtXqMteyRIpuRVJIwrSsXmL8oFqU5ajJ&_dt_no_comment=1&origin=11)
Ming-3.0-tts 提供两个语音生成模型:**0.5b**,一个稠密模型,以及 **tiny**,一个基于 Ling V3 构建的 MoE 模型(总参数 7.9B · 激活参数 1.3B)。
## 🏆 Artificial Analysis 排行榜
在独立的 [Artificial Analysis](https://artificialanalysis.ai/text-to-speech) 文本转语音排行榜上(2026 年 10 月),Ming-3.0-tts 在**受控语音类别中位列开源权重模型第 1 名**,在**发音稳健性类别中位列总榜第 1 名**,并在**提供方语音类别中位列开源权重模型第 3 名**。排名会随着新模型的加入而变化。
| 排行榜 | 设置 | Ming-3.0-tts | 排名 |
| --- | --- | --- | --- |
| [受控语音](https://artificialanalysis.ai/text-to-speech/leaderboard/controlled-voice) | 竞技场偏好 Elo;每个模型克隆相同的 8 个声音(4 个美式、4 个英式) | 1069 | **开源权重模型第 1 名** · 总榜第 13 名 |
| [发音稳健性](https://artificialanalysis.ai/text-to-speech) | 英文文本中高亮片段被人工评审判定为发音正确的比例 | 93.2% | **总榜第 1 名** |
| [提供方语音](https://artificialanalysis.ai/text-to-speech/leaderboard/provider-voice) | 竞技场偏好 Elo;每个模型使用其自带的原生声音 | 1188 | **开源权重模型第 3 名** · 总榜第 17 名 |
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/aa_leaderboards.png)
在发音稳健性类别中,Ming-3.0-tts 还在扩展缩写(文本规范化,96.5%)和保留精确序列(序列保真度,90.6%)两项上位列**第 1 名**。
## 主要能力
- **声音创建:**克隆参考声音、根据文本设计声音,或从 100 多个中文内置声音和五个英文内置声音中选择。
- **自由形式控制:**用自然语言指令引导语音表达。
- **副语言控制:**插入 14 个内联标签,用于中文或英文中的笑声、叹息及其他发声表达。
- **语言与方言:**合成中文、英文、德文、西班牙文、法文、韩文和日文,以及四川话和粤语。
- **情感与属性:**控制情感,或调整语速、音高和音量。
- **多说话人对话:**根据脚本和说话人参考生成对话(**仅 tiny**)。
各项能力使用独立的任务模式,而非任意组合。支持的输入请参阅 [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook),基准测试请参阅 [Performance](https://github.com/inclusionAI/Ming-3.0-tts#performance),音频示例请参阅 [博客与演示](https://nomorecoding.github.io/Ming-3.0-tts/)。
## 演示视频
tts11\_under10mb.mp4
## 架构
文本和任务特定的控制 token 作为条件输入到一个语言模型,该模型带有 DiT patch head 和音频分词器。两个模型共享这一生成框架。
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/ming-3.0-tts.png)
## 新闻
- 🔥 **\[2026.10\]** Ming-3.0-tts-0.5b 和 Ming-3.0-tts-tiny 发布,包含推理代码、任务 cookbook 以及下方的评估表格。
## 路线图
- Ming-3.0-tts-0.5b 权重与推理代码
- Ming-3.0-tts-tiny 权重与推理代码
- Ming-3.0-tts-tiny 基准测试
- vLLM-Omni 加速推理
- 技术报告
## 目录
- [演示视频](https://github.com/inclusionAI/Ming-3.0-tts#demo-video)
- [模型](https://github.com/inclusionAI/Ming-3.0-tts#models)
- [快速开始](https://github.com/inclusionAI/Ming-3.0-tts#quick-start-)
- [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook)
- [内置声音](https://github.com/inclusionAI/Ming-3.0-tts#built-in-voices)
- [性能](https://github.com/inclusionAI/Ming-3.0-tts#performance)
- [局限性](https://github.com/inclusionAI/Ming-3.0-tts#limitations)
- [使用免责声明](https://github.com/inclusionAI/Ming-3.0-tts#usage-disclaimer)
## 模型
| 模型 | 架构 | 参数 | 下载 |
| --- | --- | --- | --- |
| Ming-3.0-tts-0.5b | 稠密 | 0.5b | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) |
| Ming-3.0-tts-tiny | MoE | 总参数 7.9B · 激活参数 1.3B | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-tiny)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-tiny) |
每个发布检查点都打包了 backbone、DiT head、音频 tokenizer、说话人编码器和文本 tokenizer。0.5b 发布版的目录结构如下所示;tiny 使用分片权重,配合 `model.safetensors.index.json`。
```
Ming-3.0-tts-0.5b/
├── config.json # model_type: dense, auto_map -> BailingMMConfig
├── configuration.json # ModelScope task descriptor
├── model.safetensors # backbone + audio VAE + DiT head + aggregator
├── campplus.onnx # speaker embedding extractor
├── tokenizer.json # plus vocab.json / merges.txt / added_tokens.json
├── tokenizer_config.json # special_tokens_map.json / chat_template.jinja
└── README.md
```
## 快速开始 🚀
### 安装
git clone https://github.com/inclusionAI/Ming-3.0-tts.git
cd Ming-3.0-tts
pip install -r requirements.txt # Ming-3.0-tts-0.5b
或者构建 0.5b Docker 镜像(torch 2.6、CUDA 12.4、`transformers==4.46.3`):
docker build -f docker/Dockerfile -t ming-tts .
0.5b 环境已在 NVIDIA H800-80GB 和 H20-96G 上测试过。对于 0.5b,`flash-attn` 是可选的;推理会回退到 PyTorch SDPA。
Tiny 使用单独的参考推理环境:Python **3.10.15**、PyTorch 和 torchaudio **2.7.1+cu118**(CUDA 11.8 wheels),以及 `transformers==4.46.3`。请在专用环境中从 [requirements-tiny.txt](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/requirements-tiny.txt) 安装固定版本的依赖,因为其他依赖版本与 0.5b 不同。我们建议使用 NVIDIA H800 进行推理。
conda create -n ming-tiny python=3.10.15 -y && conda activate ming-tiny
pip install -r requirements-tiny.txt
### 模型下载
pip install "huggingface\_hub\[cli\]"
hf download inclusionAI/Ming-3.0-tts-0.5b --local-dir Ming-3.0-tts-0.5b
在中国大陆,ModelScope 通常更快:
pip install modelscope
modelscope download --model inclusionAI/Ming-3.0-tts-0.5b --local\_dir Ming-3.0-tts-0.5b
将 `0.5b` 替换为 `tiny` 即可获取另一个变体。
### 文本规范化
数字、日期、单位和符号通过 FST 语法进行规范化,这需要 `pynini`。它没有通用 wheel:
conda install -c conda-forge pynini==2.1.6 # or: pip install pynini==2.1.6
它是延迟加载的;传入 `--no_text_normalization` 可跳过它。
### Ming-3.0-tts-0.5b
python infer.py \\
--model\_path inclusionAI/Ming-3.0-tts-0.5b \\
--task TTS \\
--language\_id Chinese \\
--text "傍晚的风从山口吹过来,远处的云慢慢染成了金色。" \\
--prompt\_wav\_path data/voice\_presets/voice\_audio/lingguang.wav \\
--prompt\_text "诶,你今天有空吗?我们能不能聊聊这件事啊?" \\
--use\_spk\_emb \\
--output output/tts.wav
或者从 Python 中:
from ming\_tts import MingTTS
model \= MingTTS("inclusionAI/Ming-3.0-tts-0.5b")
model.synthesize(
task\="TTS",
language\_id\="Chinese", \# English for English text
text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。",
prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav",
prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?",
use\_spk\_emb\=True,
output\_path\="output/tts.wav",
)
### Ming-3.0-tts-tiny
每个任务在 [examples/one\_click/](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/examples/one_click) 中都有一个单行脚本;诸如 `MODEL`、`TEXT` 和 `OUT` 这样的变量可以从环境变量中覆盖:
bash examples/one\_click/run\_seed.sh
MODEL=/path/to/Ming-3.0-tts-tiny OUT=outputs/yue.wav bash examples/one\_click/run\_dialect.sh
从 Python 中调用同样如此。加载时会配置发布版运行时(bf16、可移植 MoE 循环、fp32 循环状态)并对照检查检查点;`generate()` 默认为 `cfg_scale=2.0`、`temperature=0`、`flow_steps=10`、`sigma=0.25`,随机种子为 1234(`podcast` 为 1235):
from ming\_tiny import MingTinyTTS
tts \= MingTinyTTS.from\_pretrained("inclusionAI/Ming-3.0-tts-tiny", device\="cuda:0")
tts.generate(
task\="seed",
text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。",
prompt\_wav\="data/voice\_presets/voice\_audio/lingguang.wav",
prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?",
target\_language\="Chinese", \# English for English text
output\_path\="outputs/seed.wav",
)
tts.generate(
task\="podcast",
text\="speaker\_1: 你最近是不是又去那家面馆了?\\nspeaker\_2: 对啊,排了半个小时的队,不过真的值得。",
prompt\_wav\=\["data/voice\_presets/voice\_audio/lingguang.wav", "data/voice\_presets/voice\_audio/db30.wav"\],
prompt\_text\=\["诶,你今天有空吗?我们能不能聊聊这件事啊?", "感谢你的认可。"\],
output\_path\="outputs/podcast.wav",
)
每次调用选择一个任务,并接受下面列出的输入:
| 任务 | 参考音频 | 参考文本 | `target_language` | 控制项 |
| --- | --- | --- | --- | --- |
| `seed` | 一个 | 必需 | `Chinese` / `English`,必需 | — |
| `multilingual` | 一个 | 必需 | `German` / `Spanish` / `French` / `Korean` / `Japanese` | — |
| `paralinguistic` | 一个 | 必需 | `Chinese` / `English` | `text` 中的内联标签 |
| `podcast` | 每个说话人一个 | 每个说话人一个,顺序相同 | 从文本推断 | 每行开头的 `speaker_N:` |
| `basic` | 一个 | 不接受 | — | `speed`、`pitch`、`volume`,全部三个 |
| `emotion` | 一个 | 不接受 | — | `emotion` |
| `dialect` | 一个 | 不接受 | — | `dialect`:`川渝话` / `广粤话` |
| `freeform` | 默认无(推荐);可选 | 不接受 | — | `instruction` |
| `voice_design` | 无 | — | — | `style` |
| `ip` | 无 | — | — | `ip` |
`python infer_tiny.py --help` 为命令行列出了相同的参数;对于播客,重复 `--prompt_wav` 和 `--prompt_text`。
| | Ming-3.0-tts-0.5b | Ming-3.0-tts-tiny |
| --- | --- | --- |
| Python API | `ming_tts.MingTTS` | `ming_tiny.MingTinyTTS` |
| CLI | `infer.py` | `infer_tiny.py`、`examples/one_click/` |
| 预检 | `tools/preflight.py` | `tools/preflight_tiny.py` |
| 任务 | TTS(包含内联副语言标签)、FREEFORM、VOICE\_DESIGN、MULTILINGUAL、BASIC、EMOTION、DIALECT、IP | 与小写任务相同的能力(`seed`、`paralinguistic`、……),外加 `podcast` |
### 验证你的设置
在不加载权重的情况下检查检查点和 tokenizer:
python tools/preflight.py --model\_path /path/to/Ming-3.0-tts-0.5b
python tools/preflight\_tiny.py --model\_path /path/to/Ming-3.0-tts-tiny
## Cookbook
按照一行语音的生成顺序排列:谁在说话、如何说、超越文字的内容,以及在哪里听到。每个示例对应一个任务;来自其他任务的控制项会被拒绝。文本和控制项与 [演示页面](https://nomorecoding.github.io/Ming-3.0-tts/) 上的一致,因此你可以与在那里听到的内容进行对比。0.5b API;完整取值列表见 [English](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md) / [简体中文](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instructions.md),每个任务的端到端示例见 [cookbooks/cookbook.ipynb](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/cookbook.ipynb)。
### 谁在说话
**零样本克隆**:[快速开始](https://github.com/inclusionAI/Ming-3.0-tts#ming-30-tts-05b) 中的 `TTS` 调用会克隆 `prompt_wav_path` 的声音。
**声音设计**:描述声音;无需参考音频。
model.synthesize(
task\="VOICE\_DESIGN",
text\="冰川在漫长岁月中缓缓移动,也悄悄记录着气候变化的痕迹。",
controls\={"style": "成熟沉稳的男性纪录片旁白,吐字清晰,节奏舒缓,语气理性而有厚度。"},
use\_zero\_spk\_emb\=True,
)
**内置声音**:推荐 **US\_James**、**US\_Stella**、**灵小柔** 和 **灵楷**。参见 [完整声音列表](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices)。
model.synthesize(
task\="IP",
text\="The garden will remain open until sunset, provided the weather stays clear.",
controls\={"ip": "US\_James"},
use\_zero\_spk\_emb\=True,
)
### 如何说
**自由形式**:用一条自然语言指令描述声音和表达方式。两个模型默认都在无参考音频的情况下生成;不建议添加参考。不与声音设计、内置声音、情感、方言或属性控制项组合使用。
model.synthesize(
task\="FREEFORM",
text\="The old radio crackled softly as the storm moved across the valley.",
instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.",
use\_zero\_spk\_emb\=True,
)
对于 Tiny,省略 `prompt_wav` 和 `prompt_text`:
tts.generate(
task\="freeform",
text\="The old radio crackled softly as the storm moved across the valley.",
instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.",
)
**情感**:`高兴 / 悲伤 / 愤怒 / 恐惧 / 厌恶 / 惊讶 / 中性`。
model.synthesize(
task\="EMOTION",
text\="所有测试都顺利通过了,我们终于可以把这个好消息告诉大家!",
controls\={"emotion": "高兴"},
prompt\_wav\_path\="data/wavs/emotion\_prompt.wav",
use\_spk\_emb\=True,
)
**语速、音高和音量**:语速 `慢速 / 中速 / 快速`,音高和音量 `低 / 中 / 高`。
model.synthesize(
task\="BASIC",
text\="今晚的观测将在云层散开后开始,请耐心等待下一次通知。",
controls\={"speed": "慢速", "pitch": "中", "volume": "中"},
spk\_emb\_path\="/path/to/speaker\_embedding.pt",
use\_spk\_emb\=True,
)
### 超越文字
**副语言标签**:可内联,出现在句子任意位置,支持中文或英文。`[Breathing]`、`[Cough]`、`[Sigh]`、`[Laughter]`、`[Surprise-ah]`、`[Surprise-oh]`、`[Dissatisfaction-hnn]`、`[Uhm]`、`[Confirmation-en]`、`[Question-ei]`、`[Question-ah]`、`[Question-en]`、`[Question-oh]`、`[Question-huh]`。
model.synthesize(
task\="TTS",
text\="\[Sigh\]雨还是没有停,\[Cough\]我们恐怕只能把活动改到室内了。",
prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav",
prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?",
use\_spk\_emb\=True,
)
### 适用场景
**多语言**:五种新语言需要 `language_id`;短代码和全名均可使用。
model.synthesize(
task\="MULTILINGUAL",
text\="週末、駅前の小さな喫茶店で、季節限定の抹茶パフェを食べました",
language\_id\="Japanese", \# ja / ko / de / fr / es
prompt\_wav\_path\="/path/to/japanese\_prompt.wav",
prompt\_text\="参考音频对应的日语文本。",
use\_spk\_emb\=True,
)
**方言**:`chuanyu dialect` 或 `cantonese`;此任务不接受 `language_id`。
model.synthesize(
task\="DIALECT",
text\="今日天气几好,我哋食完饭一齐去海边行下啦。",
controls\={"dialect": "cantonese"},
prompt\_wav\_path\="data/wavs/yue\_prompt.wav",
use\_spk\_emb\=True,
)
## 内置音色
### 推荐音色
| 音色 | 语言 | 角色 |
| --- | --- | --- |
| US\_James | 英语(美国) | 男声。旁白、讲解和纪录片。 |
| US\_Stella | 英语 | — |
| 灵小柔 | 中文 | 明亮甜美的年轻女声,吐字清晰。适用于助手和客服。 |
| 灵楷 | 中文 | 温暖浑厚的年轻男声。适用于电台、纪录片和有声书。 |
提供超过 100 种中文音色和五种英文音色。完整列表(包括情感特定变体)请参见[音色目录](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices)。
## 性能
公开基准如下;独立竞技场结果见[Artificial Analysis 排行榜](https://github.com/inclusionAI/Ming-3.0-tts#artificial-analysis-leaderboards)。
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_basic_1018.png)
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_control_1018.png)
下文中,**tiny** 指 Ming-3.0-tts-tiny,**0.5b** 指 Ming-3.0-tts-0.5b。排名和标记按指标在所列开源模型中计算;Seed-TTS 和 NVbench-GT 为参考行,不参与排名。若历史 Ming-omni-tts 结果优于加粗的 Ming-3.0-tts 数值,将在表下注明,且不声称 SOTA。**加粗**表示每列最佳值,下划线表示次佳值;并列共享同一标记。其他系统的结果使用其报告的推理设置。
### 总结
1. **副语言:**两个模型合计在**全部六项 NVBench 单标签指标上达到 SOTA**;tiny 在**全部三项英文多标签指标上达到 SOTA**。
2. **方言:**0.5b 在粤语 Base CER 和 ACC 上达到 **SOTA**;tiny 在四川话 Hard CER 和 Easy ACC 上达到 **SOTA**。
3. **多语言:**tiny 在德语和西班牙语 WER 上达到 **SOTA**,法语 WER 排名**第 2**,日语 CER 并列**第 2**。
4. **情感:**tiny 在 CV3 文本相关 Happy 上并列 **SOTA**,并在两个 CV3N Sad 子集上达到 **SOTA**;0.5b 在 CV3 文本相关 Angry 上达到 **SOTA**。tiny 的 CV3 平均排名**第 2**。
5. **属性控制:**0.5b 在平均 ACC 和音高上领先当前对比,而 tiny 在 WER 和 SIM 上领先。历史 Ming-omni-tts 结果仍保持相应的总体记录。
6. **播客:**tiny 取得 CER 2.03%、cpSIM 0.561 和 UTMOS 2.12,分别排名**第 1**、**第 2**和**第 2**;历史 Ming-omni-tts 保持更低的 CER。
7. **音色设计:**tiny 在平均 ACC(75.37%)、APS(85.10%)和 DSD(79.90%)上排名**第 2**。
8. **中文和英文:**0.5b 的中文 WER 排名**第 3**;tiny 在中文和英文 SIM 上达到 **SOTA**。
9. **下一步:**改进多语言和粤语说话人相似度、tiny 的韩语准确率和音高控制、文本无关情感、角色扮演指令遵循,以及预测的播客音频质量。
### 中文和英文(seed-tts-eval)
| 模型 | seed-tts-eval-zh | seed-tts-eval-en |
|---|
| WER ↓ | SIM ↑ | WER ↓ | SIM ↑ |
|---|
| Seed-TTS | 1.110% | 0.796 | 2.240% | 0.762 |
| GLM-TTS | 1.030% | 0.761 | 2.230% | 0.672 |
| Qwen3-TTS | 1.070% | — | 1.390% | — |
| Qwen3-TTS-12Hz-0.6B-Base | 0.920% | — | 1.320% | — |
| Qwen3-TTS-12Hz-1.7B-Base | 0.770% | — | 1.240% | — |
| CosyVoice3-0.5B | 1.160% | 0.780 | 2.020% | 0.718 |
| CosyVoice3-1.5B | 0.710% | 0.775 | 1.450% | 0.695 |
| F5-TTS | 1.560% | 0.741 | 1.830% | 0.647 |
| Ming-3.0-tts-0.5b | 0.800% | 0.758 | 1.584% | 0.691 |
| Ming-3.0-tts-tiny | 0.820% | 0.781 | 1.666% | 0.728 |
Tiny 在中文和英文上实现了**SOTA 说话人相似度**。0.5b 在两种语言上的 WER 都更低,并在中文 WER 上排名**第 3**。
### 多语言(MiniMaxAI TTS-Multilingual-Test-Set)
| 模型 | 德语 | 西班牙语 | 法语 | 韩语 | 日语 |
|---|
| WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | CER ↓ | SIM ↑ | CER ↓ | SIM ↑ |
|---|
| dots.tts | 0.452% | 0.798 | 2.593% | 0.839 | 19.878% | 0.789 | 4.837% | 0.853 | 5.134% | 0.838 |
| VoxCPM2 | 0.963% | 0.790 | 1.324% | 0.836 | 3.380% | 0.761 | 3.547% | 0.839 | 4.358% | 0.825 |
| Fish Audio S2 | 1.045% | 0.714 | 1.216% | 0.748 | 3.636% | 0.698 | 1.675% | 0.761 | 3.427% | 0.755 |
| FireRedTTS3 | 0.348% | 0.825 | 1.086% | 0.868 | 4.366% | 0.807 | 1.644% | 0.864 | 3.418% | 0.825 |
| Ming-3.0-tts-0.5b | 0.365% | 0.753 | 1.099% | 0.763 | 4.264% | 0.675 | 1.728% | 0.793 | 3.765% | 0.765 |
| Ming-3.0-tts-tiny | 0.325% | 0.791 | 0.971% | 0.797 | 3.470% | 0.733 | 6.000% | 0.797 | 3.427% | 0.780 |
每个开源模型均在其推荐的推理设置下,采用统一的基于 ASR 的 WER/CER 评估协议进行评估。
Tiny 在**德语和西班牙语上实现了 SOTA WER**,在法语 WER 上排名**第 2**,在日语 CER 上并列**第 2**。0.5b 在韩语上更准确;两个模型的说话人相似度仍有差距。
### 副语言控制(NVBench)
报告 CER、PCER(副语言 CER)和 OCER。NVbench-GT 为人工标注的参考结果,仅用于提供背景,不参与排名。
| 模型 | 单标签 | 多标签 |
|---|
| 中文 | 英文 | 中文 | 英文 |
|---|
| CER | PCER | OCER | CER | PCER | OCER | CER | PCER | OCER | CER | PCER | OCER |
|---|
| NVbench-GT | 3.86 | 9.38 | 4.07 | 6.73 | 8.31 | 6.90 | 3.79 | 23.71 | 5.07 | 7.26 | 21.41 | 8.62 |
| Orpheus-TTS | 11.36 | 88.77 | 13.91 | 9.03 | 71.92 | 10.63 | 19.83 | 84.85 | 24.38 | 8.68 | 71.46 | 11.89 |
| SMIIP-NV-CV2 | 8.80 | 75.64 | 11.34 | 17.92 | 56.80 | 19.47 | 10.66 | 77.20 | 14.79 | 20.93 | 54.49 | 23.87 |
| Emilia-NV-CV2 | 5.05 | 40.00 | 6.64 | 12.50 | 55.30 | 13.21 | 5.54 | 48.74 | 8.09 | 11.71 | 60.28 | 14.63 |
| CosyVoice3 | 3.85 | 57.69 | 5.86 | 7.87 | 62.75 | 9.06 | 4.75 | 61.94 | 8.26 | 6.39 | 57.84 | 10.69 |
| NV-FlexiVoice | 6.98 | 31.08 | 8.15 | 11.88 | 50.43 | 13.21 | 8.20 | 39.37 | 10.39 | 9.60 | 51.32 | 13.76 |
| NV-CV3 | 3.80 | 27.69 | 4.90 | 8.33 | 46.13 | 9.44 | 3.44 | 30.04 | 4.84 | 6.70 | 47.13 | 10.10 |
| Ming-3.0-tts-0.5b | 2.70 | 22.80 | 3.20 | 7.60 | 37.00 | 8.90 | 3.50 | 36.60 | 8.10 | 7.90 | 47.30 | 10.40 |
| Ming-3.0-tts-tiny | 2.80 | 17.23 | 3.20 | 5.86 | 32.95 | 7.05 | 4.49 | 32.88 | 5.81 | 5.12 | 46.13 | 7.68 |
两个模型共同在**全部六项中英文单标签指标上实现了 SOTA**;tiny 还在全部三项英文多标签指标上领先。中文多标签控制仍落后于 NV-CV3。
### 方言
| 模型 | WSC-Eval-TTS-easy | WSC-Eval-TTS-hard | WSYue-TTS-eval-Base | WSYue-TTS-eval-Coverage |
|---|
| CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ |
|---|
| Qwen3-TTS | 4.13% | — | — | 7.35% | — | — | — | — | — | — | — | — |
| CosyVoice3-0.5B | 3.17% | 0.696 | 76.05% | 4.07% | 0.723 | 81.41% | 8.36% | 0.611 | 91.70% | 8.95% | 0.658 | 95.80% |
| CosyVoice3-1.5B + DiffRO-EMO | 7.14% | 0.703 | — | 9.06% | 0.601 | — | 14.38% | 0.812 | — | 13.74% | 0.826 | — |
| CosyVoice2-WSC | 4.28% | 0.728 | — | 8.87% | 0.626 | — | — | — | — | — | — | — |
| CosyVoice2-WSC-SFT | 4.08% | 0.788 | — | 7.22% | 0.680 | — | — | — | — | — | — | — |
| CosyVoice2-Yue | — | — | — | — | — | — | 10.33% | 0.821 | — | 9.49% | 0.834 | — |
| Sparks-TTS | 10.83% | 0.677 | — | 12.52% | 0.545 | — | 27.79% | 0.762 | — | 24.25% | 0.781 | — |
| Ming-3.0-tts-0.5b | 2.27% | 0.689 | 80.68% | 3.35% | 0.691 | 87.64% | 6.15% | 0.586 | 98.00% | 8.15% | 0.622 | 95.27% |
| Ming-3.0-tts-tiny | 2.53% | 0.721 | 86.09% | 3.11% | 0.738 | 88.94% | 7.40% | 0.605 | 92.90% | 8.67% | 0.653 | 95.16% |
Tiny 在四川话控制准确度上更强,实现了**SOTA 四川话 Hard CER 和 Easy ACC**,而 0.5b 实现了**SOTA 粤语 Base CER 和 ACC**。粤语说话人相似度仍低于 CosyVoice2-Yue。历史 Ming-omni-tts 结果在四川话 Easy CER(2.25%)、Hard SIM(0.750)和 Hard ACC(90.45%)以及粤语 Coverage CER(7.87%)上仍然更强。
### 情感控制(CV3)
| 模型 | 平均 | 文本相关 | 文本无关 |
|---|
| 开心 | 悲伤 | 愤怒 | 开心 | 悲伤 | 愤怒 |
|---|
| CosyVoice3-0.5B | 66.33% | 92% | 70% | 72% | 64% | 42% | 58% |
| CosyVoice3-1.5B | 63.00% | 86% | 64% | 72% | 64% | 44% | 48% |
| CosyVoice3-1.5B + DiffRO-EMO | 77.67% | 98% | 68% | 84% | 98% | 50% | 68% |
| F5-TTS | 64.67% | 92% | 52% | 72% | 80% | 28% | 64% |
| Sparks-TTS | 55.33% | 80% | 56% | 50% | 50% | 60% | 36% |
| Ming-3.0-tts-0.5b | 72.33% | 96% | 76% | 94% | 52% | 36% | 80% |
| Ming-3.0-tts-tiny | 74.00% | 98% | 82% | 90% | 60% | 36% | 78% |
Tiny 和 0.5b 在平均 ACC 上分别排名**第 2**和**第 3**。Tiny 在文本相关的开心和悲伤上更强,而 0.5b 在愤怒上更强;文本无关情感仍是两者的共同局限。历史 Ming-omni-tts 结果在文本相关悲伤(86%)和文本无关愤怒(82%)上仍然更高。
### 情感控制(CV3N,困难情感控制)
| 模型 | 平均 | 文本相关 | 文本无关 |
|---|
| 开心 | 悲伤 | 愤怒 | 开心 | 悲伤 | 愤怒 |
|---|
| CosyVoice3-0.5B | 40.00% | 68% | 30% | 78% | 14% | 4% | 46% |
| Ming-3.0-tts-0.5b | 37.33% | 70% | 36% | 70% | 14% | 4% | 30% |
| Ming-3.0-tts-tiny | 30.67% | 60% | 40% | 44% | 18% | 10% | 12% |
0.5b 和 tiny 在平均 ACC 上分别排名**第 2**和**第 3**。Tiny 在**两个悲伤子集上均实现了 SOTA**,而 0.5b 的整体准确率更高。历史 Ming-omni-tts 结果在文本相关开心(78%)和文本无关开心(30%)上仍然更高。
### 音色设计(InstructTTSEval-ZH)
| 模型 | 平均 | APS | DSD | RP |
| --- | --- | --- | --- | --- |
| Qwen3-TTS | **77.13%** | **85.20%** | **81.10%** | **65.10%** |
| Mimo-Audio-7B-Instruct | 70.50% | 75.70% | 74.30% | 61.50% |
| VoiceSculptor | 67.30% | 75.70% | 64.70% | 61.50% |
| VoxInstruct | 47.47% | 47.50% | 52.30% | 42.60% |
| **Ming-3.0-tts-0.5b** | 72.29% | 78.54% | 77.28% | 61.06% |
| **Ming-3.0-tts-tiny** | 75.37% | 85.10% | 79.90% | 61.10% |
Tiny 和 0.5b 在平均 ACC 上分别排名**第 2**和**第 3**,落后于 Qwen3-TTS。角色扮演指令遵循仍是两个模型的主要差距。
### 基础属性控制
| 模型 | 语速 | 音量 | 音高 | 平均 | WER ↓ | SIM ↑ |
| --- | --- | --- | --- | --- | --- | --- |
| CosyVoice3-0.5B | **100.00%** | **97.67%** | 65.33% | 87.67% | 1.210% | 0.580 |
| **Ming-3.0-tts-0.5b** | 99.00% | 95.00% | **86.00%** | **93.30%** | 0.317% | 0.712 |
| **Ming-3.0-tts-tiny** | 93.00% | 93.00% | 78.70% | 88.23% | **0.274%** | **0.742** |
0.5b 的控制精度更高,而 tiny 的 WER 更低、说话人相似度更高。历史 Ming-omni-tts 模型在平均 ACC、音高、WER 和 SIM 上保留了整体记录。
### 播客
| 模型 | CER ↓ | cpSIM ↑ | UTMOS ↑ |
| --- | --- | --- | --- |
| FireRedTTS2 | 3.34% | 0.512 | 1.90 |
| SoulX-Podcast | 2.20% | **0.599** | 2.09 |
| VibeVoice-1.5B | 12.87% | 0.455 | 1.74 |
| MOSS-TTSD | 8.62% | 0.421 | 1.70 |
| ZipVoice-Dia | 3.39% | 0.553 | **2.24** |
| MoonCast | 27.43% | 0.441 | 1.76 |
| **Ming-3.0-tts-tiny** | **2.03%** | 0.561 | 2.12 |
Tiny 在 CER 上排名**第 1**,在 cpSIM 和 UTMOS 上排名**第 2**。历史 Ming-omni-tts 保持了更低的 CER 和更高的 UTMOS。
## 局限性
- **说话人相似度。** Tiny 在德语 SIM 上排名第三(0.791),但两个模型在其他四种新语言中均未进入前三。法语 SIM 为 0.5b 的 0.675 和 tiny 的 0.733;两者在粤语 SIM 上也落后于 CosyVoice2-Yue。
- **一行中的多个事件。** NVBench _Multi_ 可用,但落后于 _Single_;每句包含多个标签时的稳定性仍在改进中。
- **仅凭表达传递情感。** 当文字本身不携带情感信号时(CV3-Eval Text-Unrelated,以及困难集中的大部分),准确率远低于 Text-Related 的一半。
- **tiny 的音高。** 音高是两个模型中最不准确的属性。Tiny 的 78.70% 排名第 2,比 0.5b 的 86.00% 低 7.30 个百分点;历史 Ming-omni-tts-0.5B 的得分为 91.33%。
- **角色扮演音色设计。** RP 是音色设计子项中得分最低的,tiny 排名第 4(61.10%),0.5b 排名第 5(61.06%)。
- **tiny 的韩语。** Tiny 的韩语 CER(6.000%)排名第 6,远落后于 0.5b 的 1.728%(第 3)。
- **播客音频质量。** Tiny 的 UTMOS 为 2.12,排名第 2,低于 ZipVoice-Dia 的 2.24 和历史 Ming-omni-tts-0.5B 的 2.25。
## 使用免责声明
- 语音克隆和内置角色音色仅供研究和合法创意用途使用。请勿使用它们未经同意冒充真实人物、进行欺骗或用于任何非法目的。
- 在可能被误认为真实录音的场合,请将合成语音标注为合成语音。
- 作者对滥用不承担任何责任。如果您发现对本项目的滥用行为,请向我们报告。
## 致谢
- [CAM++](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common) 用于说话人嵌入提取
- [Ming-omni-tts](https://github.com/inclusionAI/Ming-omni-tts) 用于音频分词器
## 许可证
依据 [MIT](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/LICENSE) 许可证发布。
## 正文 · 原文
Multilingual speech synthesis with free-form and paralinguistic control.
🤗 [Hugging Face](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)| 🤖 [ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) | 📑 [Blog & Demos](https://nomorecoding.github.io/Ming-3.0-tts/) | 💬 [DingTalk(钉钉)](https://qr.dingtalk.com/action/joingroup?code=v1,k1,O7MiZqrOrB2c7PnUZVBNvmDh/6tghNLMEtXqMteyRIpuRVJIwrSsXmL8oFqU5ajJ&_dt_no_comment=1&origin=11)
Ming-3.0-tts provides two speech-generation models: **0.5b**, a dense model, and **tiny**, a MoE model built on Ling V3 (7.9B total · 1.3B active).
## 🏆 Artificial Analysis Leaderboards
On the independent [Artificial Analysis](https://artificialanalysis.ai/text-to-speech) text-to-speech leaderboards (October 2026), Ming-3.0-tts ranks **1st among open-weights models on Controlled Voice**, **1st overall on Pronunciation Robustness**, and **3rd among open-weights models on Provider Voice**. Ranks change as new models are added.
| Leaderboard | Setting | Ming-3.0-tts | Rank |
| --- | --- | --- | --- |
| [Controlled Voice](https://artificialanalysis.ai/text-to-speech/leaderboard/controlled-voice) | Arena preference Elo; every model clones the same 8 voices (4 US, 4 UK) | 1069 | **1st among open-weights** · 13th overall |
| [Pronunciation Robustness](https://artificialanalysis.ai/text-to-speech) | Share of highlighted spans in English text that human reviewers judged correctly pronounced | 93.2% | **1st overall** |
| [Provider Voice](https://artificialanalysis.ai/text-to-speech/leaderboard/provider-voice) | Arena preference Elo; each model uses its own native voices | 1188 | **3rd among open-weights** · 17th overall |
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/aa_leaderboards.png)
Within Pronunciation Robustness, Ming-3.0-tts also ranks **1st** on Expanding Shorthand (text normalization, 96.5%) and Preserving Exact Sequences (sequence fidelity, 90.6%).
## Featured Abilities
- **Voice creation:** clone a reference voice, design one from text, or choose from 100+ Chinese and five English built-in voices.
- **Free-form control:** guide speech delivery with natural-language instructions.
- **Paralinguistic control:** insert 14 inline tags for laughter, sighs, and other vocal expressions in Chinese or English.
- **Languages and dialects:** synthesize Chinese, English, German, Spanish, French, Korean and Japanese, plus Sichuanese and Cantonese.
- **Emotion and attributes:** control emotion, or adjust speed, pitch and volume.
- **Multi-speaker dialogue:** generate a conversation from a script and speaker references (**tiny only**).
Capabilities use separate task modes, not arbitrary combinations. See the [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook) for supported inputs, [Performance](https://github.com/inclusionAI/Ming-3.0-tts#performance) for benchmarks, and [Blog & Demos](https://nomorecoding.github.io/Ming-3.0-tts/) for audio examples.
## Demo Video
tts11\_under10mb.mp4
## Architecture
Text and task-specific control tokens condition a language model with a DiT patch head and an audio tokenizer. Both models share this generation framework.
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/ming-3.0-tts.png)
## News
- 🔥 **\[2026.10\]** Ming-3.0-tts-0.5b and Ming-3.0-tts-tiny are released, with inference code, a task cookbook and the evaluation tables below.
## Roadmap
- Ming-3.0-tts-0.5b weights and inference code
- Ming-3.0-tts-tiny weights and inference code
- Ming-3.0-tts-tiny benchmarks
- vLLM-Omni accelerated inference
- Technical report
## Contents
- [Demo Video](https://github.com/inclusionAI/Ming-3.0-tts#demo-video)
- [Models](https://github.com/inclusionAI/Ming-3.0-tts#models)
- [Quick Start](https://github.com/inclusionAI/Ming-3.0-tts#quick-start-)
- [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook)
- [Built-in voices](https://github.com/inclusionAI/Ming-3.0-tts#built-in-voices)
- [Performance](https://github.com/inclusionAI/Ming-3.0-tts#performance)
- [Limitations](https://github.com/inclusionAI/Ming-3.0-tts#limitations)
- [Usage Disclaimer](https://github.com/inclusionAI/Ming-3.0-tts#usage-disclaimer)
## Models
| Model | Architecture | Parameters | Download |
| --- | --- | --- | --- |
| Ming-3.0-tts-0.5b | Dense | 0.5b | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) |
| Ming-3.0-tts-tiny | MoE | 7.9B total · 1.3B active | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-tiny)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-tiny) |
Each release checkpoint bundles the backbone, DiT head, audio tokenizer, speaker encoder and text tokenizer. The 0.5b release layout is shown below; tiny uses sharded weights with `model.safetensors.index.json`.
```
Ming-3.0-tts-0.5b/
├── config.json # model_type: dense, auto_map -> BailingMMConfig
├── configuration.json # ModelScope task descriptor
├── model.safetensors # backbone + audio VAE + DiT head + aggregator
├── campplus.onnx # speaker embedding extractor
├── tokenizer.json # plus vocab.json / merges.txt / added_tokens.json
├── tokenizer_config.json # special_tokens_map.json / chat_template.jinja
└── README.md
```
## Quick Start 🚀
### Installation
git clone https://github.com/inclusionAI/Ming-3.0-tts.git
cd Ming-3.0-tts
pip install -r requirements.txt # Ming-3.0-tts-0.5b
Or build the 0.5b Docker image (torch 2.6, CUDA 12.4, `transformers==4.46.3`):
docker build -f docker/Dockerfile -t ming-tts .
The 0.5b environment has been tested on NVIDIA H800-80GB and H20-96G. For 0.5b, `flash-attn` is optional; inference falls back to PyTorch SDPA.
Tiny uses a separate reference inference environment: Python **3.10.15**, PyTorch and torchaudio **2.7.1+cu118** (CUDA 11.8 wheels), and `transformers==4.46.3`. Install the pinned dependencies from [requirements-tiny.txt](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/requirements-tiny.txt) in a dedicated environment, as other dependency versions differ from 0.5b. We recommend using NVIDIA H800 for inference.
conda create -n ming-tiny python=3.10.15 -y && conda activate ming-tiny
pip install -r requirements-tiny.txt
### Model Download
pip install "huggingface\_hub\[cli\]"
hf download inclusionAI/Ming-3.0-tts-0.5b --local-dir Ming-3.0-tts-0.5b
In mainland China, ModelScope is usually faster:
pip install modelscope
modelscope download --model inclusionAI/Ming-3.0-tts-0.5b --local\_dir Ming-3.0-tts-0.5b
Replace `0.5b` with `tiny` for the other variant.
### Text Normalization
Numbers, dates, units and symbols are normalized with FST grammars, which need `pynini`. It has no universal wheel:
conda install -c conda-forge pynini==2.1.6 # or: pip install pynini==2.1.6
It is loaded lazily; pass `--no_text_normalization` to skip it.
### Ming-3.0-tts-0.5b
python infer.py \\
--model\_path inclusionAI/Ming-3.0-tts-0.5b \\
--task TTS \\
--language\_id Chinese \\
--text "傍晚的风从山口吹过来,远处的云慢慢染成了金色。" \\
--prompt\_wav\_path data/voice\_presets/voice\_audio/lingguang.wav \\
--prompt\_text "诶,你今天有空吗?我们能不能聊聊这件事啊?" \\
--use\_spk\_emb \\
--output output/tts.wav
Or from Python:
from ming\_tts import MingTTS
model \= MingTTS("inclusionAI/Ming-3.0-tts-0.5b")
model.synthesize(
task\="TTS",
language\_id\="Chinese", \# English for English text
text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。",
prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav",
prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?",
use\_spk\_emb\=True,
output\_path\="output/tts.wav",
)
### Ming-3.0-tts-tiny
Every task has a one-line script in [examples/one\_click/](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/examples/one_click); variables such as `MODEL`, `TEXT` and `OUT` can be overridden from the environment:
bash examples/one\_click/run\_seed.sh
MODEL=/path/to/Ming-3.0-tts-tiny OUT=outputs/yue.wav bash examples/one\_click/run\_dialect.sh
The same call from Python. Loading configures the release runtime (bf16, portable MoE loop, fp32 recurrent state) and checks the checkpoint against it; `generate()` defaults to `cfg_scale=2.0`, `temperature=0`, `flow_steps=10`, `sigma=0.25`, and seed 1234 (1235 for `podcast`):
from ming\_tiny import MingTinyTTS
tts \= MingTinyTTS.from\_pretrained("inclusionAI/Ming-3.0-tts-tiny", device\="cuda:0")
tts.generate(
task\="seed",
text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。",
prompt\_wav\="data/voice\_presets/voice\_audio/lingguang.wav",
prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?",
target\_language\="Chinese", \# English for English text
output\_path\="outputs/seed.wav",
)
tts.generate(
task\="podcast",
text\="speaker\_1: 你最近是不是又去那家面馆了?\\nspeaker\_2: 对啊,排了半个小时的队,不过真的值得。",
prompt\_wav\=\["data/voice\_presets/voice\_audio/lingguang.wav", "data/voice\_presets/voice\_audio/db30.wav"\],
prompt\_text\=\["诶,你今天有空吗?我们能不能聊聊这件事啊?", "感谢你的认可。"\],
output\_path\="outputs/podcast.wav",
)
Each call selects one task and accepts the inputs listed below:
| Task | Reference audio | Reference text | `target_language` | Controls |
| --- | --- | --- | --- | --- |
| `seed` | one | required | `Chinese` / `English`, required | — |
| `multilingual` | one | required | `German` / `Spanish` / `French` / `Korean` / `Japanese` | — |
| `paralinguistic` | one | required | `Chinese` / `English` | inline tags in `text` |
| `podcast` | one per speaker | one per speaker, same order | inferred from the text | `speaker_N:` at the start of every line |
| `basic` | one | not accepted | — | `speed`, `pitch`, `volume`, all three |
| `emotion` | one | not accepted | — | `emotion` |
| `dialect` | one | not accepted | — | `dialect`: `川渝话` / `广粤话` |
| `freeform` | none by default (recommended); optional | not accepted | — | `instruction` |
| `voice_design` | none | — | — | `style` |
| `ip` | none | — | — | `ip` |
`python infer_tiny.py --help` lists the same arguments for the command line; repeat `--prompt_wav` and `--prompt_text` for podcast.
| | Ming-3.0-tts-0.5b | Ming-3.0-tts-tiny |
| --- | --- | --- |
| Python API | `ming_tts.MingTTS` | `ming_tiny.MingTinyTTS` |
| CLI | `infer.py` | `infer_tiny.py`, `examples/one_click/` |
| Preflight | `tools/preflight.py` | `tools/preflight_tiny.py` |
| Tasks | TTS (inline paralinguistic tags included), FREEFORM, VOICE\_DESIGN, MULTILINGUAL, BASIC, EMOTION, DIALECT, IP | the same abilities as lower-case tasks (`seed`, `paralinguistic`, ...), plus `podcast` |
### Verify Your Setup
Check the checkpoint and tokenizer without loading weights:
python tools/preflight.py --model\_path /path/to/Ming-3.0-tts-0.5b
python tools/preflight\_tiny.py --model\_path /path/to/Ming-3.0-tts-tiny
## Cookbook
Ordered the way a line gets made: who speaks, how it is said, what goes beyond the words, and where it is heard. Each example is one task; controls from another task are rejected. Texts and controls are the ones on the [demo page](https://nomorecoding.github.io/Ming-3.0-tts/), so you can compare against what you hear there. 0.5b API; full value lists in [English](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md) / [简体中文](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instructions.md), every task end to end in [cookbooks/cookbook.ipynb](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/cookbook.ipynb).
### Who speaks
**Zero-shot Cloning**: the `TTS` call in [Quick Start](https://github.com/inclusionAI/Ming-3.0-tts#ming-30-tts-05b) clones the voice of `prompt_wav_path`.
**Voice Design**: describe the voice; no reference audio.
model.synthesize(
task\="VOICE\_DESIGN",
text\="冰川在漫长岁月中缓缓移动,也悄悄记录着气候变化的痕迹。",
controls\={"style": "成熟沉稳的男性纪录片旁白,吐字清晰,节奏舒缓,语气理性而有厚度。"},
use\_zero\_spk\_emb\=True,
)
**Built-in Voices**: recommended **US\_James**, **US\_Stella**, **灵小柔** and **灵楷**. See the [full voice list](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices).
model.synthesize(
task\="IP",
text\="The garden will remain open until sunset, provided the weather stays clear.",
controls\={"ip": "US\_James"},
use\_zero\_spk\_emb\=True,
)
### How it is said
**Freeform**: describe the voice and delivery with one natural-language instruction. Both models default to generation without reference audio; adding a reference is not recommended. Not combined with voice design, a built-in voice, emotion, dialect or attribute controls.
model.synthesize(
task\="FREEFORM",
text\="The old radio crackled softly as the storm moved across the valley.",
instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.",
use\_zero\_spk\_emb\=True,
)
For Tiny, omit `prompt_wav` and `prompt_text`:
tts.generate(
task\="freeform",
text\="The old radio crackled softly as the storm moved across the valley.",
instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.",
)
**Emotion**: `高兴 / 悲伤 / 愤怒 / 恐惧 / 厌恶 / 惊讶 / 中性`.
model.synthesize(
task\="EMOTION",
text\="所有测试都顺利通过了,我们终于可以把这个好消息告诉大家!",
controls\={"emotion": "高兴"},
prompt\_wav\_path\="data/wavs/emotion\_prompt.wav",
use\_spk\_emb\=True,
)
**Speed, Pitch and Volume**: speed `慢速 / 中速 / 快速`, pitch and volume `低 / 中 / 高`.
model.synthesize(
task\="BASIC",
text\="今晚的观测将在云层散开后开始,请耐心等待下一次通知。",
controls\={"speed": "慢速", "pitch": "中", "volume": "中"},
spk\_emb\_path\="/path/to/speaker\_embedding.pt",
use\_spk\_emb\=True,
)
### Beyond the words
**Paralinguistic Tags**: inline, anywhere in the sentence, in Chinese or English. `[Breathing]`, `[Cough]`, `[Sigh]`, `[Laughter]`, `[Surprise-ah]`, `[Surprise-oh]`, `[Dissatisfaction-hnn]`, `[Uhm]`, `[Confirmation-en]`, `[Question-ei]`, `[Question-ah]`, `[Question-en]`, `[Question-oh]`, `[Question-huh]`.
model.synthesize(
task\="TTS",
text\="\[Sigh\]雨还是没有停,\[Cough\]我们恐怕只能把活动改到室内了。",
prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav",
prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?",
use\_spk\_emb\=True,
)
### Where it is heard
**Multilingual**: `language_id` is required for the five new languages; short codes and full names both work.
model.synthesize(
task\="MULTILINGUAL",
text\="週末、駅前の小さな喫茶店で、季節限定の抹茶パフェを食べました",
language\_id\="Japanese", \# ja / ko / de / fr / es
prompt\_wav\_path\="/path/to/japanese\_prompt.wav",
prompt\_text\="参考音频对应的日语文本。",
use\_spk\_emb\=True,
)
**Dialects**: `chuanyu dialect` or `cantonese`; this task takes no `language_id`.
model.synthesize(
task\="DIALECT",
text\="今日天气几好,我哋食完饭一齐去海边行下啦。",
controls\={"dialect": "cantonese"},
prompt\_wav\_path\="data/wavs/yue\_prompt.wav",
use\_spk\_emb\=True,
)
## Built-in voices
### Recommended voices
| Voice | Language | Character |
| --- | --- | --- |
| US\_James | English (US) | Male. Narration, explainers and documentary. |
| US\_Stella | English | — |
| 灵小柔 | Chinese | Bright, sweet young female voice with clear diction. Assistants and customer service. |
| 灵楷 | Chinese | Warm, resonant young male voice. Radio, documentary and audiobooks. |
Over 100 Chinese voices and five English voices are available. See [the voice catalog](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices) for the full list, including emotion-specific variants.
## Performance
Open benchmarks are below; independent arena results are in [Artificial Analysis Leaderboards](https://github.com/inclusionAI/Ming-3.0-tts#artificial-analysis-leaderboards).
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_basic_1018.png)
[](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_control_1018.png)
Below, **tiny** is Ming-3.0-tts-tiny and **0.5b** is Ming-3.0-tts-0.5b. Ranks and marks are computed per metric among the listed open-source models; Seed-TTS and NVbench-GT are reference rows and are not ranked. Where a historical Ming-omni-tts result beats a bolded Ming-3.0-tts value, it is noted below the table and no SOTA is claimed. **Bold** marks the best value and underline the second-best value in each column; ties share a mark. Other systems' results use their reported inference settings.
### Summary
1. **Paralinguistic:** together, the two models reach **SOTA on all six NVBench single-tag metrics**; tiny reaches **SOTA on all three English multi-tag metrics**.
2. **Dialect:** 0.5b reaches **SOTA** on Cantonese Base CER and ACC; tiny reaches **SOTA** on Sichuanese Hard CER and Easy ACC.
3. **Multilingual:** tiny reaches **SOTA** on German and Spanish WER, ranks **2nd** on French WER, and ties for **2nd** on Japanese CER.
4. **Emotion:** tiny ties for **SOTA** on CV3 text-related Happy and reaches **SOTA** on both CV3N Sad subsets; 0.5b reaches **SOTA** on CV3 text-related Angry. Tiny's CV3 average ranks **2nd**.
5. **Attribute control:** 0.5b leads the current comparison on average ACC and pitch, while tiny leads on WER and SIM. Historical Ming-omni-tts results still hold the corresponding overall records.
6. **Podcast:** tiny achieves CER 2.03%, cpSIM 0.561 and UTMOS 2.12, ranking **1st**, **2nd** and **2nd**; historical Ming-omni-tts retains lower CER.
7. **Voice Design:** tiny ranks **2nd** on average ACC (75.37%), APS (85.10%) and DSD (79.90%).
8. **Chinese and English:** 0.5b's Chinese WER ranks **3rd**; tiny reaches **SOTA** on Chinese and English SIM.
9. **Next:** improve multilingual and Cantonese speaker similarity, tiny's Korean accuracy and pitch control, text-unrelated emotion, role-play instruction following, and predicted podcast audio quality.
### Chinese and English (seed-tts-eval)
| Model | seed-tts-eval-zh | seed-tts-eval-en |
|---|
| WER ↓ | SIM ↑ | WER ↓ | SIM ↑ |
|---|
| Seed-TTS | 1.110% | 0.796 | 2.240% | 0.762 |
| GLM-TTS | 1.030% | 0.761 | 2.230% | 0.672 |
| Qwen3-TTS | 1.070% | — | 1.390% | — |
| Qwen3-TTS-12Hz-0.6B-Base | 0.920% | — | 1.320% | — |
| Qwen3-TTS-12Hz-1.7B-Base | 0.770% | — | 1.240% | — |
| CosyVoice3-0.5B | 1.160% | 0.780 | 2.020% | 0.718 |
| CosyVoice3-1.5B | 0.710% | 0.775 | 1.450% | 0.695 |
| F5-TTS | 1.560% | 0.741 | 1.830% | 0.647 |
| Ming-3.0-tts-0.5b | 0.800% | 0.758 | 1.584% | 0.691 |
| Ming-3.0-tts-tiny | 0.820% | 0.781 | 1.666% | 0.728 |
Tiny achieves **SOTA speaker similarity** in Chinese and English. 0.5b has lower WER in both languages and ranks **3rd** on Chinese WER.
### Multilingual (MiniMaxAI TTS-Multilingual-Test-Set)
| Model | German | Spanish | French | Korean | Japanese |
|---|
| WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | CER ↓ | SIM ↑ | CER ↓ | SIM ↑ |
|---|
| dots.tts | 0.452% | 0.798 | 2.593% | 0.839 | 19.878% | 0.789 | 4.837% | 0.853 | 5.134% | 0.838 |
| VoxCPM2 | 0.963% | 0.790 | 1.324% | 0.836 | 3.380% | 0.761 | 3.547% | 0.839 | 4.358% | 0.825 |
| Fish Audio S2 | 1.045% | 0.714 | 1.216% | 0.748 | 3.636% | 0.698 | 1.675% | 0.761 | 3.427% | 0.755 |
| FireRedTTS3 | 0.348% | 0.825 | 1.086% | 0.868 | 4.366% | 0.807 | 1.644% | 0.864 | 3.418% | 0.825 |
| Ming-3.0-tts-0.5b | 0.365% | 0.753 | 1.099% | 0.763 | 4.264% | 0.675 | 1.728% | 0.793 | 3.765% | 0.765 |
| Ming-3.0-tts-tiny | 0.325% | 0.791 | 0.971% | 0.797 | 3.470% | 0.733 | 6.000% | 0.797 | 3.427% | 0.780 |
Each open-source model was evaluated using its recommended inference settings under a unified ASR-based WER/CER evaluation protocol.
Tiny achieves **SOTA WER in German and Spanish**, ranks **2nd** in French WER and ties for **2nd** in Japanese CER. 0.5b is more accurate in Korean; speaker similarity remains a gap for both models.
### Paralinguistic control (NVBench)
CER, PCER (paralinguistic CER) and OCER are reported. NVbench-GT is human-labelled reference, shown for context and not ranked.
| Model | Single | Multi |
|---|
| Chinese | English | Chinese | English |
|---|
| CER | PCER | OCER | CER | PCER | OCER | CER | PCER | OCER | CER | PCER | OCER |
|---|
| NVbench-GT | 3.86 | 9.38 | 4.07 | 6.73 | 8.31 | 6.90 | 3.79 | 23.71 | 5.07 | 7.26 | 21.41 | 8.62 |
| Orpheus-TTS | 11.36 | 88.77 | 13.91 | 9.03 | 71.92 | 10.63 | 19.83 | 84.85 | 24.38 | 8.68 | 71.46 | 11.89 |
| SMIIP-NV-CV2 | 8.80 | 75.64 | 11.34 | 17.92 | 56.80 | 19.47 | 10.66 | 77.20 | 14.79 | 20.93 | 54.49 | 23.87 |
| Emilia-NV-CV2 | 5.05 | 40.00 | 6.64 | 12.50 | 55.30 | 13.21 | 5.54 | 48.74 | 8.09 | 11.71 | 60.28 | 14.63 |
| CosyVoice3 | 3.85 | 57.69 | 5.86 | 7.87 | 62.75 | 9.06 | 4.75 | 61.94 | 8.26 | 6.39 | 57.84 | 10.69 |
| NV-FlexiVoice | 6.98 | 31.08 | 8.15 | 11.88 | 50.43 | 13.21 | 8.20 | 39.37 | 10.39 | 9.60 | 51.32 | 13.76 |
| NV-CV3 | 3.80 | 27.69 | 4.90 | 8.33 | 46.13 | 9.44 | 3.44 | 30.04 | 4.84 | 6.70 | 47.13 | 10.10 |
| Ming-3.0-tts-0.5b | 2.70 | 22.80 | 3.20 | 7.60 | 37.00 | 8.90 | 3.50 | 36.60 | 8.10 | 7.90 | 47.30 | 10.40 |
| Ming-3.0-tts-tiny | 2.80 | 17.23 | 3.20 | 5.86 | 32.95 | 7.05 | 4.49 | 32.88 | 5.81 | 5.12 | 46.13 | 7.68 |
Together, the two models achieve **SOTA on all six Chinese and English single-tag metrics**; tiny also leads all three English multi-tag metrics. Chinese multi-tag control remains behind NV-CV3.
### Dialect
| Model | WSC-Eval-TTS-easy | WSC-Eval-TTS-hard | WSYue-TTS-eval-Base | WSYue-TTS-eval-Coverage |
|---|
| CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ |
|---|
| Qwen3-TTS | 4.13% | — | — | 7.35% | — | — | — | — | — | — | — | — |
| CosyVoice3-0.5B | 3.17% | 0.696 | 76.05% | 4.07% | 0.723 | 81.41% | 8.36% | 0.611 | 91.70% | 8.95% | 0.658 | 95.80% |
| CosyVoice3-1.5B + DiffRO-EMO | 7.14% | 0.703 | — | 9.06% | 0.601 | — | 14.38% | 0.812 | — | 13.74% | 0.826 | — |
| CosyVoice2-WSC | 4.28% | 0.728 | — | 8.87% | 0.626 | — | — | — | — | — | — | — |
| CosyVoice2-WSC-SFT | 4.08% | 0.788 | — | 7.22% | 0.680 | — | — | — | — | — | — | — |
| CosyVoice2-Yue | — | — | — | — | — | — | 10.33% | 0.821 | — | 9.49% | 0.834 | — |
| Sparks-TTS | 10.83% | 0.677 | — | 12.52% | 0.545 | — | 27.79% | 0.762 | — | 24.25% | 0.781 | — |
| Ming-3.0-tts-0.5b | 2.27% | 0.689 | 80.68% | 3.35% | 0.691 | 87.64% | 6.15% | 0.586 | 98.00% | 8.15% | 0.622 | 95.27% |
| Ming-3.0-tts-tiny | 2.53% | 0.721 | 86.09% | 3.11% | 0.738 | 88.94% | 7.40% | 0.605 | 92.90% | 8.67% | 0.653 | 95.16% |
Tiny is stronger in Sichuanese control accuracy and achieves **SOTA Sichuanese Hard CER and Easy ACC**, while 0.5b achieves **SOTA Cantonese Base CER and ACC**. Cantonese speaker similarity remains below CosyVoice2-Yue. Historical Ming-omni-tts results remain stronger on Sichuanese Easy CER (2.25%), Hard SIM (0.750) and Hard ACC (90.45%), and on Cantonese Coverage CER (7.87%).
### Emotion control (CV3)
| Model | Average | Text-Related | Text-Unrelated |
|---|
| Happy | Sad | Angry | Happy | Sad | Angry |
|---|
| CosyVoice3-0.5B | 66.33% | 92% | 70% | 72% | 64% | 42% | 58% |
| CosyVoice3-1.5B | 63.00% | 86% | 64% | 72% | 64% | 44% | 48% |
| CosyVoice3-1.5B + DiffRO-EMO | 77.67% | 98% | 68% | 84% | 98% | 50% | 68% |
| F5-TTS | 64.67% | 92% | 52% | 72% | 80% | 28% | 64% |
| Sparks-TTS | 55.33% | 80% | 56% | 50% | 50% | 60% | 36% |
| Ming-3.0-tts-0.5b | 72.33% | 96% | 76% | 94% | 52% | 36% | 80% |
| Ming-3.0-tts-tiny | 74.00% | 98% | 82% | 90% | 60% | 36% | 78% |
Tiny and 0.5b rank **2nd** and **3rd** in average ACC. Tiny is stronger on text-related Happy and Sad, while 0.5b is stronger on Angry; text-unrelated emotion remains a shared limitation. Historical Ming-omni-tts results remain higher on text-related Sad (86%) and text-unrelated Angry (82%).
### Emotion control (CV3N, hard emotion control)
| Model | Average | Text-Related | Text-Unrelated |
|---|
| Happy | Sad | Angry | Happy | Sad | Angry |
|---|
| CosyVoice3-0.5B | 40.00% | 68% | 30% | 78% | 14% | 4% | 46% |
| Ming-3.0-tts-0.5b | 37.33% | 70% | 36% | 70% | 14% | 4% | 30% |
| Ming-3.0-tts-tiny | 30.67% | 60% | 40% | 44% | 18% | 10% | 12% |
0.5b and tiny rank **2nd** and **3rd** in average ACC. Tiny achieves **SOTA on both Sad subsets**, while 0.5b has higher overall accuracy. Historical Ming-omni-tts results remain higher on text-related Happy (78%) and text-unrelated Happy (30%).
### Voice Design (InstructTTSEval-ZH)
| Model | Average | APS | DSD | RP |
| --- | --- | --- | --- | --- |
| Qwen3-TTS | **77.13%** | **85.20%** | **81.10%** | **65.10%** |
| Mimo-Audio-7B-Instruct | 70.50% | 75.70% | 74.30% | 61.50% |
| VoiceSculptor | 67.30% | 75.70% | 64.70% | 61.50% |
| VoxInstruct | 47.47% | 47.50% | 52.30% | 42.60% |
| **Ming-3.0-tts-0.5b** | 72.29% | 78.54% | 77.28% | 61.06% |
| **Ming-3.0-tts-tiny** | 75.37% | 85.10% | 79.90% | 61.10% |
Tiny and 0.5b rank **2nd** and **3rd** in average ACC, behind Qwen3-TTS. Role-play instruction following remains the main gap for both models.
### Basic attribute control
| Model | Speed | Volume | Pitch | Average | WER ↓ | SIM ↑ |
| --- | --- | --- | --- | --- | --- | --- |
| CosyVoice3-0.5B | **100.00%** | **97.67%** | 65.33% | 87.67% | 1.210% | 0.580 |
| **Ming-3.0-tts-0.5b** | 99.00% | 95.00% | **86.00%** | **93.30%** | 0.317% | 0.712 |
| **Ming-3.0-tts-tiny** | 93.00% | 93.00% | 78.70% | 88.23% | **0.274%** | **0.742** |
0.5b has higher control accuracy, while tiny has lower WER and higher speaker similarity. Historical Ming-omni-tts models retain the overall records for average ACC, pitch, WER and SIM.
### Podcast
| Model | CER ↓ | cpSIM ↑ | UTMOS ↑ |
| --- | --- | --- | --- |
| FireRedTTS2 | 3.34% | 0.512 | 1.90 |
| SoulX-Podcast | 2.20% | **0.599** | 2.09 |
| VibeVoice-1.5B | 12.87% | 0.455 | 1.74 |
| MOSS-TTSD | 8.62% | 0.421 | 1.70 |
| ZipVoice-Dia | 3.39% | 0.553 | **2.24** |
| MoonCast | 27.43% | 0.441 | 1.76 |
| **Ming-3.0-tts-tiny** | **2.03%** | 0.561 | 2.12 |
Tiny ranks **1st** in CER and **2nd** in cpSIM and UTMOS. Historical Ming-omni-tts retains lower CER and higher UTMOS.
## Limitations
- **Speaker similarity.** Tiny ranks third on German SIM (0.791), but neither model reaches the top three in the other four new languages. French SIM is 0.675 for 0.5b and 0.733 for tiny; both also trail CosyVoice2-Yue on Cantonese SIM.
- **Several events in one line.** NVBench _Multi_ works, but trails _Single_; stability with many tags per sentence is still improving.
- **Emotion from delivery alone.** When the words carry no emotional signal (CV3-Eval Text-Unrelated, and most of the hard set), accuracy drops well below the Text-Related half.
- **Pitch on tiny.** Pitch is the least accurate attribute for both models. Tiny's 78.70% ranks 2nd, 7.30 percentage points below 0.5b's 86.00%; the historical Ming-omni-tts-0.5B score is 91.33%.
- **Role-play voice design.** RP is the lowest voice design sub-score, ranking 4th for tiny (61.10%) and 5th for 0.5b (61.06%).
- **Korean on tiny.** Tiny's Korean CER (6.000%) ranks 6th, well behind 0.5b's 1.728% (3rd).
- **Podcast audio quality.** Tiny's UTMOS of 2.12 ranks 2nd, below ZipVoice-Dia's 2.24 and the historical Ming-omni-tts-0.5B score of 2.25.
## Usage Disclaimer
- Voice cloning and built-in character voices are provided for research and legitimate creative use. Do not use them to impersonate a real person without consent, to deceive, or for any illegal purpose.
- Label synthetic speech as synthetic where it could be mistaken for a real recording.
- The authors accept no liability for misuse. If you find abuse of this project, please report it to us.
## Acknowledgements
- [CAM++](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common) for speaker embedding extraction
- [Ming-omni-tts](https://github.com/inclusionAI/Ming-omni-tts) for the audio tokenizer
## License
Released under the [MIT](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/LICENSE) license.