蚂蚁 inclusionAI 开源 Ming-3.0-tts 语音合成模型

来源:蚂蚁 inclusionAI:GitHub 新仓库 2026年10月19日 15:03 AIHOT 评分:72 精选
摘要:蚂蚁 inclusionAI 在 GitHub 发布 Ming-3.0-tts,包含 0.5b dense 和基于 Ling V3 的 tiny(7.9B 总参数、1.3B 激活)两个语音生成模型,MIT 协议开源,权重可在 Hugging Face 和 ModelScope 下载。
推荐理由:官方仓库给出两版模型的榜单名次、完整任务列表和部署细节,读者可据此评估其在多语言与副语言控制上的实际可用性。
# 蚂蚁 inclusionAI 开源 Ming-3.0-tts 语音合成模型 > 原标题:inclusionAI/Ming-3.0-tts - 来源:蚂蚁 inclusionAI:GitHub 新仓库 - 发布时间:2026-10-19T07:03:25.000Z - AIHOT:https://aihot.news/items/uvgayscfmc9ujjak94ymsnxnd - 原文:https://github.com/inclusionAI/Ming-3.0-tts ## 摘要 蚂蚁 inclusionAI 在 GitHub 发布 Ming-3.0-tts,包含 0.5b dense 和基于 Ling V3 的 tiny(7.9B 总参数、1.3B 激活)两个语音生成模型,MIT 协议开源,权重可在 Hugging Face 和 ModelScope 下载。 ## 推荐理由 官方仓库给出两版模型的榜单名次、完整任务列表和部署细节,读者可据此评估其在多语言与副语言控制上的实际可用性。 ## 正文 · 中文译文 支持自由形式与副语言控制的多语言语音合成。 🤗 [Hugging Face](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)| 🤖 [ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) | 📑 [博客与演示](https://nomorecoding.github.io/Ming-3.0-tts/) | 💬 [DingTalk(钉钉)](https://qr.dingtalk.com/action/joingroup?code=v1,k1,O7MiZqrOrB2c7PnUZVBNvmDh/6tghNLMEtXqMteyRIpuRVJIwrSsXmL8oFqU5ajJ&_dt_no_comment=1&origin=11) Ming-3.0-tts 提供两个语音生成模型:**0.5b**,一个稠密模型,以及 **tiny**,一个基于 Ling V3 构建的 MoE 模型(总参数 7.9B · 激活参数 1.3B)。 ## 🏆 Artificial Analysis 排行榜 在独立的 [Artificial Analysis](https://artificialanalysis.ai/text-to-speech) 文本转语音排行榜上(2026 年 10 月),Ming-3.0-tts 在**受控语音类别中位列开源权重模型第 1 名**,在**发音稳健性类别中位列总榜第 1 名**,并在**提供方语音类别中位列开源权重模型第 3 名**。排名会随着新模型的加入而变化。 | 排行榜 | 设置 | Ming-3.0-tts | 排名 | | --- | --- | --- | --- | | [受控语音](https://artificialanalysis.ai/text-to-speech/leaderboard/controlled-voice) | 竞技场偏好 Elo;每个模型克隆相同的 8 个声音(4 个美式、4 个英式) | 1069 | **开源权重模型第 1 名** · 总榜第 13 名 | | [发音稳健性](https://artificialanalysis.ai/text-to-speech) | 英文文本中高亮片段被人工评审判定为发音正确的比例 | 93.2% | **总榜第 1 名** | | [提供方语音](https://artificialanalysis.ai/text-to-speech/leaderboard/provider-voice) | 竞技场偏好 Elo;每个模型使用其自带的原生声音 | 1188 | **开源权重模型第 3 名** · 总榜第 17 名 | [![Artificial Analysis leaderboards: Controlled Voice and Provider Voice (open weights), Pronunciation Robustness](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/aa_leaderboards.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/aa_leaderboards.png) 在发音稳健性类别中,Ming-3.0-tts 还在扩展缩写(文本规范化,96.5%)和保留精确序列(序列保真度,90.6%)两项上位列**第 1 名**。 ## 主要能力 - **声音创建:**克隆参考声音、根据文本设计声音,或从 100 多个中文内置声音和五个英文内置声音中选择。 - **自由形式控制:**用自然语言指令引导语音表达。 - **副语言控制:**插入 14 个内联标签,用于中文或英文中的笑声、叹息及其他发声表达。 - **语言与方言:**合成中文、英文、德文、西班牙文、法文、韩文和日文,以及四川话和粤语。 - **情感与属性:**控制情感,或调整语速、音高和音量。 - **多说话人对话:**根据脚本和说话人参考生成对话(**仅 tiny**)。 各项能力使用独立的任务模式,而非任意组合。支持的输入请参阅 [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook),基准测试请参阅 [Performance](https://github.com/inclusionAI/Ming-3.0-tts#performance),音频示例请参阅 [博客与演示](https://nomorecoding.github.io/Ming-3.0-tts/)。 ## 演示视频 tts11\_under10mb.mp4 ## 架构 文本和任务特定的控制 token 作为条件输入到一个语言模型,该模型带有 DiT patch head 和音频分词器。两个模型共享这一生成框架。 [![](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/ming-3.0-tts.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/ming-3.0-tts.png) ## 新闻 - 🔥 **\[2026.10\]** Ming-3.0-tts-0.5b 和 Ming-3.0-tts-tiny 发布,包含推理代码、任务 cookbook 以及下方的评估表格。 ## 路线图 - Ming-3.0-tts-0.5b 权重与推理代码 - Ming-3.0-tts-tiny 权重与推理代码 - Ming-3.0-tts-tiny 基准测试 - vLLM-Omni 加速推理 - 技术报告 ## 目录 - [演示视频](https://github.com/inclusionAI/Ming-3.0-tts#demo-video) - [模型](https://github.com/inclusionAI/Ming-3.0-tts#models) - [快速开始](https://github.com/inclusionAI/Ming-3.0-tts#quick-start-) - [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook) - [内置声音](https://github.com/inclusionAI/Ming-3.0-tts#built-in-voices) - [性能](https://github.com/inclusionAI/Ming-3.0-tts#performance) - [局限性](https://github.com/inclusionAI/Ming-3.0-tts#limitations) - [使用免责声明](https://github.com/inclusionAI/Ming-3.0-tts#usage-disclaimer) ## 模型 | 模型 | 架构 | 参数 | 下载 | | --- | --- | --- | --- | | Ming-3.0-tts-0.5b | 稠密 | 0.5b | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) | | Ming-3.0-tts-tiny | MoE | 总参数 7.9B · 激活参数 1.3B | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-tiny)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-tiny) | 每个发布检查点都打包了 backbone、DiT head、音频 tokenizer、说话人编码器和文本 tokenizer。0.5b 发布版的目录结构如下所示;tiny 使用分片权重,配合 `model.safetensors.index.json`。 ``` Ming-3.0-tts-0.5b/ ├── config.json # model_type: dense, auto_map -> BailingMMConfig ├── configuration.json # ModelScope task descriptor ├── model.safetensors # backbone + audio VAE + DiT head + aggregator ├── campplus.onnx # speaker embedding extractor ├── tokenizer.json # plus vocab.json / merges.txt / added_tokens.json ├── tokenizer_config.json # special_tokens_map.json / chat_template.jinja └── README.md ``` ## 快速开始 🚀 ### 安装 git clone https://github.com/inclusionAI/Ming-3.0-tts.git cd Ming-3.0-tts pip install -r requirements.txt # Ming-3.0-tts-0.5b 或者构建 0.5b Docker 镜像(torch 2.6、CUDA 12.4、`transformers==4.46.3`): docker build -f docker/Dockerfile -t ming-tts . 0.5b 环境已在 NVIDIA H800-80GB 和 H20-96G 上测试过。对于 0.5b,`flash-attn` 是可选的;推理会回退到 PyTorch SDPA。 Tiny 使用单独的参考推理环境:Python **3.10.15**、PyTorch 和 torchaudio **2.7.1+cu118**(CUDA 11.8 wheels),以及 `transformers==4.46.3`。请在专用环境中从 [requirements-tiny.txt](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/requirements-tiny.txt) 安装固定版本的依赖,因为其他依赖版本与 0.5b 不同。我们建议使用 NVIDIA H800 进行推理。 conda create -n ming-tiny python=3.10.15 -y && conda activate ming-tiny pip install -r requirements-tiny.txt ### 模型下载 pip install "huggingface\_hub\[cli\]" hf download inclusionAI/Ming-3.0-tts-0.5b --local-dir Ming-3.0-tts-0.5b 在中国大陆,ModelScope 通常更快: pip install modelscope modelscope download --model inclusionAI/Ming-3.0-tts-0.5b --local\_dir Ming-3.0-tts-0.5b 将 `0.5b` 替换为 `tiny` 即可获取另一个变体。 ### 文本规范化 数字、日期、单位和符号通过 FST 语法进行规范化,这需要 `pynini`。它没有通用 wheel: conda install -c conda-forge pynini==2.1.6 # or: pip install pynini==2.1.6 它是延迟加载的;传入 `--no_text_normalization` 可跳过它。 ### Ming-3.0-tts-0.5b python infer.py \\ --model\_path inclusionAI/Ming-3.0-tts-0.5b \\ --task TTS \\ --language\_id Chinese \\ --text "傍晚的风从山口吹过来,远处的云慢慢染成了金色。" \\ --prompt\_wav\_path data/voice\_presets/voice\_audio/lingguang.wav \\ --prompt\_text "诶,你今天有空吗?我们能不能聊聊这件事啊?" \\ --use\_spk\_emb \\ --output output/tts.wav 或者从 Python 中: from ming\_tts import MingTTS model \= MingTTS("inclusionAI/Ming-3.0-tts-0.5b") model.synthesize( task\="TTS", language\_id\="Chinese", \# English for English text text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。", prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav", prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?", use\_spk\_emb\=True, output\_path\="output/tts.wav", ) ### Ming-3.0-tts-tiny 每个任务在 [examples/one\_click/](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/examples/one_click) 中都有一个单行脚本;诸如 `MODEL`、`TEXT` 和 `OUT` 这样的变量可以从环境变量中覆盖: bash examples/one\_click/run\_seed.sh MODEL=/path/to/Ming-3.0-tts-tiny OUT=outputs/yue.wav bash examples/one\_click/run\_dialect.sh 从 Python 中调用同样如此。加载时会配置发布版运行时(bf16、可移植 MoE 循环、fp32 循环状态)并对照检查检查点;`generate()` 默认为 `cfg_scale=2.0`、`temperature=0`、`flow_steps=10`、`sigma=0.25`,随机种子为 1234(`podcast` 为 1235): from ming\_tiny import MingTinyTTS tts \= MingTinyTTS.from\_pretrained("inclusionAI/Ming-3.0-tts-tiny", device\="cuda:0") tts.generate( task\="seed", text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。", prompt\_wav\="data/voice\_presets/voice\_audio/lingguang.wav", prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?", target\_language\="Chinese", \# English for English text output\_path\="outputs/seed.wav", ) tts.generate( task\="podcast", text\="speaker\_1: 你最近是不是又去那家面馆了?\\nspeaker\_2: 对啊,排了半个小时的队,不过真的值得。", prompt\_wav\=\["data/voice\_presets/voice\_audio/lingguang.wav", "data/voice\_presets/voice\_audio/db30.wav"\], prompt\_text\=\["诶,你今天有空吗?我们能不能聊聊这件事啊?", "感谢你的认可。"\], output\_path\="outputs/podcast.wav", ) 每次调用选择一个任务,并接受下面列出的输入: | 任务 | 参考音频 | 参考文本 | `target_language` | 控制项 | | --- | --- | --- | --- | --- | | `seed` | 一个 | 必需 | `Chinese` / `English`,必需 | — | | `multilingual` | 一个 | 必需 | `German` / `Spanish` / `French` / `Korean` / `Japanese` | — | | `paralinguistic` | 一个 | 必需 | `Chinese` / `English` | `text` 中的内联标签 | | `podcast` | 每个说话人一个 | 每个说话人一个,顺序相同 | 从文本推断 | 每行开头的 `speaker_N:` | | `basic` | 一个 | 不接受 | — | `speed`、`pitch`、`volume`,全部三个 | | `emotion` | 一个 | 不接受 | — | `emotion` | | `dialect` | 一个 | 不接受 | — | `dialect`:`川渝话` / `广粤话` | | `freeform` | 默认无(推荐);可选 | 不接受 | — | `instruction` | | `voice_design` | 无 | — | — | `style` | | `ip` | 无 | — | — | `ip` | `python infer_tiny.py --help` 为命令行列出了相同的参数;对于播客,重复 `--prompt_wav` 和 `--prompt_text`。 | | Ming-3.0-tts-0.5b | Ming-3.0-tts-tiny | | --- | --- | --- | | Python API | `ming_tts.MingTTS` | `ming_tiny.MingTinyTTS` | | CLI | `infer.py` | `infer_tiny.py`、`examples/one_click/` | | 预检 | `tools/preflight.py` | `tools/preflight_tiny.py` | | 任务 | TTS(包含内联副语言标签)、FREEFORM、VOICE\_DESIGN、MULTILINGUAL、BASIC、EMOTION、DIALECT、IP | 与小写任务相同的能力(`seed`、`paralinguistic`、……),外加 `podcast` | ### 验证你的设置 在不加载权重的情况下检查检查点和 tokenizer: python tools/preflight.py --model\_path /path/to/Ming-3.0-tts-0.5b python tools/preflight\_tiny.py --model\_path /path/to/Ming-3.0-tts-tiny ## Cookbook 按照一行语音的生成顺序排列:谁在说话、如何说、超越文字的内容,以及在哪里听到。每个示例对应一个任务;来自其他任务的控制项会被拒绝。文本和控制项与 [演示页面](https://nomorecoding.github.io/Ming-3.0-tts/) 上的一致,因此你可以与在那里听到的内容进行对比。0.5b API;完整取值列表见 [English](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md) / [简体中文](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instructions.md),每个任务的端到端示例见 [cookbooks/cookbook.ipynb](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/cookbook.ipynb)。 ### 谁在说话 **零样本克隆**:[快速开始](https://github.com/inclusionAI/Ming-3.0-tts#ming-30-tts-05b) 中的 `TTS` 调用会克隆 `prompt_wav_path` 的声音。 **声音设计**:描述声音;无需参考音频。 model.synthesize( task\="VOICE\_DESIGN", text\="冰川在漫长岁月中缓缓移动,也悄悄记录着气候变化的痕迹。", controls\={"style": "成熟沉稳的男性纪录片旁白,吐字清晰,节奏舒缓,语气理性而有厚度。"}, use\_zero\_spk\_emb\=True, ) **内置声音**:推荐 **US\_James**、**US\_Stella**、**灵小柔** 和 **灵楷**。参见 [完整声音列表](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices)。 model.synthesize( task\="IP", text\="The garden will remain open until sunset, provided the weather stays clear.", controls\={"ip": "US\_James"}, use\_zero\_spk\_emb\=True, ) ### 如何说 **自由形式**:用一条自然语言指令描述声音和表达方式。两个模型默认都在无参考音频的情况下生成;不建议添加参考。不与声音设计、内置声音、情感、方言或属性控制项组合使用。 model.synthesize( task\="FREEFORM", text\="The old radio crackled softly as the storm moved across the valley.", instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.", use\_zero\_spk\_emb\=True, ) 对于 Tiny,省略 `prompt_wav` 和 `prompt_text`: tts.generate( task\="freeform", text\="The old radio crackled softly as the storm moved across the valley.", instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.", ) **情感**:`高兴 / 悲伤 / 愤怒 / 恐惧 / 厌恶 / 惊讶 / 中性`。 model.synthesize( task\="EMOTION", text\="所有测试都顺利通过了,我们终于可以把这个好消息告诉大家!", controls\={"emotion": "高兴"}, prompt\_wav\_path\="data/wavs/emotion\_prompt.wav", use\_spk\_emb\=True, ) **语速、音高和音量**:语速 `慢速 / 中速 / 快速`,音高和音量 `低 / 中 / 高`。 model.synthesize( task\="BASIC", text\="今晚的观测将在云层散开后开始,请耐心等待下一次通知。", controls\={"speed": "慢速", "pitch": "中", "volume": "中"}, spk\_emb\_path\="/path/to/speaker\_embedding.pt", use\_spk\_emb\=True, ) ### 超越文字 **副语言标签**:可内联,出现在句子任意位置,支持中文或英文。`[Breathing]`、`[Cough]`、`[Sigh]`、`[Laughter]`、`[Surprise-ah]`、`[Surprise-oh]`、`[Dissatisfaction-hnn]`、`[Uhm]`、`[Confirmation-en]`、`[Question-ei]`、`[Question-ah]`、`[Question-en]`、`[Question-oh]`、`[Question-huh]`。 model.synthesize( task\="TTS", text\="\[Sigh\]雨还是没有停,\[Cough\]我们恐怕只能把活动改到室内了。", prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav", prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?", use\_spk\_emb\=True, ) ### 适用场景 **多语言**:五种新语言需要 `language_id`;短代码和全名均可使用。 model.synthesize( task\="MULTILINGUAL", text\="週末、駅前の小さな喫茶店で、季節限定の抹茶パフェを食べました", language\_id\="Japanese", \# ja / ko / de / fr / es prompt\_wav\_path\="/path/to/japanese\_prompt.wav", prompt\_text\="参考音频对应的日语文本。", use\_spk\_emb\=True, ) **方言**:`chuanyu dialect` 或 `cantonese`;此任务不接受 `language_id`。 model.synthesize( task\="DIALECT", text\="今日天气几好,我哋食完饭一齐去海边行下啦。", controls\={"dialect": "cantonese"}, prompt\_wav\_path\="data/wavs/yue\_prompt.wav", use\_spk\_emb\=True, ) ## 内置音色 ### 推荐音色 | 音色 | 语言 | 角色 | | --- | --- | --- | | US\_James | 英语(美国) | 男声。旁白、讲解和纪录片。 | | US\_Stella | 英语 | — | | 灵小柔 | 中文 | 明亮甜美的年轻女声,吐字清晰。适用于助手和客服。 | | 灵楷 | 中文 | 温暖浑厚的年轻男声。适用于电台、纪录片和有声书。 | 提供超过 100 种中文音色和五种英文音色。完整列表(包括情感特定变体)请参见[音色目录](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices)。 ## 性能 公开基准如下;独立竞技场结果见[Artificial Analysis 排行榜](https://github.com/inclusionAI/Ming-3.0-tts#artificial-analysis-leaderboards)。 [![](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/benchmark_basic_1018.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_basic_1018.png) [![](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/benchmark_control_1018.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_control_1018.png) 下文中,**tiny** 指 Ming-3.0-tts-tiny,**0.5b** 指 Ming-3.0-tts-0.5b。排名和标记按指标在所列开源模型中计算;Seed-TTS 和 NVbench-GT 为参考行,不参与排名。若历史 Ming-omni-tts 结果优于加粗的 Ming-3.0-tts 数值,将在表下注明,且不声称 SOTA。**加粗**表示每列最佳值,下划线表示次佳值;并列共享同一标记。其他系统的结果使用其报告的推理设置。 ### 总结 1. **副语言:**两个模型合计在**全部六项 NVBench 单标签指标上达到 SOTA**;tiny 在**全部三项英文多标签指标上达到 SOTA**。 2. **方言:**0.5b 在粤语 Base CER 和 ACC 上达到 **SOTA**;tiny 在四川话 Hard CER 和 Easy ACC 上达到 **SOTA**。 3. **多语言:**tiny 在德语和西班牙语 WER 上达到 **SOTA**,法语 WER 排名**第 2**,日语 CER 并列**第 2**。 4. **情感:**tiny 在 CV3 文本相关 Happy 上并列 **SOTA**,并在两个 CV3N Sad 子集上达到 **SOTA**;0.5b 在 CV3 文本相关 Angry 上达到 **SOTA**。tiny 的 CV3 平均排名**第 2**。 5. **属性控制:**0.5b 在平均 ACC 和音高上领先当前对比,而 tiny 在 WER 和 SIM 上领先。历史 Ming-omni-tts 结果仍保持相应的总体记录。 6. **播客:**tiny 取得 CER 2.03%、cpSIM 0.561 和 UTMOS 2.12,分别排名**第 1**、**第 2**和**第 2**;历史 Ming-omni-tts 保持更低的 CER。 7. **音色设计:**tiny 在平均 ACC(75.37%)、APS(85.10%)和 DSD(79.90%)上排名**第 2**。 8. **中文和英文:**0.5b 的中文 WER 排名**第 3**;tiny 在中文和英文 SIM 上达到 **SOTA**。 9. **下一步:**改进多语言和粤语说话人相似度、tiny 的韩语准确率和音高控制、文本无关情感、角色扮演指令遵循,以及预测的播客音频质量。 ### 中文和英文(seed-tts-eval)
模型seed-tts-eval-zhseed-tts-eval-en
WER ↓SIM ↑WER ↓SIM ↑
Seed-TTS1.110%0.7962.240%0.762
GLM-TTS1.030%0.7612.230%0.672
Qwen3-TTS1.070%—1.390%—
Qwen3-TTS-12Hz-0.6B-Base0.920%—1.320%—
Qwen3-TTS-12Hz-1.7B-Base0.770%—1.240%—
CosyVoice3-0.5B1.160%0.7802.020%0.718
CosyVoice3-1.5B0.710%0.7751.450%0.695
F5-TTS1.560%0.7411.830%0.647
Ming-3.0-tts-0.5b0.800%0.7581.584%0.691
Ming-3.0-tts-tiny0.820%0.7811.666%0.728
Tiny 在中文和英文上实现了**SOTA 说话人相似度**。0.5b 在两种语言上的 WER 都更低,并在中文 WER 上排名**第 3**。 ### 多语言(MiniMaxAI TTS-Multilingual-Test-Set)
模型德语西班牙语法语韩语日语
WER ↓SIM ↑WER ↓SIM ↑WER ↓SIM ↑CER ↓SIM ↑CER ↓SIM ↑
dots.tts0.452%0.7982.593%0.83919.878%0.7894.837%0.8535.134%0.838
VoxCPM20.963%0.7901.324%0.8363.380%0.7613.547%0.8394.358%0.825
Fish Audio S21.045%0.7141.216%0.7483.636%0.6981.675%0.7613.427%0.755
FireRedTTS30.348%0.8251.086%0.8684.366%0.8071.644%0.8643.418%0.825
Ming-3.0-tts-0.5b0.365%0.7531.099%0.7634.264%0.6751.728%0.7933.765%0.765
Ming-3.0-tts-tiny0.325%0.7910.971%0.7973.470%0.7336.000%0.7973.427%0.780
每个开源模型均在其推荐的推理设置下,采用统一的基于 ASR 的 WER/CER 评估协议进行评估。 Tiny 在**德语和西班牙语上实现了 SOTA WER**,在法语 WER 上排名**第 2**,在日语 CER 上并列**第 2**。0.5b 在韩语上更准确;两个模型的说话人相似度仍有差距。 ### 副语言控制(NVBench) 报告 CER、PCER(副语言 CER)和 OCER。NVbench-GT 为人工标注的参考结果,仅用于提供背景,不参与排名。
模型单标签多标签
中文英文中文英文
CERPCEROCERCERPCEROCERCERPCEROCERCERPCEROCER
NVbench-GT3.869.384.076.738.316.903.7923.715.077.2621.418.62
Orpheus-TTS11.3688.7713.919.0371.9210.6319.8384.8524.388.6871.4611.89
SMIIP-NV-CV28.8075.6411.3417.9256.8019.4710.6677.2014.7920.9354.4923.87
Emilia-NV-CV25.0540.006.6412.5055.3013.215.5448.748.0911.7160.2814.63
CosyVoice33.8557.695.867.8762.759.064.7561.948.266.3957.8410.69
NV-FlexiVoice6.9831.088.1511.8850.4313.218.2039.3710.399.6051.3213.76
NV-CV33.8027.694.908.3346.139.443.4430.044.846.7047.1310.10
Ming-3.0-tts-0.5b2.7022.803.207.6037.008.903.5036.608.107.9047.3010.40
Ming-3.0-tts-tiny2.8017.233.205.8632.957.054.4932.885.815.1246.137.68
两个模型共同在**全部六项中英文单标签指标上实现了 SOTA**;tiny 还在全部三项英文多标签指标上领先。中文多标签控制仍落后于 NV-CV3。 ### 方言
模型WSC-Eval-TTS-easyWSC-Eval-TTS-hardWSYue-TTS-eval-BaseWSYue-TTS-eval-Coverage
CER ↓SIM ↑ACC ↑CER ↓SIM ↑ACC ↑CER ↓SIM ↑ACC ↑CER ↓SIM ↑ACC ↑
Qwen3-TTS4.13%——7.35%————————
CosyVoice3-0.5B3.17%0.69676.05%4.07%0.72381.41%8.36%0.61191.70%8.95%0.65895.80%
CosyVoice3-1.5B + DiffRO-EMO7.14%0.703—9.06%0.601—14.38%0.812—13.74%0.826—
CosyVoice2-WSC4.28%0.728—8.87%0.626———————
CosyVoice2-WSC-SFT4.08%0.788—7.22%0.680———————
CosyVoice2-Yue——————10.33%0.821—9.49%0.834—
Sparks-TTS10.83%0.677—12.52%0.545—27.79%0.762—24.25%0.781—
Ming-3.0-tts-0.5b2.27%0.68980.68%3.35%0.69187.64%6.15%0.58698.00%8.15%0.62295.27%
Ming-3.0-tts-tiny2.53%0.72186.09%3.11%0.73888.94%7.40%0.60592.90%8.67%0.65395.16%
Tiny 在四川话控制准确度上更强,实现了**SOTA 四川话 Hard CER 和 Easy ACC**,而 0.5b 实现了**SOTA 粤语 Base CER 和 ACC**。粤语说话人相似度仍低于 CosyVoice2-Yue。历史 Ming-omni-tts 结果在四川话 Easy CER(2.25%)、Hard SIM(0.750)和 Hard ACC(90.45%)以及粤语 Coverage CER(7.87%)上仍然更强。 ### 情感控制(CV3)
模型平均文本相关文本无关
开心悲伤愤怒开心悲伤愤怒
CosyVoice3-0.5B66.33%92%70%72%64%42%58%
CosyVoice3-1.5B63.00%86%64%72%64%44%48%
CosyVoice3-1.5B + DiffRO-EMO77.67%98%68%84%98%50%68%
F5-TTS64.67%92%52%72%80%28%64%
Sparks-TTS55.33%80%56%50%50%60%36%
Ming-3.0-tts-0.5b72.33%96%76%94%52%36%80%
Ming-3.0-tts-tiny74.00%98%82%90%60%36%78%
Tiny 和 0.5b 在平均 ACC 上分别排名**第 2**和**第 3**。Tiny 在文本相关的开心和悲伤上更强,而 0.5b 在愤怒上更强;文本无关情感仍是两者的共同局限。历史 Ming-omni-tts 结果在文本相关悲伤(86%)和文本无关愤怒(82%)上仍然更高。 ### 情感控制(CV3N,困难情感控制)
模型平均文本相关文本无关
开心悲伤愤怒开心悲伤愤怒
CosyVoice3-0.5B40.00%68%30%78%14%4%46%
Ming-3.0-tts-0.5b37.33%70%36%70%14%4%30%
Ming-3.0-tts-tiny30.67%60%40%44%18%10%12%
0.5b 和 tiny 在平均 ACC 上分别排名**第 2**和**第 3**。Tiny 在**两个悲伤子集上均实现了 SOTA**,而 0.5b 的整体准确率更高。历史 Ming-omni-tts 结果在文本相关开心(78%)和文本无关开心(30%)上仍然更高。 ### 音色设计(InstructTTSEval-ZH) | 模型 | 平均 | APS | DSD | RP | | --- | --- | --- | --- | --- | | Qwen3-TTS | **77.13%** | **85.20%** | **81.10%** | **65.10%** | | Mimo-Audio-7B-Instruct | 70.50% | 75.70% | 74.30% | 61.50% | | VoiceSculptor | 67.30% | 75.70% | 64.70% | 61.50% | | VoxInstruct | 47.47% | 47.50% | 52.30% | 42.60% | | **Ming-3.0-tts-0.5b** | 72.29% | 78.54% | 77.28% | 61.06% | | **Ming-3.0-tts-tiny** | 75.37% | 85.10% | 79.90% | 61.10% | Tiny 和 0.5b 在平均 ACC 上分别排名**第 2**和**第 3**,落后于 Qwen3-TTS。角色扮演指令遵循仍是两个模型的主要差距。 ### 基础属性控制 | 模型 | 语速 | 音量 | 音高 | 平均 | WER ↓ | SIM ↑ | | --- | --- | --- | --- | --- | --- | --- | | CosyVoice3-0.5B | **100.00%** | **97.67%** | 65.33% | 87.67% | 1.210% | 0.580 | | **Ming-3.0-tts-0.5b** | 99.00% | 95.00% | **86.00%** | **93.30%** | 0.317% | 0.712 | | **Ming-3.0-tts-tiny** | 93.00% | 93.00% | 78.70% | 88.23% | **0.274%** | **0.742** | 0.5b 的控制精度更高,而 tiny 的 WER 更低、说话人相似度更高。历史 Ming-omni-tts 模型在平均 ACC、音高、WER 和 SIM 上保留了整体记录。 ### 播客 | 模型 | CER ↓ | cpSIM ↑ | UTMOS ↑ | | --- | --- | --- | --- | | FireRedTTS2 | 3.34% | 0.512 | 1.90 | | SoulX-Podcast | 2.20% | **0.599** | 2.09 | | VibeVoice-1.5B | 12.87% | 0.455 | 1.74 | | MOSS-TTSD | 8.62% | 0.421 | 1.70 | | ZipVoice-Dia | 3.39% | 0.553 | **2.24** | | MoonCast | 27.43% | 0.441 | 1.76 | | **Ming-3.0-tts-tiny** | **2.03%** | 0.561 | 2.12 | Tiny 在 CER 上排名**第 1**,在 cpSIM 和 UTMOS 上排名**第 2**。历史 Ming-omni-tts 保持了更低的 CER 和更高的 UTMOS。 ## 局限性 - **说话人相似度。** Tiny 在德语 SIM 上排名第三(0.791),但两个模型在其他四种新语言中均未进入前三。法语 SIM 为 0.5b 的 0.675 和 tiny 的 0.733;两者在粤语 SIM 上也落后于 CosyVoice2-Yue。 - **一行中的多个事件。** NVBench _Multi_ 可用,但落后于 _Single_;每句包含多个标签时的稳定性仍在改进中。 - **仅凭表达传递情感。** 当文字本身不携带情感信号时(CV3-Eval Text-Unrelated,以及困难集中的大部分),准确率远低于 Text-Related 的一半。 - **tiny 的音高。** 音高是两个模型中最不准确的属性。Tiny 的 78.70% 排名第 2,比 0.5b 的 86.00% 低 7.30 个百分点;历史 Ming-omni-tts-0.5B 的得分为 91.33%。 - **角色扮演音色设计。** RP 是音色设计子项中得分最低的,tiny 排名第 4(61.10%),0.5b 排名第 5(61.06%)。 - **tiny 的韩语。** Tiny 的韩语 CER(6.000%)排名第 6,远落后于 0.5b 的 1.728%(第 3)。 - **播客音频质量。** Tiny 的 UTMOS 为 2.12,排名第 2,低于 ZipVoice-Dia 的 2.24 和历史 Ming-omni-tts-0.5B 的 2.25。 ## 使用免责声明 - 语音克隆和内置角色音色仅供研究和合法创意用途使用。请勿使用它们未经同意冒充真实人物、进行欺骗或用于任何非法目的。 - 在可能被误认为真实录音的场合,请将合成语音标注为合成语音。 - 作者对滥用不承担任何责任。如果您发现对本项目的滥用行为,请向我们报告。 ## 致谢 - [CAM++](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common) 用于说话人嵌入提取 - [Ming-omni-tts](https://github.com/inclusionAI/Ming-omni-tts) 用于音频分词器 ## 许可证 依据 [MIT](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/LICENSE) 许可证发布。 ## 正文 · 原文 Multilingual speech synthesis with free-form and paralinguistic control. 🤗 [Hugging Face](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)| 🤖 [ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) | 📑 [Blog & Demos](https://nomorecoding.github.io/Ming-3.0-tts/) | 💬 [DingTalk(钉钉)](https://qr.dingtalk.com/action/joingroup?code=v1,k1,O7MiZqrOrB2c7PnUZVBNvmDh/6tghNLMEtXqMteyRIpuRVJIwrSsXmL8oFqU5ajJ&_dt_no_comment=1&origin=11) Ming-3.0-tts provides two speech-generation models: **0.5b**, a dense model, and **tiny**, a MoE model built on Ling V3 (7.9B total · 1.3B active). ## 🏆 Artificial Analysis Leaderboards On the independent [Artificial Analysis](https://artificialanalysis.ai/text-to-speech) text-to-speech leaderboards (October 2026), Ming-3.0-tts ranks **1st among open-weights models on Controlled Voice**, **1st overall on Pronunciation Robustness**, and **3rd among open-weights models on Provider Voice**. Ranks change as new models are added. | Leaderboard | Setting | Ming-3.0-tts | Rank | | --- | --- | --- | --- | | [Controlled Voice](https://artificialanalysis.ai/text-to-speech/leaderboard/controlled-voice) | Arena preference Elo; every model clones the same 8 voices (4 US, 4 UK) | 1069 | **1st among open-weights** · 13th overall | | [Pronunciation Robustness](https://artificialanalysis.ai/text-to-speech) | Share of highlighted spans in English text that human reviewers judged correctly pronounced | 93.2% | **1st overall** | | [Provider Voice](https://artificialanalysis.ai/text-to-speech/leaderboard/provider-voice) | Arena preference Elo; each model uses its own native voices | 1188 | **3rd among open-weights** · 17th overall | [![Artificial Analysis leaderboards: Controlled Voice and Provider Voice (open weights), Pronunciation Robustness](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/aa_leaderboards.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/aa_leaderboards.png) Within Pronunciation Robustness, Ming-3.0-tts also ranks **1st** on Expanding Shorthand (text normalization, 96.5%) and Preserving Exact Sequences (sequence fidelity, 90.6%). ## Featured Abilities - **Voice creation:** clone a reference voice, design one from text, or choose from 100+ Chinese and five English built-in voices. - **Free-form control:** guide speech delivery with natural-language instructions. - **Paralinguistic control:** insert 14 inline tags for laughter, sighs, and other vocal expressions in Chinese or English. - **Languages and dialects:** synthesize Chinese, English, German, Spanish, French, Korean and Japanese, plus Sichuanese and Cantonese. - **Emotion and attributes:** control emotion, or adjust speed, pitch and volume. - **Multi-speaker dialogue:** generate a conversation from a script and speaker references (**tiny only**). Capabilities use separate task modes, not arbitrary combinations. See the [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook) for supported inputs, [Performance](https://github.com/inclusionAI/Ming-3.0-tts#performance) for benchmarks, and [Blog & Demos](https://nomorecoding.github.io/Ming-3.0-tts/) for audio examples. ## Demo Video tts11\_under10mb.mp4 ## Architecture Text and task-specific control tokens condition a language model with a DiT patch head and an audio tokenizer. Both models share this generation framework. [![](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/ming-3.0-tts.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/ming-3.0-tts.png) ## News - 🔥 **\[2026.10\]** Ming-3.0-tts-0.5b and Ming-3.0-tts-tiny are released, with inference code, a task cookbook and the evaluation tables below. ## Roadmap - Ming-3.0-tts-0.5b weights and inference code - Ming-3.0-tts-tiny weights and inference code - Ming-3.0-tts-tiny benchmarks - vLLM-Omni accelerated inference - Technical report ## Contents - [Demo Video](https://github.com/inclusionAI/Ming-3.0-tts#demo-video) - [Models](https://github.com/inclusionAI/Ming-3.0-tts#models) - [Quick Start](https://github.com/inclusionAI/Ming-3.0-tts#quick-start-) - [Cookbook](https://github.com/inclusionAI/Ming-3.0-tts#cookbook) - [Built-in voices](https://github.com/inclusionAI/Ming-3.0-tts#built-in-voices) - [Performance](https://github.com/inclusionAI/Ming-3.0-tts#performance) - [Limitations](https://github.com/inclusionAI/Ming-3.0-tts#limitations) - [Usage Disclaimer](https://github.com/inclusionAI/Ming-3.0-tts#usage-disclaimer) ## Models | Model | Architecture | Parameters | Download | | --- | --- | --- | --- | | Ming-3.0-tts-0.5b | Dense | 0.5b | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5b)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-0.5b) | | Ming-3.0-tts-tiny | MoE | 7.9B total · 1.3B active | [🤗 HuggingFace](https://huggingface.co/inclusionAI/Ming-3.0-tts-tiny)
[🤖 ModelScope](https://modelscope.cn/models/inclusionAI/Ming-3.0-tts-tiny) | Each release checkpoint bundles the backbone, DiT head, audio tokenizer, speaker encoder and text tokenizer. The 0.5b release layout is shown below; tiny uses sharded weights with `model.safetensors.index.json`. ``` Ming-3.0-tts-0.5b/ ├── config.json # model_type: dense, auto_map -> BailingMMConfig ├── configuration.json # ModelScope task descriptor ├── model.safetensors # backbone + audio VAE + DiT head + aggregator ├── campplus.onnx # speaker embedding extractor ├── tokenizer.json # plus vocab.json / merges.txt / added_tokens.json ├── tokenizer_config.json # special_tokens_map.json / chat_template.jinja └── README.md ``` ## Quick Start 🚀 ### Installation git clone https://github.com/inclusionAI/Ming-3.0-tts.git cd Ming-3.0-tts pip install -r requirements.txt # Ming-3.0-tts-0.5b Or build the 0.5b Docker image (torch 2.6, CUDA 12.4, `transformers==4.46.3`): docker build -f docker/Dockerfile -t ming-tts . The 0.5b environment has been tested on NVIDIA H800-80GB and H20-96G. For 0.5b, `flash-attn` is optional; inference falls back to PyTorch SDPA. Tiny uses a separate reference inference environment: Python **3.10.15**, PyTorch and torchaudio **2.7.1+cu118** (CUDA 11.8 wheels), and `transformers==4.46.3`. Install the pinned dependencies from [requirements-tiny.txt](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/requirements-tiny.txt) in a dedicated environment, as other dependency versions differ from 0.5b. We recommend using NVIDIA H800 for inference. conda create -n ming-tiny python=3.10.15 -y && conda activate ming-tiny pip install -r requirements-tiny.txt ### Model Download pip install "huggingface\_hub\[cli\]" hf download inclusionAI/Ming-3.0-tts-0.5b --local-dir Ming-3.0-tts-0.5b In mainland China, ModelScope is usually faster: pip install modelscope modelscope download --model inclusionAI/Ming-3.0-tts-0.5b --local\_dir Ming-3.0-tts-0.5b Replace `0.5b` with `tiny` for the other variant. ### Text Normalization Numbers, dates, units and symbols are normalized with FST grammars, which need `pynini`. It has no universal wheel: conda install -c conda-forge pynini==2.1.6 # or: pip install pynini==2.1.6 It is loaded lazily; pass `--no_text_normalization` to skip it. ### Ming-3.0-tts-0.5b python infer.py \\ --model\_path inclusionAI/Ming-3.0-tts-0.5b \\ --task TTS \\ --language\_id Chinese \\ --text "傍晚的风从山口吹过来,远处的云慢慢染成了金色。" \\ --prompt\_wav\_path data/voice\_presets/voice\_audio/lingguang.wav \\ --prompt\_text "诶,你今天有空吗?我们能不能聊聊这件事啊?" \\ --use\_spk\_emb \\ --output output/tts.wav Or from Python: from ming\_tts import MingTTS model \= MingTTS("inclusionAI/Ming-3.0-tts-0.5b") model.synthesize( task\="TTS", language\_id\="Chinese", \# English for English text text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。", prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav", prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?", use\_spk\_emb\=True, output\_path\="output/tts.wav", ) ### Ming-3.0-tts-tiny Every task has a one-line script in [examples/one\_click/](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/examples/one_click); variables such as `MODEL`, `TEXT` and `OUT` can be overridden from the environment: bash examples/one\_click/run\_seed.sh MODEL=/path/to/Ming-3.0-tts-tiny OUT=outputs/yue.wav bash examples/one\_click/run\_dialect.sh The same call from Python. Loading configures the release runtime (bf16, portable MoE loop, fp32 recurrent state) and checks the checkpoint against it; `generate()` defaults to `cfg_scale=2.0`, `temperature=0`, `flow_steps=10`, `sigma=0.25`, and seed 1234 (1235 for `podcast`): from ming\_tiny import MingTinyTTS tts \= MingTinyTTS.from\_pretrained("inclusionAI/Ming-3.0-tts-tiny", device\="cuda:0") tts.generate( task\="seed", text\="傍晚的风从山口吹过来,远处的云慢慢染成了金色。", prompt\_wav\="data/voice\_presets/voice\_audio/lingguang.wav", prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?", target\_language\="Chinese", \# English for English text output\_path\="outputs/seed.wav", ) tts.generate( task\="podcast", text\="speaker\_1: 你最近是不是又去那家面馆了?\\nspeaker\_2: 对啊,排了半个小时的队,不过真的值得。", prompt\_wav\=\["data/voice\_presets/voice\_audio/lingguang.wav", "data/voice\_presets/voice\_audio/db30.wav"\], prompt\_text\=\["诶,你今天有空吗?我们能不能聊聊这件事啊?", "感谢你的认可。"\], output\_path\="outputs/podcast.wav", ) Each call selects one task and accepts the inputs listed below: | Task | Reference audio | Reference text | `target_language` | Controls | | --- | --- | --- | --- | --- | | `seed` | one | required | `Chinese` / `English`, required | — | | `multilingual` | one | required | `German` / `Spanish` / `French` / `Korean` / `Japanese` | — | | `paralinguistic` | one | required | `Chinese` / `English` | inline tags in `text` | | `podcast` | one per speaker | one per speaker, same order | inferred from the text | `speaker_N:` at the start of every line | | `basic` | one | not accepted | — | `speed`, `pitch`, `volume`, all three | | `emotion` | one | not accepted | — | `emotion` | | `dialect` | one | not accepted | — | `dialect`: `川渝话` / `广粤话` | | `freeform` | none by default (recommended); optional | not accepted | — | `instruction` | | `voice_design` | none | — | — | `style` | | `ip` | none | — | — | `ip` | `python infer_tiny.py --help` lists the same arguments for the command line; repeat `--prompt_wav` and `--prompt_text` for podcast. | | Ming-3.0-tts-0.5b | Ming-3.0-tts-tiny | | --- | --- | --- | | Python API | `ming_tts.MingTTS` | `ming_tiny.MingTinyTTS` | | CLI | `infer.py` | `infer_tiny.py`, `examples/one_click/` | | Preflight | `tools/preflight.py` | `tools/preflight_tiny.py` | | Tasks | TTS (inline paralinguistic tags included), FREEFORM, VOICE\_DESIGN, MULTILINGUAL, BASIC, EMOTION, DIALECT, IP | the same abilities as lower-case tasks (`seed`, `paralinguistic`, ...), plus `podcast` | ### Verify Your Setup Check the checkpoint and tokenizer without loading weights: python tools/preflight.py --model\_path /path/to/Ming-3.0-tts-0.5b python tools/preflight\_tiny.py --model\_path /path/to/Ming-3.0-tts-tiny ## Cookbook Ordered the way a line gets made: who speaks, how it is said, what goes beyond the words, and where it is heard. Each example is one task; controls from another task are rejected. Texts and controls are the ones on the [demo page](https://nomorecoding.github.io/Ming-3.0-tts/), so you can compare against what you hear there. 0.5b API; full value lists in [English](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md) / [简体中文](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instructions.md), every task end to end in [cookbooks/cookbook.ipynb](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/cookbook.ipynb). ### Who speaks **Zero-shot Cloning**: the `TTS` call in [Quick Start](https://github.com/inclusionAI/Ming-3.0-tts#ming-30-tts-05b) clones the voice of `prompt_wav_path`. **Voice Design**: describe the voice; no reference audio. model.synthesize( task\="VOICE\_DESIGN", text\="冰川在漫长岁月中缓缓移动,也悄悄记录着气候变化的痕迹。", controls\={"style": "成熟沉稳的男性纪录片旁白,吐字清晰,节奏舒缓,语气理性而有厚度。"}, use\_zero\_spk\_emb\=True, ) **Built-in Voices**: recommended **US\_James**, **US\_Stella**, **灵小柔** and **灵楷**. See the [full voice list](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices). model.synthesize( task\="IP", text\="The garden will remain open until sunset, provided the weather stays clear.", controls\={"ip": "US\_James"}, use\_zero\_spk\_emb\=True, ) ### How it is said **Freeform**: describe the voice and delivery with one natural-language instruction. Both models default to generation without reference audio; adding a reference is not recommended. Not combined with voice design, a built-in voice, emotion, dialect or attribute controls. model.synthesize( task\="FREEFORM", text\="The old radio crackled softly as the storm moved across the valley.", instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.", use\_zero\_spk\_emb\=True, ) For Tiny, omit `prompt_wav` and `prompt_text`: tts.generate( task\="freeform", text\="The old radio crackled softly as the storm moved across the valley.", instruction\="Speak in a warm, intimate storytelling voice, with a gentle pace and a hint of nostalgia.", ) **Emotion**: `高兴 / 悲伤 / 愤怒 / 恐惧 / 厌恶 / 惊讶 / 中性`. model.synthesize( task\="EMOTION", text\="所有测试都顺利通过了,我们终于可以把这个好消息告诉大家!", controls\={"emotion": "高兴"}, prompt\_wav\_path\="data/wavs/emotion\_prompt.wav", use\_spk\_emb\=True, ) **Speed, Pitch and Volume**: speed `慢速 / 中速 / 快速`, pitch and volume `低 / 中 / 高`. model.synthesize( task\="BASIC", text\="今晚的观测将在云层散开后开始,请耐心等待下一次通知。", controls\={"speed": "慢速", "pitch": "中", "volume": "中"}, spk\_emb\_path\="/path/to/speaker\_embedding.pt", use\_spk\_emb\=True, ) ### Beyond the words **Paralinguistic Tags**: inline, anywhere in the sentence, in Chinese or English. `[Breathing]`, `[Cough]`, `[Sigh]`, `[Laughter]`, `[Surprise-ah]`, `[Surprise-oh]`, `[Dissatisfaction-hnn]`, `[Uhm]`, `[Confirmation-en]`, `[Question-ei]`, `[Question-ah]`, `[Question-en]`, `[Question-oh]`, `[Question-huh]`. model.synthesize( task\="TTS", text\="\[Sigh\]雨还是没有停,\[Cough\]我们恐怕只能把活动改到室内了。", prompt\_wav\_path\="data/voice\_presets/voice\_audio/lingguang.wav", prompt\_text\="诶,你今天有空吗?我们能不能聊聊这件事啊?", use\_spk\_emb\=True, ) ### Where it is heard **Multilingual**: `language_id` is required for the five new languages; short codes and full names both work. model.synthesize( task\="MULTILINGUAL", text\="週末、駅前の小さな喫茶店で、季節限定の抹茶パフェを食べました", language\_id\="Japanese", \# ja / ko / de / fr / es prompt\_wav\_path\="/path/to/japanese\_prompt.wav", prompt\_text\="参考音频对应的日语文本。", use\_spk\_emb\=True, ) **Dialects**: `chuanyu dialect` or `cantonese`; this task takes no `language_id`. model.synthesize( task\="DIALECT", text\="今日天气几好,我哋食完饭一齐去海边行下啦。", controls\={"dialect": "cantonese"}, prompt\_wav\_path\="data/wavs/yue\_prompt.wav", use\_spk\_emb\=True, ) ## Built-in voices ### Recommended voices | Voice | Language | Character | | --- | --- | --- | | US\_James | English (US) | Male. Narration, explainers and documentary. | | US\_Stella | English | — | | 灵小柔 | Chinese | Bright, sweet young female voice with clear diction. Assistants and customer service. | | 灵楷 | Chinese | Warm, resonant young male voice. Radio, documentary and audiobooks. | Over 100 Chinese voices and five English voices are available. See [the voice catalog](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/cookbooks/instruction_en.md#built-in-premium-voices) for the full list, including emotion-specific variants. ## Performance Open benchmarks are below; independent arena results are in [Artificial Analysis Leaderboards](https://github.com/inclusionAI/Ming-3.0-tts#artificial-analysis-leaderboards). [![](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/benchmark_basic_1018.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_basic_1018.png) [![](https://github.com/inclusionAI/Ming-3.0-tts/raw/main/figures/benchmark_control_1018.png)](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/figures/benchmark_control_1018.png) Below, **tiny** is Ming-3.0-tts-tiny and **0.5b** is Ming-3.0-tts-0.5b. Ranks and marks are computed per metric among the listed open-source models; Seed-TTS and NVbench-GT are reference rows and are not ranked. Where a historical Ming-omni-tts result beats a bolded Ming-3.0-tts value, it is noted below the table and no SOTA is claimed. **Bold** marks the best value and underline the second-best value in each column; ties share a mark. Other systems' results use their reported inference settings. ### Summary 1. **Paralinguistic:** together, the two models reach **SOTA on all six NVBench single-tag metrics**; tiny reaches **SOTA on all three English multi-tag metrics**. 2. **Dialect:** 0.5b reaches **SOTA** on Cantonese Base CER and ACC; tiny reaches **SOTA** on Sichuanese Hard CER and Easy ACC. 3. **Multilingual:** tiny reaches **SOTA** on German and Spanish WER, ranks **2nd** on French WER, and ties for **2nd** on Japanese CER. 4. **Emotion:** tiny ties for **SOTA** on CV3 text-related Happy and reaches **SOTA** on both CV3N Sad subsets; 0.5b reaches **SOTA** on CV3 text-related Angry. Tiny's CV3 average ranks **2nd**. 5. **Attribute control:** 0.5b leads the current comparison on average ACC and pitch, while tiny leads on WER and SIM. Historical Ming-omni-tts results still hold the corresponding overall records. 6. **Podcast:** tiny achieves CER 2.03%, cpSIM 0.561 and UTMOS 2.12, ranking **1st**, **2nd** and **2nd**; historical Ming-omni-tts retains lower CER. 7. **Voice Design:** tiny ranks **2nd** on average ACC (75.37%), APS (85.10%) and DSD (79.90%). 8. **Chinese and English:** 0.5b's Chinese WER ranks **3rd**; tiny reaches **SOTA** on Chinese and English SIM. 9. **Next:** improve multilingual and Cantonese speaker similarity, tiny's Korean accuracy and pitch control, text-unrelated emotion, role-play instruction following, and predicted podcast audio quality. ### Chinese and English (seed-tts-eval)
Modelseed-tts-eval-zhseed-tts-eval-en
WER ↓SIM ↑WER ↓SIM ↑
Seed-TTS1.110%0.7962.240%0.762
GLM-TTS1.030%0.7612.230%0.672
Qwen3-TTS1.070%—1.390%—
Qwen3-TTS-12Hz-0.6B-Base0.920%—1.320%—
Qwen3-TTS-12Hz-1.7B-Base0.770%—1.240%—
CosyVoice3-0.5B1.160%0.7802.020%0.718
CosyVoice3-1.5B0.710%0.7751.450%0.695
F5-TTS1.560%0.7411.830%0.647
Ming-3.0-tts-0.5b0.800%0.7581.584%0.691
Ming-3.0-tts-tiny0.820%0.7811.666%0.728
Tiny achieves **SOTA speaker similarity** in Chinese and English. 0.5b has lower WER in both languages and ranks **3rd** on Chinese WER. ### Multilingual (MiniMaxAI TTS-Multilingual-Test-Set)
ModelGermanSpanishFrenchKoreanJapanese
WER ↓SIM ↑WER ↓SIM ↑WER ↓SIM ↑CER ↓SIM ↑CER ↓SIM ↑
dots.tts0.452%0.7982.593%0.83919.878%0.7894.837%0.8535.134%0.838
VoxCPM20.963%0.7901.324%0.8363.380%0.7613.547%0.8394.358%0.825
Fish Audio S21.045%0.7141.216%0.7483.636%0.6981.675%0.7613.427%0.755
FireRedTTS30.348%0.8251.086%0.8684.366%0.8071.644%0.8643.418%0.825
Ming-3.0-tts-0.5b0.365%0.7531.099%0.7634.264%0.6751.728%0.7933.765%0.765
Ming-3.0-tts-tiny0.325%0.7910.971%0.7973.470%0.7336.000%0.7973.427%0.780
Each open-source model was evaluated using its recommended inference settings under a unified ASR-based WER/CER evaluation protocol. Tiny achieves **SOTA WER in German and Spanish**, ranks **2nd** in French WER and ties for **2nd** in Japanese CER. 0.5b is more accurate in Korean; speaker similarity remains a gap for both models. ### Paralinguistic control (NVBench) CER, PCER (paralinguistic CER) and OCER are reported. NVbench-GT is human-labelled reference, shown for context and not ranked.
ModelSingleMulti
ChineseEnglishChineseEnglish
CERPCEROCERCERPCEROCERCERPCEROCERCERPCEROCER
NVbench-GT3.869.384.076.738.316.903.7923.715.077.2621.418.62
Orpheus-TTS11.3688.7713.919.0371.9210.6319.8384.8524.388.6871.4611.89
SMIIP-NV-CV28.8075.6411.3417.9256.8019.4710.6677.2014.7920.9354.4923.87
Emilia-NV-CV25.0540.006.6412.5055.3013.215.5448.748.0911.7160.2814.63
CosyVoice33.8557.695.867.8762.759.064.7561.948.266.3957.8410.69
NV-FlexiVoice6.9831.088.1511.8850.4313.218.2039.3710.399.6051.3213.76
NV-CV33.8027.694.908.3346.139.443.4430.044.846.7047.1310.10
Ming-3.0-tts-0.5b2.7022.803.207.6037.008.903.5036.608.107.9047.3010.40
Ming-3.0-tts-tiny2.8017.233.205.8632.957.054.4932.885.815.1246.137.68
Together, the two models achieve **SOTA on all six Chinese and English single-tag metrics**; tiny also leads all three English multi-tag metrics. Chinese multi-tag control remains behind NV-CV3. ### Dialect
ModelWSC-Eval-TTS-easyWSC-Eval-TTS-hardWSYue-TTS-eval-BaseWSYue-TTS-eval-Coverage
CER ↓SIM ↑ACC ↑CER ↓SIM ↑ACC ↑CER ↓SIM ↑ACC ↑CER ↓SIM ↑ACC ↑
Qwen3-TTS4.13%——7.35%————————
CosyVoice3-0.5B3.17%0.69676.05%4.07%0.72381.41%8.36%0.61191.70%8.95%0.65895.80%
CosyVoice3-1.5B + DiffRO-EMO7.14%0.703—9.06%0.601—14.38%0.812—13.74%0.826—
CosyVoice2-WSC4.28%0.728—8.87%0.626———————
CosyVoice2-WSC-SFT4.08%0.788—7.22%0.680———————
CosyVoice2-Yue——————10.33%0.821—9.49%0.834—
Sparks-TTS10.83%0.677—12.52%0.545—27.79%0.762—24.25%0.781—
Ming-3.0-tts-0.5b2.27%0.68980.68%3.35%0.69187.64%6.15%0.58698.00%8.15%0.62295.27%
Ming-3.0-tts-tiny2.53%0.72186.09%3.11%0.73888.94%7.40%0.60592.90%8.67%0.65395.16%
Tiny is stronger in Sichuanese control accuracy and achieves **SOTA Sichuanese Hard CER and Easy ACC**, while 0.5b achieves **SOTA Cantonese Base CER and ACC**. Cantonese speaker similarity remains below CosyVoice2-Yue. Historical Ming-omni-tts results remain stronger on Sichuanese Easy CER (2.25%), Hard SIM (0.750) and Hard ACC (90.45%), and on Cantonese Coverage CER (7.87%). ### Emotion control (CV3)
ModelAverageText-RelatedText-Unrelated
HappySadAngryHappySadAngry
CosyVoice3-0.5B66.33%92%70%72%64%42%58%
CosyVoice3-1.5B63.00%86%64%72%64%44%48%
CosyVoice3-1.5B + DiffRO-EMO77.67%98%68%84%98%50%68%
F5-TTS64.67%92%52%72%80%28%64%
Sparks-TTS55.33%80%56%50%50%60%36%
Ming-3.0-tts-0.5b72.33%96%76%94%52%36%80%
Ming-3.0-tts-tiny74.00%98%82%90%60%36%78%
Tiny and 0.5b rank **2nd** and **3rd** in average ACC. Tiny is stronger on text-related Happy and Sad, while 0.5b is stronger on Angry; text-unrelated emotion remains a shared limitation. Historical Ming-omni-tts results remain higher on text-related Sad (86%) and text-unrelated Angry (82%). ### Emotion control (CV3N, hard emotion control)
ModelAverageText-RelatedText-Unrelated
HappySadAngryHappySadAngry
CosyVoice3-0.5B40.00%68%30%78%14%4%46%
Ming-3.0-tts-0.5b37.33%70%36%70%14%4%30%
Ming-3.0-tts-tiny30.67%60%40%44%18%10%12%
0.5b and tiny rank **2nd** and **3rd** in average ACC. Tiny achieves **SOTA on both Sad subsets**, while 0.5b has higher overall accuracy. Historical Ming-omni-tts results remain higher on text-related Happy (78%) and text-unrelated Happy (30%). ### Voice Design (InstructTTSEval-ZH) | Model | Average | APS | DSD | RP | | --- | --- | --- | --- | --- | | Qwen3-TTS | **77.13%** | **85.20%** | **81.10%** | **65.10%** | | Mimo-Audio-7B-Instruct | 70.50% | 75.70% | 74.30% | 61.50% | | VoiceSculptor | 67.30% | 75.70% | 64.70% | 61.50% | | VoxInstruct | 47.47% | 47.50% | 52.30% | 42.60% | | **Ming-3.0-tts-0.5b** | 72.29% | 78.54% | 77.28% | 61.06% | | **Ming-3.0-tts-tiny** | 75.37% | 85.10% | 79.90% | 61.10% | Tiny and 0.5b rank **2nd** and **3rd** in average ACC, behind Qwen3-TTS. Role-play instruction following remains the main gap for both models. ### Basic attribute control | Model | Speed | Volume | Pitch | Average | WER ↓ | SIM ↑ | | --- | --- | --- | --- | --- | --- | --- | | CosyVoice3-0.5B | **100.00%** | **97.67%** | 65.33% | 87.67% | 1.210% | 0.580 | | **Ming-3.0-tts-0.5b** | 99.00% | 95.00% | **86.00%** | **93.30%** | 0.317% | 0.712 | | **Ming-3.0-tts-tiny** | 93.00% | 93.00% | 78.70% | 88.23% | **0.274%** | **0.742** | 0.5b has higher control accuracy, while tiny has lower WER and higher speaker similarity. Historical Ming-omni-tts models retain the overall records for average ACC, pitch, WER and SIM. ### Podcast | Model | CER ↓ | cpSIM ↑ | UTMOS ↑ | | --- | --- | --- | --- | | FireRedTTS2 | 3.34% | 0.512 | 1.90 | | SoulX-Podcast | 2.20% | **0.599** | 2.09 | | VibeVoice-1.5B | 12.87% | 0.455 | 1.74 | | MOSS-TTSD | 8.62% | 0.421 | 1.70 | | ZipVoice-Dia | 3.39% | 0.553 | **2.24** | | MoonCast | 27.43% | 0.441 | 1.76 | | **Ming-3.0-tts-tiny** | **2.03%** | 0.561 | 2.12 | Tiny ranks **1st** in CER and **2nd** in cpSIM and UTMOS. Historical Ming-omni-tts retains lower CER and higher UTMOS. ## Limitations - **Speaker similarity.** Tiny ranks third on German SIM (0.791), but neither model reaches the top three in the other four new languages. French SIM is 0.675 for 0.5b and 0.733 for tiny; both also trail CosyVoice2-Yue on Cantonese SIM. - **Several events in one line.** NVBench _Multi_ works, but trails _Single_; stability with many tags per sentence is still improving. - **Emotion from delivery alone.** When the words carry no emotional signal (CV3-Eval Text-Unrelated, and most of the hard set), accuracy drops well below the Text-Related half. - **Pitch on tiny.** Pitch is the least accurate attribute for both models. Tiny's 78.70% ranks 2nd, 7.30 percentage points below 0.5b's 86.00%; the historical Ming-omni-tts-0.5B score is 91.33%. - **Role-play voice design.** RP is the lowest voice design sub-score, ranking 4th for tiny (61.10%) and 5th for 0.5b (61.06%). - **Korean on tiny.** Tiny's Korean CER (6.000%) ranks 6th, well behind 0.5b's 1.728% (3rd). - **Podcast audio quality.** Tiny's UTMOS of 2.12 ranks 2nd, below ZipVoice-Dia's 2.24 and the historical Ming-omni-tts-0.5B score of 2.25. ## Usage Disclaimer - Voice cloning and built-in character voices are provided for research and legitimate creative use. Do not use them to impersonate a real person without consent, to deceive, or for any illegal purpose. - Label synthetic speech as synthetic where it could be mistaken for a real recording. - The authors accept no liability for misuse. If you find abuse of this project, please report it to us. ## Acknowledgements - [CAM++](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common) for speaker embedding extraction - [Ming-omni-tts](https://github.com/inclusionAI/Ming-omni-tts) for the audio tokenizer ## License Released under the [MIT](https://github.com/inclusionAI/Ming-3.0-tts/blob/main/LICENSE) license.