# Artificial Analysis 发布 NVIDIA Vera Rubin NVL72 首批 AA-AgentPerf 智能体推理评测
> 原标题:First AA-AgentPerf results for NVIDIA Vera Rubin NVL72
- 来源:Artificial Analysis 完整文章
- 发布时间:2026-10-14T16:00:00.000Z
- AIHOT:https://aihot.news/items/cfa00lfy4h5vywbpnf1ixt6m0
- 原文:https://artificialanalysis.ai/articles/vera-rubin-nvl72-agentperf
## 摘要
Artificial Analysis 公布首批 AA-AgentPerf 结果,测试 NVIDIA Vera Rubin NVL72 在智能体编码工作负载下的服务效率,对比 GB300 NVL72。
## 推荐理由
Artificial Analysis 实测四种模型在 Vera Rubin NVL72 与 GB300 上的每瓦吞吐,读者可据此评估新硬件对智能体推理服务成本的影响。
## 正文 · 中文译文
## Agentic 编码工作负载难以高效服务
## Vera Rubin NVL72 每兆瓦性能领先 GB300 NVL72 最高约 5.7 倍
Vera Rubin NVL72 延续了 GB300 NVL72 的机架级设计,在单个 NVLink 域中集成 72 个 GPU 和 36 个 CPU。每个 Rubin GPU 拥有与 Blackwell Ultra 相同的 288 GB 内存,但内存带宽提升 2.4 倍(得益于 HBM4),NVLink 带宽提升 1.7 倍,密集 NVFP4 算力提升 2.3 倍,而 Vera CPU 将其与 GPU 之间的 NVLink-C2C 带宽翻倍,这条链路承载了所有卸载到 CPU 内存的 KV 缓存。
| 每 GPU | Rubin(Vera Rubin NVL72) | Blackwell Ultra(GB300 NVL72) |
| --- | --- | --- |
| GPU 内存 | 288 GB HBM4 | 288 GB HBM3E |
| 内存带宽 | 19.2 TB/s | 8 TB/s |
| NVLink 带宽 | 3.0 TB/s(NVLink 6) | 1.8 TB/s(NVLink 5) |
| 密集 NVFP4 算力 | 35 PFLOPS | 15 PFLOPS |
| 稀疏 NVFP4 算力 | 50 PFLOPS | 20 PFLOPS |
| CPU | Vera,88 核 | Grace,72 核 |
| 每超级芯片的 CPU 到 GPU 链路 | 1.8 TB/s NVLink-C2C | 0.9 TB/s NVLink-C2C |
| 这些结果中的实测功耗 | 0.5 至 1.9 kW | 0.3 至 1.0 kW |
规格来自 NVIDIA 截至 2026 年 10 月的 [Vera Rubin NVL72](https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/)、[Blackwell Ultra](https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/) 和 [GB300 NVL72](https://www.nvidia.com/en-us/data-center/gb300-nvl72/) 页面。实测功耗为每个 GPU 在负载下的自身功耗(GPU 及其 HBM,依据 NVIDIA 的 DCGM/SMI 遥测报告,不含 CPU、网络和散热),覆盖各系统在四个模型前沿上的配置。
### 四个模型的结果
下面每张图都将每兆瓦实测 GPU 功耗的总吞吐量与每用户输出速度进行对比。总吞吐量计入输入 token(缓存的和新预填充的)和输出 token,其中约 96% 为缓存输入,每用户速度是各请求的第 25 百分位数(p25),不含首 token 时间,这也是我们全文引用的速度。每个点代表设定智能体数量下的一个服务配置,箭头标出 Vera Rubin 相对 GB300 的最大领先幅度。
- 在峰值吞吐量下,Vera Rubin NVL72 在 DeepSeek V4 Pro 上每兆瓦服务 272M tokens/s,而 GB300 NVL72 为 239M(1.14 倍);在 DeepSeek V4.1 Flash 上为 571M 对 395M(1.45 倍);在 GLM-5.2 上为 272M 对 201M(1.35 倍);在 Kimi K3 上为 70M 对 56M(1.26 倍)
- Vera Rubin 的领先在 GB300 吞吐量下滑之处最为明显:在 DeepSeek V4 Pro 上,每用户约 190 tok/s 时达到约 5.7 倍;在 DeepSeek V4.1 Flash 上,约 580 tok/s 时达到约 5.4 倍;在 GLM-5.2 上,约 165 tok/s 时达到约 3.0 倍;在 Kimi K3 上,约 67 tok/s 时达到约 2.8 倍。在 DeepSeek V4 Pro 上,每用户速度从约 120 提升到约 165 tok/s,会让 GB300 损失其每兆瓦吞吐量的 81%,而 Vera Rubin 仅损失 16%
- 按每 GPU 计算,Vera Rubin 的领先幅度更大:峰值吞吐量下为 2.0 倍至 2.5 倍,在 DeepSeek V4 Pro 上每用户约 165 tok/s 时最高约 12 倍,在 DeepSeek V4.1 Flash 上约 580 tok/s 时约 18 倍,在 GLM-5.2 上约 200 至约 310 tok/s 之间约 7 倍,在 Kimi K3 上约 176 tok/s 时约 7 倍
- Vera Rubin 最快的配置在我们的排行榜上创下了 DeepSeek V4.1 Flash(906 tok/s,所有模型中最快的结果)、GLM-5.2(408 tok/s)和 Kimi K3(255 tok/s)的最高每用户速度
更高的能效降低了服务每个 token 的运营成本,而且随着 AI 数据中心越来越受限于可获取的电力而非可安装的加速器数量,它还能让同一数据中心服务更多 agent。例如,在每用户约 165 tok/s 的速度下,相同的 GPU 功耗预算在 Vera Rubin NVL72 上可服务的 DeepSeek V4 Pro token 数量约为 GB300 NVL72 的 5 倍。我们排行榜上的每一个数据点都附带了其背后的服务配置,可在我们的[配置浏览器](https://artificialanalysis.ai/hardware-inference-stack/configs)中查看,因此团队可以从针对自身硬件调优好的配置入手,而不必自行开展性能研究。
## DeepSeek V4 Pro 的结果在高速下展现出最大领先优势
## DeepSeek V4.1 Flash 的结果在我们排行榜上创下最高吞吐量和速度
## GLM-5.2 的结果在更高速下展现出约 3 倍的领先优势
## Kimi K3 的结果在中等速度下展现出约 2.8 倍的领先优势
## NVIDIA 针对 agentic 工作负载调优了其 Vera Rubin 技术栈
## 各模型 Vera Rubin 配置概览
| | DeepSeek V4 Pro | DeepSeek V4.1 Flash | GLM-5.2 | Kimi K3 |
| --- | --- | --- | --- | --- |
| 服务栈 | TensorRT-LLM 搭配 NVIDIA Dynamo | vLLM 搭配 NVIDIA Dynamo | TensorRT-LLM 搭配 NVIDIA Dynamo | TensorRT-LLM,以及用于 1 个 agent 的 vLLM,均搭配 NVIDIA Dynamo |
| 权重 | NVIDIA NVFP4 checkpoint | DeepSeek 发布 checkpoint(MXFP4) | NVIDIA NVFP4 checkpoint | NVIDIA NVFP4 checkpoint |
| Prefill 与 decode | 分离式。DEP4 prefill worker,宽专家并行 decode(DEP16 或 DEP32),34 个 agent 时 TP4/EP4 decode | 聚合式 TP4 或 TP2,用于 1 和 2 个 agent。从 64 个 agent 起采用分离式,使用一到两个 TP2 prefill worker 和 DEP4 decode(或五个 TP2 decode worker) | 分离式。1 到 3 个 prefill worker(TP2/EP2 或 DEP4),decode 采用 TP8、TP4/EP4 或 DEP8 | 1 个 agent 时在 vLLM 上采用聚合式 TP8。从 16 个 agent 起采用分离式:16 和 30 个 agent 时使用一个 TP8 prefill worker 搭配三个 TP8 decode worker,从 512 个 agent 起使用 DWDP16 prefill(一到两组)搭配 DEP16 decode |
| 投机解码 | DSpark,3 到 7 个 draft token | 内置 DSpark drafting,6 个 draft token | MTP,4 到 8 个 draft token | DSpark,3 到 7 个 draft token |
| KV cache | FP8,在 prefill worker 上卸载到主机内存 | 保留在 GPU 内存中,模型的 Engram 内存卸载到主机内存 | FP8,除一个配置外,均在 prefill worker 上卸载到主机内存 | FP8,在 TensorRT-LLM 配置中卸载到主机内存 |
| 路由 | KV 感知,带会话亲和性 | 在 1,024 个 agent 时跨两个 prefill worker 进行 KV 感知,否则使用单个 prefill worker | KV 感知,带会话亲和性 | KV 感知,带会话亲和性 |
| 每个配置的 GPU 数量 | 32 到 56 | 2 到 12 | 10 到 20 | 8 到 48 |
## 延伸阅读
## 正文 · 原文
## Agentic coding workloads are hard to serve efficiently
## Vera Rubin NVL72 leads GB300 NVL72 by up to ~5.7x per MW
Vera Rubin NVL72 keeps the rack-scale design of GB300 NVL72, with 72 GPUs and 36 CPUs in a single NVLink domain. Each Rubin GPU has the same 288 GB of memory as Blackwell Ultra, but 2.4x the memory bandwidth (from HBM4), 1.7x the NVLink bandwidth and 2.3x the dense NVFP4 compute, and the Vera CPU doubles the NVLink-C2C bandwidth to its GPUs, which is the link that carries any KV cache offloaded to CPU memory.
| Per GPU | Rubin (Vera Rubin NVL72) | Blackwell Ultra (GB300 NVL72) |
| --- | --- | --- |
| GPU memory | 288 GB HBM4 | 288 GB HBM3E |
| Memory bandwidth | 19.2 TB/s | 8 TB/s |
| NVLink bandwidth | 3.0 TB/s (NVLink 6) | 1.8 TB/s (NVLink 5) |
| Dense NVFP4 compute | 35 PFLOPS | 15 PFLOPS |
| Sparse NVFP4 compute | 50 PFLOPS | 20 PFLOPS |
| CPU | Vera, 88 cores | Grace, 72 cores |
| CPU to GPU link per superchip | 1.8 TB/s NVLink-C2C | 0.9 TB/s NVLink-C2C |
| Measured power in these results | 0.5 to 1.9 kW | 0.3 to 1.0 kW |
Specifications from NVIDIA’s [Vera Rubin NVL72](https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/), [Blackwell Ultra](https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/) and [GB300 NVL72](https://www.nvidia.com/en-us/data-center/gb300-nvl72/) pages as of October 2026. Measured power is each GPU’s own power draw under load (the GPU and its HBM, as reported by NVIDIA’s DCGM/SMI telemetry, excluding CPUs, networking and cooling), across the configs on each system’s frontier for the four models.
### Results across the four models
Each chart below plots total throughput per MW of measured GPU power against per-user output speed. Total throughput counts input tokens (cached and newly prefilled) and output tokens, with ~96% of them cached input, and per-user speed is the 25th percentile (p25) across requests excluding time to first token, which is the speed we quote throughout. Each point is a serving config at a set number of agents, and the arrow marks Vera Rubin’s largest lead over GB300.
- At peak throughput, Vera Rubin NVL72 serves 272M tokens/s per MW on DeepSeek V4 Pro vs 239M for GB300 NVL72 (1.14x), 571M vs 395M on DeepSeek V4.1 Flash (1.45x), 272M vs 201M on GLM-5.2 (1.35x) and 70M vs 56M on Kimi K3 (1.26x)
- Vera Rubin’s lead is largest where GB300’s throughput drops away, reaching ~5.7x on DeepSeek V4 Pro at ~190 tok/s per user, ~5.4x on DeepSeek V4.1 Flash at ~580 tok/s, ~3.0x on GLM-5.2 at ~165 tok/s and ~2.8x on Kimi K3 at ~67 tok/s. Going from ~120 to ~165 tok/s per user on DeepSeek V4 Pro costs GB300 81% of its throughput per MW, compared to 16% for Vera Rubin
- Per GPU, Vera Rubin’s lead is wider again: 2.0x to 2.5x at peak throughput, and up to ~12x on DeepSeek V4 Pro at ~165 tok/s per user, ~18x on DeepSeek V4.1 Flash at ~580 tok/s, ~7x on GLM-5.2 between ~200 and ~310 tok/s and ~7x on Kimi K3 at ~176 tok/s
- Vera Rubin’s fastest configs set the highest per-user speeds on our leaderboard for DeepSeek V4.1 Flash (906 tok/s, the fastest result for any model), GLM-5.2 (408 tok/s) and Kimi K3 (255 tok/s)
Greater power efficiency lowers the operating cost of serving each token, and as AI datacenters are increasingly limited by the power they can draw rather than the number of accelerators they can install, it also allows more agents to be served out of the same datacenter. At ~165 tok/s per user, for example, the same GPU power budget serves ~5x as many DeepSeek V4 Pro tokens on Vera Rubin NVL72 as on GB300 NVL72. Every point on our leaderboard comes with the serving config behind it in our [configurations browser](https://artificialanalysis.ai/hardware-inference-stack/configs), so teams can start from a tuned setup for their own hardware rather than running their own performance study.
## DeepSeek V4 Pro results show the widest lead at high speeds
## DeepSeek V4.1 Flash results set the highest throughput and speeds on our leaderboard
## GLM-5.2 results show a ~3x lead at faster speeds
## Kimi K3 results show a ~2.8x lead at moderate speeds
## NVIDIA tuned its Vera Rubin stack for agentic workloads
## Overview of the Vera Rubin configs for each model
| | DeepSeek V4 Pro | DeepSeek V4.1 Flash | GLM-5.2 | Kimi K3 |
| --- | --- | --- | --- | --- |
| Serving stack | TensorRT-LLM with NVIDIA Dynamo | vLLM with NVIDIA Dynamo | TensorRT-LLM with NVIDIA Dynamo | TensorRT-LLM, and vLLM for 1 agent, both with NVIDIA Dynamo |
| Weights | NVIDIA NVFP4 checkpoint | DeepSeek release checkpoint (MXFP4) | NVIDIA NVFP4 checkpoint | NVIDIA NVFP4 checkpoint |
| Prefill and decode | Disaggregated. DEP4 prefill workers, wide expert parallel decode (DEP16 or DEP32), TP4/EP4 decode at 34 agents | Aggregated TP4 or TP2 for 1 and 2 agents. Disaggregated from 64 agents, with one or two TP2 prefill workers and DEP4 decode (or five TP2 decode workers) | Disaggregated. 1 to 3 prefill workers (TP2/EP2 or DEP4), decode on TP8, TP4/EP4 or DEP8 | Aggregated TP8 on vLLM for 1 agent. Disaggregated from 16 agents: a TP8 prefill worker with three TP8 decode workers at 16 and 30 agents, and DWDP16 prefill (one or two groups) with DEP16 decode from 512 agents |
| Speculative decoding | DSpark, 3 to 7 draft tokens | Built-in DSpark drafting, 6 draft tokens | MTP, 4 to 8 draft tokens | DSpark, 3 to 7 draft tokens |
| KV cache | FP8, offloaded to host memory on prefill workers | Kept in GPU memory, with the model’s Engram memory offloaded to host memory | FP8, offloaded to host memory on prefill workers in all but one config | FP8, offloaded to host memory in the TensorRT-LLM configs |
| Routing | KV-aware, with session affinity | KV-aware across two prefill workers at 1,024 agents, a single prefill worker otherwise | KV-aware, with session affinity | KV-aware, with session affinity |
| GPUs per config | 32 to 56 | 2 to 12 | 10 to 20 | 8 to 48 |
## Further reading