Anthropic 用 Claude 对 123 个开源库完成汇编级形式化安全证明

来源:Anthropic:Research(发表成果 · 网页) 2026年10月21日 00:00 AIHOT 评分:85 精选
摘要:Anthropic 与 Theorem 合作,用 Claude 将 OSS-Fuzz 模糊测试转化为规格,对 123 个开源库的 239 个入口点证明了任意 64 MB 以下输入不崩溃,生成约 1.8 亿行 Rocq 证明。
推荐理由:Anthropic 用 Claude 对 123 个 OSS-Fuzz 开源库在 RISC-V 汇编层面写 Rocq 证明,读者可以据此了解 LLM 规模化形式化验证的实际流程与局限。
# Anthropic 用 Claude 对 123 个开源库完成汇编级形式化安全证明 > 原标题:LLMs are making formal software verification feasible at a much larger scale - 来源:Anthropic:Research - 发布时间:2026-10-21T16:00:00.000Z - AIHOT:https://aihot.news/items/jsr18rtgxl0kvb0k7np7lpyou - 原文:https://www.anthropic.com/research/formal-verification ## 摘要 Anthropic 与 Theorem 合作,用 Claude 将 OSS-Fuzz 模糊测试转化为规格,对 123 个开源库的 239 个入口点证明了任意 64 MB 以下输入不崩溃,生成约 1.8 亿行 Rocq 证明。 ## 推荐理由 Anthropic 用 Claude 对 123 个 OSS-Fuzz 开源库在 RISC-V 汇编层面写 Rocq 证明,读者可以据此了解 LLM 规模化形式化验证的实际流程与局限。 ## 正文 · 中文译文 _Keane Lucas, Aengus Lynch, Luna Tong, Luke Champine, Zygi Straznickas, Dino Distefano, Harshikaa Agrawal Evyatar Ben Asher, Ava Howell, Sami Jawhar, Chris Rohlf, Nickolai Zeldovich Nicholas Carlini, Rajashree Agrawal, Jason Gross_ _大多数软件都充斥着漏洞,而 AI 现在可以轻松地[发现](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/)并[利用](https://www.anthropic.com/research/exploit-evals)它们。针对网络攻击的标准防御手段已不再足够:我们需要能够证明代码没有漏洞的方法。形式化软件验证可以提供这种保证,但它在历史上需要专家投入数年努力,对大多数现实项目而言并不实用。然而,在与 [Theorem](https://theorem.dev/) 的合作中,我们表明情况已不再如此:LLM 现在可以大规模生成形式化证明,从而支持包含可证明安全保证的新软件开发工作流。借助 Claude,我们在 100 多个真实世界开源软件库中,于汇编代码层面(RISC-V)证明了安全属性。**据我们所知,这是首次大规模尝试使用 AI 将形式化验证应用于汇编代码层面的生产软件。**在编写证明的过程中,Claude 还发现了多年来一直未被检测到的漏洞,尽管已经进行了[数万亿次测试运行](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html)。随着模型能力持续提升,我们预计形式化验证将成为安全软件开发中不可或缺的一部分。_ ## 引言 即使是成熟且经过大量测试的软件也包含漏洞,而 LLM [越来越擅长](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/)发现它们。Linux 内核——当今测试最充分的软件之一,部署在全球数十亿台设备上——最近披露了[超过 1,000 个漏洞](https://lwn.net/Articles/1097401/)。随着前沿模型越来越有能力将[看似无害的漏洞](https://www.anthropic.com/research/exploit-evals)串联成严重漏洞利用,我们需要能够在这些漏洞于现实世界中被利用之前检测并修复它们的技术。 理想情况下,这类技术能够发现并修补_所有漏洞_,然后提供某种证明,表明再也找不到更多漏洞了。_形式化验证_是实现这一点的一种方法。在形式化验证中,开发者将代码转换为数学模型,然后尝试证明它始终遵循某组已定义的安全规则。如果无法证明这一点,就意味着代码中可能存在违反规则的漏洞,而失败的性质暗示了漏洞可能是什么。重复这一过程(每次消除失败原因之后),最终将产生证明,表明代码中不存在任何会违反这些规则的漏洞。 然而,形式化验证并不容易。证明可能比原始代码大许多倍,编写证明需要大量努力以及小众且高度专业化的技能组合。例如,实现 seL4 操作系统内核花费了两人年,但[对其进行形式化验证](https://sel4.systems/Research/pdfs/comprehensive-formal-verification-os-microkernel.pdf)花费了_20_人年。因此,形式化验证仅在真正关键任务的应用中才具有实用性,例如[空客的航空电子代码](https://www.astree.ens.fr/papers/astree_airbus_sas2007.pdf)。 但 LLM 让这件事变得容易得多。前沿模型现在能够完全自主地为真实世界的软件生成正确性证明。为了演示这一点,我们让 Claude 为 123 个开源库证明有用的安全属性,包括证明它们的某些可执行产物在某些条件下_绝不会_崩溃。具体而言,Claude 证明了:在特定假设下(即程序以单线程静态二进制形式运行、内存预算固定、运行在采用 Linux 操作系统的 RISC-V1 CPU 上),对于所有小于 64 MB 的输入以及所有建模的 Linux 操作系统(OS)响应和内存布局,这些程序将始终正常退出,不会触发内存安全错误或断言失败。这距离修复_所有_可能的 bug 仍很遥远,但确实让我们比以往更接近目标,并提供了通往该目标的路线图。 我们正在发布[这些证明的一个子集](https://github.com/theorem-labs/oss-verify-proofs),以便任何人都能验证和复用它们,同时发布[RISC-V 和操作系统语义](https://github.com/theorem-labs/rocq-linux-process-semantics)。我们在该项目上的合作方 [Theorem](https://theorem.dev/) 也提出为[其他开源项目](https://theorem.dev/oss-verify/)运行相同的流水线,并发布了一个 [Claude skill](https://github.com/theorem-labs/oss-verify/blob/main/.claude/skills/add-project/SKILL.md),帮助开发者开始验证自己的代码。 形式化验证是一个涵盖众多方法论的广泛领域。下文我们还将描述使用其他形式化验证方法进行的初步探索,以加速其他安全或软件开发任务,例如评估 zlib 压缩算法的安全性、证明用 Rust 编写的 Python 解释器的行为等价性,以及根除关键软件中长期存在的 bug。 形式化验证的前景广阔。随着模型变得更智能,编写证明的成本正在下降。此外,随着更多代码被形式化、实现安全保障的方法变得更加标准化,跨代码库复用或修改证明的能力将摊薄成本。我们鼓励所有开发者用 LLM 探索形式化验证——_尤其是_那些此前用能力较弱的模型尝试过并发现行不通的人。 在本文的剩余部分,我们将更详细地介绍我们的形式化验证实验。 ## 将测试转化为证明 要证明一段软件是正确的,我们首先必须说明“正确”究竟意味着什么。正确性由_规约_(spec)定义。在某些程序中,有一种清晰的方式来形式化地规定程序的预期行为。例如,[PNG](https://en.wikipedia.org/wiki/PNG) 解码器接收一串字节,并(遵循 [RFC 2083](https://www.rfc-editor.org/info/rfc2083/))将这些字节解码为对应的图像。但对于大多数程序来说,事情并没有这么简单。文本编辑器或网页浏览器“正确”意味着什么?获得全面且无歧义的规约的难度,是在更大规模上验证软件的一大障碍。 为了针对我们试图证明的开源库代码解决这一问题,我们采取的方法是:自动将其已有的可编译测试套件转换为一份规范,具体做法是把“该测试套件始终通过”这一规则编码进去,并采用一个通用模型来刻画在 Linux 上执行任何 RISC-V 编译二进制文件的含义。从历史上看,证明这类规范一直是难以处理的,原因在于编写一个能够涵盖并论证数万条汇编指令在任意输入下的所有可能行为和交互的证明本身具有内在复杂性(以及由此导致证明往往极其庞大)。这正是当今 LLM 的用武之地:为达成一个定义明确的目标而处理数百万行代码,正是 LLM 极其擅长的事情。 在这个项目中,我们从_模糊测试_入手。在这类测试中,开发者在其代码中编写一个_入口点_2来接收输入,任何导致崩溃的输入都是作者所关心的缺陷。随后,_模糊测试器_这一测试引擎会在该入口点生成海量的半随机和变异输入,以试图发现缺陷(如图 1 所示)。模糊测试器运行的时间越长,能发现的缺陷就越多。但这本质上是一种暴力过程,无论尝试多少输入,都无法保证覆盖每一条可能的代码路径。 _**模糊测试的工作原理是将大量生成的输入通过一个入口点运行,以找出导致崩溃的输入。**输入是无限的,因此搜索永远无法穷尽。_ 为了找出模糊测试本会暴露的所有缺陷,我们将该模糊测试转换为一份定义在编译后的 RISC-V 指令之上的规范。该规范指出,对于任何小于 64 MB 的输入,模糊测试都不会崩溃。3如果 Claude 能够证明这份规范,那么针对 RISC-V 的模糊测试就变得多余了:对该入口点进行模糊测试将永远找不到任何缺陷。 我们在 Google [OSS-Fuzz](https://github.com/google/oss-fuzz) 计划中的数百个项目中开展了这项工作,该计划为符合条件的开源项目提供持续的模糊测试。OSS-Fuzz 包含数千个由开发者编写的入口点,它们通过触发崩溃(例如断言失败)来查找代码属性被破坏的情况。OSS-Fuzz 会自动用 AddressSanitizer4(ASan)对 C 和 C++ 程序进行插桩,使其在内存被不当访问时崩溃(这是[软件漏洞的一个主要原因](https://www.cisa.gov/news-events/news/cisa-nsa-fbi-and-international-cybersecurity-authorities-publish-guide-case-memory-safe-roadmaps))。像内存安全这样的广谱规范几乎适用于所有软件。 **_该开源库及其现有的模糊测试被编译为 RISC-V 可执行文件。该可执行文件,连同 Theorem 的 RISC-V 与操作系统形式化语义以及所期望的无崩溃属性,机械地生成一份证明必须满足的规范。Claude 编写证明;Rocq 内核依据该规范对其进行检查。绿色标记表示已证明的内容,其成立以受信任组件正确为前提。_** 在几周的时间里,**Claude 成功修补了它发现的漏洞,随后证明:对于 123 个不同 OSS-Fuzz 库中的 239 个入口点,在任何输入(最大 64 MB)下都不会发生崩溃。**这些证明作用于编译后的 RISC-V 指令(即可执行二进制文件),总计约 1.8 亿行 Rocq 代码,典型证明约 60 万行,为二进制文件提供安全保障,而这些二进制文件的中位长度为 18,000 条 RISC-V 指令。这些证明是通过使用 Theorem 的 RISC-V 指令和 Linux 进程语义,将 RISC-V 汇编代码转换为数学模型而构建的。这意味着这些证明目前假设程序在运行 Linux 操作系统的 RISC-V CPU 上,以单线程静态二进制形式运行,并具有固定的内存预算。扩大这些证明的范围完全触手可及,因为这将涉及为更多汇编架构和其他操作系统进程标准化语义。请参阅[专家对 Claude 所写证明的评论](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf)。 _**对所有输入的形式化无崩溃证明排除了从入口点可达的任何路径上的所有崩溃。**在迭代其证明的过程中,Claude 还可以发现并修复任何剩余的、可导致崩溃的漏洞。_ 尽管证明代码量巨大,但几乎不需要人工审查。Rocq 证明检查器确保证明正确地满足了规范。因此,开发者唯一需要信任的产物是: - RISC-V 和 Linux 进程的形式化模型; - 所证明的确切定理陈述; - Rocq 证明检查器本身。5 **这并不意味着这些库不包含漏洞。**相反,这意味着任何**剩余的漏洞要么存在于上述列出的产物中,要么超出了 Claude 证明的范围**。加固这些仍可能存在漏洞的可信组件(Rocq、RISC-V 和 Linux 语义)并非易事,但这比调试大规模证明所覆盖的每一行代码要容易得多——尤其是因为这种方法可以在不增加可信组件规模的情况下保护任意数量的项目。 我们欢迎对我们的证明进行独立验证,并鼓励改进我们的方法。为此,我们发布了为本项目制作但尚未公开的可信组件(上述);Claude 为无需修补的模糊测试提供的完整 Rocq 证明;以及我们无法证明的其他模糊测试的规范。 ## 发现并修复漏洞 上文我们展示了,你可以将许多开源库开发者编写的测试(模糊测试)转化为规范,Claude 可以证明这些规范始终通过,有时需要先修补它发现的剩余漏洞。因此,尝试证明这些规范的过程发现了数百个漏洞。 在 Claude 尝试证明数百个入口点的过程中,出现了大量漏洞。为确保 Claude 不必每次遇到这种情况都停下来等待人工处理,我们让 Claude 修复它发现的漏洞:它修补代码,导致规范随修补后的代码重新生成,然后 Claude 继续证明新的规范。6 我们发现的许多 bug 是模糊测试器不太可能找到的。模糊测试器会变异小输入,[通常上限为 1 MB](https://github.com/AFLplusplus/AFLplusplus/blob/stable/include/config.h#L314),并且通常依赖代码覆盖率来引导它们发现 bug。在证明某个库中的一个入口点时,Claude 在其解压器副本中发现了一个计数器下溢,导致越界写入,但仅在约 20 MB、具有一种罕见特定布局的输入上才会触发。其他 bug 包括:一个图像解码器中的越界读取,需要 64 MB 输入和超过 2 GB 内存;另一个库中的越界读取,需要对同一文件的两个句柄进行特定调用序列;以及在 libexpat 中重新发现 CVE-2026-56132,该漏洞需要深度嵌套的输入。Claude 还发现了根本不会导致崩溃的逻辑 bug。例如,在证明某个库是内存安全的过程中,它注意到一个用于限制应用程序可加载哪些文件的功能并未始终被强制执行。 这个项目的最终目标是获得证明:经 RISC-V 编译的模糊测试已没有剩余的崩溃可被发现。然而,上述这类 bug 表明,无论该目标是否达成,这个过程还有另一个好处:即使只是尝试这个过程,也会产生需要修复的真实 bug。 ### 证明超越内存安全 证明一个模糊测试不再有崩溃可被发现,其意义比听起来更大。在现实世界的代码中,模糊测试常常用于检验有意义的高层属性是否被违反,例如编码-解码往返、幂等性、代数定律、差分测试、自洽性或与参考实现的等价性。例如,针对 LZO 压缩算法的模糊测试断言:如果你压缩一个输入,然后解压它,结果与原始输入相同。如果该属性不成立,程序就会崩溃——因此证明测试永不崩溃,就等同于证明该属性本身。 即使这些属性并非由开发者直接指定,这种情况也可能发生。例如,为了让 Claude 完成针对 cpp-httplib 的 OSS-Fuzz 证明,必须排除因 std::set 析构函数中递归过多而导致栈溢出的可能性。这个递归深度受树的最大深度限制,因此 Claude 必须(在机器码上!)证明 libc++ 的红黑树再平衡算法被正确优化,以保持树的深度在对数范围内有界。 ### 其他形式化验证案例研究 形式化验证是一个广阔的领域。上述从模糊测试到证明的例子,是 LLM 通过直接将证明应用于机器码,使形式化验证更广泛可用的一个实例。 我们还看到在其他方面将 LLM 与形式化验证结合的成功案例。例如,我们让 Claude 为 [RFC 1950](https://www.rfc-editor.org/info/rfc1950/) 编写形式化规范,该技术文档定义了 zlib 压缩文件格式。然后,我们让 Claude 证明 Linux 内核中的 zlib 实现正确实现了该规范。与上述示例一样,Claude 在二进制层面证明了这一性质,只不过这次不是 RISC-V,而是使用 Claude 生成(并经过大量测试)的 x86 ISA 形式化语义,直接证明了 x86 编译的二进制文件。7 随后,我们对另外六个广泛使用的库重复了这一过程。在此过程中,我们在边缘情况中发现了一些轻微(不可利用)的 bug,目前正在向上游提交。我们从这次实践中得出两个结论:(1) 这些经过极其充分测试的代码库已经几乎完全正确;(2) 对真实现有代码库证明端到端正确性性质是可能的。 在另一项实验中,我们让 Claude 用 Rust 开发一个支持大部分 Python 子集的解释器,并证明(使用 Verus,一个 Rust 验证工具)它符合一篇学术论文中概述的 CPython 语义。然后,我们通过运行超过 170 万个随机生成的 Python 程序,实证测量了其对 CPython 行为的符合程度,发现 99.95% 的测试与 CPython 行为完全一致,其余测试差异均为意料之中的已知边缘情况。 我们还通过对简单安全性质的形式化验证尝试,在几个广泛使用且关键的应用中发现了问题。其中一个应用里,我们发现了一个潜伏十多年、此前未知的逻辑 bug。在验证一个广泛使用的 PDF 阅读器的过程中,我们一周内发现的安全 bug 数量相当于通常一年报告的数量。 有关其中部分工作的更多细节,请参阅[附录](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf)。 ## 这仅仅是开始 LLM 辅助形式化验证的关键限制之一是,证明可能需要消耗数亿个 token。如果我们的目标是让这些方法惠及整个软件生态系统,成本效率就至关重要。幸运的是,有充分理由预期效率会提升。 ### 更新的模型需要更少的 token 我们基于[真实的 OSS-Fuzz 代码](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf)构建了 36 个小型验证任务。Opus 5.5 证明其中一半任务所用的 token 比 Opus 5 少约 40%。随着模型在长验证任务上接受训练,我们预计这一趋势将持续。 ![Line chart of the share of 36 verification tasks completed by token budget per agent for Opus 5.5, Fable 5.1, and Opus 5. Opus 5.5 finishes proofs with the fewest tokens.](https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F02207cbc6f46e272fe9a0fa7929cb8ec29578e74-1920x980.png&w=3840&q=75) _**Opus 5.5 完成证明的速度快于早期模型。**在给定 token 预算下,单个智能体证明的 36 个形式化验证任务所占比例。每个任务在 500 万 token 预算下获得三次独立的单智能体尝试,首次尝试完成即计为已证明。_ ### 证明可以复用 当智能体各自独立工作时,它们往往会重复大量工作。证明与软件一样,是由更小的证明递归构建而成——而证明越小,往往越容易复用。为了衡量这一点的影响,我们让智能体在证明各自目标时互相复制彼此的证明。在获得这个共享工作区后,智能体所需的 token 比单独工作时减少了 17%,一些智能体甚至在单独尝试失败后取得了成功。 ![Chart of 36 proofs in finishing order, with arcs joining proofs that share a file and bars showing the tokens spent on each proof compared with an agent working alone.](https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F5078468fa7e09774b1ab6e86984ec5691d3d11fa-1080x712.gif&w=3840&q=75) _**来自同一共享工作区的 36 个证明,按完成顺序排列。**弧线连接任意两个共享未更改文件的证明;条形表示编写每个证明所消耗的 token,黑线表示同一任务由单个智能体独立完成时的平均成本,在 34 个已完成的证明中有 30 个的独立成本更高(独立工作的智能体有两个证明未能完成)。_ 我们在 OSS-Fuzz 证明工作中大规模观察到了同样的现象。证明的大部分内容涉及每个程序都包含的代码(尤其是 C 运行时和 AddressSanitizer 的初始化),因此差异很大的程序最终也可能有大量重叠的证明。智能体团队利用了这一点,复制另一个团队已完成的证明,然后将其适配到自己的程序(图 6)。 ![Block chart of the proofs for libcups, freerdp, and libzmq, showing how much of each part was identical, lightly edited, edited more or new.](https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F8e295614d599d548fbd2ff97ed3cc3e62f501b2b-1920x1248.png&w=3840&q=75) _**智能体复制现有证明并将其适配到每个新程序。**每一列代表一个程序完整证明的字节占比。方块是证明的各个部分,每个方块显示有多少内容保持完全相同、轻度编辑(同一文件,大小差异在 2% 以内)、编辑较多或全新编写。共享部分几乎没有新增代码。_ 证明的这种可复用性正是 [Mathlib](https://leanprover-community.github.io/) 得以成长的方式——它是 Lean 中的形式化数学共享库,已发展到由 700 多名贡献者提交的近 [300,000 个定理](https://leanprover-community.github.io/mathlib_stats.html):每个新证明都建立在现有成果之上。我们在 OSS-Fuzz 工作中也积累了类似的证明库,其中一些将包含在我们的发布中(我们不会发布可能暴露开源软件中未修补漏洞的证明)。 随着我们从这些初步实验中逐步推进,我们需要一种策略来验证海量的_现有_软件,从最关键的依赖项开始。正如我们所展示的,这可以通过使用通用规范和二进制层面的工作大规模实现,从而避免用另一种语言重写整个代码库。 这种方法还促进了包级别粒度的证明复用,这意味着每个依赖项只需验证一次。依赖关系遵循幂律:即使只证明一小部分共享库,也能让大量软件变得可证明。更多细节请参见 [Theorem 的路线图](https://theorem.dev/blog/verified-software-stack/),该路线图旨在验证整个 Nix 生态系统,其中包括 glibc、OpenSSL 和 cURL 等关键基础设施。 ## 局限性 在实践中,形式化验证有时对代码安全性的证明可能比我们期望的要少。 **“形式化验证”并不意味着“没有缺陷”。**Claude 可以编写一个证明,表明编译后程序的行为满足形式化规范的要求(该证明由外部证明检查器验证)。然而,证明检查器无法告诉你你的规范是否与现实相符。要与现实相符,规范必须正确涵盖并建模已验证程序所运行的真实机器的所有方面,包括 CPU 和操作系统。当已验证代码中出现缺陷时,是因为规范或假定的机器语义与现实不同。幸运的是,这也意味着我们知道_在哪里_寻找剩余缺陷,以及如何设计一个系统来最小化这些仍可能存在缺陷的缺口。以下是这如何适用于我们的证明,以及未来工作如何弥合这些缺口的想法。 **真实机器会做出模型未描述的行为。**我们的证明针对的是单线程、静态链接到 musl libc 的 RISC-V 二进制文件。然而,大多数人是在 x86 或 ARM 上运行这些库,并将它们动态链接到 glibc。要弥合这一差距,我们需要更接近实际部署系统的系统模型。例如,为 x86 生成精确语义就是朝着这个方向迈出的一步。 **该主张可能并未涵盖我们关心的所有内容。**我们证明了在 Google OSS-Fuzz 的许多模糊测试入口点上,对于任何小于 64 MB 的输入,崩溃都不会触发,包括 ASan 崩溃。然而,ASan 并不能检测所有内存安全缺陷,而且我们并未验证每个库的所有入口点。即使我们验证了,验证所有入口点也无法覆盖整个库(尤其是在存在死代码的情况下)。此外,真实程序可能误用该库,导致_调用方_出现内存损坏。要弥合这一差距,我们需要规定调用方对每个公开函数必须做什么,并且调用方必须严格遵守。 我们预计,随着我们对如何利用 LLM 进行形式化方法的研究不断深入,以及模型持续改进,这些局限将变得可控,我们将能够验证我们所依赖代码中更大比例的部分。 ## 结论 趋势很明确:形式化验证过去只在最安全关键的代码中用于定制专家打造的保证,而很快它将像今天的模糊测试一样,对几乎所有生产软件都变得切实可行。就像静态类型、测试框架和其他正确性工具一样,验证将从“外挂式”成熟为“内置式”,这不仅由 LLM 驱动,也由语言设计和开发工作流的创新驱动。 在当今世界——LLM 自动扫描并发现越来越多的代码漏洞——我们必须采用能够_保证_代码安全属性的流程。LLM 规模化的形式化验证是一种有前景的方法,能够以以前无法达到的范围和规模来确保这些保证。我们鼓励其他人进行实验,以便我们共同构建一个世界:稳健的软件广泛普及,软件缺陷成为被充分理解且可快速处理的罕见现象。 [阅读完整附录。](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf) ## 正文 · 原文 _Keane Lucas, Aengus Lynch, Luna Tong, Luke Champine, Zygi Straznickas, Dino Distefano, Harshikaa Agrawal Evyatar Ben Asher, Ava Howell, Sami Jawhar, Chris Rohlf, Nickolai Zeldovich Nicholas Carlini, Rajashree Agrawal, Jason Gross_ _Most software is riddled with bugs, which AI can now easily [find](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/) and [exploit](https://www.anthropic.com/research/exploit-evals). The standard defenses against cyberattacks are no longer sufficient: we need ways to prove that code is free of bugs. Formal software verification could provide this guarantee, but it has historically required years of expert effort, making it impractical for most real-world projects. In a collaboration with [Theorem](https://theorem.dev/), however, we show this is no longer the case: LLMs can now produce formal proofs at large scale, enabling new software development workflows that include proven security guarantees. Using Claude, we proved security properties at the assembly code level (RISC-V) in over 100 real-world open-source software libraries. **To our knowledge, this represents the first large-scale effort to use AI to apply formal verification to production software at the assembly code level.** While writing the proofs, Claude also discovered bugs that had eluded detection for years, despite [trillions of test runs](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html). As model capabilities continue to improve, we expect formal verification to become an essential part of secure software development._ ## Introduction Even mature, heavily tested software contains bugs, and LLMs are [increasingly adept](https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/) at finding them. The Linux kernel—one of the most well-tested pieces of software today, deployed on billions of devices worldwide—recently had [over 1,000 vulnerabilities](https://lwn.net/Articles/1097401/) disclosed. With frontier models becoming more capable of [chaining seemingly benign bugs](https://www.anthropic.com/research/exploit-evals) into critical exploits, we need techniques that can detect and fix these bugs before they can be exploited in the wild. Ideally, such techniques could find and patch _all bugs,_ and then provide some proof that there are no more to be found. _Formal verification_ is one way to do this. In formal verification, developers translate their code into a mathematical model, then try to prove that it always follows some defined set of security rules. If they fail to prove this, that means there could be a bug in the code that would violate a rule, and the nature of the failure implies what the bug might be. Repeating this process (after eliminating the cause of failure each time) will ultimately yield proof that the code is devoid of any bug that would violate those rules. However, formal verification isn’t easy. Proofs can be many times larger than the original code, and writing them requires significant effort and a niche, highly specialized skill set. For instance, it took two person-years to implement the seL4 operating system kernel, but _20_ person-years [to formally verify it](https://sel4.systems/Research/pdfs/comprehensive-formal-verification-os-microkernel.pdf). Accordingly, formal verification has been practical only in truly mission-critical applications, such as [Airbus’s avionics code](https://www.astree.ens.fr/papers/astree_airbus_sas2007.pdf). But LLMs are making it far easier. Frontier models can now produce correctness proofs for real-world software fully autonomously. To demonstrate this, we asked Claude to prove useful security properties across 123 open-source libraries, including showing that some of their executable artifacts can _never_ crash under certain conditions. Specifically, Claude proved that these programs will always exit normally without triggering memory-safety errors or failing assertions, for every input less than 64 MB and every modeled Linux operating system (OS) response and memory layout, under certain assumptions (namely that the program runs as a single-threaded static binary with a fixed memory budget on a RISC-V1 CPU using Linux OS). This is still far away from fixing _all_ possible bugs, but it does get us much closer than before and provides a roadmap toward that end. We’re publishing a [subset of these proofs](https://github.com/theorem-labs/oss-verify-proofs) so that anyone can verify and reuse them, along with the [RISC-V and OS semantics](https://github.com/theorem-labs/rocq-linux-process-semantics). [Theorem](https://theorem.dev/), our collaborator on this project, is also offering to run the same pipeline for [other open-source projects](https://theorem.dev/oss-verify/), and is publishing a [Claude skill](https://github.com/theorem-labs/oss-verify/blob/main/.claude/skills/add-project/SKILL.md) to help developers get started with verifying their own code. Formal verification is a broad field with many methodologies. Below, we also describe our initial explorations using other formal verification methods to accelerate other security or software development tasks, such as assessing the security of the zlib compression algorithm, proving behavioral equivalence of a Python interpreter written in Rust, and rooting out long-standing bugs in critical software. The road ahead for formal verification is promising. The cost of writing proofs is falling as models become more intelligent. Moreover, as more code is formalized and methods of achieving security guarantees become more standardized, the ability to reuse or modify proofs across codebases will amortize the cost. We encourage all developers to explore formal verification with LLMs—_especially_ those who previously tried it with less capable models and found it didn’t work. In the rest of this post, we cover our formal verification experiments in more detail. ## Turning tests into proofs To prove that a piece of software is correct, we must first state what “correct” even means. Correctness is defined by the _specification_ (spec). In some programs, there is a clear way to formally specify the program’s intended behavior. For example, a [PNG](https://en.wikipedia.org/wiki/PNG) decoder takes a sequence of bytes and (following [RFC 2083](https://www.rfc-editor.org/info/rfc2083/)) decodes these bytes into the corresponding image. But for most programs, things aren’t so straightforward. What would it mean for a text editor or a web browser to be “correct”? The difficulty of obtaining a comprehensive and unambiguous spec is a significant obstacle to verifying software at a larger scale. To address this for the code in the open-source libraries that we sought to prove, we take the approach of automatically converting their preexisting compilable test suites into a spec by encoding the rule “this test suite always passes,” using a general model of what it means to execute any RISC-V-compiled binary on Linux. Historically, proving such specs was intractable, given the inherent complexity of writing a proof that can encompass and make an argument over all possible behaviors and interactions of tens of thousands of assembly instructions over any input (and the fact that proofs tend to be extremely large as a result). This is where today’s LLMs come in: churning through millions of lines of code in pursuit of a well-defined goal is a task that LLMs are extremely well suited for. For this project, we started with _fuzz tests_. In these tests, developers write an _entry point_2 into their code that takes inputs, such that any input that causes a crash is a bug the authors care about. Then the _fuzzer,_ a testing engine, generates enormous numbers of semirandom and mutated inputs at the entry point to try to find a bug (visualized in Figure 1). The longer a fuzzer runs, the more bugs it can find. But this is inherently a brute-force process, with no guarantee of exercising every possible code path, regardless of how many inputs you try. _**Fuzzing works by running huge numbers of generated inputs through one entry point to find the inputs that cause a crash.** There are infinite inputs, so the search can never be complete._ To find all the bugs that would’ve been surfaced by a fuzz test, we turned the fuzz test into a spec defined over the compiled RISC-V instructions. The spec says that the fuzz test won’t crash for _any_ input less than 64 MB.3 If Claude can prove this spec, the fuzz test on RISC-V becomes superfluous: fuzzing that entry point will never find any bugs. We did this for hundreds of projects in Google’s [OSS-Fuzz](https://github.com/google/oss-fuzz) initiative, which provides continuous fuzz testing for qualifying open-source projects. OSS-Fuzz contains thousands of developer-authored entry points that look for broken code properties by crashing (for example, on a failed assertion). OSS-Fuzz automatically instruments C and C++ programs with AddressSanitizer4 (ASan), causing them to crash whenever memory is accessed improperly (which is a [leading cause of software vulnerabilities](https://www.cisa.gov/news-events/news/cisa-nsa-fbi-and-international-cybersecurity-authorities-publish-guide-case-memory-safe-roadmaps)). Broad-spectrum specifications like memory safety apply to almost all software. **_The open-source library and its existing fuzz test are compiled to a RISC-V executable. This executable, along with Theorem’s RISC-V and OS formal semantics and the desired no-crash property, mechanically yield a specification the proof must satisfy. Claude writes the proof; the Rocq kernel checks it against the specification. Green marks what’s proven, conditional on the trusted components being correct._** Over the span of a few weeks, **Claude successfully patched the bugs it found and then proved that there are no crashes on any input (up to 64 MB) for 239 entry points across 123 different OSS-Fuzz libraries.** These proofs operate on the compiled RISC-V instructions (i.e., executable binaries), and they comprise about 180 million lines of Rocq, with a typical proof running to about 600,000 lines to provide security guarantees on binaries with a median length of 18,000 RISC-V instructions. These proofs are built by translating the RISC-V assembly code to a mathematical model using Theorem’s RISC-V instruction and Linux process semantics. This means these proofs currently assume that the program runs as a single-threaded static binary with a fixed memory budget on a RISC-V CPU using Linux OS. Expanding the scope of these proofs is well within reach, as it would involve standardizing semantics for more assembly architectures and other operating system processes. See [an expert’s take](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf) on the proofs Claude wrote. _**A formal no-crash proof over all inputs rules out all crashes along any path reachable from the entry point.** While iterating on its proof, Claude can also find and fix any remaining reachable crash-producing bug._ While this is an enormous volume of proof code, virtually none of it needs to be manually reviewed. The Rocq proof checker ensures that the spec is correctly satisfied by the proof. As a result, the only artifacts that a developer needs to trust are: - The formal models of RISC-V and of Linux processes; - The exact theorem statements that were proved; - The Rocq proof checker itself.5 **This doesn’t mean these libraries contain no bugs.** Instead, it means that any **remaining bugs either are to be found within the artifacts listed above or are outside the scope of Claude’s proof**. Hardening those still-possibly-buggy trusted components (Rocq, RISC-V, and Linux semantics) is not insignificant, but it’s a far more tractable task than debugging every line of code the scaled-out proofs cover—especially as this approach can secure an arbitrary number of projects without increasing the size of the trusted components. We welcome independent verification of our proofs and encourage refinement of our approach. To that end, we are publishing the trusted components (above) made for this project that are not already public; Claude’s complete Rocq proofs for fuzz tests that did not require patching; and specifications for other fuzz tests we were not able to prove. ## Finding and fixing bugs Above, we showed that you can turn many open-source libraries’ developer-written tests (fuzz tests) into specs that Claude can prove will always pass, sometimes after patching the remaining bugs it found. For this reason, the process of attempting to prove these specs discovered hundreds of bugs. Lots of bugs showed up during Claude’s attempts to prove hundreds of entry points. To ensure that Claude didn’t have to stop for a human every time this happened, we let Claude fix the bugs it found: it patched the code, which caused the spec to be regenerated with that patched code, and then Claude proceeded to prove that new spec.6 Many of the bugs we found are ones that a fuzzer would be unlikely to ever find. Fuzzers mutate small inputs, [typically capped at 1 MB](https://github.com/AFLplusplus/AFLplusplus/blob/stable/include/config.h#L314), and typically rely on code coverage to guide them toward bugs. While proving an entry point in one library, Claude found a counter underflow in its copy of a decompressor that led to an out-of-bounds write, but only on inputs of around 20 MB that had one rare, specific layout. Other bugs include an out-of-bounds read in an image decoder that needed a 64 MB input and over 2 GB of memory, an out-of-bounds read in another library that needed a particular sequence of calls across two handles to the same file, and a rediscovery of CVE-2026-56132 in libexpat, which needed deeply nested input. Claude also found logic bugs that never crash at all. While proving that one library was memory-safe, for example, it noticed that a feature for restricting which files an application can load isn’t always enforced. The ultimate goal of this project was to achieve a proof that the RISC-V-compiled fuzz tests had no remaining crashes to find. However, bugs like the above show that the process has another benefit, regardless of whether that goal is achieved: even just attempting this process yields real bugs that need to be fixed. ### Proving more than memory safety Proving that a fuzz test has no more crashes to find is more significant than it sounds. It’s common for fuzz tests in real-world code to test for violations of meaningful high-level properties, such as encode-decode roundtripping, idempotence, algebraic laws, differential testing, self-consistency, or equivalence to reference implementations. For example, a fuzz test for the LZO compression algorithm asserts that if you compress an input, then decompress it, the result is the same as the original. If the property does not hold, the program crashes—so proving the test never crashes is equivalent to proving the property itself. This can also occur even when the properties are not directly specified by the developer. For example, in order for Claude to complete its OSS-Fuzz proof for cpp-httplib, it was necessary to rule out the possibility of a stack overflow due to too much recursion in the std::set destructor. This recursion depth is bounded by the maximum depth of a tree, so Claude had to prove (on machine code!) that libc++’s red-black tree rebalancing algorithm is correctly optimized to keep the tree’s depth logarithmically bounded. ### Other formal verification case studies Formal verification is an expansive domain. The fuzz-test-to-proof example above is one instance of LLMs making formal verification more widely usable by directly applying proofs to machine code. We’ve also seen success using LLMs with formal verification in other ways. For example, we asked Claude to write a formal spec of [RFC 1950](https://www.rfc-editor.org/info/rfc1950/), the technical document that defines the zlib compression file format. Then, we had Claude prove that the zlib implementation in the Linux kernel correctly implements that spec. As with the examples above, Claude proved this property at the binary level, except instead of RISC-V, Claude directly proved the x86-compiled binary using Claude-generated (and heavily tested) formal semantics of the x86 ISA.7 We then repeated this process for half a dozen other widely used libraries. Along the way, we found some minor (unexploitable) bugs in corner cases, which are in the process of being filed upstream. We draw two conclusions from this exercise: (1) these extremely well-tested codebases are already almost fully correct, and (2) it is possible to prove end-to-end correctness properties about real existing codebases. In another experiment, we asked Claude to develop an interpreter for a large subset of Python in Rust, and to prove (using Verus, a verification tool for Rust) that it adheres to the CPython semantics outlined in an academic paper. We then measured the adherence to CPython’s behavior empirically, running over 1.7 million randomly generated Python programs, finding identical behavior with CPython in 99.95% of tests, with remaining test differences as unsurprising known corner cases. We also found issues in several widely used and critical applications via formal verification attempts of simple security properties. In one of them, we discovered a previously unknown logic bug that had been latent for more than a decade. During an effort to verify a widely used PDF reader, we found as many security bugs in one week as are typically reported in a year. See [the Appendix](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf) for more details on some of these efforts. ## This is just the beginning One of the key limitations of LLM-assisted formal verification is that the proofs can take hundreds of millions of tokens to produce. Cost-efficiency is critical if our goal is to make these methods available to the entire software ecosystem. Fortunately, there are compelling reasons to expect efficiency to improve. ### Newer models need fewer tokens We built 36 small verification tasks [from real OSS-Fuzz code](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf). Opus 5.5 proves half of them with about 40% fewer tokens than Opus 5. We expect this trend to continue as models are trained on long verification tasks. ![Line chart of the share of 36 verification tasks completed by token budget per agent for Opus 5.5, Fable 5.1, and Opus 5. Opus 5.5 finishes proofs with the fewest tokens.](https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F02207cbc6f46e272fe9a0fa7929cb8ec29578e74-1920x980.png&w=3840&q=75) _**Opus 5.5 finishes proofs faster than earlier models.** Share of 36 formal verification tasks proved by a single agent at a given token budget. Each task received three independent single-agent attempts at a 5M-token budget, and counts as proven when the first attempt finishes._ ### Proofs can be reused When agents work separately, they often end up duplicating a lot of effort. Proofs, like software, are recursively built from smaller proofs—and the smaller the proofs are, the more reusable they tend to be. To measure the impact of this, we let agents copy each other’s proofs as they worked on proving their own targets. When given access to this shared workspace, agents needed 17% fewer tokens than when they worked alone, and some agents even succeeded after previously failing in a solo attempt. ![Chart of 36 proofs in finishing order, with arcs joining proofs that share a file and bars showing the tokens spent on each proof compared with an agent working alone.](https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F5078468fa7e09774b1ab6e86984ec5691d3d11fa-1080x712.gif&w=3840&q=75) _**The 36 proofs from one shared workspace, in the order they finished.** An arc joins any two proofs that share an unchanged file; bars show the tokens spent writing each proof, and black lines show the same task’s average cost for an agent working alone, which was higher in 30 of the 34 completed proofs (agents working alone were not able to complete two of the proofs)._ We observed the same phenomenon occurring at scale during our OSS-Fuzz proof effort. Most of a proof concerns code that every program contains (particularly initialization of the C runtime and AddressSanitizer), so very different programs can end up with significantly overlapping proofs. Agent teams took advantage of this by copying another team’s finished proof and then adapting it to their own program (Figure 6). ![Block chart of the proofs for libcups, freerdp, and libzmq, showing how much of each part was identical, lightly edited, edited more or new.](https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F8e295614d599d548fbd2ff97ed3cc3e62f501b2b-1920x1248.png&w=3840&q=75) _**Agents copied an existing proof and adapted it to each new program.** Each column represents one program’s complete proof by share of bytes. Blocks are the parts of the proof and each block shows how much was kept identical, lightly edited (same file, size within 2%), edited more, or newly written. The shared parts gained almost no new code._ This reusability of proofs is how [Mathlib](https://leanprover-community.github.io/), a shared library of formal mathematics in Lean, grew to nearly [300,000 theorems](https://leanprover-community.github.io/mathlib_stats.html) submitted by over 700 contributors: each new proof builds on the existing body of work. We accumulated a similar body of proofs in the course of our OSS-Fuzz effort, some of which we’re including in our release (we are not releasing proofs that may reveal an unpatched bug in the open-source software). As we graduate from these initial experiments, we need a strategy for verifying the enormous body of _existing_ software, starting with the most critical dependencies. As we’ve shown, this can be achieved at scale by using generic specs and working at the binary level, sidestepping the need to rewrite entire codebases in another language. This approach also facilitates the reuse of proofs at package-level granularity, which means that each dependency only needs to be verified once. Dependencies follow a power law: proving even a small set of shared libraries makes a huge swath of software feasible to prove. For more details, see [Theorem’s roadmap](https://theorem.dev/blog/verified-software-stack/) for verifying the entirety of the Nix ecosystem, which includes critical infrastructure like glibc, OpenSSL, and cURL. ## Limitations In practice, formal verification can sometimes prove less about the security of the code than we would like. **“Formally verified” does not mean “bug-free.”** Claude can write a proof showing that a compiled program’s behavior satisfies the requirements of a formal spec (which is verified by an external proof checker). However, a proof checker can’t tell you whether your spec matches reality. To match reality, the spec must correctly encompass and model everything about the real-world machine that the verified program runs on, including the CPU and operating system. When bugs show up in verified code, it’s because the spec, or the assumed semantics of the machine, differs from reality. Fortunately, this also means we know _where_ to look for remaining bugs, and how to design a system where those still-possibly-buggy gaps are minimized. Below are the ways this applies to our proofs, and ideas for future work that would close those gaps. **The real machine does something the model doesn’t describe.** Our proofs are about single-threaded, static RISC-V binaries linked against musl libc. However, most people run these libraries on x86 or ARM and dynamically link them against glibc. To close this gap, we’d need system models that more closely resemble deployed systems. For example, a step toward this would be generating accurate semantics for x86. **The claim may not cover everything we care about.** We proved that a crash never triggers for any input under 64 MB on many fuzzing entry points in Google OSS-Fuzz, including ASan crashes. However, ASan doesn’t detect all memory safety bugs, and we didn’t verify all the entry points for each library. Even if we had, the whole library wouldn’t be covered by verifying all entry points (especially if there is dead code). Additionally, real programs might misuse the library, leading to memory corruption in the _caller_. To close that gap, we’d need to specify what callers must do for each public function, and callers would have to strictly adhere to it. We expect that as we learn more about leveraging LLMs for formal methods, and as models continue to improve, these limitations will become tractable, and we’ll be able to verify a much larger percentage of the code we rely upon. ## Conclusion The trajectory is clear: formal verification, which was historically practical only for bespoke expert-crafted guarantees in the most safety-critical code, will soon be as practical as fuzzing is today for almost all production software. Just like static types, testing frameworks, and other correctness tooling, verification will mature from “bolted-on” to “built-in,” driven not only by LLMs but also by innovations in language design and development workflows. In today’s world—where LLMs automatically scan for and find increasingly more code vulnerabilities—we must use processes that can _guarantee_ code security properties. LLM-scaled formal verification is a promising approach toward securing these guarantees at a scope and scale that were previously unattainable. We encourage others to experiment so we can collectively build a world in which robust software has proliferated very widely and software bugs are a well-understood and quickly addressable rarity. [Read the full appendix.](https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf)