OpenAI 推出两项自建评测,展示 GPT-6 系列在 COBOL 等低资源编程语言上的进步

来源:OpenAI:官网动态(RSS · 排除企业/客户案例) 2026年10月19日 20:00 AIHOT 评分:64 精选
摘要:OpenAI 宣布最新模型在 COBOL、Fortran、Visual Basic 等低资源编程语言上能力显著提升,并推出两项自建评测。
推荐理由:原文给出低资源语言的两套自建评测和逐语言数据,读者可据此评估新模型在遗留系统维护与迁移上的可用性。
# OpenAI 发布 MultiLangBench 与 COBOL 迁移评测,GPT-6 系列大幅提升低资源编程语言能力 > 原标题:Advancing Coding in Low-Resource Programming Languages - 来源:OpenAI:官网动态 - 发布时间:2026-10-19T12:00:00.000Z - AIHOT:https://aihot.news/items/tzuiybo7a45sm87wj4rec2mvo - 原文:https://openai.com/index/low-resource-programming-languages ## 摘要 OpenAI 宣布其最新模型在 COBOL、Fortran、Visual Basic 等低资源编程语言上的构建与迁移能力显著提升,并开源评测套件。 ## 推荐理由 OpenAI 用自建评测量化了新模型在 COBOL 等低资源语言上的构建与迁移能力,并开源了测试套件,读者可对照自己使用的语言查看具体成绩。 ## 正文 · 中文译文 ## ![Image 1](https://openai.com/_next/static/immutable/media/lab.44a_dymau51y3.webp)\[ COBOL \]\[ OCaml\]![Image 6](https://openai.com/_next/static/immutable/media/laptop.2t2hneg7lf1gz.webp)\[ Fortran\]\[ Visual Basic \]![Image 8](https://openai.com/_next/static/immutable/media/chip.1t86cqmsrm7g2.webp)推进低资源编程语言的编码能力 加载中… [音频 1](https://openai.com/index/low-resource-programming-languages) 分享 1. 从零构建 SQLite(MultiLangBench) 我们最新的模型进展为 COBOL、Fortran 和 Visual Basic 等低资源编程语言的软件构建与迁移带来了更强的能力。与 Python 和 JavaScript 等语言相比,这些语言可供模型学习的代码和文档更少,但它们支撑着全球关键基础设施和基本服务。COBOL [支撑着⁠(在新窗口中打开)](https://www.ibm.com/think/topics/cobol)超过 40% 的网上银行系统、80% 的线下信用卡交易和 95% 的 ATM 交易,而 Fortran 被用于[天气预报⁠(在新窗口中打开)](https://repository.library.noaa.gov/view/noaa/71120)和[核反应堆安全分析⁠(在新窗口中打开)](https://www.nrc.gov/education-regulatory-research/research/computer-codes)。提升 AI 在这些语言上的工作能力,可以帮助团队维护和现代化基本服务所依赖的软件。 这项工作以开发者和企业客户的反馈为指导,他们告诉我们,编码模型在这些语言上往往表现不佳。维护和现代化这类软件可能很困难:文档可能不完整,最初的开发者可能已经离开公司,重要的业务规则可能只存在于代码中。无论团队是在修复缺陷还是迁移到另一种语言,他们都需要保留系统所依赖的行为。 为了衡量进展,我们开发了两项评估。MultiLangBench 测试模型能否在 20 种不同的编程语言中从零构建一个兼容 SQLite 的数据库引擎。我们的 COBOL 迁移评估测试模型能否将现有程序迁移到另一种语言,同时保留其行为。两者共同测试了两项互补的技能:构建新软件,以及保留现有软件的行为。 我们最新的模型在这两项评估中都显示出大幅提升,以更低的成本实现了更高的准确率。 在 MultiLangBench 上,GPT‑6 Astra 在 20 种编程语言中平均测试通过率达到 87.4%,高于 GPT‑5.6 Sol 的 26.8%,且所有测试语言均有提升。GPT‑6 Luna 也优于 GPT‑5.6 Sol,达到 40.3%,同时估计 API 成本降低约 78%。 在我们的 COBOL 迁移评估中,GPT‑6.1 Sol 在低推理强度下达到 86.6% 的准确率,而 GPT‑5.6 Sol 仅为 5.4%,同时成本降低 39%。在更高推理强度下,准确率达到 97.9%,几乎使该评估饱和。 这些结果共同表明,我们的模型在处理多种编程语言方面正变得显著更强,同时也更具成本效益。 ## 1\. 从零构建 SQLite(MultiLangBench) MultiLangBench 要求模型用指定的编程语言构建一个兼容 SQLite 的数据库引擎。模型必须组织代码库、管理状态、处理错误,并在保留现有功能的同时添加新功能。 为了跨语言比较实现,我们使用数据库文件作为共享接口。候选程序接收一个初始数据库和一条 SQL 查询,然后生成一个修改后的数据库。我们使用参考 SQLite 实现来检查生成的数据库是否具有预期内容。这使我们能够应用相同的测试,无论构建引擎使用的是哪种语言。 我们将 SQLite 现有测试套件中的测试改编为这种格式。该评估衡量模型在长时间编码会话中能构建多少可用功能,同时报告测试通过率和 API 成本。 在最高推理强度下,GPT‑6 Astra 在 20 种语言上的平均测试通过率达到 87.4%,高于 GPT‑5.6 Sol 的 26.8%。所有受测语言均有提升,包括 COBOL(10.9% 至 72.1%)、OCaml(25.6% 至 85.8%)、Fortran(21.4% 至 83.5%)和 Visual Basic(27.5% 至 89.4%)。Astra 在 20 种语言中的 19 种得分超过 83%,COBOL 仍是最难的,为 72%。 GPT‑6 Luna 提供了一种成本更低的选项,在所有受测语言上也均优于 GPT‑5.6 Sol。在最高推理强度下,其平均测试通过率达到 40.3%。提升包括 COBOL(10.9% 至 17.1%)、OCaml(25.6% 至 34.1%)、Fortran(21.4% 至 31.4%)和 Visual Basic(27.5% 至 41.6%)。这些改进的估计平均 API 成本为每个样本 $0.77,而 GPT‑5.6 Sol 为 $3.48,约低 78%。 C C#C++COBOL Common Lisp Fortran Go Java JavaScript Lean OCaml PHP PowerShell Python Ruby Rust Scala Swift Visual Basic Zig 我们正在 [GitHub⁠(在新窗口中打开)](https://github.com/openai/MultiLangBench) 上发布 MultiLangBench 评估设置和测试套件,以及每种语言的结果。开发者可以查看评估测试的内容,审阅其使用语言的性能,并找出对其自身工作重要的差距。 ## 2\. 迁移 COBOL 代码库 我们的第二项评估要求模型将 COBOL 程序迁移到 C++、Java、Kotlin、Python 或 Rust,同时保留每个程序的功能。 成功的迁移需要理解程序的业务逻辑和 COBOL 的特定语言行为。COBOL 程序通常处理金融交易和业务记录,其中舍入规则和缺失记录处理等细节反映了业务需求。例如,COBOL 的定点算术会丢弃多余的小数位,除非明确要求舍入。若重写时改为对这些值进行舍入,可能运行无错误,却产生不同的财务结果。 为了测试这些行为是否被保留,我们在隐藏测试上评估模型,这些测试要求可观察输出逐字节一致。只有所有测试通过,迁移才被视为成功。这类保持行为的能力对维护和现代化旧系统的团队很有用,但将其应用于生产软件仍需专家审查,并针对系统的业务规则和依赖项进行测试。 在这项评估中,GPT‑6.1 Sol 在低推理强度下达到 86.6% 的准确率,而 GPT‑5.6 Sol 在相同设置下为 5.4%,且 API 成本仅为后者的 61%。在更高推理强度下,准确率达到 97.9%,使该评估接近饱和。这些评估结果展示了在为现代化业务所依赖软件的团队实现更准确、更经济的代码迁移方面的进展。 ## 提升企业软件能力 这些评估,连同开发者和企业客户的直接反馈,帮助我们了解模型在哪些方面正在改进,以及仍在哪些方面存在不足。 我们的目标是帮助模型将软件工程技能应用于更多语言:理解不熟悉的代码库、规划变更、使用构建工具、运行测试以及调试失败。 你可以通过 API 在团队所用语言的编程任务中试用我们的最新模型。你对哪些方面有效、模型在哪些地方存在困难,以及我们接下来应重点关注哪些语言或工作流程的反馈,有助于塑造我们训练和评估模型的方式,从而更好地支持你所依赖的软件。 - [软件与工程](https://openai.com/research/index/?tags=software-engineering) - [GPT](https://openai.com/research/index/?tags=gpt) - [学习范式](https://openai.com/research/index/?tags=learning-paradigms) - [2026](https://openai.com/research/index/?tags=2026) ## 作者 OpenAI ## 正文 · 原文 ## ![Image 1](https://openai.com/_next/static/immutable/media/lab.44a_dymau51y3.webp)\[ COBOL \]\[ OCaml\]![Image 6](https://openai.com/_next/static/immutable/media/laptop.2t2hneg7lf1gz.webp)\[ Fortran\]\[ Visual Basic \]![Image 8](https://openai.com/_next/static/immutable/media/chip.1t86cqmsrm7g2.webp)Advancing coding in low‑resource programming languages Loading… [Audio 1](https://openai.com/index/low-resource-programming-languages) Share 1. Building SQLite from scratch (MultiLangBench) Our latest model advances bring stronger capabilities for building and migrating software in low-resource programming languages such as COBOL, Fortran, and Visual Basic. These languages have less code and documentation available for models to learn from than languages like Python and JavaScript, yet support critical infrastructure and essential services around the world. COBOL [supports⁠(opens in a new window)](https://www.ibm.com/think/topics/cobol) more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, while Fortran is used in [weather forecasting⁠(opens in a new window)](https://repository.library.noaa.gov/view/noaa/71120) and [nuclear reactor safety analysis⁠(opens in a new window)](https://www.nrc.gov/education-regulatory-research/research/computer-codes). Improving AI’s ability to work in these languages could help teams maintain and modernize the software that essential services depend on. This work is guided by feedback from developers and enterprise customers who have told us that coding models often struggle with these languages. Maintaining and modernizing this software can be difficult: documentation may be incomplete, original developers may have left the company, and important business rules may only exist in code. Whether teams are fixing bugs or migrating to another language, they need to preserve the behavior their systems depend on. To measure progress, we developed two evaluations. MultiLangBench tests whether a model can build a SQLite-compatible database engine from scratch across 20 different programming languages. Our COBOL migration evaluation tests whether a model can migrate existing programs to another language while preserving their behavior. Together, they test two complementary skills: building new software and preserving what existing software does. Our latest models show substantial improvements across both evaluations, delivering much higher accuracy at lower cost. On MultiLangBench, GPT‑6 Astra achieves an average test pass rate of 87.4% across 20 programming languages, up from 26.8% for GPT‑5.6 Sol, with improvements in every language tested. GPT‑6 Luna also outperforms GPT‑5.6 Sol, reaching 40.3% at approximately 78% lower estimated API cost. On our COBOL migration evaluation, GPT‑6.1 Sol achieves 86.6% accuracy at low reasoning effort, compared with just 5.4% for GPT‑5.6 Sol, while costing 39% less. At higher reasoning effort, accuracy reaches 97.9%, nearly saturating the evaluation. Together, these results show that our models are becoming significantly better at working with a wide range of programming languages, while also becoming more cost-efficient. ## 1\. Building SQLite from scratch (MultiLangBench) MultiLangBench asks a model to build a SQLite-compatible database engine in a specified programming language. The model must organize a codebase, manage state, handle errors, and add new features while preserving existing functionality. To compare implementations across languages, we use database files as a shared interface. The candidate program receives an initial database and a SQL query, then produces a modified database. We use the reference SQLite implementation to check whether the resulting database has the expected contents. This lets us apply the same tests regardless of the language used to build the engine. We adapted tests from SQLite’s existing test suite to this format. The evaluation measures how much working functionality a model can build over an extended coding session, reporting both test pass rate and API cost. At maximum reasoning effort, GPT‑6 Astra achieves an average test pass rate of 87.4% across the 20 languages, up from 26.8% for GPT‑5.6 Sol. Improvements extend to every language tested, including COBOL (10.9% to 72.1%), OCaml (25.6% to 85.8%), Fortran (21.4% to 83.5%), and Visual Basic (27.5% to 89.4%). Astra scores above 83% in 19 of the 20 languages, with COBOL remaining the hardest at 72%. GPT‑6 Luna offers a lower-cost option that also improves on GPT‑5.6 Sol in every language tested. At maximum reasoning effort, it reaches an average test pass rate of 40.3%. Gains include COBOL (10.9% to 17.1%), OCaml (25.6% to 34.1%), Fortran (21.4% to 31.4%), and Visual Basic (27.5% to 41.6%). These improvements come at an estimated average API cost of $0.77 per sample, compared with $3.48 for GPT‑5.6 Sol, approximately 78% lower. C C#C++COBOL Common Lisp Fortran Go Java JavaScript Lean OCaml PHP PowerShell Python Ruby Rust Scala Swift Visual Basic Zig We are releasing the MultiLangBench evaluation setup and test suite on [GitHub⁠(opens in a new window)](https://github.com/openai/MultiLangBench), along with results for each language. Developers can inspect what the evaluation tests, review performance in the languages they use, and identify gaps that matter for their own work. ## 2\. Migrating a COBOL codebase Our second evaluation asks models to migrate COBOL programs to C++, Java, Kotlin, Python, or Rust while preserving what each program does. Successful migration requires understanding both the program’s business logic and COBOL’s specific language behavior. COBOL programs often process financial transactions and business records, where details such as rounding rules and missing-record handling reflect business requirements. For example, COBOL’s fixed-point arithmetic discards excess decimal digits unless rounding is explicitly requested. A rewrite that rounds those values instead can run without errors while producing different financial results. To test whether these behaviors are preserved, we evaluate our models on hidden tests that require byte-for-byte parity on observable outputs. A migration is considered successful only if all tests pass. Behavior-preserving capabilities like these are useful for teams maintaining and modernizing older systems, though applying it to production software still requires expert review and testing against the system’s business rules and dependencies. On this evaluation, GPT‑6.1 Sol achieves 86.6% accuracy at low reasoning effort, compared with GPT‑5.6 Sol’s 5.4% at the same setting at just 61% of the API cost. At higher reasoning effort, accuracy reaches 97.9%, bringing the evaluation close to saturation. These evaluation results demonstrate progress toward more accurate, affordable code migration for teams modernizing the software their businesses depend on. ## Advancing capabilities for enterprise software These evaluations, alongside direct feedback from developers and enterprise customers, help us see where our models are improving and where they still fall short. Our goal is to help models apply software engineering skills across more languages: understanding unfamiliar repositories, planning changes, using build tools, running tests, and debugging failures. You can try our latest models through the API on coding tasks in the languages your team uses. Your feedback on what works, where the models struggle, and which languages or workflows we should focus on next helps shape how we train and evaluate models to better support the software you depend on. - [Software & Engineering](https://openai.com/research/index/?tags=software-engineering) - [GPT](https://openai.com/research/index/?tags=gpt) - [Learning Paradigms](https://openai.com/research/index/?tags=learning-paradigms) - [2026](https://openai.com/research/index/?tags=2026) ## Author OpenAI