# Google Research 三个月专利起草实验发现AI辅助未必能培养初级律师的专业判断
> 原标题:Does better work always mean better workers?
- 来源:Google Research:Blog
- 发布时间:2026-10-07T00:00:00.000Z
- AIHOT:https://aihot.news/items/tyhn4y95ez3mvj0p7xovpaq61
- 原文:https://research.google/blog/does-better-work-always-mean-better-workers/
## 摘要
Google Research 在 NBER 发表的论文报告了一项三个月随机田野实验,向11家知识产权律所的133名律师随机开放当时未发布的 AI 专利写作助手(现属 Gemini Notebook)。
## 推荐理由
原文用三个月随机田野实验区分了AI辅助产出与独立能力变化,给出初级与资深律师分化的具体证据。
## 正文 · 中文译文
判断力是区分资深专业人士与初级同行的关键。培养判断力需要在工作中投入数千小时的学习,通常是通过与相对常规的工作进行枯燥但具有塑造性的接触,并且往往是在经验丰富的员工指导下完成的。但AI已经在[改变](https://cepr.org/publications/dp21872)在职学习的方式。一方面,[放射学](https://www.insead.edu/faculty-research/publications/working-papers/predictive-ai-can-support-human-learning-while)、[商业问题解决](https://arxiv.org/abs/2608.04198)、[求职者写作](https://arxiv.org/abs/2502.02880)和[法律教育](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6525800)领域的实证研究表明,当AI工具被深思熟虑地嵌入培训工作流程时,经验较少的员工可以提高其独立工作表现。
另一方面,近期针对[软件工程师](https://arxiv.org/abs/2601.20245)、[管理咨询顾问](https://ssrn.com/abstract=4944588)、[高中生](https://www.pnas.org/doi/10.1073/pnas.2422633122)和[临床医生](https://pubmed.ncbi.nlm.nih.gov/40816301/)的实验表明,虽然AI充当了临时外骨骼,能提升即时产出,但一旦移除该工具,这些收益往往无法持续。要理解AI如何侵蚀或增强专业能力,需要三个很少同时具备的要素:(1)将AI持续使用嵌入日常专业工作中,持续数月而非数小时;(2)在AI不可用时对无辅助判断力进行可信评估;(3)由领域专家进行盲评。
在[美国国家经济研究局](https://www.nber.org/)发表的《[AI辅助是增强还是侵蚀专业能力?来自专利撰写三个月实地实验的证据](https://www.nber.org/papers/w35720)》一文中,我们描述了一项实地实验,用于捕捉在知识产权法这一高度专业密集的职业中,因使用AI而发生的短期生产力和长期技能培养两方面的变化。在我们的实验中,我们在11家与Google有常规、非排他性业务往来的知识产权律师事务所的133名律师中,随机分配了一款当时尚未发布的[Google Labs](https://labs.google/) AI专利撰写辅助工具(现已成为[Gemini Notebook](https://notebook.google/)的一部分)的使用权限。每家律所三分之二的律师(“实验”组)获得了该工具的早期使用权,其余律师(“对照”组)接受了一些关于如何使用AI的培训,但在三个月研究期结束前被禁止使用该工具。
## 用AI做出更好的工作
在为期10天的人工智能辅助之后,我们向所有参与的律师发送了一包关于一项假设发明的发明人材料,并要求他们起草一份专利。我们在90天后也做了同样的事情,但使用了另一组模拟的发明人材料。我们在两个时间段都观察到了起草任务中预期的人工智能提升效果。在第10天,获得人工智能工具访问权限使起草得分(由独立法律专家根据包含[李克特量表](https://en.wikipedia.org/wiki/Likert_scale)的评分标准,对五个不同质量维度进行评分)提高了0.34个标准差,相当于在对照组得分中百分位排名上升了10个点;到第90天,这一提升增至0.38个标准差,意味着百分位排名上升了11个点。这些改进在所有质量维度上都表现出显著的一致性。值得注意的是,更好的表现来自低分频率的降低和高分频率的提高,而优秀得分的频率没有变化。从形式上看,获得人工智能访问权限的律师的得分从底部移动到了中下和中等等分位,但卓越得分的数量没有明显变化。这一模式在初级律师中尤为明显,他们质量的提升还伴随着显著的时间节省(例如,在10天任务中,比对照组平均124分钟的速度快了18分钟)。
但专利律师不仅仅起草专利;他们还会相互审查彼此的工作,以发现可能导致专利在法庭上无法执行的关键错误(有时称为“专利亵渎”)——例如,过度主张发明的新颖性或范围。因此,我们在第90天增加了另一项任务,要求受试者对一份包含许多实质性及文体错误的现有假设专利进行红线标注(手动标记并更正)。这项测试任务反映了更资深律师日常使用的专业判断,而他们通常无法奢侈地依赖人工智能。关键的是,虽然治疗组律师被鼓励在起草任务中使用未发布的人工智能工具或任何其他人工智能工具,但没有人被允许在红线标注任务中使用人工智能,这使得他们在该任务上的得分成为衡量其技能发展程度的指标。对于所有任务,无论是起草还是红线标注,我们都与第三方专利专业人士合作,从五个质量维度对提交内容进行评分:(1) 可执行性,(2) 准确性,(3) 策略性模糊(权利要求的战术性界定),(4) 完整性,以及 (5) 清晰度。
## 当人工智能被移除时
当人工智能助手在第90天的红线标注任务中被移除时,这种拉平效应消失了。获得人工智能工具访问权限的律师在这项无辅助任务中比对照组高出0.32个标准差(相当于百分位排名上升9个点),但这种效应完全由资深律师驱动,他们比对照组受试者高出0.45个标准差(上升13个点),而初级律师则没有表现出明显改善。相反,初级律师的得分出现了分化:极低分更多,平庸分更少,良好分更多,而优秀分没有增加。
这些结果对学习意味着什么?获得AI工具使用权限的资深律师,显然在过去三个月里通过使用该工具,从专业判断中收获了显著的价值。但初级律师的情况则更为复杂。部分分数有所提高,暗示AI具备跨越式促进学习的能力。另一些分数则向分布的低端扩散,表明在某些场景下AI能帮助初级律师交付合格的工作,但他们借助机器提升的表现,并未转化为更快地掌握那些让资深专业人士独具价值的专业知识。
考察定性编辑模式,可以洞察不同经验水平的专业人士如何与生成式AI工具互动。在实验组和对照组中,初级律师的提交都遵循僵化的形式主义:他们从上到下按顺序工作,常常在触及主要专利权利要求之前,就把时间耗尽在低风险引言部分的文字编辑上。初级律师还专注于表面层面的同义词替换,而非商业范围的改进。即使初级律师发现了严重缺陷,他们也往往留下描述性评论来诊断问题,而不是执行修订来修复它。由于这种“只诊断不执行”的姿态在未借助AI的对照组初级律师中同样普遍,它代表了一种初级律师的基线缺陷,而三个月依赖AI执行改写并未弥补这一缺陷。
相比之下,资深律师则持续从AI的接触中受益。接受处理的资深律师在修订上花费的时间比初级律师更长,他们跳过低风险的文字,从头重建权利要求,剔除可能无意中缩小法律权利或限制保护范围的语言,并将许多修改与明确的法律原则相配对。在后续访谈中,资深律师描述他们并非将AI输出视为成品,而是视为“逻辑审计员”。它削弱了对现有文字的依恋,迫使他们阐明结构性修改的原因和方式,从而激活并磨砺了他们的基础专业知识。
## 进一步的方向
提升绩效与建立持久的专业判断是不同的目标。尤其对初级专业人士而言,二者可能相互冲突。基础专业知识可能是缺失的一环,使AI辅助的重复练习能够在职业生涯中转化为老练的专业判断。即使模型能力不断进步,这种判断力很可能依然重要:律师将继续在与法官、客户和同事的互动中运用他们的判断力。他们无法在所有场合都依赖AI工具来协助自己。这个AI时代的一个关键挑战是,确保那些旨在今天产出更好工作的工具,不会阻碍初级专业人士成为我们明天所需的专家。
研究专业人士在其工作环境中的表现对研究人员来说是一项挑战。我们的实验仅包含有限数量的专利律师样本,这反映了该领域顶级专业人士高昂的小时计费费率。我们仅对他们进行了为期三个月的观察,与培养持久专业知识所需的数年时间相比,这一窗口期很短。此外,过去一年中模型能力和基线AI熟悉度(以及AI赋能产品在法律领域的渗透率)的快速提升,如果现在重新进行试验,可能会影响我们的结果。这些局限性为进一步研究提供了机会,我们希望在未来几个月内开展其中一些研究。尽管如此,我们工作得出的一个明确结论是,理解AI辅助对专业技能的影响,需要将使用工具的效果与无辅助用户表现的效果区分开来的测试。
## 致谢
_衷心感谢共同作者Josh Martin、Zanna Iscenko、Scott Strand、David Pearl和Melissa Ferere;感谢James Manyika、Ruth Porat、Kent Walker、Josh Woodward、Anu Madgavkar、Daniel Rock和Fabien Curto Millet_ _的指导与支持;感谢Google Labs的Tom Shane、Mauricio Carneiro、Johnny Jacobs、Victor Lavrenko、Yinghao Sun、Samuel Yang、Daniel Nishi、Eric Wang、Kevin Li、Hao Zheng和Franz Och与我们合作开展实验;感谢Kiera Russell提供产品访问权限和可信测试者支持;感谢Steve Gong和Taylor Montgomery提供法律指导和招聘支持;感谢我们的传播、营销和研究博客合作伙伴,包括Kerry McHugh、Zoe Ortiz、Emily Short、Joseph Mack、Esmeralda Cardenas和Leanne Trujillo推动此次发布。_
## 正文 · 原文
Judgment is what sets senior professionals apart from their junior peers. Building judgment requires thousands of hours learning on the job, usually through tedious but formative engagement with relatively routine work, often under guided supervision by experienced workers. But AI is already [changing](https://cepr.org/publications/dp21872) how on-the-job learning works. On the one hand, empirical studies in [radiology](https://www.insead.edu/faculty-research/publications/working-papers/predictive-ai-can-support-human-learning-while), [business problem-solving](https://arxiv.org/abs/2608.04198), [job-seeker writing](https://arxiv.org/abs/2502.02880), and [legal education](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6525800) show that when AI tools are thoughtfully embedded into training workflows, less-experienced workers can improve their independent performance.
On the other hand, recent experiments with [software engineers](https://arxiv.org/abs/2601.20245), [management consultants](https://ssrn.com/abstract=4944588), [high school students](https://www.pnas.org/doi/10.1073/pnas.2422633122), and [clinicians](https://pubmed.ncbi.nlm.nih.gov/40816301/) demonstrate that while AI serves as a temporary exoskeleton that boosts immediate output, those gains often fail to persist once the tool is removed. Understanding how AI can erode or enhance expertise requires three elements rarely found together: (1) sustained AI usage embedded in ordinary professional work over months rather than hours, (2) a credible evaluation of unassisted judgment when AI is unavailable, and (3) blinded grading by domain experts.
In “[Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting](https://www.nber.org/papers/w35720)”, published by the [National Bureau of Economic Research](https://www.nber.org/), we describe a field experiment used to capture changes in both short-term productivity and longer-term skill building that occur as a result of AI usage in the highly expertise-intensive profession of intellectual property law. In our experiment, we randomized access to a then-unreleased [Google Labs](https://labs.google/) AI patent writing assistant tool, now part of [Gemini Notebook](https://notebook.google/), among 133 lawyers at eleven intellectual property law firms that are engaged in regular, non-exclusive business with Google. Two-thirds of lawyers at each firm (“treatment” group) were given early access to the tool, while the remainder ("control" group) were given some training on how to use AI but were withheld from using the tool until after the three-month study period.
## Better work with AI
After 10 days of AI assistance, we sent all participating lawyers a packet of inventor materials for a hypothetical invention and asked them to draft a patent. We did the same after 90 days, with a different set of simulated inventor materials. We saw the expected AI boost in performance on the drafting tasks at both time periods. At 10 days, AI tool access raised drafting scores (graded on a rubric with [Likert scales](https://en.wikipedia.org/wiki/Likert_scale) for five different aspects of quality by independent legal experts) by 0.34 standard deviations, equivalent to a 10-point climb in percentile ranking among control group scores; by 90 days, this improvement increased to 0.38 SD, implying an 11-percentile point increase. These improvements occurred with remarkable consistency across all quality dimensions. Tellingly, better performance came from lower frequency of poor scores and higher frequency of good scores, and no change in the frequency of excellent scores. Formally, scores among lawyers allocated AI access moved from the bottom to the lower-middle and middle quintiles, but there were no noticeable changes in the number of exceptional scores. This pattern was particularly clear for junior lawyers, whose improvements in quality were also matched by significant time savings (18 minutes faster than the control group’s average speed of 124 minutes on the 10-day task, for example).
But patent lawyers don’t just draft patents; they also review each other’s work for critical mistakes (sometimes called “patent profanity”) that can make patents unenforceable in court — for instance, by overclaiming the novelty or scope of an invention. We therefore added another task at 90 days, which required subjects to redline (to manually mark up and correct) an existing hypothetical patent that contained many errors, both substantive and stylistic. This test task mirrors the kind of professional judgment that more senior lawyers routinely use and for which they most often do not have the luxury of relying on AI. Critically, while treatment group lawyers were encouraged to use the unreleased AI tool or any other AI tool on the drafting tasks, no one was allowed to use AI for the redlining task, making their scores on this task a measure of how well their skills had developed. For all tasks, drafting as well as redlining, we worked with third party patent professionals to score submissions on five dimensions of quality: (1) enforceability, (2) accuracy, (3) strategic ambiguity (the tactical scoping of claims), (4) completeness, and (5) clarity.
## When AI is removed
When the AI assistant was removed for the 90-day redlining task, the leveling effect vanished. Lawyers given AI tool access outperformed controls by 0.32 SD on this unassisted task (a 9-point equivalent climb in percentile rankings), but this effect was driven entirely by senior lawyers, who outperformed control subjects by 0.45 SD (a 13-point climb), while juniors showed no discernible improvement. Instead, scores of juniors bifurcated: more very low scores, fewer mediocre ones, more good ones, and no more excellent ones.
What do these results mean for learning? Seniors who had been given AI tool access clearly harvested significant value in terms of their professional judgment from their use of the tool over the previous three months. But the junior-level picture is more nuanced. Some scores improved, hinting at AI’s ability to leapfrog learning. Other scores spread to the lower rungs of the distribution, showing that in some scenarios AI can help juniors deliver adequate work, but their higher machine-assisted performance does not translate into faster acquisition of the expertise that makes senior professionals distinctly valuable.
Examining qualitative editing patterns provides some insight into how professionals of different experience levels engage with generative AI tools. Across both treatment and control groups, junior submissions followed rigid formalism: juniors worked sequentially from top to bottom, frequently exhausting their time copy-editing low-stakes introductory sections before reaching the main patent claims. Juniors also focused on surface-level synonym swapping rather than commercial scope improvement. Even when juniors spotted serious flaws, they often left descriptive comments diagnosing the defect rather than executing the redline to fix it. Because this "diagnose-without-execute" posture was equally common among unassisted control juniors, it represents a baseline junior deficit that three months of relying on AI to execute rewrites did not remediate.
Senior lawyers, by contrast, benefited consistently from AI exposure. Treated seniors spent longer on redlining than juniors, bypassing low-stakes prose to rebuild the claims from scratch, strip out language that could inadvertently narrow legal rights or limit protective scope, and pair many edits with explicit legal doctrines. In follow-up interviews, seniors described treating AI output not as a finished product, but as a "logic auditor". It weakened attachment to existing prose and forced them to articulate the why and how of structural edits, activating and sharpening their foundational expertise.
## Further directions
Improving performance and building durable professional judgment are different goals. For junior professionals in particular, they may be in tension with one another. Foundational expertise may be the missing link that allows AI-assisted repetition to translate into seasoned professional judgment over the course of a career. Such judgment will likely continue to be important even as model capabilities advance: lawyers will continue to use their judgment in their interactions with judges, clients and colleagues. They will not be able to rely on an AI tool to assist them in all settings. A critical challenge of this AI moment is ensuring that tools designed to produce better work today do not prevent junior professionals from becoming the experts we need tomorrow.
Studying professionals in their working environments is a challenge for researchers. Our experiment includes a limited sample of patent lawyers, a reflection of the high hourly billing rate of top-tier professionals in the field. We only observe them over three months, a small window compared with the years they would need to develop durable expertise. Moreover, the rapid advance in both model capabilities and baseline AI familiarity (as well as AI-enabled product penetration in the legal domain) over the past year might affect our results if we re-ran the trial now. These limitations give rise to opportunities for further research, some of which we hope to pursue in the months ahead. Nevertheless, a clear takeaway from our work is that understanding the effect of AI assistance on professional skill requires tests that separate the effect of using tools from the effect of unassisted user performance.
## Acknowledgements
_A huge thank you to co-authors Josh Martin, Zanna Iscenko, Scott Strand, David Pearl and Melissa Ferere; to James Manyika, Ruth Porat, Kent Walker, Josh Woodward, Anu Madgavkar, Daniel Rock, and Fabien Curto Millet_ _for their guidance and support; to Tom Shane, Mauricio Carneiro, Johnny Jacobs, Victor Lavrenko, Yinghao Sun, Samuel Yang, Daniel Nishi, Eric Wang, Kevin Li, Hao Zheng and Franz Och of Google Labs for partnering with us on the experiment; to Kiera Russell for product access and trusted tester support; to Steve Gong and Taylor Montgomery for legal guidance and recruitment support; and to our Comms, Marketing, and Research Blog partners including Kerry McHugh, Zoe Ortiz, Emily Short, Joseph Mack, Esmeralda Cardenas and Leanne Trujillo for driving the launch._