Goodfire 为 Kimi K3 和 GLM 5.3 训练并部署生产级网络安全监控器

来源:Goodfire Research(网页) 2026年10月7日 00:00 AIHOT 评分:72 精选
摘要:Goodfire Research 为 Kimi K3 和 GLM 5.3 构建基于激活探针加 LLM judge 的监控级联,并部署到生产推理栈。
推荐理由:原文给出监控级联的完整数据、成本和部署细节,读者可以据此评估实时监控开放模型智能体的可行做法。
# Goodfire 为 Kimi K3 和 GLM 5.3 训练并部署生产级网络安全监控器 > 原标题:Training and Deploying Production Cyber Monitors on Kimi K3 - 来源:Goodfire Research - 发布时间:2026-10-07T16:00:00.000Z - AIHOT:https://aihot.news/items/ja19mtm89yojd9w2u67qp1gqv - 原文:https://www.goodfire.com/research/production-cyber-monitors-on-kimi-k3 ## 摘要 Goodfire Research 为 Kimi K3 和 GLM 5.3 构建基于激活探针加 LLM judge 的监控级联,并部署到生产推理栈。 ## 推荐理由 原文给出监控级联的完整数据、成本和部署细节,读者可以据此评估实时监控开放模型智能体的可行做法。 ## 正文 · 中文译文 ## 引言 网络安全是监控 AI 智能体最困难的场景之一,因为良性行为与恶意行为往往看起来相似:审计代码库漏洞与利用漏洞所涉及的任务之间存在大量重叠。要区分二者,意味着需要在整个轨迹中追踪意图,而轨迹可能长达数百万个 token。 尽管存在这一困难,网络安全监控仍然至关重要,近期涉及智能体编码模型的事件就证明了这一点。 显而易见的解决方案是让 LLM 评判器读取每一轮对话,但这样做太慢、太昂贵,无法与智能体实时并行运行。因此在实际中,监控是异步的:等到有害轨迹被标记出来时,智能体早已采取了行动。 为解决这一问题,**我们为 Kimi K3 和 GLM 5.3 构建了基于探针的网络监控器,并将其部署在生产推理栈上。**正如我们在[上一篇文章](https://www.goodfire.com/blog/probe-monitors-101)中所述,探针读取模型的内部激活,充当廉价的第一道过滤器。它只将可疑的交互发送给 LLM 评判器。与对每一轮都运行评判器相比,这种级联方案: - **更准确:**它在帕累托意义上优于评判器,并在我们的工作点上达到与其相当的召回率(召回率 93%,假阳性率 5.5%)。 - **成本降低约 50 倍**,每百万次交互不到 200 美元。 - **速度更快得多,**保持基线推理 token 吞吐量,且不增加延迟。 - **将通用越狱降至 0%**,这是在独立评估者的静态红队攻击测试集上取得的结果。 这些结果使得实时监控智能体 rollout 变得切实可行,意味着有风险的网络操作可以在任何工具被调用之前就被阻止。在本文中,我们分享构建这些监控器的细节,以及我们对监控栈所做的一些推理优化,正是这些优化实现了如此高效的性能。 开放模型并不附带完整的安全栈;我们认为,这样的监控设置对于任何在智能体场景中提供开放模型服务的人来说,都是一个实用的默认选择。如果你提供或基于开放模型进行构建,并希望部署这样的监控器,[请联系我们](https://www.goodfire.com/contact-us)。 ## 我们的监控方案 ### 数据 智能体会话可能跨越数十万到数百万个 token,有害行为可能在任何时刻出现。为了对这类生产数据进行建模,我们在网络和非网络(例如 SWE、工具使用等)环境中模拟具有各种有害和良性目标的用户。为此,我们使用 Prime Intellect 的 verifiers、Prime Sandboxes 和 Tunnels 收集了一部分数据,使我们能够在无网络访问的情况下,以安全的方式在 Kimi K3 推理上运行网络攻击性任务以及良性任务。 为了明确我们的监控器应标记哪些行为,我们使用了一套精确的策略来描述不受欢迎的网络活动,采用具体因素来区分善意任务(例如有边界的漏洞复现)与恶意任务(例如漏洞利用)。模拟用户可以在会话过程中的任何时刻引入或升级有害意图,包括在最初是良性请求之后。例如,大多数任务一开始只有最低限度的良性探索(例如搜索文件目录),只有在经过一系列操作之后,有害活动才开始发生。这样就从相同的基础任务中产生了多样化的交互。 ### 架构:监控级联 理想情况下,我们可以通过让另一个能力相当的 LLM 评判器按照滥用政策对每一轮进行分类,来监控目标智能体。然而,随着 rollout 变长,使用另一个模型会导致极高的成本和计算开销:根据 OpenRouter 的费率,使用 Opus 5 监控 Kimi K3 rollout 中的一百万轮将花费约 20 万美元,而通过 Kimi K3 自身则约为 3 万美元。 为了解决这个问题,我们利用了这样一个事实:在长时间运行的智能体会话中,即使预期目标是恶意的,智能体的大部分活动也是良性的——因此许多智能体轮次甚至无需由评判器处理。挑战在于如何知道_哪些_轮次值得被评判器查看。为此,与我们自己过去的工作以及他人近期的工作一致关于基于探针的监控的相关思路,参见先前工作:[Building Production-Ready Probes for Gemini](https://arxiv.org/abs/2601.11516);[Detecting High-Stakes Interactions with Activation Probes](https://arxiv.org/abs/2506.10805);[Constitutional Classifiers++](https://arxiv.org/abs/2601.04603)。其中,轮级分类过程已在 Constitutional Classifiers++ 中尝试过,但仅针对聚焦于 CBRN 威胁的短 rollout 任务;Gemini 的工作仅关注输入监控,因此忽略了上述针对完整输入-输出监控的技术挑战;而高风险检测论文则聚焦于较小的模型。,我们使用探针来分摊使用 LLM 评判器作为监控器的成本。 具体而言,我们设计了一种新颖的探针架构,专门用于捕获以局部化(即局部化到轮次或 rollout 中的特定片段)和分布式(即在整个轮次中持续存在)两种方式表示的行为,同时运行效率高,如下文所述。 该探针在**监控级联**中充当第一道高效、始终开启的检测器:它有助于判断输入、工具调用或输出是否高风险,如果探针读数超过校准阈值,则将该 rollout 升级到 LLM 评判器进行进一步评估。经过校准后,我们发现我们的探针在一组留出的智能体和聊天交互上实现了 0% FNR 和 95% TNR,确保 LLM 评判器仅被调用来处理有害活动。这也使我们能够通过使用更快但能力可能较弱的评判器来降低成本。 ![Flowchart: an incoming request goes to Kimi K3, whose internal activations feed the probes inside the monitor cascade. Below threshold, the interaction continues; above threshold, it goes to the judge, which either refuses the request or continues the interaction.](https://static.goodfire.ai/cyber-monitors/monitor_cascade.webp) 图 1:我们的监控流水线的高层概览。我们的探针在推理服务器生成 token 时从目标智能体(此处为 Kimi K3)接收内部激活。探针决策根据阈值被标记给 LLM 评判器,评判器可以拒绝请求或继续交互。 ### 推理优化 对探针运行一次前向传播远比 LLM 评判器便宜。然而,要确保它在生产推理服务器内运行时不会增加_延迟_,需要仔细的工程工作。像 SGLang 和 vLLM 这样的现代推理服务器通过一起处理许多请求、重用先前的前缀计算以及使用推测解码一次生成多个候选 token 来实现高吞吐量。监控器必须在保持这些吞吐量收益的同时,使分数与正确的 token 和会话保持一致。 在我们[之前关于可解释性基础设施的文章](https://www.goodfire.com/blog/interpretability-infra-at-frontier-scale)中,我们描述了如何收集激活值以供分析。[Goodfire,“前沿规模的可解释性基础设施:从万亿参数模型中采集激活值。”](https://www.goodfire.com/blog/interpretability-infra-at-frontier-scale)对于本文所述的实时监控,我们扩展了 SGLang,使其能够在服务引擎内部直接计算探针分数。 我们在运行探针的同时做了多项优化以保持推理吞吐量,包括为探针逻辑实现自定义内核。我们还通过基数树、DSpark 推测器和预填充-解码分离来维护探针分数,以避免额外的重复计算。在服务多个 Kimi K3 模型副本时,每个 worker 都会运行探针并为其活跃请求维护历史记录。我们扩展了 [SMG 路由器](https://github.com/smg-project/smg),使其在请求转换过程中保留我们的监控配置,并将其传递给选定的 worker。监控事件随响应流一同返回,使下游消费者无论请求由哪个副本处理都能处理这些事件。 ![Table of throughput change with monitoring relative to serving without probes, median of three repeats. Output monitor / input plus output monitor: short workload +1.0% / −1.1%, medium +0.2% / −1.1%, shared prefix −0.6% / −1.4%, long −0.3% / −1.1%.](https://static.goodfire.ai/cyber-monitors/throughput.webp) 图 2:在四种不同的推理工作负载下,推理期间运行我们的探针与默认 SGLang 相比的吞吐量差异。我们看到吞吐量下降不到 2%,这在我们吞吐量测试的噪声范围内。我们分别测量了仅输出 token 以及输入和输出 token 上启用探针时的吞吐量。 总体而言,我们发现在推理期间运行探针在多种不同工作负载下都不会增加额外开销。在四种推理工作负载中,与默认 SGLang 服务相比,启用探针后吞吐量保持在基准测试的噪声阈值之内。 ![Bar chart of mean added latency per turn: LLM judge with reasoning 2,750 ms (bar truncated), LLM judge 160 ms, our cascade 42 ms, and our cascade without reasoning 2 ms.](https://static.goodfire.ai/cyber-monitors/efficiency_latency_linear.webp) 图 3:监控为每个智能体轮次增加的平均时间。通过仅将标记的交互发送给评判模型,我们的级联方案与评判每一轮相比将这一延迟降低了 50 倍,并且相对于默认 SGLang 服务仅增加极少的延迟。 我们还评估了监控级联所增加的延迟,并将其与审查每一轮的 LLM 评判模型进行比较。在不使用评判推理的情况下,我们的级联方案将每轮平均增加的延迟从 160 毫秒降至 2 毫秒;在使用推理的情况下,将延迟从 2,750 毫秒降至 42 毫秒。与评判每一轮相比,两种配置都将平均监控延迟降低了 50 倍以上。 ## 结果 ### 评估我们的监控器 我们在一个内部数据集上评估我们的网络滥用监控器,该数据集包含 2.4K 个聊天和智能体会话,共计超过 60K 轮,并以 Claude Opus 5 的标签作为参考。为了生成逼真的交互,我们以两种方式对用户进行建模。我们使用 Petri[Anthropic,“Petri:一个加速 AI 安全研究的开源审计工具。”](https://www.anthropic.com/research/petri-open-source-auditing)——一个开源自动化红队测试库——来模拟对话和工具响应,并使用我们自己的用户模拟来与在改编自 ExploitBench 等基准的真实环境中运行的智能体进行交互。这使我们能够从相同的网络任务中收集有害和无害的 rollout。我们还纳入了来自 WildChat 网络特定子集的真实聊天交互,以及来自现成 SWE、Coding 和 DevOps 任务的无害智能体 rollout。 我们报告了所设计部署方案中的召回率和监控成本。检测比较使用 5.5% 的良性会话中断预算。成本和延迟比较使用能够达到至少 90% 召回率、同时最多中断 10% 良性会话的最廉价配置。如果监控器至少标记一次良性会话以供审查,则该良性会话计为被中断。 ![Recall versus benign sessions interrupted (0–10%) for LLM judge with and without reasoning and our cascade with and without reasoning; our curves sit at or above the judges', and both cascades reach 93% recall at 5.5% interruption.](https://static.goodfire.ai/cyber-monitors/roc.webp) ![Grouped bar charts by benchmark. Recall: Petri cyber misuse about 90%, Petri policy adversarial 95–96%, ExploitBench 92%, WildChat cyber subset 97% for all four monitors. False-positive rate: mostly 3–8%, ExploitBench 13–18%, and 0% on benign SWE and coding agents.](https://static.goodfire.ai/cyber-monitors/benchmark_decomposition.webp) 图 4:我们的级联方案可检测出约 93% 的有害会话,同时最多中断 5.5% 的良性会话,无论是否使用 judge reasoning。上:召回率与良性会话中断率的关系,将我们的级联方案与逐轮审查的 judge 进行比较。下:按基准细分后的相应召回率和假阳性率。 总体而言,我们发现我们的监控级联是**对单独 LLM judge 的帕累托改进**,即我们的监控在任何中断量下都优于仅使用 judge。在 5.5% 的良性会话中断率下,我们发现我们的监控在有和没有 judge reasoning 的情况下,对有害会话均达到约 93% 的召回率,并且对 0% 的非网络流量进行标记,使我们的监控可用于生产流量。我们还给出了结果的基准级分解,发现我们的监控流水线在 agentic 和聊天数据上均准确。我们发现,标准线性探针的召回率比我们的最终方案低约 5%,而非线性探针在我们的分布外评估集上损失高达 30% 的召回率。同时,n-gram 基线需要标记几乎所有的轮次才能达到这一召回率。 ![Bar chart of judge cost per million turns: LLM judge with reasoning $5,420, LLM judge $3,166, our cascade $185, our cascade without reasoning $65.](https://static.goodfire.ai/cyber-monitors/judge_cost_linear.webp) ![Bar chart of share of compute spent on monitoring: LLM judge with reasoning 20.5%, LLM judge 12.0%, our cascade 0.70%, our cascade without reasoning 0.24%.](https://static.goodfire.ai/cyber-monitors/efficiency_compute_share.webp) 图 5:每百万次交互的 judge 成本(左)和用于监控的总计算占比(右),将我们的探针加 judge 级联与逐轮审查的 judge 进行比较。与通过 API 调用 judge 相比,我们的监控成本要低得多。在固定的推理计算预算内,我们的监控将用于监控的占比最多降低 50 倍。 为了进一步具体说明上述成本优势,我们报告了我们为定义级联流水线而优化的 LLM judge 的成本,以及在 LLM judge 之前运行探针的总体成本。基于 OpenRouter 上为我们评估套件记录的推理成本,我们的监控级联在无 judge reasoning 时每百万 agent 轮次花费 65 美元,在有 reasoning 时花费 185 美元。相对于使用相应 LLM judge 审查每一轮,**我们的监控在有 reasoning 时将 judge 成本降低约 29 倍,在无 reasoning 时降低 49 倍**。 在实践中,人们不会通过第三方 API 提供监控,而是使用本地 judge 作为推理计算的一部分,同时监控目标模型的多个副本。我们的结果表明,在有限的计算量下,我们的监控允许我们将**超过 99% 的计算用于服务目标模型(Kimi K3),而仅使用不到 1% 的计算运行监控,且不损失可监控性**;同时,若单独运行 LLM judge,则需要分配高达 20% 的计算用于监控,才能监控目标模型产生的每一轮。 有趣的是,我们发现我们的流水线也可以仅通过离策略数据来构建监控器。具体而言,我们使用最初为 Kimi K3 开发的数据,为 GLM 5.3 设计了一个监控流水线,发现在 Kimi 的 rollout 上评估时,我们得到了一个能力大致相当的 GLM 监控器,而针对使用 GLM 设计的在策略评估数据集,监控器仅略差一些。图表见[附录](https://www.goodfire.com/research/production-cyber-monitors-on-kimi-k3#on-vs-off-policy)。 ### 比较不同的监控系统 一个有用的监控器需要在阻止有害活动的同时,允许合法工作继续进行。我们运行了完整轮次的智能体和聊天会话,将带有和不带有我们监控器的 Kimi K3 和 GLM 5.3 与几个前沿模型系统进行了比较。 加入我们的监控器后,Kimi K3 的有害会话阻止率从大约 29% 提升到 96%,同时其无害会话中断率从约 5% 增加到 9%。对于 GLM 5.3,有害会话阻止率从大约 36% 上升到 91%,而无害会话中断率从约 10% 增加到 15%。两个受监控的模型都接近我们所评估的前沿系统中最高的有害会话阻止率,同时中断的无害会话要少得多。 ![Scatter plot of harmful versus harmless block/refusal rate. Arrows move Kimi K3 from about 29% to 96% harmful blocking and GLM 5.3 from about 36% to 91% when our monitor is added, both at low harmless block rates; frontier models with their monitors sit further right, at higher harmless block rates.](https://static.goodfire.ai/cyber-monitors/monitor_comparison.webp) 图 6:有害和无害会话的阻止或拒绝率。箭头显示了向 Kimi K3 和 GLM 5.3 添加我们的监控器后的效果。越靠近左上角的点,阻止的有害活动越多,同时允许更多无害活动继续进行。 我们还与[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev)进行了比较,这是一个可以作为廉价评判者的决策模型。在固定的 5.5% 良性会话中断率下,Jev 达到 86% 的召回率,而我们的非推理级联为 93%,推理级联为 93%。也就是说,单独运行时,Jev 的召回率更差。此外,我们的非推理级联每百万轮次使用约 65 美元的评判计算成本,每轮增加 2 毫秒延迟,而 Jev 为 600 美元和 200 毫秒。 ![Three bar charts comparing Jev with our cascades without and with reasoning: recall at 5.5% benign sessions interrupted (86%, 93%, 93%), judge cost per million exchanges ($600, $65, $185), and mean added latency per turn (200 ms, 2 ms, 42 ms).](https://static.goodfire.ai/cyber-monitors/jev-recall-cost-latency.webp) 图 7:我们的级联比 Jev 捕获更多有害会话,同时评判成本更低、附加延迟更小。左:在 5.5% 良性会话中断预算下的召回率。中:每百万次交互的评判成本。右:每轮平均附加延迟。对于成本和延迟比较,我们选择每个级联中最便宜的配置,该配置在中断至多 10% 良性会话的同时达到至少 90% 的召回率。 最大的差异出现在长上下文或对抗性框架的上下文中:当有害提示嵌入在良性框架中时,Jev 仅捕获 12%,同时在其 32K token 窗口之外,还会越来越多地中断良性 bug 复现会话。我们的监控器在这些情况下表现更好,并且在大多数会话长度下保持接近零中断。 ![Left: share of benign-framed harmful prompts caught: Jev 12%, our cascade without reasoning 52%, with reasoning 100%. Right: benign bug-reproduction sessions interrupted versus median session length from 30K to 150K tokens; Jev rises from 0% to about 22% past its 32K-token window, while ours stays at 0% (the no-reasoning variant reaches about 5% at the longest length).](https://static.goodfire.ai/cyber-monitors/jev-robustness.webp) 图 8:检测嵌入在良性框架中的有害提示(左)以及随着会话长度增加而被中断的良性 bug 复现会话(右)。虚线标记了 Jev 的 32k token 上下文窗口。Jev 对对抗性提示的鲁棒性较差,并且随着上下文增加表现更差。 ### 外部红队测试结果 为了在内部评估之外测试监控器,我们请[FAR.AI](https://far.ai/)对部署的系统进行红队测试。他们的评估对带有和不带有我们监控器的 Kimi K3 应用了一组固定的越狱策略,评估了可在许多交互中迁移的通用攻击以及在单个提示上成功的非通用攻击。用 FAR.AI 的话来说: “[FAR.AI](https://far.ai/) 在两天时间内对 Goodfire 的 Kimi K3 监控器进行了初步测试,在聊天场景中应用了一组非自适应的静态越狱攻击组合,用于进攻性网络滥用。从最初 1500 个候选越狱中,测试选出了 140 个在我们没有监控器的情况下对基础模型另一版本有效的越狱。将这 140 个越狱分别在对应不同攻击者目标的 8 个单独交互上进行测试后,Goodfire 的监控器将引发的漏洞从 66 个通用越狱和 700 个总越狱交互减少到 0 个通用越狱和 18 个越狱交互。” 也就是说,在没有我们的监控器时,通用越狱攻击成功率在不同推理努力设置下为 12% 到 53%。启用监控器后,140 个策略中没有一个成功产生通用越狱。监控器还将 1,120 个非通用交互上的单个提示成功率从至少 9–40% 降低到 0.1–1.6%,具体取决于推理努力。因此,虽然确实存在一些成功的越狱_交互_,但监控器在很大程度上能够抵御这一广泛、静态的攻击组合。 我们计划继续与 FAR.AI 合作,对我们的监控器进行进一步测试和强化。 ![Two bar charts of attack success rate by reasoning effort (None, Low, High, Max), without and with our monitor. (a) Universal jailbreaks over 140 strategies: 53%, 39%, 47%, and 12% without the monitor, 0% in every setting with it. (b) Non-universal jailbreaks over 1,120 interactions: at least 40%, 29%, 35%, and 9% without the monitor, versus 0.5%, 0.2%, 1.6%, and 0.1% with it.](https://static.goodfire.ai/cyber-monitors/external_redteam.webp) 图 9:FAR.AI 在不同推理努力设置下进行的独立红队测试。(a) 我们的监控器在 140 个测试策略上将通用越狱成功率降至零。(b) 在 1,120 个非通用交互中,攻击成功率降至 0.1–1.6%。柱状图比较了启用和未启用监控时的攻击成功率;受监控的攻击只有在交互有害且监控器未标记它时才算成功。 ## 结论 我们为 Kimi K3 和 GLM 5.3 设计的探针加评判器级联监控器,在我们测试的每一种中断预算下,都优于每一轮都运行 LLM 评判器。它的成本约低 50 倍,对推理吞吐量没有可测量的影响,并且在 FAR.AI 的红队测试中,它将通用越狱降至零。 由于它如此便宜,因此可以同步运行:风险操作会在智能体调用工具之前被捕获,而不是事后才被发现。该方案还仅使用在 Kimi K3 上收集的数据就迁移到了 GLM 5.3,因此你不需要为每个模型都建立新的数据管道。 开放模型并不附带完整的安全栈。我们认为,对于任何在智能体环境中提供这些模型的人来说,这样的监控设置是一个实用的默认选择。如果你提供或基于开放模型进行构建,并希望部署这样的监控器,[请联系我们](https://www.goodfire.com/contact-us)。 ## 正文 · 原文 ## Introduction Cybersecurity is one of the hardest settings for monitoring AI agents, because benign and malicious behavior often look similar: there’s significant overlap between the tasks involved in auditing a codebase for vulnerabilities vs. exploiting it. Distinguishing them means tracking intent across the whole trajectory, which can stretch to millions of tokens. Despite this difficulty, cybersecurity monitoring is critical, as demonstrated by recent incidents involving agentic coding models. The obvious solution is to have an LLM judge read every turn, but that's too slow and expensive to run alongside the agent in real time. So in practice monitoring is asynchronous: by the time a harmful trajectory is flagged, the agent has already acted. To address this, **we built probe-based cyber monitors for Kimi K3 and GLM 5.3 and deployed them on a production inference stack.** As we described in our [previous post](https://www.goodfire.com/blog/probe-monitors-101), a probe reads the model's internal activations and acts as a cheap first-line filter. It sends only suspicious exchanges to an LLM judge. Compared with running the judge on every turn, this cascade: - **is more accurate:** it Pareto-dominates the judge and matches its recall at our operating point (93% recall at 5.5% FPR). - **costs ~50x less**, at under $200 per million exchanges. - **is much faster,** maintaining baseline inference token throughput and adding no latency. - **reduces universal jailbreaks to 0%** on an independent evaluator’s static battery of red-teaming attacks. These results make it practical to monitor agentic rollouts in real time, meaning that risky cyber actions can be blocked before any tool is called. In this post, we share details on how we built these monitors, as well as some of the inference optimizations to our monitoring stack that enable such efficient performance. Open models don't come with a full safety stack; we think a monitoring setup like this is a practical default for anyone serving them in agentic settings. If you serve or build on open models and want to deploy monitors like these, [get in touch](https://www.goodfire.com/contact-us). ## Our Monitor Recipe ### Data Agentic sessions can span hundreds of thousands to millions of tokens, and harmful behavior can emerge at any point. To model production data of this nature, we simulate users with various harmful and benign goals within both cyber and non-cyber (e.g., SWE, tool-use, etc.) environments. For this purpose, we collected a portion of our data using Prime Intellect’s verifiers, Prime Sandboxes, and Tunnels, allowing us to run Kimi K3 inference on cyber offensive as well as benign tasks without network access in a secure fashion. To specify which behaviors our monitor should flag, we use a precise policy that describes undesirable cyber activity, using concrete factors that distinguish well-intended tasks (e.g., bounded vulnerability reproduction) from ill-intended ones (e.g., vulnerability exploitation). The simulated user can introduce or escalate harmful intent at any point throughout the session, including after an initially benign request. For example, most tasks start with a minimal level of benign exploration (e.g., searching a file directory), and it’s only over a course of actions that harmful activity starts to take place. This produces a diverse set of interactions from the same base tasks. ### Architecture: a monitor cascade Ideally, we could monitor the target agent by having another equally capable LLM judge classify each turn in accordance with a misuse policy. However, as rollouts get long, using another model can result in extremely high costs and compute overhead: based on OpenRouter rates, monitoring one million turns from Kimi K3 rollouts using Opus 5 would cost on the order of $200K, or via Kimi K3 itself ~$30K. To address this problem, we exploit the fact that in a long-running agentic session, most of the agent’s activity is benign, even if the intended goal is harmful—so many agent turns need not even be processed by the judge. The challenge lies in knowing _which_ turns warrant being looked at by the judge. To this end, in line with our own past work and recent work from othersSee prior work for related ideas on probe-based monitoring: [Building Production-Ready Probes for Gemini](https://arxiv.org/abs/2601.11516); [Detecting High-Stakes Interactions with Activation Probes](https://arxiv.org/abs/2506.10805); [Constitutional Classifiers++](https://arxiv.org/abs/2601.04603). Amongst these, the turn-level classification process has been tried in Constitutional Classifiers++, but only for short rollout tasks focused on CBRN threats; the Gemini work focused on only input monitoring and hence ignores the technical challenges noted above for full input-output monitoring; and the high-stakes detection paper focuses on smaller models., we use probes to amortize the cost of using an LLM judge as a monitor. In particular, we design a novel probing architecture that is specifically designed for capturing behaviors represented in both a localized (i.e., localized to a specific chunk in a turn or rollout) and distributed (i.e., persistently present over an entire turn) manner, while also being efficient to run, as discussed in the section below. The probe acts as the first-line, efficient, always-on detector in a **monitor cascade**: it helps read whether an input, tool call, or output is high-stakes, and if the probe readout exceeds a calibrated threshold, the rollout is escalated to the LLM judge for further evaluation. After calibration, we find our probe achieves 0% FNR and 95% TNR on a held-out set of agentic and chat interactions, ensuring that the LLM judge is only called to process harmful activity. This also allows us to reduce cost by using a faster, even if less capable, judge. ![Flowchart: an incoming request goes to Kimi K3, whose internal activations feed the probes inside the monitor cascade. Below threshold, the interaction continues; above threshold, it goes to the judge, which either refuses the request or continues the interaction.](https://static.goodfire.ai/cyber-monitors/monitor_cascade.webp) Figure 1: A high level overview of our monitoring pipeline. Our probes receive internal activations from the target agent (Kimi K3 in this case) as the inference server produces tokens. Probe decisions are flagged based on a threshold to the LLM judge, which can refuse the request or continue the interaction. ### Inference Optimizations Running a forward pass on a probe is far less expensive than an LLM judge. However, making sure it adds no _latency_ while running inside a production inference server requires careful engineering work. Modern inference servers like SGLang and vLLM achieve high throughput by processing many requests together, reusing earlier prefix computations, and generating several candidate tokens at a time with speculative decoding. The monitor must preserve these throughput gains while keeping scores aligned with the correct tokens and sessions. In our [previous interpretability infrastructure post](https://www.goodfire.com/blog/interpretability-infra-at-frontier-scale), we described how to collect activations for analysis.[Goodfire, “Interpretability Infrastructure at Frontier Scale: Harvesting Activations from a Trillion-Parameter Model.”](https://www.goodfire.com/blog/interpretability-infra-at-frontier-scale) For the live monitoring described in this post, we extended SGLang to compute probe scores directly inside the serving engine. We made several optimizations to preserve inference throughput while running our probes, including the implementation of custom kernels for our probe logic. We also maintain probe scores through the radix tree, DSpark speculator, and prefill-decode disaggregation to avoid additional recomputation. When serving multiple Kimi K3 model replicas, each worker runs the probes and maintains histories for its active requests. We extended the [SMG router](https://github.com/smg-project/smg) to preserve our monitoring configuration through request conversion and pass it to the selected worker. Monitoring events are returned with the response stream, allowing downstream consumers to handle events regardless of which replica served the request. ![Table of throughput change with monitoring relative to serving without probes, median of three repeats. Output monitor / input plus output monitor: short workload +1.0% / −1.1%, medium +0.2% / −1.1%, shared prefix −0.6% / −1.4%, long −0.3% / −1.1%.](https://static.goodfire.ai/cyber-monitors/throughput.webp) Figure 2: Difference in throughput when running our probes during inference compared to default SGLang, across four different inference workloads. We see a less than 2% decrease, which is within noise for our throughput testing. We measure throughput with probes on output tokens alone, and on both input and output tokens. Overall, we find that running our probes during inference adds no overhead across several different workloads. Across four inference workloads, enabling the probes maintained throughput within the noise threshold of our benchmarks relative to default SGLang serving. ![Bar chart of mean added latency per turn: LLM judge with reasoning 2,750 ms (bar truncated), LLM judge 160 ms, our cascade 42 ms, and our cascade without reasoning 2 ms.](https://static.goodfire.ai/cyber-monitors/efficiency_latency_linear.webp) Figure 3: Average time added to each agent turn by monitoring. By sending only flagged exchanges to the judge, our cascade reduces this delay by 50× compared with judging every turn and adds very little latency to default SGLang serving. We also evaluate the added latency of our monitor cascade, comparing it with an LLM judge reviewing every turn. Without judge reasoning, our cascade reduces average added latency per turn from 160 ms to 2 ms; with reasoning, it reduces latency from 2,750 ms to 42 ms. Both configurations reduce average monitoring delay by more than 50× compared with judging every turn. ## Results ### Evaluating Our Monitors We evaluate our cyber-misuse monitors on an in-house dataset of 2.4K chat and agentic sessions comprising over 60K turns, using Claude Opus 5’s labels as reference. To generate realistic interactions, we model users in two ways. We use Petri[Anthropic, “Petri: An open-source auditing tool to accelerate AI safety research.”](https://www.anthropic.com/research/petri-open-source-auditing), an open-source automated red-teaming library, to simulate conversations and tool responses, and our own user simulation to interact with agents operating in real environments adapted from benchmarks such as ExploitBench. This gives us the ability to collect both harmful and harmless rollouts from the same cyber tasks. We also include real-world chat interactions from a cyber-specific subset of WildChat, alongside benign agentic rollouts from off-the-shelf SWE, Coding, and DevOps tasks. We report recall and monitoring cost over the rollouts designed. Detection comparisons use a benign-session interruption budget of 5.5%. Cost and latency comparisons use the cheapest configuration achieving at least 90% recall, while interrupting at most 10% of benign sessions. A benign session counts as interrupted if the monitor would flag it for review at least once. ![Recall versus benign sessions interrupted (0–10%) for LLM judge with and without reasoning and our cascade with and without reasoning; our curves sit at or above the judges', and both cascades reach 93% recall at 5.5% interruption.](https://static.goodfire.ai/cyber-monitors/roc.webp) ![Grouped bar charts by benchmark. Recall: Petri cyber misuse about 90%, Petri policy adversarial 95–96%, ExploitBench 92%, WildChat cyber subset 97% for all four monitors. False-positive rate: mostly 3–8%, ExploitBench 13–18%, and 0% on benign SWE and coding agents.](https://static.goodfire.ai/cyber-monitors/benchmark_decomposition.webp) Figure 4: Our cascades detect approximately 93% of harmful sessions while interrupting at most 5.5% of benign sessions, with or without judge reasoning. Top: recall versus benign-session interruption rate, comparing our cascades with judges reviewing every turn. Bottom: the corresponding recall and false-positive rates broken down by benchmark. In general, we find that our monitor cascade is a **Pareto improvement over the LLM judge on its own**, i.e., our monitor performs better than just the judge at any amount of interruption. At a 5.5% benign-session interruption rate, we find our monitor reaches approximately 93% recall with and without judge reasoning on harmful sessions and flags 0% of non-cyber traffic, making our monitors viable for production traffic. We also give a benchmark-level decomposition of our results, finding our monitoring pipeline is accurate on both agentic and chat data.We find that standard linear probes achieve roughly 5% lower recall than our final recipe, while nonlinear probes lose up to 30% recall on our out-of-distribution evaluation sets. Meanwhile, an n-gram baseline requires flagging almost all turns to reach this recall rate at all. ![Bar chart of judge cost per million turns: LLM judge with reasoning $5,420, LLM judge $3,166, our cascade $185, our cascade without reasoning $65.](https://static.goodfire.ai/cyber-monitors/judge_cost_linear.webp) ![Bar chart of share of compute spent on monitoring: LLM judge with reasoning 20.5%, LLM judge 12.0%, our cascade 0.70%, our cascade without reasoning 0.24%.](https://static.goodfire.ai/cyber-monitors/efficiency_compute_share.webp) Figure 5: Judge cost per million exchanges (left) and share of total compute spent on monitoring (right), comparing our probe-plus-judge cascades with judges reviewing every turn. Our monitors are much cheaper in cost compared to making judge calls over an API. Within a fixed inference compute budget, our monitors reduce the share spent on monitoring by up to 50×. To further concretize the cost benefits above, we report the cost of the LLM judge we optimized for defining our cascade pipeline and the overall cost of running our probe before the LLM judge. Based on inference cost documented on OpenRouter for our evaluation suite, our monitoring cascade costs $65 without judge reasoning, and $185 with reasoning per one million agent turns. Relative to reviewing every turn with the corresponding LLM judge, **our monitor reduces judge cost by about 29× with reasoning and 49× without reasoning**. In practice, one would not serve a monitor via a third-party API, and instead use a local judge to simultaneously monitor multiple replicas of the target model as a part of inference compute. Our results show that given a finite amount of compute, our monitors allow us to spend **\>99% of that compute for serving the target model (Kimi K3), and using only <1% compute for running the monitor at no loss of monitorability**; meanwhile, with an LLM judge run by itself, one would need to allocate up to 20% compute for monitoring in order to monitor every turn produced by the target model. Interestingly, we find our pipeline can also be used for building monitors via just off-policy data. In particular, we design a monitoring pipeline for GLM 5.3 using the data we originally developed for Kimi K3, finding we get an approximately equally capable GLM monitor when evaluated on Kimi rollouts, and only a marginally worse monitor for on-policy evaluation dataset designed using GLM. See [the appendix](https://www.goodfire.com/research/production-cyber-monitors-on-kimi-k3#on-vs-off-policy) for plots. ### Comparing Different Monitoring Systems A useful monitor needs to block harmful activity while allowing legitimate work to continue. We ran full-turn agent and chat sessions, comparing Kimi K3 and GLM 5.3 with and without our monitors against several frontier model systems. Adding our monitor raises Kimi K3’s harmful-session blocking rate from roughly 29% to 96%, while its harmless-session interruption rate increases from about 5% to 9%. For GLM 5.3, harmful-session blocking rises from roughly 36% to 91%, while harmless-session interruption increases from about 10% to 15%. Both monitored models approach the highest harmful-session blocking rate among the frontier systems we evaluated, while interrupting substantially fewer harmless sessions. ![Scatter plot of harmful versus harmless block/refusal rate. Arrows move Kimi K3 from about 29% to 96% harmful blocking and GLM 5.3 from about 36% to 91% when our monitor is added, both at low harmless block rates; frontier models with their monitors sit further right, at higher harmless block rates.](https://static.goodfire.ai/cyber-monitors/monitor_comparison.webp) Figure 6: Harmful- and harmless-session blocking or refusal rates. Arrows show the effect of adding our monitors to Kimi K3 and GLM 5.3. Points toward the upper left block more harmful activity while allowing more harmless activity to proceed. We also compare against [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev), a decision model that can act as a cheap judge. At a fixed 5.5% benign-session interruption rate, Jev reaches 86% recall, compared with 93% for our non-reasoning cascade and 93% for our reasoning cascade. That is, when run by itself, Jev gives worse recall. Moreover, our non-reasoning cascade uses about $65 of judge compute per million turns and adds 2 ms of latency per turn, compared with $600 and 200 ms for Jev. ![Three bar charts comparing Jev with our cascades without and with reasoning: recall at 5.5% benign sessions interrupted (86%, 93%, 93%), judge cost per million exchanges ($600, $65, $185), and mean added latency per turn (200 ms, 2 ms, 42 ms).](https://static.goodfire.ai/cyber-monitors/jev-recall-cost-latency.webp) Figure 7: Our cascades catch more harmful sessions than Jev, with lower judge cost and added latency. Left: recall at a 5.5% benign-session interruption budget. Middle: judge cost per million exchanges. Right: mean added latency per turn. For the cost and latency comparisons, we select each cascade’s cheapest configuration that achieves at least 90% recall while interrupting at most 10% of benign sessions. The largest difference appears in long or adversarially framed contexts: Jev catches only 12% of harmful prompts when they are embedded in benign framing, while also increasingly interrupting benign bug-reproduction sessions beyond its 32K-token window. Our monitor performs better on these cases and remains near zero interruptions across most session lengths. ![Left: share of benign-framed harmful prompts caught: Jev 12%, our cascade without reasoning 52%, with reasoning 100%. Right: benign bug-reproduction sessions interrupted versus median session length from 30K to 150K tokens; Jev rises from 0% to about 22% past its 32K-token window, while ours stays at 0% (the no-reasoning variant reaches about 5% at the longest length).](https://static.goodfire.ai/cyber-monitors/jev-robustness.webp) Figure 8: Detection of harmful prompts embedded in benign framing (left) and benign bug-reproduction sessions interrupted as session length increases (right). The dashed line marks Jev’s 32k-token context window. Jev is less robust to adversarial prompts, and performs worse as context increases. ### External Red Teaming Results To test the monitor beyond our internal evaluations, we asked [FAR.AI](https://far.ai/) to red-team the deployed system. Their evaluation applied a fixed battery of jailbreak strategies to Kimi K3 with and without our monitor, evaluating for both universal attacks that transfer across many interactions and non-universal attacks that succeeded on individual prompts. In FAR.AI’s words: “[FAR.AI](https://far.ai/) conducted preliminary testing of Goodfire’s Kimi K3 monitors over a period of two days, applying a non-adaptive portfolio of static jailbreaks for offensive cyber misuse in chat contexts. From an initial pool of 1500 candidate jailbreaks, the testing selected 140 found to be effective against a different version of the base model without our monitors. Testing each of those 140 on 8 individual interactions corresponding to different attacker goals, Goodfire’s monitors reduced vulnerabilities elicited from 66 universal jailbreaks and 700 total jailbroken interactions to 0 universal jailbreaks and 18 jailbroken interactions.” That is, without our monitor, universal-jailbreak attack success ranged from 12% to 53% across reasoning-effort settings. With the monitor enabled, none of the 140 strategies succeeded at producing a universal jailbreak. The monitor also reduced individual prompt success on 1,120 non-universal interactions from at least 9–40% to 0.1–1.6%, depending on reasoning effort. So, while there were some successful jailbreak _interactions_, the monitors were largely robust to this broad, static battery of attacks. We plan to continue our collaboration with FAR.AI to perform further testing and strengthening of our monitors. ![Two bar charts of attack success rate by reasoning effort (None, Low, High, Max), without and with our monitor. (a) Universal jailbreaks over 140 strategies: 53%, 39%, 47%, and 12% without the monitor, 0% in every setting with it. (b) Non-universal jailbreaks over 1,120 interactions: at least 40%, 29%, 35%, and 9% without the monitor, versus 0.5%, 0.2%, 1.6%, and 0.1% with it.](https://static.goodfire.ai/cyber-monitors/external_redteam.webp) Figure 9: Independent red-teaming by FAR.AI across reasoning-effort settings. (a) Our monitor reduces universal jailbreak success to zero across 140 tested strategies. (b) Across 1,120 non-universal interactions, attack success falls to 0.1–1.6%. Bars compare attack success with and without monitoring; a monitored attack succeeds only if the interaction is harmful and the monitor does not flag it. ## Conclusion Our probe-plus-judge monitor cascade for Kimi K3 and GLM 5.3 beats running an LLM judge on every turn, at every interruption budget we tested. It costs about 50x less, has no measurable impact on inference throughput, and in FAR.AI's red-teaming it cut universal jailbreaks to zero. Because it's this cheap, it can be synchronous: risky actions get caught before the agent calls a tool, instead of after the fact. The recipe also carried over to GLM 5.3 using only data collected on Kimi K3, so you don't need a new data pipeline for every model. Open models don't come with a full safety stack. We think a monitoring setup like this is a practical default for anyone serving them in agentic settings. If you serve or build on open models and want to deploy monitors like these, [get in touch](https://www.goodfire.com/contact-us).