[翻译]That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip

该文档由Doc2X翻译提供解析与翻译, 想看更多论文翻译欢迎来Doc2X
This document is provided with parsing and translation by Doc2X. For more translated papers, feel free to visit Doc2X.
原文链接 arxiv.org/pdf/2411.10…

That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design

那片芯片已启航：对芯片设计领域AI无端怀疑的批判

Anna Goldie ${}^{1,3}$ ,Azalia Mirhoseini ${}^{1,3}$ ,and Jeff Dean ${}^{1,2}$

Anna Goldie ${}^{1,3}$ ,Azalia Mirhoseini ${}^{1,3}$ ,和 Jeff Dean ${}^{1,2}$

${}^{1}$ Google DeepMind

${}^{2}$ Google Research

${}^{3}$ Department of Computer Science,Stanford University

${}^{3}$ 斯坦福大学计算机科学系

Abstract

摘要

In 2020, we introduced a deep reinforcement learning method capable of generating superhuman chip layouts, which we then published in Nature and open-sourced on GitHub. AlphaChip has inspired an explosion of work on AI for chip design, and has been deployed in state-of-the-art chips across Alphabet and extended by external chipmakers. Even so, a non-peer-reviewed invited paper at ISPD 2023 questioned its performance claims, despite failing to run our method as described in Nature. For example, it did not pre-train the RL method (removing its ability to learn from prior experience), used substantially fewer compute resources (20x fewer RL experience collectors and half as many GPUs), did not train to convergence (standard practice in machine learning), and evaluated on test cases that are not representative of modern chips. Recently, Igor Markov published a "meta-analysis" of three papers: our peer-reviewed Nature paper, the non-peer-reviewed ISPD paper, and Markov's own unpublished paper (though he does not disclose that he co-authored it). Although AlphaChip has already achieved widespread adoption and impact, we publish this response to ensure that no one is wrongly discouraged from innovating in this impactful area.

在2020年，我们介绍了一种深度强化学习方法，能够生成超越人类的芯片布局，随后我们在《自然》杂志上发表并开源到GitHub。AlphaChip激发了大量关于芯片设计AI的研究，并在Alphabet旗下的尖端芯片中得到部署，并得到了外部芯片制造商的扩展。即便如此，一篇在ISPD 2023上发表的非同行评审邀请论文质疑了其性能声明，尽管未能按照《自然》杂志中描述的方法运行我们的方法。例如，它没有对RL方法进行预训练（剥夺了其从先前经验中学习的能力），使用了显著更少的计算资源（RL经验收集器减少了20倍，GPU数量减半），没有训练到收敛（机器学习的标准做法），并在不具现代芯片代表性的测试案例上进行评估。最近，Igor Markov发表了对三篇论文的“元分析”：我们的同行评审《自然》论文，非同行评审的ISPD论文，以及Markov自己未发表的文章（尽管他没有披露自己是合著者）。尽管AlphaChip已经取得了广泛的采用和影响，我们发表这篇回应是为了确保没有人被错误地劝阻，从而不敢在这个有影响力的领域进行创新。

1 Introduction

1 引言

Following its publication in Nature, AlphaChip [30] has inspired an explosion of work on AI for chip design [41, 39, 43, 40, 10, 18, 5, 34, 8, 12, 17, 37, 7, 29]. It has also generated superhuman chip layouts used in three generations of TPU (see Figure 1), datacenter CPUs (Axion), and other chips across Alphabet, and been extended to new areas of chip design by external academics and chipmakers $\left\lbrack {{25},{11}}\right\rbrack$ .

在《自然》杂志发表后，AlphaChip [30] 激发了大量关于芯片设计的人工智能研究 [41, 39, 43, 40, 10, 18, 5, 34, 8, 12, 17, 37, 7, 29]。它还生成了超越人类的芯片布局，用于三代TPU（见图1）、数据中心CPU（Axion）以及Alphabet旗下的其他芯片，并且被外部学者和芯片制造商扩展到芯片设计的新领域 $\left\lbrack {{25},{11}}\right\rbrack$ 。

Even so, Igor Markov published a criticism of our work in the November 2024 issue of Communications of the ACM [27], which is presented as a "meta-analysis" of our Nature paper and two non-peer-reviewed papers:

尽管如此，伊戈尔·马尔科夫在2024年11月的《ACM通讯》上发表了对我们工作的批评 [27]，这篇文章被呈现为对我们《自然》论文和两篇非同行评审论文的“元分析”：

Cheng et al.: The first is an invited ISPD paper ${}^{1}$ by Cheng et al. [9]. This paper did not follow standard machine learning practices, and its reinforcement learning methodology and experimental setup diverged significantly from those described in our Nature paper.
程等人：第一篇是程等人 [9] 撰写的受邀ISPD论文 ${}^{1}$ 。这篇论文没有遵循标准的机器学习实践，其强化学习方法论和实验设置与我们在《自然》论文中描述的有显著差异。

${}^{1}$ Invited papers at ISPD are not peer-reviewed.

${}^{1}$ ISPD上的受邀论文不经过同行评审。

Figure 1: AlphaChip has been deployed in three additional generations of TPU. In each generation, it has been adopted in a greater proportion of blocks and has outperformed human experts by a wider margin.

图1：AlphaChip已被部署在另外三代TPU中。在每一代中，它都被更大比例的模块采用，并且以更大幅度超越了人类专家的表现。

Nevertheless,its hamstrung version of our method still outperformed RePlAce ${}^{2}$ [13],which was the state of the art when we published in Nature.

然而，其受限版本的方法仍然超越了RePlAce ${}^{2}$ [13]，这是我们发表《自然》论文时的最先进技术。

Markov et al.: The second "meta-analyzed" paper is an unpublished PDF with no author list [3], which is described as a "separate evaluation" performed by "Google Team 2’,but was in fact co-authored by Markov himself ${}^{3}$ ,though this is not disclosed ${}^{4}$ . This paper did not meet Google's bar for publication. In 2022, it was reviewed by an independent committee at Google, which determined that "the claims and conclusions in the draft are not scientifically backed by the experiments" [33] and "as the [AlphaChip] results on their original datasets were independently reproduced, this brought the [Markov et al.] RL results into question" [33]. We provided the committee with one-line scripts that generated significantly better RL results than those reported in Markov et al., outperforming their "stronger" simulated annealing baseline. We still do not know how Markov and his collaborators produced the numbers in their paper.
Markov 等人：第二篇“元分析”论文是一篇未发表的 PDF 文件，没有作者列表 [3]，被描述为由“Google 团队 2”进行的“单独评估”，但实际上是由 Markov 本人共同撰写的 ${}^{3}$ ，尽管这一点并未披露 ${}^{4}$ 。这篇论文未达到 Google 的发表标准。2022年，它由 Google 的一个独立委员会进行了审查，该委员会认定“草稿中的主张和结论没有得到实验的科学支持” [33]，“由于 [AlphaChip] 在其原始数据集上的结果被独立重现，这使得 [Markov 等人] 的强化学习结果受到质疑” [33]。我们向委员会提供了单行脚本，生成了比 Markov 等人报告中显著更好的强化学习结果，超越了他们“更强”的模拟退火基线。我们仍然不知道 Markov 及其合作者是如何得出论文中的数据的。

Markov's "meta-analysis" offers one additional source of concern regarding our paper: a "whistle-blower" within Google. However, this "whistleblower" admitted to a Google investigator that he had no reason to believe fraud occurred: "he stated that he suspected that the research being conducted by Goldie and Mirhoseini was fraudulent, but also stated that he did not have evidence to support his suspicion of fraud" [24].

Markov 的“元分析”对我们的论文提出了另一个关注点：Google 内部的一位“吹哨人”。然而，这位“吹哨人”向 Google 调查人员承认，他没有理由相信存在欺诈行为：“他表示怀疑 Goldie 和 Mirhoseini 进行的研究是欺诈性的，但也表示他没有证据支持他对欺诈的怀疑” [24]。

In his "meta-analysis", Markov speculates wildly and without evidence about fraud and scientific misconduct, none of which occurred. Most of Markov's criticisms are of this form: it does not look to him like our method should work, and therefore it must not work, and any evidence suggesting otherwise is fraud. Nature investigated Markov's concerns, found them to be entirely without merit, and published an Addendum upholding our work at the conclusion of this process [20].

在他的“元分析”中，Markov 对欺诈和科学不端行为进行了毫无根据的狂妄推测，而这些情况并未发生。Markov 的大部分批评都是这种形式：在他看来，我们的方法似乎不应该奏效，因此它肯定不会奏效，任何相反的证据都是欺诈。《自然》杂志调查了 Markov 的担忧，发现它们完全站不住脚，并在这一过程的最后发表了一篇附录，支持我们的工作 [20]。

As an example, in the opening paragraph of his conclusions, Markov states (emphasis his): "In the paper,we find a smorgasbord of questionable practices in ${ML}{\left\lbrack {26}\right\rbrack }^{5}$ including irreproducible research practices, multiple variants of cherry-picking, misreporting, and likely data contamination (leakage)." We did not engage in any of these practices, or any other form of scientific misconduct, and Markov provides no evidence for these allegations. Nowhere in Markov's paper does he describe any form of alleged cherry-picking, let alone multiple variants, nor does he provide evidence. Nor does he describe any form of alleged "misreporting," or explain what he means by this, or provide evidence. Nor does he provide any evidence of data contamination (leakage), aside from his speculation that it would have improved our results if it had occurred. Many of these allegations appear for the first time in his "Conclusions" section!

作为例子，在结论的开头段落中，Markov 声明（强调为他所加）：“在论文中，我们发现 ${ML}{\left\lbrack {26}\right\rbrack }^{5}$ 中存在一系列可疑的做法，包括不可复制的研究实践、多种形式的挑选、误报以及可能的数据污染（泄露）。”我们没有参与任何这些做法，也没有任何其他形式的科学不端行为，Markov 没有为这些指控提供证据。在 Markov 的论文中，他并未描述任何形式的所谓挑选，更不用说多种形式了，也没有提供证据。他也没有描述任何形式的所谓“误报”，或者解释他指的是什么，或者提供证据。他也没有提供任何数据污染（泄露）的证据，除了他推测如果发生了这种情况，可能会改善我们的结果。许多这些指控首次出现在他的“结论”部分！

${}^{2}$ Incidentally,RePlAce,as noted in a footnote of Cheng et al.,is unable to produce any result at all for 2 out of the 6 test cases in its main data table.

${}^{2}$ 顺便提一下，正如 Cheng 等人在脚注中指出的，RePlAce 在其主数据表的 6 个测试案例中有 2 个无法产生任何结果。

${}^{3}$ Markov did not disclose anywhere in his "meta-analysis" that he is an author of one of the two "separate evaluations". He also omitted his name from the paper's authors in the references section, and linked only to an anonymous PDF. When questioned on LinkedIn, Markov admitted his authorship, but later deleted the post.

${}^{3}$ Markov 在他的“元分析”中并未透露他是两个“独立评估”之一的作者。他还在参考文献部分省略了他的名字，并且只链接到一个匿名的 PDF。当在 LinkedIn 上被问及时，Markov 承认了他的作者身份，但后来删除了该帖子。

${}^{4}$ Markov also failed to disclose his role as a high-level employee at Synopsys,a company which licenses commercial tools that compete with our open-source method.

${}^{4}$ Markov 还未透露他在 Synopsys 的高级员工身份，该公司授权商业工具，与我们的开源方法竞争。

${}^{5}$ Note that Markov’s citation 26 has nothing to do with our paper,though readers may mistakenly believe that it offers corroboration.

${}^{5}$ 注意，Markov 的引用 26 与我们的论文无关，尽管读者可能误以为它提供了佐证。

In an effort to discredit our TPU deployments, Markov also suggests that Google must just be "dog-fooding" our method, allowing inferior AlphaChip placements to be used in TPU in order to prop up our research paper. This is untrue, and absurd on its face. Google cares far more about the efficiency of TPU designs - a multi-billion-dollar project that is central to Google's cloud and AI initiatives - than it does about a research paper ${}^{6}$ .

为了诋毁我们的TPU部署，Markov还暗示谷歌只是在“内部试用”我们的方法，允许使用劣质的AlphaChip布局在TPU中，以此来支撑我们的研究论文。这是不真实的，并且从表面上看是荒谬的。谷歌对TPU设计的效率——一个数十亿美元的项目，对谷歌的云和AI倡议至关重要——的关注远远超过对一篇研究论文的关注 ${}^{6}$ 。

For clarity,we present a timeline of events,including non-confidential deployments ${}^{7}$ :

为了清晰起见，我们提供了一个事件时间线，包括非机密部署 ${}^{7}$ ：

Apr 2020: Released arXiv preprint of our Nature paper [4].
2020年4月：发布了我们Nature论文的arXiv预印本 [4]。
Aug 2020: 10 AlphaChip layouts taped out in TPU v5e.
2020年8月：10个AlphaChip布局在TPU v5e中完成。
Jun 2021: Published Nature article [30].
2021年6月：发表了Nature文章 [30]。
Sep 2021: 15 AlphaChip layouts taped out in TPU v5p.
2021年9月：15个AlphaChip布局在TPU v5p中完成。
Jan 2022 - Jul 2022: Open-sourced AlphaChip [21], after ensuring compliance with export control restrictions and excising internal dependencies. This involved independent replication of the results in our Nature paper by another team at Google. See Section 4.
2022年1月至2022年7月：在确保符合出口管制限制并去除内部依赖后，开源了AlphaChip [21]。这包括由谷歌的另一支团队独立复制我们Nature论文中的结果。参见第4节。
Feb 2022: Independent committee within Google declined to publish Markov et al. as the data did not support its claims and conclusions [33].
2022年2月：谷歌内部的独立委员会拒绝发表Markov等人的文章，因为数据不支持其主张和结论 [33]。
Oct 2022: 25 AlphaChip layouts taped out in Trillium (latest public TPU).
2022年10月：25个AlphaChip布局在Trillium（最新的公开TPU）中完成。
Feb 2023: Cheng et al. posted on arXiv [9], claiming to perform "massive reimplementa-tion" of our method, despite it being fully open-source. As discussed in Sections 2 and 3, Cheng et al. did not run our method as described in Nature, among other issues.
2023年2月：Cheng等人在arXiv上发表了文章 [9]，声称对我们方法进行了“大规模重新实现”，尽管它已经完全开源。如第2节和第3节所讨论的，Cheng等人并没有按照Nature中描述的方式运行我们的方法，还有其他问题。
Jun 2023: Markov released arXiv preprint of his “meta-analysis” [28].
2023年6月：Markov发布了他的“元分析”的arXiv预印本 [28]。
Sep 2023: Nature posted Editor's note stating that they are investigating our paper, and initiated second peer review process [30].
2023年9月：Nature 发表了编辑声明，表示正在调查我们的论文，并启动了第二次同行评审过程 [30]。
Mar 2024: 7 AlphaChip layouts adopted in Google Axion Processors (ARM-based CPU).
2024年3月：7种 AlphaChip 布局被采用于 Google Axion 处理器（基于ARM的CPU）。
Apr 2024: Nature completed its investigation and post-publication review, and found entirely in our favor, concluding that "the best way forward is to publish an update to the paper in the form of an Addendum (not a 'Correction', as we have established that there is little that actually needs correcting)." [44]
2024年4月：Nature 完成了其调查和发表后的评审，完全支持我们的立场，结论是“最佳的前进方式是以增补形式（而非‘更正’，因为我们已确定实际上几乎没有需要更正的内容）发布论文的更新。” [44]
Sep 2024: Nature published Addendum upholding our work [20], removed Editor's note.
2024年9月：Nature 发布了支持我们工作的增补文件 [20]，移除了编辑声明。
Sep 2024: SVP of MediaTek announced that they extended AlphaChip to accelerate development of their most advanced chips [19].
2024年9月：MediaTek 高级副总裁宣布他们将扩展 AlphaChip 以加速其最先进芯片的开发 [19]。
Nov 2024: Markov republished his "meta-analysis", though his concerns were already found to be without merit during Nature's investigation and second peer review process.
2024年11月：Markov 重新发表了他的“元分析”，尽管他的担忧在 Nature 的调查和第二次同行评审过程中已被发现毫无根据。

In brief, Markov's paper contains no original data, and is a "meta-analysis" of just two papers. The first is presented with no author list (though Markov was an author), was never published, made claims that were not scientifically backed by the data, and could not be reproduced. The second, Cheng et al., is the only substantive content in Markov's "meta-analysis", so we devote the remainder of this paper to describing significant issues in its purported reproduction of our method.

简而言之，Markov 的论文没有原始数据，是对仅两篇论文的“元分析”。第一篇没有作者名单（尽管 Markov 是作者之一），从未发表，提出的论断没有科学数据的支持，且无法复现。第二篇，Cheng 等人的论文，是 Markov “元分析”中唯一实质性的内容，因此我们将在本文的剩余部分描述其在所谓复现我们方法中的重大问题。

${}^{6}$ In reality,we had to work for a long time to build enough trust for the TPU team to use our layouts,even after AlphaChip was outperforming human experts on the metrics, and this makes sense - their job is to deliver TPU chips and make them as efficient and reliable as possible, and they cannot afford to take unnecessary risks.

${}^{6}$ 实际上，即使 AlphaChip 在各项指标上已经超越了人类专家，我们仍需花费很长时间建立足够的信任，使 TPU 团队使用我们的布局，这是有道理的——他们的工作是交付 TPU 芯片并使其尽可能高效和可靠，他们不能承担不必要的风险。

${}^{7}$ AlphaChip has been deployed in other hardware across Alphabet that we cannot yet disclose.

${}^{7}$ AlphaChip 已经部署在 Alphabet 内部的其他硬件中，但我们目前无法透露。

2 Errors In Cheng et al.'s Reproduction of Our Method

2 程等人复现我们方法中的错误

Cheng et al. claim to evaluate our method against alternative approaches on new test cases. Unfortunately, Cheng et al. did not run our method as described in Nature, so it is unsurprising that they report different results. In this section, we describe major errors in their purported reproduction:

程等人声称在新测试案例上评估我们的方法与替代方法的对比。不幸的是，程等人并未按照《自然》杂志中的描述运行我们的方法，因此他们报告不同结果并不令人意外。在本节中，我们描述了他们在所谓复现中的主要错误：

Did not pre-train the RL method. The ability to learn from prior experience is the key advantage of our learning-based method, and to remove it is to evaluate a different and inferior approach. Incidentally, pre-training also gives rise to the impressive capabilities of large language models like Gemini [36] and ChatGPT [32] (the “P” in “GPT” stands for “pre-trained”). See Section 2.1.
未进行RL方法的预训练。从先前经验中学习的能力是我们基于学习的方法的关键优势，去除这一步骤实际上是评估了一个不同且劣等的方法。顺便提一下，预训练也是大型语言模型如Gemini [36] 和ChatGPT [32]（“GPT”中的“P”代表“预训练”）展现出令人印象深刻能力的原因。参见第2.1节。
Used an order of magnitude fewer compute resources: 20x fewer RL experience collectors (26 vs 512 in Nature) and 2x fewer GPUs (8 vs 16 in Nature). See Section 2.2.
使用的计算资源数量级少于我们：RL经验收集器少20倍（26个对比《自然》中的512个），GPU少2倍（8个对比《自然》中的16个）。参见第2.2节。
Did not train to convergence. Training to convergence is standard practice in machine learning, as not doing so is well known to harm performance [1]. See Section 2.3.
未训练至收敛。训练至收敛是机器学习中的标准做法，因为不这样做众所周知会损害性能 [1]。参见第2.3节。
Evaluated on non-representative, irreproducible benchmarks. Cheng et al.'s benchmarks have much older and larger technology node sizes $({45}\mathrm{\;{nm}}$ and ${12}\mathrm{\;{nm}}$ vs sub- $7\mathrm{\;{nm}}$ in Nature), and differ substantially from a physical design perspective. Additionally, the authors were unable or unwilling to share the synthesized netlists necessary to replicate the results in their main data table. See Sections 2.4 and 4.2.
在非代表性、不可复现的基准上进行评估。程等人的基准具有更老且更大的技术节点尺寸 $({45}\mathrm{\;{nm}}$ 和 ${12}\mathrm{\;{nm}}$ 对比《自然》中的亚 $7\mathrm{\;{nm}}$ ，并且从物理设计角度来看存在显著差异。此外，作者无法或不愿分享合成网表，这是复现其主数据表结果所必需的。参见第2.4节和第4.2节。
Performed "massive reimplementation" of our method, which may have introduced errors. We recommend instead using our open-source code. See Section 4.
对我们的方法进行了“大规模重实现”，这可能引入了错误。我们建议使用我们的开源代码。参见第4节。

These major methodological differences unfortunately invalidate Cheng et al.'s comparisons with and conclusions about our method. If Cheng et al. had reached out to the corresponding authors of the Nature paper ${}^{8}$ ,we would have gladly helped them to correct these issues prior to publication ${}^{9}$ .

这些主要的方法论差异不幸地使程等人对我们方法的比较和结论无效。如果程等人曾联系《自然》论文的通讯作者 ${}^{8}$ ，我们很乐意在发表前帮助他们纠正这些问题 ${}^{9}$ 。

2.1 No Pre-Training Performed for RL Method

2.1 未对RL方法进行预训练

Unlike prior approaches, AlphaChip is a learning-based method, meaning that it becomes better and faster as it solves more instances of the chip placement problem. This is achieved by pre-training, which consists of training on "practice" blocks (training data) prior to running on the held-out test cases (test data).

与先前的方法不同，AlphaChip是一种基于学习的方法，意味着随着它解决更多芯片放置问题的实例，它会变得更好、更快。这是通过预训练实现的，即在运行保留测试案例（测试数据）之前，对“练习”块（训练数据）进行训练。

As we showed in Figure 5 of our Nature paper (reproduced below as Figure 2), the larger the training dataset is, the better the method becomes at placing new blocks. As described in our Nature article, we pre-trained on 20 blocks in our main data table (Nature Table 1).

正如我们在《自然》论文的图5中所展示的（如下复现为图2），训练数据集越大，该方法在放置新块方面的表现就越好。正如我们在《自然》文章中所描述的，我们在主数据表（自然表1）中对20个块进行了预训练。

Cheng et al. did not pre-train at all (i.e., no training data), meaning that the RL agent had never seen a chip before and had to learn how to perform placement from scratch for each of the test cases. This removed the key advantage of our method, namely its ability to learn from prior experience.

程等人根本没有进行预训练（即没有训练数据），这意味着RL代理从未见过芯片，并且必须从头开始学习如何为每个测试案例进行放置。这消除了我们方法的关键优势，即其从先前经验中学习的能力。

By analogy to other well-known work on reinforcement learning, this would be like evaluating a version of AlphaGo [14] that had never seen a game of Go before (instead of being pre-trained on millions of games), and then concluding that AlphaGo is not very good at Go.

通过与强化学习的其他知名工作进行类比，这就像评估一个从未见过围棋游戏的AlphaGo版本（而不是在数百万局游戏中进行预训练），然后得出结论说AlphaGo不擅长围棋。

We discussed the importance of pre-training at length in our Nature paper (e.g. the word "pretrain" appeared 37 times), and empirically demonstrated its impact. For example, Nature Figure 4 (reproduced here as Figure 3) showed that pre-training improves placement quality and convergence speed. On the open-source Ariane RISC-V CPU [16], it took 48 hours for the non-pretrained RL

我们在《自然》杂志的论文中详细讨论了预训练的重要性（例如，“预训练”一词出现了37次），并通过实证展示了其影响。例如，《自然》杂志图4（此处复现为图3）显示，预训练提高了布局质量和收敛速度。在开源的Ariane RISC-V CPU [16]上，未经预训练的RL需要48小时

${}^{8}$ Prior to publication of Cheng et al.,our last correspondence with any of its authors was in August of 2022 when we reached out to share our new contact information.

${}^{8}$ 在程等人发表文章之前，我们与其中任何一位作者的最后一次通信是在2022年8月，当时我们联系他们以分享我们的新联系信息。

${}^{9}$ In contrast,prior to publishing in Nature,we corresponded extensively with Andrew Kahng,senior author of Cheng et al. and of the prior state of the art (RePlAce), to ensure that we were using the appropriate settings for RePlAce.

${}^{9}$ 相比之下，在《自然》杂志发表之前，我们与程等人及先前技术水平（RePlAce）的高级作者Andrew Kahng进行了广泛通信，以确保我们使用了RePlAce的适当设置。

—— 更多内容请到Doc2X翻译查看

—— For more content, please visit Doc2X for translations