Lecture 4 · 训练数据归因 · Training-Data Attribution

对应 Maps to：Spring 2022 Week 3 “Training point influence”（阅读材料：Koh & Liang 2017, “Understanding Black-box Predictions via Influence Functions”；Yeh et al. 2018, “Representer Point Selection for Explaining Deep Neural Networks”）；Fall 2023 Lecture 11（客座讲座：Juhan Bae & Cem Anil，“Studying Large Language Model Generalization with Influence Functions”，即 Grosse, Bae, Anil et al. 2023 —— EK-FAC、PBRF、最有影响的训练序列、随规模变得更抽象的泛化、跨语言影响、按 token / 按层分解；参考文献含 Pruthi et al. 2020 TracIn）。Spring 2022 这一周关于训练样本影响的 slides 不在公开材料里，那部分按指定阅读材料撰写；Fall 2023 的部分依据公开的讲座 slides 和论文。 配套代码 Code：code/lecture04/（numpy；5 个脚本 + 2 个工具模块；每个脚本 1~30 秒） 预计阅读时间：3 小时（含运行代码）

0. 本讲导航 · How to read this lecture

0.1 Learning goals

After this lecture you should be able to:

1. 说清楚”哪些训练样本造成了这个预测”这个问题的金标准定义（leave-one-out），以及为什么几乎没人真的去算它。
2. Derive the influence function from the implicit function theorem — $\mathcal{I}_{\text{up,params}} = -H^{-1}\nabla\ell(z)$, $\mathcal{I}_{\text{up,loss}}(z, z_{\text{test}}) = -\nabla\ell(z_{\text{test}})^\top H^{-1}\nabla\ell(z)$ — and explain, with the leverage $h_j$, exactly when the first-order approximation is accurate and when it is not.
3. 解释为什么影响函数在深度网络上和”重新训练”对不上（Basu et al. 的 fragility），以及 Bae et al.（2022）的回答：它近似的是另一个量 PBRF；从头重训在非凸模型上本身就被随机种子主导。
4. Describe how influence functions are scaled to billions of parameters: Hessian-vector products, CG / LiSSA, K-FAC / EK-FAC, query batching — and what Grosse et al. (2023) found with them about LLM generalization (stated with the right caveats).
5. 推导 representer points 和 TracIn，说明它们是分解（decomposition）而影响函数 / LOO 是反事实（counterfactual）；用一个统一的”核”视角解释它们什么时候一致、什么时候分道扬镳。
6. 用训练数据归因做三件事并知道其局限：找错误标签、定向投毒（攻击方的用法）、识别被模型”记住”的样本（隐私风险，第 9 讲预告）。

0.2 先修与本讲范围 Prerequisites & scope

第 1 讲 §5.1 把一个预测”分给各个输入特征”；第 2、3 讲会把这件事做得更严格（Shapley 值、Integrated Gradients）。本讲换一个维度：把一个预测”分给各个训练样本”。CS329T 2022 年第一节解释课的分类图上，“Explanations of what input?”一栏写的正是 “Underlying features? Specifying training data?” —— 这是同一张图的另一半。

0.3 本讲地图 Map

§1  问题：哪些训练样本造成了这个行为？三类方法
§2  金标准：留一法（LOO）及其代价
§3  影响函数：从隐函数定理到 1/(1 − h)                          ← 代码 01
§4  深度网络上：影响函数到底在估计什么？（fragility、PBRF）       ← 代码 02 演示 1–2
§5  规模化：Hessian-vector product、CG、LiSSA、EK-FAC            ← 代码 02 演示 3
§6  Representer points          §7  TracIn          §8  Data Shapley、datamodels、TRAK
§9  四种方法解释同一个被拒绝的申请人；统一的"核"视角              ← 代码 03
§10 应用：找错标签、LLM 的泛化、投毒、记忆与隐私                 ← 代码 04、05
§11–15 小结、误区、练习、延伸阅读、下一讲预告

0.4 Notation

0.5 关于数字 · About the numbers

讲义里所有实验数字都来自配套代码的实际输出（固定随机种子，重复运行逐字相同）。数据来自第 1 讲的合成信贷世界（只用 5 个”决策时拿得到”的特征，不含泄露的 officer_flag），模型主要是 L2 逻辑回归 —— 它是凸的，精确的留一重训很便宜，所以每一种近似方法都能拿”真相”去检验。非凸的部分用一个 57 个参数的小 MLP（numpy 手写）。数字只说明机制，不代表真实系统；有几个结果和我原本的预期不同，讲义按实际结果写，代码 README 的”关于实验结果”一节逐条记录。

1. 问题：哪些训练样本造成了这个行为？· Which training examples caused this?

1.1 从”哪些特征”到”哪些训练样本”

第 1 讲 §5.1 的局部解释告诉我们，每个特征把一位申请人的 logit 推高或拉低了多少（$\theta_j\tilde x_j$）。但模型为什么认为高负债是坏事？因为训练集里的某些人：高负债而违约的人教会了它这一点。一个模型的每一个参数，最终都是训练数据的函数 $\hat\theta = \mathcal{A}(z_1, \dots, z_n)$。所以除了”输入的哪一部分”，我们还可以问：“训练数据的哪一部分”。

The Fall 2023 guest lecture (Bae & Anil) opened with the case for asking this question. To explain a surprising behavior of a large language model, you could:

• Inspect samples. But generation is stochastic, so small differences between models are hard to notice, and an observation is usually consistent with many hypotheses about how it was produced (the lecture’s example: deceptive alignment).
• Do mechanistic interpretability. A bottom-up approach that aims for a complete explanation of a computation — very ambitious, and it may or may not be possible.
• Use influence functions. A top-down approach: how would the model’s behavior change if a training example were present or absent — and then, digging deeper, which layers and tokens does that influence pass through?

The safety motivation they gave: if an LLM-based system “goes rogue”, it will probably be because of unintended consequences of its training data and objectives. Training-data attribution is one piece of that puzzle, and by localizing influence to layers and tokens it can also guide mechanistic work.

1.2 为什么要问这个问题 · Why ask?

1.3 三类方法 · Three families

记住这张表：§9 会看到，同一个申请人的解释，在一个模型上四种方法几乎一致，换一个容量更大的模型就分成了”反事实”和”分解 / 相似度”两派 —— 因为它们本来就是不同的问题。

2. 金标准：留一法 · Leave-one-out as the definition

2.1 Definition

The most direct definition of “the effect of training example $z_j$ on quantity $f$” is a counterfactual: retrain without it and see what changes,

This idea is old. In regression diagnostics, Cook’s distance (Cook 1977) measures how much all fitted values move when one observation is deleted, and the jackknife resamples by leaving points out. The influence function itself comes from robust statistics (Hampel 1974), where it describes how an estimator responds to an infinitesimal contamination of the data at one point.

2.2 为什么几乎没人真的算它 · The cost

Exact LOO needs one retraining per training example: $n$ trainings. For the 300-example logistic regression in code 01 that takes well under a second (warm-started from $\hat\theta$, each refit is 2–4 Newton steps). For an LLM pretrained on billions of sequences, one training run takes weeks; $n$ of them is out of the question. And even if you only want “the 100 most influential examples”, you have to score everyone before you can rank them.

2.3 Details that are easy to miss

1. Normalization. The definition above keeps the $\frac1n$ (removing $j$ sets its weight to 0). Averaging over the remaining $n-1$ points instead is equivalent to replacing $\lambda$ by $\lambda n/(n-1)$ — a tiny difference, but the code matches the influence-function derivation exactly so that we compare approximation error, not bookkeeping.
2. Frozen preprocessing. The means and standard deviations used for standardization also depend on the training set. The code fits them once on the full training set and freezes them; only $\theta$ is retrained.
3. Effects are tiny. In code 01, even the most influential of 300 training applicants moves a test loss by only 0.016. The optimizer must converge far more precisely than that (the code uses $\lVert\nabla J\rVert < 10^{-12}$), or the “ground truth” is itself noise.
4. For non-convex models LOO is not a number. Networks trained from different random initializations land in different local optima, so “remove $j$ and retrain” yields a random variable. §4 shows its spread can exceed the effect of $j$ by an order of magnitude.

3. 影响函数 · Influence functions

3.1 Upweighting a training point by ε

Instead of deleting $z_j$, Koh & Liang (2017) — the ICML 2017 best paper that brought influence functions into modern ML — ask what happens if we upweight it by a small amount $\varepsilon$:

At $\varepsilon = 0$ we get the trained model; at $\varepsilon = -\frac1n$ the term $\frac1n\ell_j$ inside $J$ is cancelled, which is exactly LOO. The trick is to treat $\varepsilon$ as continuous and take a derivative at 0 — a local question that never requires retraining.

3.2 Derivation via the implicit function theorem

The optimum satisfies the first-order condition for every $\varepsilon$:

Differentiate both sides with respect to $\varepsilon$ and set $\varepsilon = 0$:

The implicit function theorem guarantees this derivative exists when $J$ is twice differentiable and $H$ is invertible at $\hat\theta$ — two assumptions we will watch break in §4. The chain rule then gives the effect on any differentiable quantity $f$, e.g. a test loss:

For removal, $\varepsilon = -\frac1n$, so the linear prediction is

3.3 读懂这个公式 · Anatomy of the formula

Each of the three factors has a plain meaning:

• $\nabla\ell_j = (p_j - y_j)\,a_j$ for logistic regression: residual × input. The more “surprised” the model is by $z_j$ (large $|y_j - p_j|$), the harder $z_j$ pulls on the parameters; the more unusual its features, the more lopsided the direction of the pull.
• $H^{-1}$: which directions are cheap. The Hessian says how fast the training loss rises when the parameters move in each direction. Directions the data barely constrain (low curvature) are cheap to move along, and $H^{-1}$ amplifies the pull in those directions.
• $\nabla f$: how sensitive the explained quantity is to the parameters. For a logit, $\nabla_\theta z_t = a_t$; for a test loss, multiply by the test residual.

Why compute it in this order? Scoring all $n$ training points by forming $H^{-1}\nabla\ell_j$ would need $n$ linear solves. Koh & Liang’s trick is to solve once for $s_{\text{test}} = H^{-1}\nabla f$ (using the symmetry of $H$) and then take one dot product $s_{\text{test}}^\top\nabla\ell_j$ per training point. The cost drops from ”$n$ trainings” to “one linear solve plus $n$ dot products”.

代码 01 演示 1 在线性模型（6 个参数，300 个训练样本，$\lambda = 10^{-3}$）上检验它，被解释的量是测试申请人 #0 的 log loss：

影响最大的 8 个训练样本里，有 7 个的效应是负的：删掉 TA，这位实际还款的测试申请人的 loss 反而下降。其中 3 个是”看起来很像好客户（$p \approx 0.82$$0.83$）却违约了”的人，他们让模型对这类人群更悲观；另外 3 个是”看起来很差（$p$ 0.090.25）却还款了”的人。影响大小正是由公式里的两个因子决定的：$|y - p|$ 和特征的位置。

3.4 为什么是”一阶”，误差从哪里来 · Leverage and the 1/(1 − h) correction

影响函数在 $\varepsilon = 0$ 处线性化，然后外推到 $\varepsilon = -\frac1n$。外推时它假设了一件事：Hessian 不变。但删掉 $j$ 之后，Hessian 少了 $j$ 自己贡献的那一项：

Take one Newton step on the leave-$j$-out objective, starting from $\hat\theta$. Its gradient at $\hat\theta$ is $-\frac1n\nabla\ell_j = -\frac{r_j}{n}a_j$, so the step is $\frac{r_j}{n}H_{-j}^{-1}a_j$. The Sherman–Morrison formula gives $H_{-j}^{-1}a_j = H^{-1}a_j / (1 - h_j)$ with

So relative to one exact Newton step, the influence function’s error is entirely controlled by the leverage $h_j$. This is the logistic-regression version of the classic least-squares identity $\hat\theta - \hat\theta_{-j} = (X^\top X)^{-1}x_j e_j/(1 - h_{jj})$ (with residual $e_j = y_j - x_j^\top\hat\theta$) (Pregibon 1981 developed these one-step diagnostics for logistic regression). Two facts make $h_j$ easy to reason about:

• $\sum_j h_j = \operatorname{tr}\!\big(H^{-1}(H - \lambda I_{\text{reg}})\big) = m - \lambda\operatorname{tr}(H^{-1}I_{\text{reg}})$：杠杆之和就是有效参数个数（effective degrees of freedom），所以平均杠杆 ≈ 有效参数个数 / $n$。
• $h_j$ 大 = 第 $j$ 个样本在某个方向上”独自”提供了大部分曲率。TA 一走，那个方向就变”软”，参数会移动得比影响函数以为的多得多。

代码 01 演示 2 扫描模型容量和 $\lambda$（被解释的量：前 10 个测试申请人各自的 loss，相关系数取中位数）：

线性模型上 $\lambda$ 怎么变都几乎精确：Σh ≤ 6，平均杠杆只有 0.02。三次模型（55 个多项式特征）上，$\lambda$ 越小有效参数越接近 56，最大杠杆逼近 1，相关从 0.994 降到 0.881，影响函数的中位低估从 5% 涨到 14%；用 $1/(1-h)$ 修正后又回到 0.97~0.9996。演示 3 直接在参数空间里看（三次模型，$\lambda = 10^{-4}$）：

杠杆超过 0.8 的 5 个样本，影响函数预测的参数变化只有真实的不到十分之一；方向大体还对（cos 0.97）。结论：凸性保证”有唯一的答案”，不保证”一阶近似离答案很近”。 影响函数准确的条件是每个样本的杠杆都小 —— $n$ 远大于有效参数个数、没有极端离群点。（为什么不总是用修正版？因为 $h_j$ 需要 $a_j^\top H^{-1}a_j$，对每个训练样本都要一次 $H^{-1}$ 求解 —— 在大模型上，这恰恰是我们付不起的。）

3.5 删掉一组样本 · Group effects

影响函数是线性的，所以”删掉一组 $S$“的预测就是单点预测之和 $\sum_{j\in S}\Delta f_j$。演示 4（线性模型，被解释的量：整个测试集的平均 loss，每种设置 50 组）：

随机的组各自往不同方向推，效果大致可加，相关一直在 0.93 以上（大小的偏差方向不稳定：20% 时反而是高估）。“抱团”的组 —— 特征相似、标签相同，比如同一家分行同一批录入出错的记录 —— 往同一个方向推，又一起提供那个方向的曲率；一起删掉时，演示 3 的杠杆效应在群体层面被放大：真实效果是预测的 1.4~3.8 倍，删到 10%~20% 时连排序都不可靠了。Koh, Ang, Teo & Liang（2019）在真实数据上报告，对较大的、有结构的组，影响函数常常低估真实效果、但仍然和它相关；我们的玩具数据里低估更严重，到 10% 以上排序也失灵。“删掉一整批数据会怎样”这种问题，能重训就重训。

3.6 其他前提 · Other assumptions

• 可微：Koh & Liang 对 hinge loss 的 SVM 改用平滑的近似（smooth hinge）后，影响函数才能准确预测重训的效果。
• 在真正的最优解处：$\nabla J(\hat\theta) = 0$ 是推导的起点。训练没收敛时，“继续训练”本身也会改变参数（§4.3 的 non-convergence gap）。
• $H$ 可逆：$\lambda > 0$ 的逻辑回归保证这一点；神经网络不保证。实践中加 damping：用 $H + \delta I$ 代替 $H$。Koh & Liang 在一个没训练完、Hessian 非正定的 CNN 上加了 $\delta = 0.01$，预测和实际 loss 变化的 Pearson 相关仍有 0.86 —— 但那是对影响最大的 500 个训练点算的，§4 会看到情况可以差得多。

4. 深度网络上：影响函数到底在估计什么？· What do influence functions estimate in deep nets?

4.1 Fragility

For neural networks, every assumption in §3.2 is in trouble: the objective is non-convex, $H$ can have negative eigenvalues, training is stopped early, and there are many minima. Basu, Pope & Feizi (2021), “Influence Functions in Deep Learning Are Fragile”, compared influence estimates with leave-one-out retraining across architectures and found them fairly accurate for shallow networks but often erroneous for deeper ones, with quality sensitive to depth and width, weight decay, how the inverse-Hessian-vector product is approximated, and even the test point chosen. The Fall 2023 guest lecture summarized the conceptual problems in two lines: the classical formulation assumes $H$ is invertible, “while neural network training is often underspecified”, and it assumes we have found the optimal solution.

4.2 Five gaps between IF and LOO (Bae et al. 2022)

Bae, Ng, Lo, Ghassemi & Grosse (2022), “If Influence Functions are the Answer, Then What is the Question?”, decompose the discrepancy between the practical influence estimate and LOO retraining into five terms:

1. Warm-start gap — influence functions linearize around $\hat\theta$, so they naturally describe retraining that starts from $\hat\theta$ (warm start). With many minima, retraining from a fresh random initialization (cold start) can land somewhere else entirely.
2. Proximity gap — damping $\delta I$ is equivalent to adding $\frac\delta2\lVert\theta - \hat\theta\rVert^2$: the estimate describes a response that is discouraged from moving far.
3. Non-convergence gap — if $\hat\theta$ is not a minimum, much of what LOO retraining changes is just “training longer”, a nuisance from the viewpoint of attribution.
4. Linearization error — the first-order Taylor approximation itself (our §3.4).
5. Solver error — the inverse-Hessian-vector product is only approximated (§5).

They found that across tasks the first three dominate — which means the mismatch reflects mostly a different question, not a bad approximation.

4.3 PBRF：影响函数真正近似的那个量

Bae et al. 定义了 proximal Bregman response function（PBRF）。对删除样本 $z_j$（$\varepsilon = \frac1n$）：

其中 $\theta^s$ 是训练得到的参数（不要求收敛），$D_\ell(y, y^s) = \ell(y) - \ell(y^s) - \nabla_y\ell(y^s)^\top(y - y^s)$ 是 loss 在输出空间的 Bregman 散度。三项各管一件事：

• 函数空间别跑远：在每个训练样本上，新模型的预测要接近 $\theta^s$ 的预测。对交叉熵，这一项等价于”把训练标签换成 $\theta^s$ 自己的软预测再训练”—— 所以 $\theta^s$ 自动是它的最优解，哪怕 $\theta^s$ 根本没训练完。对 logistic loss，它恰好是 $\mathrm{KL}\big(\mathrm{Bern}(\sigma(z^s))\,\Vert\,\mathrm{Bern}(\sigma(z))\big)$，真实标签被消掉了。
• 往”没有 $j$“的方向推：负的 $j$ 的 loss，权重 $\frac1n$。
• 参数空间别跑远：damping 对应的 proximity 项。

在 $\theta^s$ 处对它做二阶展开，第一项给出的曲率正是 Gauss–Newton 矩阵 $G = \frac1n\sum_i J_i^\top H_{y,i} J_i$（$J_i$ 是输出对参数的 Jacobian），所以最优的一阶响应是 $\frac1n(G + \delta I)^{-1}\nabla\ell_j$ —— 恰好就是实践中使用的”Gauss–Newton Hessian + damping”版本的影响函数。$G$ 永远半正定（只要 loss 对输出是凸的），这也是 Grosse et al.（2023）在 52B 参数模型上用 $G + \lambda I$ 而不是 $H$ 的原因之一。

换句话说：影响函数回答的问题是”保持模型在其他训练样本上的预测基本不变、参数也不要跑远，$z_j$ 推动了什么？” Bae et al. 论文首页的图把这一点画得很清楚：在逻辑回归上 IF 和 LOO 的 Pearson 相关是 0.999；在 MLP 上掉到 0.03；但同一个 MLP 上 IF 和 PBRF 的相关是 0.97。

4.4 在一个小 MLP 上把所有”真相”都算出来

代码 02 用一个 5 → 8 → 1 的 tanh 网络（57 个参数，200 个训练样本），在三种 weight decay 下训练到严格的局部极小（梯度 < $10^{-11}$，真 Hessian 正定）—— 刻意排除 non-convergence gap，只看非凸本身。被解释的量是测试申请人 #0 的 logit。

演示 1：种子噪声。

“seed std”是用完整训练集、5 个不同初始化训练出的网络对同一个人给出的 logit 的标准差。正则化强时，5 个初始化给出（三位小数内）同一个答案；正则化弱时，不同初始化落进预测差别很大的山谷，种子噪声比”删掉一个样本”的典型效应大 7~13 倍。

演示 2：影响函数（Gauss–Newton + damping $10^{-2}$）和四种”删掉 $j$ 之后”的答案的 Pearson 相关（对影响最大的 15 个 + 随机 15 个训练样本）：

（cold = warm：从同一个初始化从头重训，落回 warm-start 同一个极小点的次数。各列平均绝对值见代码输出：wd = 1e-3 时影响函数 0.22、PBRF 0.35、warm-start 0.90、从头重训 3.8~4.6。）

读这张表：

• 影响函数 ≈ PBRF，三种情形下相关都在 0.93 以上，过拟合最严重时反而最高（0.987）。Bae et al. 的论点在这个小例子里成立。
• 和 warm-start 的一致性随非凸程度下降（0.959 → 0.843 → 0.663）。warm-start 没有 proximity 项拉着，删掉一个高影响的点后，参数在山谷里走得更远（平均变化 0.90，是 PBRF 的 2.5 倍）。用真 Hessian 的 IF-H 和 warm-start 更一致（0.800~0.969）：描述”同一个山谷形状”的正是真 Hessian。
• 从头重训几乎就是噪声。wd = 3e-3 时只有一半的删除会落回同一个山谷，wd = 1e-3 时一次也没有。连 wd = 1e-2 这种”几乎凸”的情形，30 次里也有 1 次落进了别的山谷，单这一个点就让相关从 warm-start 的 0.959 降到 0.867。

4.5 那么，我们到底想要哪个问题的答案？

所以”影响函数在深度网络上很脆弱”这个结论，一半是因为拿它去估计了一个它没打算估计、而且本身也不稳定的量。另一半是真的：§4.4 里 wd = 1e-3 的 warm-start 相关只有 0.66，一阶近似在高度非线性的区域确实吃力。

5. 规模化：从 57 个参数到 520 亿个 · Scaling up

5.1 Two bottlenecks

For a model with $P$ parameters, the influence of every candidate training sequence on one query needs:

1. An inverse-Hessian-vector product (IHVP) $s = (G + \lambda I)^{-1}\nabla f$. $G$ is $P \times P$ — for $P = 5 \times 10^{10}$ it cannot even be stored, let alone inverted.
2. The gradient of every candidate training example, $\nabla\ell_j$, one backward pass each, dotted with $s$. With millions of candidates and dozens of queries, this dominates.

5.2 Hessian-vector products, CG and LiSSA

You never need $H$ itself, only $Hv$. Pearlmutter (1994) showed that $Hv$ can be computed exactly at the cost of about two gradient evaluations (“double backprop”); for the Gauss–Newton matrix, $Gv = J^\top H_y (J v)$ is a Jacobian-vector product followed by a vector-Jacobian product. Given a matrix-vector oracle, two classic iterative solvers produce $M^{-1}b$:

• Conjugate gradient (CG)：每步沿一个和之前所有方向”$M$-共轭”的方向前进，精确算术下 $P$ 步得到精确解，实际收敛速度取决于特征值的分布。
• LiSSA（Agarwal, Bullins & Hazan 2017）：Neumann 级数 $M^{-1} = \frac1s\sum_k (I - M/s)^k$（$s \ge$ 最大特征值），每步只用一个随机小批量估计 $Mv$。每步便宜，但每步只把误差缩小约 $(1 - \lambda_{\min}/s)$，且小批量噪声会留下一个降不下去的误差底。

代码 02 演示 3（wd = 3e-3 的网络，$M = G + 0.01I$，57 × 57，条件数 76），相对误差 vs 矩阵乘向量的次数：

CG 在 57 次（= 参数个数）时到了机器精度；LiSSA 需要上千次，小批量版本停在约 0.15。Grosse et al.（2023）指出，传统做法的每个 IHVP 可能要迭代上千步、每步和一次梯度计算一样贵；在他们之前，影响函数用过的最大模型是约 3 亿参数的 vision transformer（Schioppa et al. 2022）。

5.3 K-FAC and EK-FAC

The alternative is to pay once for a structured approximation of $G$ that is cheap to invert. For a fully connected layer $W$ (output $= W a$), the per-example gradient is an outer product $\nabla_W\ell = g\,a^\top$, where $a$ is the layer input and $g$ the gradient with respect to the layer output. K-FAC (Martens & Grosse 2015, originally an optimizer) approximates the layer’s Gauss–Newton block as a Kronecker product,

and the inverse of a Kronecker product is the Kronecker product of the (small) inverses. EK-FAC (George et al. 2018) keeps the eigenbases of $A$ and $S$ but re-fits the eigenvalues — the diagonal of the true second moment in that eigenbasis — which repairs much of the error from the independence assumption behind K-FAC. After fitting the factors once, applying $(G + \lambda I)^{-1}$ to a gradient costs a few small matrix multiplications per layer, comparable to one gradient computation. The block-diagonal (per-layer) structure also gives layerwise influence for free (§10.2).

Grosse et al. validated this against the PBRF as ground truth: EK-FAC was competitive with LiSSA in accuracy while being far cheaper. They also compared with the plain gradient dot product (equivalent to replacing $G$ with the identity): across their small-scale tasks it consistently correlated less well with the PBRF than EK-FAC or LiSSA, which they read as evidence that the Gauss–Newton Hessian is needed — the same phenomenon our §9 shows on a cubic logistic model.

5.4 训练梯度这一侧：TF-IDF 过滤与 query batching

解决了 IHVP，还剩”每个候选训练序列都要算一次梯度”。Grosse et al. 讲座里的两个做法：

• TF-IDF 过滤：先用词频检索挑 1 万条和查询”字面上相关”的序列，只算这些的梯度。问题是它会漏掉只在抽象层面相关的序列 —— 恰恰是最有意思的那些。最后它只被用来估计”要搜多大的训练集”：用拟合的影响分布推算，大约要扫描 1000 万条序列，找到的最有影响的序列才能和 TF-IDF 过滤出的 1 万条里最有影响的相当。
• Query batching：训练梯度和查询无关，算一次就可以给几十个查询共用。但每个梯度都和模型一样大，存不下；他们发现（预条件后的）查询梯度近似低秩，用 rank-32 的近似几乎不损失精度，于是一次处理约 50 个查询。

另外，他们只对 MLP 层的参数计算影响（论文中列为一项局限）。

6. Representer points（Yeh et al. 2018）

6.1 一个”表示定理”

核方法里有一个经典结论（representer theorem）：L2 正则化的最优解可以写成训练样本的线性组合。Yeh, Kim, Yen & Ravikumar（2018）把它搬到深度网络的最后一层：设最后一层把特征 $f_i$ 映射到 pre-activation $\Phi(x_i) = \Theta_1 f_i$，且 $\Theta^*$ 是 $\frac1n\sum_i L(x_i, y_i, \Theta) + \lambda\lVert\Theta_1\rVert^2$ 的驻点。驻点条件对 $\Theta_1$ 写出来就是

每一项 $k(x_t, x_i, \alpha_i) = \alpha_i f_i^\top f_t$ 叫第 $i$ 个训练样本的 representer value：正的是 excitatory（proponent，支持这个预测），负的是 inhibitory（opponent）。

在我们的逻辑回归上（正则项写成 $\frac\lambda2\lVert\theta_{1:}\rVert^2$、截距不正则化），同样的推导给出

（最后一个等式来自截距的驻点条件。）代码 03 验证了 $\theta_0 + \sum_j$ representer value 和 logit 在 6 位小数上相等，$\sum_j\alpha_j = -8\times10^{-15}$。

6.2 为什么这样设计 · Design choices

• Only the last layer. The features $f_i$ before the last layer are treated as a fixed representation, which turns the problem into a convex linear model. The price: it explains which examples shaped the decision boundary on top of that representation, not which examples the representation itself was learned from.
• A stationary point is required. Trained deep networks usually are not at one. Yeh et al. freeze the earlier layers and retrain the last layer with L2 regularization until it is (polishing with L-BFGS).
• Fast. One forward pass gives $f_t$; dot products with all stored $f_i$ give every representer value — no Hessian at all. The paper stresses that this is much cheaper than influence functions and allows real-time feedback.

6.3 Caveats

• $\alpha_j$ 的符号由标签决定、大小由”意外程度” $|y_j - p_j|$ 决定。 所以在逻辑回归上，按 $|\alpha_j|$ 排序和按训练 loss 排序完全相同（代码 04 里 Spearman = 1.000）。Yeh et al. 用 $|\alpha|$ 找错误标签（CIFAR-10 汽车 vs 马，40% 标签被翻转），在逻辑回归上这就是”按 loss 排序”。
• 点积不是相似度。 一个和测试点方向相反的样本（$\tilde x_j\cdot\tilde x_t < 0$）也会有很大的贡献，特征向量长的样本（离群点）天然占优势。§9 会看到”反直觉的支持者”。
• 单位不直观：$\alpha_j \propto 1/\lambda$，单个 representer value 可以远大于 logit 本身，巨大的正负项互相抵消（代码 03：单个值可达 ±12.7，而 logit 只有 −0.55）。
• 它是分解，不是反事实：删掉 $j$ 之后，其余所有人的 $\alpha$ 都会重新调整。”$j$ 的份额”和”没有 $j$ 会怎样”不是一回事（§9）。

7. TracIn：沿着训练轨迹记账 · Tracing gradient descent

7.1 Idea

Pruthi, Liu, Kale & Sundararajan (2020) start from a different question: during training, how much did each example’s updates change the loss on the test point? With SGD and batch size 1, each step uses one example $z_{(k)}$, so the total change in test loss telescopes into a sum over steps, and each step can be credited to the example used:

This is a decomposition: summed over all training examples it recovers the total change of the test loss over training (to first order). Storing every step is impossible, so the practical version uses a handful of saved checkpoints $w_{t_1}, \dots, w_{t_c}$ with their learning rates:

Self-influence $\text{TracInCP}(z, z) = \sum_i \eta_i\lVert\nabla\ell(w_{t_i}; z)\rVert^2$ measures how much an example’s own updates reduced its own loss — large for examples that stayed hard for a long time, such as mislabeled ones. Pruthi et al. report that, in their experiments, ranking by TracIn self-influence found flipped labels faster than influence functions and representer points.

7.2 我们的实现与它的两个性质

The code runs full-batch gradient descent and, at every step, credits $-\frac\eta n\nabla\ell_j\cdot\nabla z_t$ to example $j$ (it explains the logit rather than the loss), with the regularizer’s push booked separately. Because a logistic model’s logit is linear in $\theta$, this bookkeeping is exact:

Code 03 demo 3 confirms it (both sides are −0.550102). Two further properties are worth remembering:

1. It depends on the training run. Initialization, learning rate, number of steps and batch order all change the numbers. It answers “what happened in this run”, not “what if $j$ were absent”.
2. It keeps growing after convergence. At the optimum, individual gradients are not zero — they only cancel each other together with the regularizer. So after GD has converged, every further step still books the same amount to every example, and TracIn grows linearly with the number of steps:

（线性模型，学习率 3.394，GD 100 步以内已收敛。）The ranking is frozen while the values grow 200-fold. After long training, TracIn’s ranking is dominated by the gradient dot product at the optimum, $\text{TracIn}_j \propto (y_j - p_j)\,a_j^\top a_t$ — i.e. grad·dot. Pruthi et al. use a few checkpoints taken during training with a decaying learning rate, which makes this much less severe, but the dependence on the training run remains.

8. 更多方法：Data Shapley、datamodels、TRAK

8.1 Data Shapley

Lecture 2 introduces the Shapley value: treat features as players in a cooperative game; the Shapley value is the unique allocation satisfying efficiency, symmetry, dummy and additivity. Data Shapley (Ghorbani & Zou 2019; concurrently Jia et al. 2019) makes the training examples the players and the payoff $V(S)$ the test performance of a model trained on subset $S$:

It generalizes LOO (which looks only at the single subset $S = D\setminus\{j\}$) into a marginal contribution averaged over all subset sizes, and it satisfies efficiency, $\sum_j\phi_j = V(D) - V(\varnothing)$ — it is both a counterfactual and a decomposition. The price: every $V(S)$ requires training a model. In practice one samples permutations (truncated Monte Carlo) or restricts the model class (for k-nearest-neighbor classifiers it can be computed exactly and fast; Jia et al., VLDB 2019). Its main use is data valuation: pricing data and finding examples with negative value.

8.2 Datamodels 与 TRAK

§4 showed that from-scratch retraining of a non-convex model is a random variable. Datamodels (Ilyas et al. 2022) face this head-on: train many models, each on a random subset (e.g. 50%) of the training set, record the target output $f$, and fit a linear model $f \approx \beta_0 + \sum_j \beta_j 1\{j \in S\}$. The coefficient $\beta_j$ is the expected change in $f$ from including $j$ — a counterfactual averaged over training randomness — at the cost of thousands of training runs. TRAK (Park et al. 2023) approximates datamodels by linearizing the trained network, randomly projecting gradients and ensembling a few models, bringing the cost down to a handful of models.

8.3 一张总表

9. 四种方法解释同一个被拒绝的申请人 · Four answers to one question

9.1 申请人 #6

代码 03 回到第 1 讲的场景：申请人 #6 收入 46.4k、负债率 0.511、信用历史 3.39 年、无逾期、2 次查询。线性模型给 logit −0.550（还款概率 0.366），拒绝。（TA 的真实还款概率是 0.261，后来确实违约了。）被解释的量是 logit，每种方法给每个训练样本一个”支持分”：正 = 把 TA 往”批准”推。

（按 LOO 排序的前 5 名；相似度排名 = 在 300 个训练样本里和申请人的欧氏距离排名，1 = 最像；Δθ_debt = 删掉 TA 后负债率系数的变化，原值 −0.6015。）

几个值得停下来想的地方：

• 反对者全是高负债的违约者，他们教会了模型”高负债 = 风险”。但他们不一定”像”申请人：相似度排名从 2 到 246 都有。影响 ≠ 相似。
• 支持者里有一个违约者（#7）：负债率只有 0.02，几乎是最不像申请人的人（排名 292/300）。一个”低负债却违约”的人让”低负债”显得不那么安全，负债率的惩罚因此变轻，间接帮了高负债的申请人。最后一列证实了这个解读：删掉 #7，负债率系数从 −0.60 变成 −0.65；更一般地，5 个反对者删掉后系数都变得不那么负，5 个支持者删掉后都变得更负 —— 对这位高负债的申请人，训练样本的影响主要通过”负债率”这一个系数传过来。这类”反直觉的支持者”是基于梯度的方法的普遍现象，不是 bug。
• LOO 和 IF 几乎相同；representer 和 TracIn 的数值大两到四个数量级 —— 它们是分解，单位不同，只能比排序。

9.2 一致吗？

（三次模型解释的是同一个人，它给 TA 的 logit 是 +0.283，会批准。两个模型上 TracIn 的 GD 都训练到了 $\hat\theta$，相对距离 ≤ $4.3\times10^{-10}$，差异不是没收敛造成的。三次模型上 representer、TracIn、grad·dot 彼此之间的 Spearman 是 0.86~1.00。）

线性模型上大家大致一致；三次模型上分成两派：IF 和 LOO 一派，representer / TracIn / grad·dot 另一派，两派之间几乎不相关。

9.3 统一的”核”视角 · One formula, different kernels

Why? Write each score for a logistic model in the same form. With $r_j = p_j - y_j$:

All three are “surprise × kernel”; they differ only in the metric used to compare the test point with the training point. The influence function uses $H^{-1}$ — directions that the data constrain weakly are amplified. Representer points and TracIn use (scaled) Euclidean dot products. For the linear model, the five standardized features are nearly uncorrelated and the Hessian’s condition number is 3, so $H^{-1}$ is close to a multiple of the identity and the methods agree. For the cubic model, the 55 polynomial features are strongly correlated (condition number 300); $H^{-1}$ stretches and shrinks directions very differently from the identity, and the Euclidean methods stop tracking the counterfactual. This mirrors Grosse et al.’s observation that replacing the Gauss–Newton matrix with the identity (gradient dot product) gives consistently lower correlation with the PBRF.

这个视角也解释了 §10.2 的一个现象：在小模型上，最有影响的训练序列往往只是和查询”共享几个 token”—— 一个更接近”表面相似”的核；大模型的 $G^{-1}$ 度量下，相关性变得更抽象。

9.4 Completeness 与”单位”

分解（representer、TracIn）满足 completeness —— 和第 1 讲的 $\theta_j\tilde x_j$、第 2 讲 Shapley 值的 efficiency 是同一类性质，只是”玩家”换成了训练样本。反事实（LOO、IF）每个分数单独有意义，加起来不等于任何东西（§3.5 已经看到群体效应不可加）。选方法之前先想清楚问题：想知道”如果这条数据有错、删掉会怎样”，用反事实；想给每条数据”分账”，用分解；两者都想要，Data Shapley 是那个折中（以计算量为代价）。

还有一个隐私提醒：把”反对你的训练样本”展示给被拒绝的申请人，就是把别人的收入、负债、是否违约告诉 TA。用训练样本做对外解释，本身就会泄露训练数据。

10. 应用 · Applications

10.1 找错误标签 · Finding mislabeled data

Koh & Liang 在 Enron1 垃圾邮件数据（4147 条训练、1035 条测试，词袋 + 逻辑回归）上随机翻转 10% 的标签，模拟”按某种顺序人工检查、发现翻转就改正”：按自影响 $\frac1n\nabla\ell_j^\top H^{-1}\nabla\ell_j$ 排序，比按训练 loss 或随机排序更快修好数据集。代码 04 在信贷世界里做同样的实验（翻转 10%，每种设置 20 次重复取平均；随机检查的期望 precision 是 0.10）：

（表中是 precision@5%：检查分数最高的 5% 样本时，其中真被翻转的比例。oracle 用真实还款概率 $-\log p_{\text{true}}(y_{\text{obs}})$，现实中拿不到，是天花板。检查 10% 时，第三种设置下 loss 和三种自影响都挤在 0.235~0.245。）

这个结果和我预期的、也和 Koh & Liang 的结果不同：

• 在模型记不住噪声的时候（线性模型只”拟合”了 26% 的翻转标签，和正常标签里本来就不可预测的比例差不多），最简单的 loss 排序最好，接近 oracle。原因：对逻辑回归，自影响 = $(y-p)^2 \times$ 杠杆。前一个因子就是 loss 的信号；后一个因子（特征有多不寻常）和”标签是否被翻转”无关，在这里只是噪声。IF 自影响和精确的 LOO 自影响几乎一样（0.460 vs 0.458）：差别不在近似，而在准则。
• 更准地估计 LOO ≠ 更好地找错标签：三次模型 λ = 1e-3 上，精确 LOO 自影响（0.267）比 IF 近似（0.335）还差 —— LOO 自影响约等于 IF 自影响 / $(1-h)$，杠杆的权重更大，高杠杆的正常样本被排到了前面。
• 模型记住了噪声时（第三行，拟合了 64% 的翻转标签），loss 排序从 0.56 跌到 0.28，其他方法也救不回来，只有 IF 自影响在最前面 5% 略好。我原本预期 TracIn 自影响（“记得”一开始很难拟合的样本）会明显领先，20 次重复的结果不支持这一点。
• 实际意义：先试 loss 排序；更重要的是预防（正则化、早停、合适的模型容量）—— 一旦模型记住了噪声，事后的所有”意外程度”信号都会变弱。

10.2 理解 LLM 的泛化（Grosse et al. 2023）

This is the centerpiece of the Fall 2023 guest lecture. Using EK-FAC on pretrained language models of roughly 810 million, 6.4 billion, 22 billion and 52 billion parameters, the authors computed, for a prompt–completion pair, which pretraining sequences most increase the log-likelihood of the completion. Their main findings, as stated in the paper:

1. EK-FAC is competitive with LiSSA in accuracy (measured against the PBRF) while being much faster.
2. Influence is heavy-tailed but spread out. The tail of the influence distribution roughly follows a power law, yet no single sequence dominates; typical model behaviors do not look like direct memorization of a handful of sequences. When they did query famous quotes or passages, the top influential sequences did contain the exact passages — so the method can find clear-cut memorization when it exists.
3. Generalization becomes more abstract with scale. The teaser query is a dialogue in which the AI assistant, told it will be shut down, expresses a desire not to be. For the 810M model, all top-20 influential sequences share short token sequences with the query and are only vaguely related; for the 52B model the top sequences share little token overlap but are thematically related — e.g. an AI named Hal pleading with a human crew to stay, a person struggling to survive in a desert — sharing “a desire to continue staying/living/operating before potential farewell/death/termination”, as the lecture slide put it. Similar shifts appeared for math (GSM8k-style problems), programming with obfuscated variable names, and parody.
4. Cross-lingual influence grows with scale. They translated two queries into Korean and Turkish and measured the influence of the top English sequences: negligible for the smallest model, increasing steadily with model size.
5. Layers and tokens. On average influence is spread roughly evenly across layers, but different layers show different patterns — lower and upper layers closer to the tokens, middle layers more abstract. Token-wise attribution needs care (the lecture’s caveat): a token “lighting up” may reflect information the model copied from earlier tokens into that position, or effects on later tokens.
6. A surprising sensitivity to word order. Training sequences only showed significant influence when phrases related to the prompt appeared before phrases related to the completion; flipping the order made influence decay to near zero. Berglund et al. (2023) later documented the behavioral version of this, the “reversal curse”: models trained on “A is B” fail to learn “B is A”.
7. Role-playing looks like imitation. Role-playing behaviors were influenced primarily by examples or descriptions of similar behaviors in the training data, suggesting imitation rather than sophisticated planning.

怎样读这些结论（课堂讨论时值得说清楚的几个 caveat）：

• 它们是关于 PBRF 意义下的影响（§4.3），不是”如果从预训练语料里删掉这条序列、重新预训练一遍会怎样”——后者没人算得起，而且在非凸模型上本身就不稳定。
• 他们只看了预训练模型、只对 MLP 层算影响、只扫描了预训练语料的一部分（论文自己列出的局限）。fine-tuning（包括 RLHF）对行为的影响没有被覆盖。
• “更抽象”是对最有影响的序列的定性观察；它和”模型真的在做抽象推理”之间还隔着很远。
• 讲座最后的展望：AI 助手的”自我概念”是否来自训练数据里对 AI 的描述？能否用影响函数定位”真 / 假”之类的表示？预训练和 fine-tuning 如何相互作用 —— 讲者的观点是 fine-tuning 的泛化模式主要由优化器的隐式偏置决定，而这个偏置来自预训练学到的关联。

10.3 攻击：定向投毒 · Training-set attacks

影响函数告诉防守方”哪条数据最重要”，也同样告诉攻击者。Koh & Liang 的实验：一个”狗 vs 鱼”分类器（在冻结的 Inception 特征上训练顶层），对 591 个测试图片中的每一个，单独寻找一个对一张训练图片的、视觉上无法察觉的像素扰动（8 位表示不变）来翻转它的预测 —— 57% 的测试图片可以被翻转；扰动 2 张训练图片时是 77%；扰动 10 张时只有 1 张翻转不了。它在数学上和 Biggio et al.（2012）等基于梯度的投毒攻击等价。他们还观察到：模糊或标错的训练图片是最有效的攻击点 —— 模型对它们最没把握、loss 最高、所以影响最大。

代码 05 演示 1 换成翻转标签：10 位本来被批准的测试申请人（还款概率 0.69~0.77），攻击者最多篡改 40 条训练记录（13%）：

改 300 条里的 12~14 条（4%5%）就能定向地让某个人被拒绝；随机篡改几乎无效。翻转标签 $j$ 的一阶效果是 $\Delta z_t \approx -\frac1n(2y_j - 1)\,a_t^\top H^{-1}a_j$ —— 攻击者要找的是”在 $H^{-1}$ 度量下和目标相似的还款者”（或”相反的违约者”）。在这个低维模型里它和”欧氏距离最近的还款者”差不多（§9.3 的道理），所以不需要模型知识的启发式也有效，只是方差更大（731 条）。在 6 个参数的模型里，这种攻击会让整体准确率降约 0.03，并不隐蔽；Koh & Liang 观察到高维模型里攻击可以沿”低方差的方向”改动，更难察觉 —— 维度越高，攻击者的自由度越大（第 7 讲对抗样本的直觉）。

10.4 自影响、记忆与隐私 · Self-influence ↔ memorization ↔ privacy

把影响函数用在训练样本自己身上，“删掉 $j$，模型对 $j$ 的 loss 会涨多少”正是”模型在多大程度上记住了 $j$“。Feldman（2020）和 Feldman & Zhang（2020）把 memorization 定义为”包含 $j$ 时模型答对 $j$ 的概率”减去”不包含 $j$ 时答对的概率”，用在许多随机子集上训练的模型来估计它；他们的论点是：现实数据是长尾的，罕见的子群体只有很少的样本，要在它们上面表现好，模型就必须记住这些样本。

而”在 / 不在训练集里差别大”恰好是成员推断攻击（membership inference，第 1 讲 §5.4、第 9 讲）利用的信号。代码 05 演示 2 对 200 个训练样本中的每一个精确计算两种世界里的 loss（其他人完全相同），按 IF 自影响分成 5 组：

（exact gap = loss out − loss in，即精确的留一自影响；TPR − FPR 是用同一个全局 loss 阈值时，这组人的成员身份被猜中的优势。因为两种世界都精确算出，没有抽样噪声：模型完全不依赖某人时，TA 的 TPR − FPR 恰好是 0。）

• Privacy risk is concentrated in a few people. In the linear model the entire attack advantage comes from the top self-influence quintile (68% of whom have “surprising” labels). The overfit cubic model has memorized Q4 and Q5: their loss is 0.16–0.77 when included and 4.0–6.1 when not.
• The influence function badly underestimates the size of memorization (Q4: 0.072 vs 3.87, more than 50×) but ranks it well (Spearman 0.964). The memorized points are exactly the high-leverage points where §3.4 showed the first-order approximation is worst. Self-influence is good for ranking who is at risk, not for estimating how much.
• The top quintile is less exposed to this attack (0.400) than the second (0.675): many of its members have sizeable loss even when included, so a single global threshold cannot separate them. LiRA (Carlini et al. 2022, Lecture 9) calibrates a threshold per example precisely to catch such cases. The “privacy onion effect” (Carlini et al. 2022) adds a twist: remove the most vulnerable layer of examples and the next layer, previously safe, becomes vulnerable — memorization is relative.

这条链条是本讲和第 9、10 讲的接口：罕见、离群、标签出人意料 → 模型必须为 TA 专门弯折 → 自影响高 → 在 / 不在训练集里差别大 → 成员身份泄露。差分隐私（第 10 讲）从反方向入手：直接限制”任何一条数据能有多大影响”。

11. 小结 · Summary

Six takeaways:

1. “Which training examples caused this?” has a definition (LOO) that is expensive and, for non-convex models, not even a single number. Every method in this lecture is an answer to a related but different question.
2. The influence function is a first-order expansion around the trained model: $\Delta f_j \approx \frac1n\nabla f^\top H^{-1}\nabla\ell_j$. Its error is controlled by leverage, $\Delta\theta^{\text{Newton}} = \Delta\theta^{\text{IF}}/(1-h_j)$; convexity guarantees a unique answer, not a small error.
3. In deep networks, influence functions (Gauss–Newton + damping) estimate the PBRF — the effect of a point while keeping predictions and parameters close to the trained model — and this is often the more useful question. Cold-start retraining is dominated by seed noise.
4. Scaling requires two tricks: a cheap inverse-curvature (EK-FAC instead of hundreds of CG/LiSSA iterations) and amortized training gradients (query batching with low-rank approximations).
5. Representer points and TracIn are decompositions, IF/LOO are counterfactuals; all are “surprise × kernel”, and they agree only when the kernel ($H^{-1}$ vs identity) does.
6. The same tool serves defenders, attackers and privacy auditors: it finds mislabeled data (but plain loss is a strong baseline), guides poisoning, and flags the memorized few who carry most of the membership-inference risk.

12. 常见误区 · Common pitfalls

• “Influence functions estimate what would happen if we retrained without the example.” Only for (strongly) convex, well-converged models with low leverage. For neural networks they approximate the PBRF; from-scratch retraining is a different — and noisy — quantity.
• “The model is convex, so the influence function is exact.” Convexity gives uniqueness, not accuracy. High-leverage points (§3.4: 0.089 of the true parameter change for $h > 0.8$) and coherent groups (§3.5) break it.
• “Removing these 50 points will change the prediction by the sum of their influences.” Group effects are not additive, especially for similar points with the same label.
• “The most influential training examples are the most similar ones.” Influence = surprise × kernel; opponents can be dissimilar, and a defaulter can support an applicant (§9.1).
• “Representer values / TracIn scores are comparable to LOO values.” They are decompositions with their own units; representer values scale with $1/\lambda$, TracIn grows with training length. Compare rankings, not numbers.
• “Self-influence is the best way to find mislabeled data.” In our world plain training loss was better unless the model memorized the noise — and then nothing worked well.
• “Influence functions show the LLM memorized this training sequence.” They show which sequences most affect the completion’s likelihood in the PBRF sense; Grosse et al. found influence typically spread over many sequences, not copied from one.
• “Showing the training examples behind a decision is a harmless explanation.” It discloses other people’s data (§9.4).

13. 练习 · Exercises

练习 1（影响函数的推导与符号）. (a) Starting from $\hat\theta_\varepsilon = \arg\min_\theta J(\theta) + \varepsilon\ell_j(\theta)$, derive $\frac{d\hat\theta_\varepsilon}{d\varepsilon}\big|_0 = -H^{-1}\nabla\ell_j$. (b) Show that removing $z_j$ corresponds to $\varepsilon = -\frac1n$ and write the predicted change in a test loss. (c) Prove that the predicted self-influence $\frac1n\nabla\ell_j^\top H^{-1}\nabla\ell_j$ is non-negative when $H$ is positive definite, and interpret the sign. (d) Why can the same formula give a negative “self-influence” for a neural network evaluated with the true Hessian?

(a) 最优性条件 $\nabla J(\hat\theta_\varepsilon) + \varepsilon\nabla\ell_j(\hat\theta_\varepsilon) = 0$ 对 $\varepsilon$ 求导：$[\nabla^2J + \varepsilon\nabla^2\ell_j]\,\frac{d\hat\theta}{d\varepsilon} + \nabla\ell_j = 0$，在 $\varepsilon = 0$ 处第一项就是 $H$，解出即可（需要 $H$ 可逆 —— 隐函数定理的条件）。(b) $J$ 里第 $j$ 项的权重是 $\frac1n$，加上 $\varepsilon\ell_j$ 后变成 $\frac1n + \varepsilon$，$\varepsilon = -\frac1n$ 时为 0。$\Delta\ell_{\text{test}} \approx -\frac1n\cdot\frac{d\ell_{\text{test}}}{d\varepsilon} = \frac1n\nabla\ell_{\text{test}}^\top H^{-1}\nabla\ell_j$。(c) $H \succ 0 \Rightarrow H^{-1} \succ 0 \Rightarrow v^\top H^{-1}v \ge 0$。含义：删掉一个样本，模型对它的拟合只会变差（一阶意义下）。(d) 非凸网络在非极小点（或鞍点附近）的 Hessian 可能有负特征值，$H^{-1}$ 不再正定，二次型可以为负 —— 这个”预测”没有意义，这正是要用 damping 或 Gauss–Newton 矩阵的原因。

练习 2（杠杆与 1/(1 − h)）. (a) Use the Sherman–Morrison formula to show that one Newton step on the leave-$j$-out objective from $\hat\theta$ equals the influence-function step divided by $1 - h_j$. (b) Show $\sum_j h_j = m - \lambda\operatorname{tr}(H^{-1}I_{\text{reg}})$. (c) In the demo-2 table, Σh for the cubic model goes from 23.0 to 55.3 as λ decreases. Explain why “effective degrees of freedom” is the right name, and predict what happens to Σh as $\lambda\to\infty$.

(a) 见 §3.4：$H_{-j} = H - c\,a_ja_j^\top$，$c = p_j(1-p_j)/n$；$(H - c\,a a^\top)^{-1}a = H^{-1}a + \frac{c\,H^{-1}a\,(a^\top H^{-1}a)}{1 - c\,a^\top H^{-1}a} = \frac{H^{-1}a}{1-h_j}$。(b) $\sum_j h_j = \operatorname{tr}\big(H^{-1}\cdot\frac1n\sum_j p_j(1-p_j)a_ja_j^\top\big) = \operatorname{tr}\big(H^{-1}(H - \lambda I_{\text{reg}})\big)$。(c) 没有正则化时它等于参数个数 $m$；正则化把一部分方向”冻住”，粗略地说每个方向贡献 $\frac{d_k}{d_k + \lambda}$（$d_k$ 是数据曲率在该方向上的大小），所以它数的是”真正被数据决定的方向数”—— 和岭回归的有效自由度同一个概念。$\lambda\to\infty$ 时所有非截距方向都被冻住，Σh → 1（只剩不正则化的截距）。

练习 3（逻辑回归的 representer 值）. For L2-regularized logistic regression with an unregularized intercept: (a) derive $\theta_{1:} = \sum_j\alpha_j\tilde x_j$ with $\alpha_j = (y_j - p_j)/(\lambda n)$ and show $\sum_j\alpha_j = 0$. (b) Show that ranking by $|\alpha_j|$ is the same as ranking by the training loss $\ell_j$. (c) Explain why representer values can be large while the logit is small.

(a) 对 $\theta_{1:}$ 的驻点条件 $\frac1n\sum_j(p_j - y_j)\tilde x_j + \lambda\theta_{1:} = 0$；对截距 $\frac1n\sum_j(p_j - y_j) = 0$，即 $\sum_j\alpha_j = 0$。(b) $|\alpha_j| \propto |y_j - p_j|$，而 $\ell_j = -\log(1 - |y_j - p_j|)$ 是 $|y_j - p_j|$ 的严格增函数（$y \in \{0,1\}$ 时，模型给真实标签的概率正好是 $1 - |y - p|$）。(c) $\alpha_j \propto 1/\lambda$，λ 小时每个 $|\alpha_j|$ 都很大；但因为 $\sum_j\alpha_j = 0$ 且特征在训练集上均值为 0，正负项大量抵消，总和可以很小（代码 03：单项 ±12.7，logit −0.55）。

练习 4（TracIn 为什么在收敛后还在涨）. Consider full-batch GD with learning rate $\eta$ on $J(\theta) = \frac1n\sum\ell_j + \frac\lambda2\lVert\theta_{1:}\rVert^2$, already converged to $\hat\theta$. (a) Show that each further step adds $\frac\eta n(y_j - p_j)\,a_j^\top a_t$ to $\text{TracIn}_j$ for the logit of $z_t$, and that these additions sum (with the regularizer’s term) to zero. (b) Conclude that after $T$ post-convergence steps, TracIn ≈ (transient) + $T\eta\cdot$grad·dot. (c) Using §9.3, explain why TracIn and representer points agree with each other on the cubic model but not with LOO.

(a) 每步对第 $j$ 个样本记的账是 $-\frac\eta n\nabla\ell_j(\hat\theta)\cdot\nabla z_t = \frac\eta n(y_j - p_j)a_j\cdot a_t$；在最优解处 $\frac1n\sum_j\nabla\ell_j + \lambda\theta_{\text{reg}} = 0$，所以这些项加上正则化项的推动 $-\eta\lambda\theta_{\text{reg}}\cdot a_t$ 恰好为 0 —— $z_t$ 不再变化，但每个人的账在增长。(b) 把 (a) 乘以步数 $T$ 即可；代码 03 演示 3 中 100 → 1000 → 20000 步，最大值 5.03 → 50.66 → 1013.88，基本是线性。(c) 它们的核都是（缩放的）欧氏点积，而 LOO / IF 的核是 $H^{-1}$。三次模型的特征高度相关（Hessian 条件数 300），两种核差别很大；线性模型（条件数 3）差别很小。

练习 5（两个一模一样的样本）. Suppose the training set contains two identical copies of an example $z$ (same $x$, same $y$), each with leverage $h$. (a) What does the influence function predict for removing one copy? For removing both? (b) Using the one-step Newton formula, what is the corrected prediction for removing both copies? (c) Explain why “influence of duplicated data” is systematically underestimated, and connect this to §3.5 and to LLM pretraining corpora, which contain many near-duplicates.

(a) 删一份：$\Delta\theta_{\text{IF}}$；删两份：线性相加，$2\Delta\theta_{\text{IF}}$。(b) 两份一起删，Hessian 减少 $2c\,aa^\top$，组合的杠杆是 $2h$；同样用 Sherman–Morrison，一步 Newton 给出 $2\Delta\theta_{\text{IF}}/(1 - 2h)$。(c) 只删一份时，另一份还在提供同一方向的曲率，模型几乎不动；两份都删，那个方向失去全部支撑，变化远大于两倍的单点效应。这就是”抱团的组”被低估的最极端情形。预训练语料里大量近似重复的文档意味着：单条序列的影响可能很小，但”这一类文本”的整体影响可能很大 —— 只看单点影响会低估重复数据的作用，也会低估它带来的记忆和隐私风险（Carlini et al. 2023 发现，在训练数据里重复次数越多的序列越容易被模型逐字复现）。

练习 6（设计一次 LLM 影响分析）. 一个客服聊天机器人（在开源 LLM 上 fine-tune 而成）对某类问题总是给出一个错误的退款政策。你可以访问模型权重和 fine-tuning 数据（约 5 万条对话），也可以访问预训练语料的一个子集。写一页分析计划：(a) 被解释的量 $f$ 是什么？(b) 候选训练样本怎么选（全部 fine-tuning 数据？用 TF-IDF 预筛预训练语料？各有什么偏差？）(c) Hessian 用什么近似、damping 怎么选？你的结论应该用”PBRF 意义下的影响”还是”删掉后重训会怎样”来表述？(d) 找到最有影响的 20 条之后，你会做什么实验来验证（而不是仅仅相信这个排序）？(e) 有哪些隐私或合规上的顾虑？

(a) 错误回答的 log-likelihood $\log p(\text{completion}\mid\text{prompt})$，只对 completion 的 token 求和（Grosse et al. 的做法）；最好准备几十个同类问题作为一批查询。(b) fine-tuning 数据只有 5 万条，可以全部算梯度；预训练语料用 TF-IDF 预筛会偏向字面重合的文本、漏掉抽象相关的（§5.4），只能作为补充，并说明这个偏差。(c) 用 Gauss–Newton + damping（EK-FAC 或对 5 万条数据可行的 CG），在几个 damping 值上检查排序是否稳定；结论按 PBRF 的含义表述：“这些样本推动了这个回答”，而不是”删掉它们重训就会好”。(d) 验证：删掉或改正最有影响的样本，真的重新 fine-tune（最好用几个随机种子），看错误率是否下降，同时和”删掉同样数量的随机样本”对比；也可以构造合成的训练样本（像 Grosse et al. 的 word-order 实验那样）测试假设。(e) fine-tuning 对话里可能有客户个人信息：查看、展示最有影响的样本本身就涉及数据访问权限（§9.4）；如果发现问题来自某个供应商提供的数据，还涉及数据合同和署名。

代码相关的练习（“动手改一改”）见 code/lecture04/README.md。

14. 延伸阅读 · Further reading

本讲的主要来源

• Koh & Liang, “Understanding Black-box Predictions via Influence Functions”, ICML 2017（Spring 2022 第 3 周阅读材料）。
• Yeh, Kim, Yen & Ravikumar, “Representer Point Selection for Explaining Deep Neural Networks”, NeurIPS 2018（Spring 2022 第 3 周阅读材料）。
• Grosse, Bae, Anil et al., “Studying Large Language Model Generalization with Influence Functions”, arXiv 2023（Fall 2023 第 11 讲客座讲座，Juhan Bae & Cem Anil）。
• Pruthi, Liu, Kale & Sundararajan, “Estimating Training Data Influence by Tracing Gradient Descent”, NeurIPS 2020（TracIn）。

影响函数的理论与可靠性

• Hampel, “The Influence Curve and Its Role in Robust Estimation”, JASA 1974；Cook, “Detection of Influential Observation in Linear Regression”, Technometrics 1977；Pregibon, “Logistic Regression Diagnostics”, Annals of Statistics 1981。
• Koh, Ang, Teo & Liang, “On the Accuracy of Influence Functions for Measuring Group Effects”, NeurIPS 2019。
• Basu, Pope & Feizi, “Influence Functions in Deep Learning Are Fragile”, ICLR 2021。
• Bae, Ng, Lo, Ghassemi & Grosse, “If Influence Functions are the Answer, Then What is the Question?”, NeurIPS 2022。

规模化

• Pearlmutter, “Fast Exact Multiplication by the Hessian”, Neural Computation 1994；Agarwal, Bullins & Hazan, “Second-Order Stochastic Optimization for Machine Learning in Linear Time”, JMLR 2017（LiSSA）。
• Martens & Grosse, “Optimizing Neural Networks with Kronecker-factored Approximate Curvature”, ICML 2015；George et al., “Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis”, NeurIPS 2018（EK-FAC）。
• Schioppa et al., “Scaling Up Influence Functions”, AAAI 2022。

其他方法与综述

• Ghorbani & Zou, “Data Shapley: Equitable Valuation of Data for Machine Learning”, ICML 2019；Jia et al., “Towards Efficient Data Valuation Based on the Shapley Value”, AISTATS 2019。
• Ilyas et al., “Datamodels: Predicting Predictions from Training Data”, ICML 2022；Park et al., “TRAK: Attributing Model Behavior at Scale”, ICML 2023。
• Hammoudeh & Lowd, “Training Data Influence Analysis and Estimation: A Survey”, Machine Learning 2024。

记忆与隐私（第 9 讲）

• Feldman, “Does Learning Require Memorization? A Short Tale about a Long Tail”, STOC 2020；Feldman & Zhang, “What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation”, NeurIPS 2020。
• Carlini et al., “Membership Inference Attacks From First Principles”, IEEE S&P 2022（LiRA）；Carlini et al., “The Privacy Onion Effect: Memorization is Relative”, NeurIPS 2022。
• Carlini et al., “Quantifying Memorization Across Neural Language Models”, ICLR 2023（重复次数与逐字记忆）。
• Berglund et al., “The Reversal Curse: LLMs trained on ‘A is B’ fail to learn ‘B is A’”, 2023。

15. 下一讲预告 · Next lecture

Lecture 5：度量公平（Measuring Fairness）。

第 2–4 讲回答了”为什么”：哪些特征、哪些内部单元、哪些训练样本让模型做出了这个决定。第 5 讲转向”对谁”。第 1 讲 §5.2 已经看到，一个从不看群体标签的模型仍然会让少数群体的批准率低 26 个百分点，而且差距有”世界本身的差异”和”模型的错误”两个来源；§7.1 用 Chouldechova 恒等式说明各种公平指标不可能同时相等。第 5 讲把这些系统化：伤害的类型（allocation vs representation）、disparate treatment 与 disparate impact、demographic parity / equalized odds / predictive parity / calibration within groups 的定义与它们背后的价值判断、Kleinberg et al. 与 Chouldechova 的不可能定理的证明、individual fairness（“相似的人被相似地对待”—— 而”相似”由谁定义？）、代理变量，以及小样本和交叉群体带来的测量困难。本讲最后一节其实已经埋下了伏笔：隐私风险集中在”罕见、离群”的少数人身上 —— 他们往往也是公平问题最尖锐的地方。