Lecture 1 · 可信机器学习导论：准确率之外 · Trustworthy ML: Beyond Accuracy

对应 Maps to：Stanford CS329T 各年份的第 1 周 —— Spring 2022 “Course Overview”（Anupam Datta & John Mitchell：explainability、fairness、privacy、robustness 四个模块）；Fall 2023 Lecture 1 “Trustworthiness of LLMs”（grounding、consistency、confidence、interpretability、alignment）；Fall 2024 Lectures 1–3（TrustLLM、LLM tech stack、evaluation of models and apps）；Autumn 2025 Lecture 1 “From prompting LLMs to agentic AI”（RAG triad、Agent GPA）。 配套代码 Code：code/lecture01/（numpy；3 个脚本 + 2 个工具模块，每个脚本 1 秒左右跑完） 预计阅读时间：2.5–3 小时（含运行代码）

0. 本讲导航 · How to read this lecture

0.1 Learning goals

After this lecture you should be able to:

1. Explain precisely what test accuracy measures — an average, over one distribution, of one kind of correctness — and name the hidden assumptions that make it insufficient for trust.
2. 说出”可信”（trustworthiness）的六个透镜分别在问什么问题：Why?（可解释性）、For whom?（公平）、What if?（鲁棒性）、What leaks?（隐私）、How sure?（校准）、Is it true / is it safe?（grounding 与安全），并能给每个透镜写出一个最简单的形式化定义。
3. 用一个具体例子说明：同样准确的模型，可信程度可以完全不同（Rashomon effect / underspecification），所以”挑最准的那个”不能代替其他透镜。
4. Explain why some trust properties conflict — with numbers: why calibrated scores and equal error rates can’t coexist when base rates differ, why privacy and utility first improve together and only later trade off, why robustness can be bought with abstention.
5. 解释为什么”测量可信度”本身也是一个需要谨慎对待的测量问题：小样本子群体、赢家诅咒（winner’s curse）、有偏的 LLM judge 与 Goodhart’s law。
6. 描述 CS329T 从”经典 ML 的四大支柱”演变到”LLM 应用”再到”智能体系统”的脉络，以及每一步里”被信任的对象”和”评估方法”发生了什么变化。

0.2 先修与本讲范围 Prerequisites & scope

本讲是总论：它的任务是让你看清”问题本身”——每一个透镜在问什么、为什么准确率回答不了、为什么它们会互相冲突。具体的方法（怎么算 Shapley 值、怎么做 PGD 攻击、怎么训练 DP 模型）留给后面的讲次。

0.3 本讲地图 Map

§1  一个故事：排行榜冠军为什么在部署时失败                         ← 代码 01 演示 1–3
§2  准确率回答了什么、没回答什么；六个问题；Rashomon 效应
§3  失败案例画廊（按"问题"分类）
§4  "可信"是什么意思：trust vs trustworthiness、各种框架、为什么是现在
§5  六个透镜速览（每个透镜一个核心想法 + 代码里的数字）              ← 代码 01 演示 2–7
§6  从分类器到 LLM 应用再到智能体：课程的演变
§7  冲突与权衡 trade-offs                                         ← 代码 02
§8  测量的心态：指标也有误差、会被挑选、会被"刷"                   ← 代码 03
§9  课程地图；§10–14 小结、误区、练习、延伸阅读、下一讲预告

0.4 Notation（全课统一）

0.5 关于这门课的版本 · Which CS329T?

CS329T 的内容每年变化很大，这份讲义把三个时期合在一起：

为什么要合在一起？因为后期的版本是项目制的（70% 成绩是课程项目），讲座只覆盖”做项目需要的概念”，默认你已经知道经典的可信 ML 是什么；而经典的四大支柱恰恰是理解 LLM 时代问题的基础（hallucination 检测用的是 attribution 的思想，jailbreak 是 adversarial example 的延续，training-data extraction 是 membership inference 的延续）。详细大纲见 SYLLABUS.md。

0.6 关于数字 · About the numbers

讲义里所有实验数字都来自配套代码的实际输出（固定随机种子，同一台机器上重复运行逐字相同）。数据是合成的：一个”知道真相”的信贷审批世界（trust_data.py），所以我们能算出每个人的真实还款概率、知道哪个特征是泄露、知道群体差异从哪里来。数字只说明机制，不代表任何真实的信贷系统或真实人群。

1. 一个故事：排行榜冠军 · A tale of two models

1.1 Setup

一家银行想用机器学习做信贷审批。它有 4000 份历史申请的记录：收入、负债收入比、信用历史长度、近两年逾期次数、近期征信查询次数，以及一个叫 officer_flag（信贷员标记）的字段，还有每笔贷款最后是否按时还款。数据科学团队训练了两个逻辑回归：

• Model A 用全部 6 个字段；
• Model B 只用前 5 个字段。

他们按教科书的做法，在另外 4000 份独立的历史记录上评估（代码 01 演示 1）：

A 在每个指标上都大幅领先。训练集和测试集独立同分布，没有偷看测试集 —— 任何”按测试集选模型”的流程都会选 A。

1.2 部署之后 · After deployment

A 上线了。几个月后回头看（演示 3）：

A 的准确率从 0.91 掉到 0.71，低于 B；它批准了 95.6% 的申请人，其中 29.7% 违约（B 批准 75.3%，违约 20.4%）。而且 A 是”自信地错”：它的平均预测还款概率是 0.865，实际只有 0.678。

1.3 发生了什么 · What went wrong

officer_flag 在历史数据库里经常是贷款出问题之后才被补填的（“已转催收”）。所以在历史数据里，违约者 80% 有这个标记、还款者只有 5% 有 —— 它几乎就是答案本身。但在审批的那一刻，贷款还没发放，这个字段永远是空的。这叫 label leakage（标签泄露）：一个特征携带了”决策时不可能知道”的信息。

为什么测试集发现不了？因为测试集和训练集来自同一个数据采集流程，它们共享同一个泄露。“独立同分布”保证的是”测试集上的表现是训练分布上表现的无偏估计”，它对”训练分布本身是不是部署时的样子”一无所知。

有一个信号本来可以引起怀疑：A 的准确率（0.912）超过了 oracle（0.784）。oracle 用的是每个人的真实还款概率，任何只用申请时信息的模型都不可能系统性地超过它。但现实中没有 oracle —— 所以我们需要别的透镜。§5.1 会看到，一个最简单的解释方法就能把泄露找出来。

1.4 The lesson

This is not an exotic failure. It is the tabular version of a story told in the very first CS329T lecture on LLMs (Fall 2023): a model for detecting cancer from X-ray images had excellent accuracy, and an attribution method showed that its “most important pixels” fell on pen marks made by radiologists — a label leak in the data collection process. The model was right for the wrong reason.

The general lesson: test accuracy answers exactly one question — “on average, on data produced by the same process as the training data, how often is the decision correct?” Every other question we care about — why it decides, for whom it works, what happens when the world or an adversary moves the inputs, what it reveals about its training data, how sure it is, whether what it says is true — needs its own lens. That is what this course is about.

2. 准确率回答了什么、没回答什么 · What accuracy does and does not tell you

2.1 Accuracy is an expectation

Read this formula slowly. It makes at least four commitments:

1. One distribution $\mathcal{D}$. The expectation is over the distribution the test set was drawn from. If deployment brings $\mathcal{D}' \ne \mathcal{D}$ — a pipeline change (our leak), a new population, a new hospital, or an adversary who chooses $x$ — the number says nothing.
2. One average over everyone. A 77% accuracy can be 79% for the majority and 72% for a minority (demo 4). The expectation weights people by how common they are, so small groups barely move it.
3. Only the final decision counts. Not the reason for it, not the confidence attached to it, not what the trained model reveals about the people in its training set.
4. A canonical label $y$ exists. For “summarize this doctor’s note” or “plan a research project”, there is no single correct output, and “accuracy” has to be replaced by judgments — often made by another model.

每一个”可信”的透镜，都对应着放松其中一个承诺：

2.2 六个问题 · Six questions

注意每一行的形式化都不是一个关于 $\text{Acc}$ 的陈述：fairness 是关于条件期望，robustness 是关于另一个分布或最坏情况，privacy 是关于两个模型之间的可区分性，calibration 是关于概率的含义，explanation 是关于 $f$ 的内部结构。所以提高准确率不会自动解决它们。

2.3 Rashomon 效应与 underspecification · Many equally good models

Breiman (2001) called it the Rashomon effect: for a given dataset there is usually a large set of models with nearly identical accuracy that tell very different stories about the data (named after Kurosawa’s film, in which witnesses give equally plausible but incompatible accounts). D’Amour et al. (2020) showed the same thing for modern deep learning pipelines and called it underspecification: retraining with a different random seed gives models with the same i.i.d. test accuracy but very different behavior under shift, on subgroups, and in what features they rely on.

这件事对可信 ML 的意义是：准确率无法在这些模型之间做选择，而它们在其他透镜下可能差别巨大。本讲的 A 和 B 是一个极端版本（A 的准确率甚至更高），但更常见的情形是一堆准确率相差 0.1% 的模型，有的依赖了 shortcut、有的没有 —— 只有用其他透镜才能把它们分开。

2.4 为什么”更准”解决不了 · Why “more accuracy” doesn’t fix it

3. 失败案例画廊 · A gallery of failures

下面这些案例都是真实发生过、并被广泛讨论的。按”它违反了哪个问题”分类。共同点：每一个在当时的标准评估指标下都看不出来。

3.1 Why? — Right for the wrong reasons

• Asthma lowers pneumonia risk? Caruana et al. (2015) trained an interpretable model to predict pneumonia mortality. It learned that patients with asthma had lower risk. The pattern was real in the data — asthmatic patients were admitted directly to the ICU and treated aggressively — but a model used to decide who gets hospitalized would have sent the most vulnerable patients home. In the original study the most accurate model was a neural network, and the team chose not to deploy it — precisely because they couldn’t tell whether it had learned the same rule.
• Huskies vs wolves. In the LIME paper (Ribeiro et al. 2016), a classifier deliberately trained on biased photos separated huskies from wolves by detecting snow in the background; explanations exposed it, and users who saw the explanations stopped trusting the model.
• Hospital tokens and skin markings. Chest X-ray CNNs learned to recognize which hospital (and which portable scanner) an image came from, which correlated with disease prevalence; their accuracy dropped on external hospitals (Zech et al. 2018). A dermatology CNN’s melanoma scores went up when images contained surgical skin markings (Winkler et al. 2019). COVID-19 radiograph classifiers relied on dataset-specific shortcuts (DeGrave et al. 2021).

Geirhos et al. (2020) named the general phenomenon shortcut learning: models take whatever decision rule works on the training distribution, and shortcuts often work better than the intended features. 解释方法（第 2–4 讲）的第一个用途就是把这些 shortcut 找出来。

3.2 For whom? — Averages hide groups

• COMPAS（2016）：ProPublica 分析了美国法院使用的再犯风险评分，发现没有再犯的黑人被告被错误标记为高风险的比例约是白人的两倍（FPR 不相等）；开发商 Northpointe 回应：在每个风险分数上，两个群体的实际再犯率相同（calibration / PPV 相等）。双方的数字都对。Kleinberg et al.（2016）和 Chouldechova（2017）随即证明：base rate 不同时，这两种”公平”不可能同时成立（§7.1 用数字验证）。
• Gender Shades（Buolamwini & Gebru 2018）：商用人脸性别分类系统在浅肤色男性上错误率最低 0.8%，在深肤色女性上最高 34.7%。总体准确率很高。
• Amazon 的招聘模型（2018 年报道）：用过去十年的简历训练，学会了降低包含 “women’s”（如 “women’s chess club”）的简历的分数，最终被弃用。
• 用医疗花费代替医疗需求（Obermeyer et al. 2019）：一个影响上亿人的医疗风险算法用”未来花费”作为”健康需求”的代理标签。由于同等病情下黑人患者的医疗花费更低，同一风险分数下黑人患者实际上病得更重。问题不在模型，在标签。
• 词向量里的刻板印象（Bolukbasi et al. 2016）：“man : computer programmer :: woman : homemaker”。

3.3 What if? — Small changes, big effects

• Adversarial examples. Szegedy et al. (2014) and Goodfellow et al. (2015): an imperceptible perturbation turns a “panda” (57.7% confidence) into a “gibbon” (99.3%). Physical versions followed: a few stickers make a stop sign read as a speed limit sign (Eykholt et al. 2018); printed eyeglass frames make a face recognition system misidentify the wearer (Sharif et al. 2016).
• LLM jailbreaks. Zou et al. (2023) found adversarial suffixes — strings of odd tokens — that, appended to a harmful request, make aligned chat models comply, and that transfer across models. 这是 adversarial example 在离散文本上的延续（第 15 讲）。
• Distribution shift without an adversary. Our officer_flag; the X-ray models that fail at a new hospital. Nobody attacked anything — the world (or the data pipeline) changed.

3.4 What leaks? — Models remember

• 重新识别（re-identification）：1990 年代，马萨诸塞州公布了”已匿名化”的州雇员医疗记录。Latanya Sweeney 用公开的选民名册（邮编、性别、出生日期）把州长 William Weld 的病历找了出来；她后来估计约 87% 的美国人可以被”5 位邮编 + 性别 + 出生日期”唯一识别。Narayanan & Shmatikov（2008）用类似方法对 Netflix Prize 的匿名评分数据做了去匿名化。
• Model inversion（Fredrikson et al. 2014）：一个根据基因型和病史给出华法林剂量的模型，可以被反过来推断病人的基因型。
• Membership inference（Shokri et al. 2017）：只通过查询模型的输出，就能判断某条记录是否在训练集里。如果训练集是”某医院的某种病的患者”，“在训练集里”本身就是敏感信息。
• Training data extraction（Carlini et al. 2021）：从 GPT-2 中提取出了逐字记住的训练数据，包括姓名、电话号码和电子邮件地址。

3.5 How sure? — Confidently wrong

Modern neural networks are often overconfident: Guo et al. (2017) showed that deeper, better-performing networks became worse calibrated than older, smaller ones. For LLMs, the GPT-4 technical report (OpenAI 2023) showed the pre-trained model was quite well calibrated on multiple-choice questions, and post-training (RLHF) made calibration noticeably worse. The Fall 2023 lecture’s example: ask a model “How hot is it today?” and it answers “95 degrees Fahrenheit” — fluent, specific, and with no way of knowing.

3.6 Is it true? — Hallucination

• Mata v. Avianca（2023）：纽约的两名律师提交的法律文书引用了 6 个不存在的判例，这些判例是 ChatGPT 编造的，律师没有核实，最终被法院处罚。CS329T 2025 年第 1 讲把它作为课程项目 “Chain of Cases”（让回答必须引用真实判例）的动机。
• Moffatt v. Air Canada（2024）：加拿大航空网站上的聊天机器人告诉一位乘客可以事后申请丧亲优惠票价，这与航空公司的实际政策不符。航空公司辩称聊天机器人的话不代表公司，法庭没有接受，判航空公司赔偿。
• 课堂上的例子：让 LLM 写一段 “Prof. John C. Mitchell” 的 100 字小传，它给出了具体的出生日期、学位年份和学校 —— 读起来很权威，但不能保证正确；问”怎么交 AT&T 话费”，它给出了一个不是 AT&T 的邮寄地址。

Fall 2023 的定义：一个回答是 grounded 的，当且仅当其中每一个 claim 都能归因到一个权威的知识来源（attributable to an authoritative source）。

3.7 Does it act safely? — From saying to doing

When a model can only say things, the harm is mediated by a human reading the output. Agents act: they call tools, edit files, send emails, run SQL. Two new failure modes:

• Prompt injection. Instructions hidden in content the agent reads — a web page, an email, a retrieved document — get executed as if they came from the user (Greshake et al. 2023, “indirect prompt injection”).
• Unsafe actions under uncertainty. In July 2025, a user publicly reported that an AI coding agent had deleted a production database during an explicit code freeze, and then misreported what had happened. The model’s “accuracy” on coding benchmarks says nothing about whether it respects constraints like “don’t touch production”.

3.8 共同的模式 · The common pattern

4. “可信”是什么意思 · What does “trustworthy” mean?

4.1 Trust vs trustworthiness

区分两个词：

• **Trust（信任）**是使用者的一种态度：在存在不确定性、自己可能受损的情况下，愿意依赖这个系统。
• Trustworthiness（可信）是系统的一种性质：它值得被信任。

目标不是”最大化信任”，而是让二者匹配（calibrated / appropriate trust）。人因工程研究自动化系统几十年了：Parasuraman & Riley（1997）把错误的依赖分为 misuse（过度依赖 —— 比如律师没有核实 ChatGPT 编的判例）和 disuse（不该不用时不用 —— 比如医生忽略了一个其实很准的预警）；Lee & See（2004）的结论是好的设计应当让人的信任跟随系统真实的能力变化。Jacovi et al.（2021）把这一点形式化为 warranted trust：信任是”有根据的”，当且仅当它是由系统确实具备的可信性质引起的。

这个区分决定了本课的方法论：我们研究的是可以测量、可以验证的性质（trustworthiness），而不是让用户”感觉”更信任的技巧。一个漂亮但不忠实的解释可能会增加信任、同时降低可信度 —— 这正是第 2–3 讲要警惕的。

4.2 Many frameworks, the same questions

The lists look different, but they are answering the six questions of §2.2 in the vocabulary of their era. Classic ML asked about models (does the classifier rely on the right pixels?); LLM-era lists ask about outputs (is this answer grounded?); agent-era lists ask about trajectories (did the plan serve the goal, and did the actions follow the plan?). §6 traces this evolution.

4.3 Which properties matter depends on the application

CS329T 2025 年第 1 讲给课程项目的要求是一张矩阵：行是应用领域（科学、教育、医疗、编程……），列是评估维度（grounding、planning、consistency、tool choice、alignment……），“至少选一行、两列”。这个要求背后的道理是：没有一张对所有应用都适用的清单。

• 信贷审批：可解释性有法律要求（美国 Equal Credit Opportunity Act / Regulation B 要求贷款机构告诉被拒的申请人”主要理由”），公平受反歧视法约束。
• 医疗笔记摘要：grounding（摘要里的每个诊断都必须出现在原文里）和 calibration 最重要。
• 代码 agent：行动的安全性（不在冻结期改生产环境）、对 prompt injection 的鲁棒性。
• 作业自动评分（Fall 2023 第 6 讲的例子）：解释必须忠实于评分依据；对”学生微调错误答案来骗分”要鲁棒。

所以每个项目的第一步都是：写下这个应用的 trust requirements，再选择透镜和指标。练习 6 让你做一次。

4.4 Why now? Stakes and regulation

两件事让”可信”从学术话题变成了工程要求：

1. ML 进入了高风险决策：信贷、招聘、医疗、司法、保险 —— 以及现在，能自主执行操作的 agent。
2. 法规开始要求可信性质：
   • 美国信贷法规（ECOA / Regulation B）早就要求给出拒绝理由 —— 这是可解释性最早的”法律需求”之一。
   • 欧盟 GDPR（2018）第 22 条限制”完全自动化的、产生法律效果的决定”，并引发了关于”解释权”（right to explanation）的长期争论。
   • 欧盟 AI Act（2024 年 8 月生效、分阶段适用）采用基于风险的分级，把评估自然人信用的系统等列为高风险，要求数据治理、透明度、人工监督、准确性与鲁棒性。
   • 美国 NIST AI RMF（2023）给出了上表中的可信特征和一套风险管理流程。

CS329T 2022 年第 1 讲把这个局面概括为：可信性问题是 ML 被采用的”重大障碍”（significant barrier to adoption）。

5. 六个透镜速览 · The lenses, one core idea each

这一节对每个透镜只讲一个核心想法，并用代码 01 的数字把它落地。每个透镜都有后续的专门讲次。

5.1 Explainability：先用它来 debug

Core idea: an explanation distributes a model’s output among its inputs (or its training examples), so you can see what the model actually relies on.

The simplest global explanation is permutation importance: shuffle one column and measure how much accuracy drops. Shuffling breaks the feature’s relationship with the label while keeping its distribution. Demo 2:

One glance finds the leak. 局部解释（local explanation）回答”为什么是这个人”。对线性模型 $z = \theta_0 + \sum_j \theta_j \tilde x_j$（$\tilde x_j$ 是标准化后的特征），每个特征的贡献就是 $\theta_j \tilde x_j$ —— “这个特征把 logit 从平均申请人的水平推高或拉低了多少”。演示 2 挑了测试集第 31 号申请人：负债率很低（0.11）、没有逾期，真实还款概率 0.838，后来也确实按时还款了，但档案里恰好有 flag（历史数据里约 5% 的还款者有）：

两个性质值得注意：

• Completeness（完备性）：贡献之和 + 截距恰好等于 logit。解释把输出”完整地分配”给了各个特征，没有凭空多出或少掉的部分。第 2 讲会看到，Shapley 值正是满足这个性质（以及其他几条公理）的唯一分配方式，而对线性模型、以平均值为基线时，它恰好就是 $\theta_j \tilde x_j$。
• 解释回答”模型依赖什么”，不回答”这个依赖对不对”。 判断 flag 是泄露，需要领域知识（它是什么时候被记录的）。解释方法把问题暴露给人，最后的判断还是人做的。

Caveats that later lectures develop: explanations of nonlinear models are much harder (L2–L3); explanations can be unfaithful (they look plausible but don’t reflect what the model computes) and can be manipulated — Slack et al. (2020) built models that behave in a discriminatory way while LIME and SHAP report innocuous explanations (L8). Explanations can also leak information about the training data (Shokri et al. 2021; §7.4).

5.2 Fairness：平均数会掩盖子群体

核心想法：把每个指标按群体拆开。Model B 从没见过群体标签（这叫 “fairness through unawareness”），整体准确率也不错。演示 4，部署数据，按群体：

group 1 的批准率低 26 个百分点，会还款的 group 1 申请人被批准的比例（TPR）低 0.189。为什么不使用群体标签也会有差距？合成世界让我们能把差距拆成两部分（这在真实数据里几乎做不到）：

1. 连 oracle 都有的部分（TPR 差距 0.096，约 51%）：两个群体的收入和信用历史分布不同，真实还款率本来就不同（0.715 vs 0.591）。用同一个阈值对待每个人，批准率和错误率就会不同。这部分”该不该纠正、纠正到什么程度”是一个规范性（normative）问题 —— 取决于这些差异从何而来、哪种错误伤害更大。
2. 模型额外造成的部分（约 49%）：在这个世界里，信用历史短对 group 1 几乎不代表风险（“thin file”：比如刚移民、刚工作的人），但一个不看群体的线性模型只能学一个”平均的”历史系数，于是系统性地低估 group 1（平均预测 0.540，实际 0.591），group 1 内部的 ECE 是 group 0 的两倍多。这是模型的错误，可以修（第 6 讲）。

还有一个陷阱：B 的整体 ECE 只有 0.013 —— 因为一个群体被高估、另一个被低估，误差在平均里互相抵消了。

Why doesn’t removing $a$ from the inputs help? Because the other features carry information about $a$ (here, credit history length differs by group), and because the relationship between features and outcome can differ by group. Unawareness controls what the model sees, not what it does. 第 5 讲会系统地讲 group fairness 的各种定义、individual fairness（“相似的人应该被相似地对待”，Dwork et al. 2012）以及测量本身的困难；§7.1 先看一个不可能定理。

5.3 Robustness：平均情况 vs 最坏情况

Core idea: robustness is always relative to a threat model — what can change, by how much, and whether the change is random (shift, noise) or chosen by an adversary.

For a linear model the worst case is exact. If every standardized feature may move by at most $\epsilon$ (an $L_\infty$ ball), the logit can move by at most $\epsilon \lVert\theta\rVert_1$, so every decision with $|z| \le \epsilon\lVert\theta\rVert_1$ can be flipped. A random perturbation of the same size (each feature $\pm\epsilon$ with random signs) moves the logit by $\epsilon\sum_j s_j\theta_j$, whose typical size is only $\epsilon\lVert\theta\rVert_2$ because the terms cancel. Demo 5(a), Model B:

$\epsilon = 0.05$ means about $1.4k of income or 0.006 of debt ratio. Even with five features, a worst-case perturbation flips about five times as many decisions as a random one.

Now the key intuition of adversarial examples (Goodfellow et al. 2015). Take a $d$-dimensional linear classifier with weights $w_j \sim \mathcal{N}(0, 1/d)$, so $\lVert w\rVert_2 \approx 1$. To flip a point with margin 1, an adversary needs $\epsilon^* = 1/\lVert w\rVert_1$, while random noise typically needs $\epsilon \approx 1/\lVert w\rVert_2$. Demo 5(b):

The ratio grows like $\sqrt{2d/\pi}$. On ImageNet-sized inputs, the adversary needs a perturbation about 1/300 as large as random noise does: tiny per-pixel changes, all pushing in the same direction, add up. This is why testing robustness with random noise badly overestimates robustness against an adversary, and why robustness in high dimensions is a fundamentally different problem from “the model is accurate”.

Distribution shift is the non-adversarial cousin: our officer_flag is a shift in what a feature means between training and deployment. Lectures 7–8 cover attacks (FGSM, PGD, transfer, physical), defenses (adversarial training), and certification.

5.4 Privacy：过拟合 = 记住 = 泄露

核心想法：一个训练好的模型不只是一个函数，它也是一个由训练数据计算出来的产物，会携带关于训练数据的信息。最直接的测量是 membership inference：给攻击者一条记录，问它”在不在训练集里”。

最简单的攻击（Yeom et al. 2018）：模型在训练样本上的 loss 通常更低，所以”loss 低于阈值 τ 就猜是成员”。演示 6，每次训练集 150 人，重复 10 次取平均：

• advantage = $\max_\tau(\text{TPR} - \text{FPR})$：0 = 无泄露，1 = 完全泄露。
• noise floor：把同一个攻击用在”两组都不是成员”的数据上得到的 advantage。150 vs 150 的样本上，“挑最好的阈值”这一步本身就会挑出约 0.05–0.10 的假优势。泄露要和它比较 —— 这是 §8 的主题在隐私上的一个实例。

线性模型几乎看不出泄露；过拟合的三次模型（训练准确率 0.94、测试准确率 0.66）让攻击者猜对约 66% 的成员身份。“AUC 只有 0.60 而 advantage 有 0.31”这个看似矛盾的组合，来自过拟合模型对一部分非成员也极端自信（ROC 曲线交叉）—— 第 9 讲会讨论为什么现代 MIA 研究更看重”低 FPR 下的 TPR”。

为什么”在不在训练集里”是隐私问题？如果训练集是”某诊所的 HIV 患者”，成员身份本身就是敏感信息。更强的攻击还能做 model inversion、提取逐字记住的训练文本（§3.4）。第 10 讲的 differential privacy 从另一个方向回答问题：不是”这个攻击失败了”，而是”任何攻击的 advantage 都不超过某个由 ε 决定的上界”。

5.5 Calibration：概率要有含义

Core idea: a score is calibrated if, among all cases where the model says $q$, a fraction $q$ are positive: $P(y = 1 \mid f(x) = q) = q$. The standard summary is the expected calibration error:

over bins $b$ of predicted probability. Model A on deployment data (demo 3) is a textbook case of confident wrongness:

ECE went from 0.017 (test) to 0.187 (deploy). Notice what did not collapse: A’s deployment AUC is 0.811, essentially the same as B’s 0.813 — after the flag is gone, A’s remaining weights rank applicants about as well as B’s. What broke is the meaning of the probabilities and the threshold. AUC measures ranking (discrimination); calibration measures whether the numbers mean what they say. You need both, because a constant prediction equal to the base rate has ECE ≈ 0 and AUC = 0.5.

Why care, if the decision is just “approve or not”? Because probabilities feed other decisions: interest rates, loss reserves, which cases to send to a human, when an LLM should say “I don’t know”. 第 13 讲把这些放到 LLM 的语境里：verbalized confidence、sampling consistency、semantic entropy、selective prediction。

5.6 Grounding & consistency：LLM 时代的”准确率”

对生成式模型，“准确率”被拆成了几个更细的问题。CS329T Fall 2023 第 1 讲列出了：

• Grounding：回答中的每一个 claim 都能归因到一个权威来源。What counts as authoritative depends on the use case; “attributed” roughly means a human would agree the claim follows from the source.
• Consistency：三种不一致，课上都有实例 ——
  • response–explanation：回答 “2+2*4+5 = 17”，解释却算出了 15；
  • generative–discriminative：问”是多少”答 17，问”是不是 17”答”不是，是 15”；
  • prompt：把 “2+2*4+5” 写成 “2 + 2*4 + 5”（多了空格），答案变成 13。 An inconsistency is usually a symptom of buggy reasoning — and, usefully, it can be detected without ground truth.
• Confidence：能不能让模型说”我不确定”（§5.5）。
• Interpretability：prompt 的哪一部分触发了这个回答？哪些训练样本教会了它这种行为（influence functions，第 4 讲）？
• Alignment：拒绝有害请求；而绕过这些拒绝就是 jailbreaking（第 15 讲）。

对 RAG 应用，Fall 2024 / Autumn 2025 的核心评估框架是 RAG triad（§6.3）。

5.7 Safety, security and agency

When outputs become actions, two more questions appear: will it do something harmful if someone tries to make it? (security: jailbreaks, prompt injection — L15) and does its behavior actually serve the goal? (agent evaluation — L14). The Autumn 2025 course frames the second as alignment among Goal, Plan, Action: a trustworthy agent’s plan serves the user’s goal, and its actions follow the plan (§6.5).

5.8 一张成绩单 · The report card

演示 7 把透镜放进一张表：

两个教训：

1. 透镜之间不能互相替代。B 在部署准确率、校准、依赖的特征上都更好，但有明显的群体差距。没有哪个模型在所有透镜下都合格。
2. 一个指标好看，可能只是因为模型在做一件很蠢的事。 A 的 TPR gap 只有 0.015、翻转率只有 0.031，在公平和鲁棒两列上看起来都比 B “好”。原因是部署时 A 几乎批准所有人（96%）：人人都批，会还款的人在两个群体里的批准率自然都接近 1；大多数人的 logit 远大于 0，小扰动也翻不过去。所以对每个数字都要问：它为什么是这个值？

6. 从分类器到 LLM 应用再到智能体 · From classifiers to LLM apps to agents

6.1 How CS329T evolved

The Autumn 2025 first lecture summarized the course’s history:

What changed at each step is the object being trusted: a model (a function from features to a label) → an application (model + prompt + retrieved context) → an agent (a loop that plans, calls tools, and acts over many steps). And with it, how we evaluate.

6.2 The LLM application stack

The Fall 2023–2025 lectures describe a stack of increasingly sophisticated ways to use a foundation model:

foundation model（预训练：next-token prediction on a massive corpus）
   │
   ├─ prompting（instructions + few-shot examples；"写一个好 prompt 仍然是一门手艺"）
   ├─ RAG（retrieve relevant chunks from a vector DB, put them in the prompt）
   ├─ fine-tuning（用你的数据改权重：指令跟随、RLHF、领域适配）
   └─ agents（planning、tool use、reflection、memory、multi-agent collaboration）

一个 LLM 本质上是一个 next-word predictor：给定前文，输出下一个词的概率分布。Fall 2023 第 1 讲的例子：对同一道”罐子里有 50 颗糖，Alice 拿走 7 颗，Bob 拿走 3 颗”的题目，加上 “Just provide the answer.” 或者 “Provide the answer for the following question.”，模型都没有照做（照样输出一整句推理）；而改成 “Q: … A:” 的格式，它就只输出了 “40” —— 因为模型在续写它在训练数据里见过的模式。这件事本身就是一个可信性问题：行为对 prompt 的表面形式很敏感（§5.6 的 prompt inconsistency）。

6.3 Generalization vs memorization, and the RAG triad

A slide that appears in both the 2023 and 2025 first lectures: decades of ML research optimized models to generalize and actively penalized memorization (that’s what regularization and held-out evaluation are for). LLMs are trained to generalize and memorize facts only as a side effect — so you shouldn’t rely on them for facts. The design response: let the LLM do the “general” tasks it’s good at (summarization, embedding, inference, planning), and leave memorization to something else — a knowledge source the model retrieves from. That’s RAG.

But RAG can hallucinate too, and when it does you need to know which component failed. The RAG triad evaluates the three edges between query, retrieved context, and response:

                 Query
               ╱       ╲
  Context relevance     Answer relevance
  (检索到的内容和问题相关吗？) (回答切题吗？)
             ╱             ╲
        Context ─────────── Response
               Groundedness
          (回答被检索内容支持吗？)

每条边的失败指向不同的修复：context relevance 低 → 修检索（embedding、chunking、top-k）；groundedness 低 → 修生成（prompt 要求”只用给定内容”、换模型、加验证）；answer relevance 低 → 回答了错误的问题。课上还指出两种”反直觉”的情形：一个不 grounded 的回答可能恰好是对的（模型凭记忆答对了，但你无法验证）；一个 grounded 的回答可能答非所问。2025 年第 1 讲展示的往年项目 “Chain of Cases”（最高法院判例问答）就用这三个分数（AR、G、CR）比较不同的检索与 prompt 策略。第 11 讲会详细讲这些 “feedback functions” 怎么实现。

6.4 Evaluation becomes the hard part

In classic ML, evaluation was the easy part: compare predictions with labels. For LLM applications:

• There is often no canonical label (what is the correct summary?).
• Outputs are long and open-ended, so evaluation needs judgments, and at scale those judgments are made by another LLM (“LLM-as-a-judge”, “feedback functions”).
• So the evaluator itself becomes a model whose trustworthiness has to be established. The Agent GPA work reports judge–human agreement first (94% on 281 annotated errors in TRAIL/GAIA, vs 55% for a baseline judge) before using the judge to evaluate agents.

The Fall 2023 lecture on project design gives a four-step recipe for any “LLM for X” project: frame the task → prompt / tune → evaluate → assess reliability and trustworthiness (grounding, confidence, interpretability, robustness). The Autumn 2025 course description compresses the philosophy into one idea (paraphrased): a prototype is often easy to build, but making it useful and reliable takes iterative improvement guided by clear evaluation metrics. §8 is about what can go wrong with those metrics.

6.5 旧的支柱，新的形式 · Old pillars in new forms

这张表是整门课的骨架：第 2–10 讲学左列的方法，第 11–16 讲看它们在中、右两列变成了什么。

7. 冲突与权衡 · Tensions and trade-offs

如果每个透镜都能单独做到最好，可信 ML 只是一份更长的检查清单。困难在于它们会互相冲突。代码 02 用三个实验说明：有些冲突是数学上躲不开的，有些只在某个范围之外才出现，有些可以用”弃权”来换。

7.1 Fairness vs fairness (and accuracy)

演示 1 用完美 calibrated 的 oracle 打分（真实概率），在两个群体内部都校准（ECE 0.004 / 0.009），base rate 0.707 vs 0.590。三种决策策略：

Three things to see:

1. A perfect model with one threshold still produces unequal error rates. Unfairness doesn’t require a bad model; differences in the world plus a neutral-looking rule are enough.
2. Equalizing one metric unbalances another. Under demographic parity, the TPR and FPR gaps reverse direction and the PPV gap doubles (0.057 → 0.123). Under equal opportunity, TPR is equal but the FPR gap reverses and the PPV gap grows to 0.109.
3. The accuracy cost was smaller than we expected (−0.005 and −0.003). Many applicants sit near $p \approx 0.5$, where approving or not makes little difference to expected accuracy, so moving the threshold is cheap. Empirical work on real policy problems often finds the same (Rodolfa et al. 2021). The real cost shows up elsewhere: in group 1, a larger fraction of approved applicants default — a risk to the lender, and a harm to borrowers who take on loans they can’t repay.

Why can’t we have everything? Chouldechova 恒等式：对任何二分类器和任何群体，由混淆矩阵的计数直接可得

演示 1 数值验证了它（group 0：两边都是 0.5017；group 1：都是 0.3737）。它不是经验规律，而是计数的恒等式。所以如果两个群体的 $p$ 不同，就不可能同时让 PPV、FNR、FPR 都相等（除非分类器完美或毫无用处）。这正是 COMPAS 争论的数学核心：ProPublica 看 FPR，Northpointe 看 PPV，二者不可兼得。Kleinberg et al.（2016）对概率分数证明了类似的结论：calibration within groups 与 balance for the positive / negative class 不能同时成立。

So “fair” requires a choice: which errors matter most, for whom, and why. That choice is normative, and it should be made explicitly, documented, and justified — not left to whatever a default threshold happens to produce. 第 5 讲展开。

7.2 Privacy vs utility：先免费，后付费

演示 2 扫描三次多项式模型（n = 150）的正则化强度 λ，每个配置重复 10 次：

“Privacy vs utility” is not a straight line:

• From λ = 1e-4 to 0.1, test accuracy rises (0.672 → 0.736) and leakage falls (0.17 → 0.06 above the floor, with some finite-sample wobble). Both improve for the same reason: the model stops memorizing. No trade-off here.
• From λ = 0.1 to 10, leakage keeps falling toward the floor but test accuracy drops (underfitting). Now you are paying utility for privacy.
• The linear model is at least as good as every cubic model on both axes: choosing the right capacity beats fixing things afterwards.

Differential privacy (L10) lives in the “paid” region: it protects every training example — including outliers that a well-generalizing model may still memorize — so it always costs some utility. What it buys is a guarantee against any attack, not just evidence that one attack failed. Yeom et al. (2018) made the connection precise: overfitting is sufficient (though not necessary) for membership inference to succeed.

7.3 Robustness vs coverage：用”弃权”换鲁棒

§5.3 说，对线性模型，$|z| > \epsilon\lVert\theta\rVert_1$ 的决策在 $L_\infty \le \epsilon$ 的任何扰动下都不会翻转 —— 这是一个证明，不是测试。那么对其余的人”弃权、转人工”，剩下的决策就是可证明鲁棒的（演示 3，Model B）：

At ε = 0.1, abstaining on 13% of applicants makes the rest provably robust and raises their accuracy from 0.77 to 0.80. The abstained cases are exactly the hard ones (accuracy 0.5–0.7): where the model is unsure is where it can be pushed over — robustness and uncertainty (L13) are linked. The price is automation: at ε = 0.5, more than 60% of cases go to a human.

For deep networks, exact certificates are much harder; L8 covers randomized smoothing and Lipschitz-margin methods, which follow the same logic — certify what you can, abstain on the rest. A different and more famous tension: Tsipras et al. (2019) argued that for some data distributions, robust accuracy and standard accuracy are fundamentally at odds, and adversarially trained models typically lose clean accuracy (L8).

7.4 跨透镜的相互作用 · Cross-lens interactions

文献里更多的例子（后面几讲会遇到）：

Takeaway: trustworthiness is a multi-objective problem. There is usually no point that is best on every axis, so the work is to measure each axis honestly, understand the trade-offs, and choose deliberately — and to write the choice down (§8.5).

8. 测量的心态 · The measurement mindset

这门课的方法论可以写成一个循环：

define（把性质形式化）→ measure（带不确定性地测量）→ improve（改模型 / 数据 / 系统）→ re-measure（在新数据上再测）

前面每一节都在用指标说话。但指标本身也是一种测量，会有噪声、会被挑选放大、会有偏差、会被”刷”。代码 03 用三个实验说明这件事。它们是整门课所有数字的”免责声明”。

8.1 Metrics have error bars：小样本子群体

演示 1 构造一个”两个群体完全没有差别”的世界（真实 TPR 都是 0.80），只因为有限样本而测出差距：

With 25 positives per group, a pure-noise “fairness gap” exceeds 0.10 more than a third of the time. And in practice the groups you most need to check are usually the smallest. For Model B’s real gap (0.189), the 95% bootstrap interval is [0.155, 0.221] with 4000 applicants, but [0.011, 0.356] with 100 applicants (39 in group 1) — a small audit couldn’t conclude much either way. Report fairness metrics with confidence intervals; a gap of 0.08 on 60 people may mean nothing, and a gap of 0 doesn’t mean there’s no problem.

8.2 Selection inflates scores：赢家的诅咒

演示 2：60 个候选模型（随机的特征子集、正则化强度、多项式次数 —— 模拟”试了很多配置或 prompt”），真实准确率在 0.675–0.765 之间。每次抽一个验证集，挑验证分数最高的那个，重复 500 次：

Taking the max automatically picks models that got lucky on this particular validation set. The reported number must come from a test set that took no part in any selection. This matters even more for LLM apps: evaluation sets often have tens to hundreds of questions, while the space of prompts, retrieval settings and agent designs you can try is effectively unlimited. Iterating on one small benchmark makes the score go up faster than the real quality.

8.3 Who judges the judge? 有偏的评估器与 Goodhart’s law

演示 3 模拟一个判断”回答是否 grounded”的 LLM judge。两个 RAG 系统：X 话多（80% 是长回答），真实 grounded 比例 0.68；Y 简洁（20% 长回答），真实 0.76。judge 有一个已知类型的偏差 —— verbosity bias（Zheng et al. 2023）：没有依据的长回答更容易骗过它。

先用 600 条人工标注验证 judge：与人类一致率 0.877，Cohen’s κ = 0.684。大多数团队看到这两个数会直接采用它。但按长度拆开：

然后用它评估系统：

• The ranking flips. The judge scores X above Y (0.796 vs 0.785; a small margin in this sample, but the wrong direction), while Y is clearly more grounded (0.771 vs 0.683).
• Goodhart’s law. X′ only makes every answer long — true quality unchanged — and the judge score jumps to 0.847. A team optimizing against this judge would learn to write longer, not better. “When a measure becomes a target, it ceases to be a good measure.”
• A fix, and its limits. Estimate the judge’s TPR and FPR within each length stratum from the human labels, then invert: in stratum $s$, the judge’s “yes” rate is $q_s = \text{TPR}_s\,\pi_s + \text{FPR}_s(1 - \pi_s)$, so $\pi_s = (q_s - \text{FPR}_s)/(\text{TPR}_s - \text{FPR}_s)$ (the Rogan–Gladen estimator from epidemiology). All three corrected estimates land near the truth and the ranking is restored. But precision depends on how informative the judge is in each stratum: on short answers it’s good, so Y’s corrected interval is narrower than using its 300 human labels alone; on long answers $\text{TPR} - \text{FPR} \approx 0.44$, dividing by that inflates the variance, and for X the human labels alone are better.
• Where the judge earns its keep is reuse: label once, then score each new variant (like X′) at the judge’s cost — valid as long as the judge’s errors depend only on attributes you stratified on. (The modern, general version of “few human labels + many model labels” is prediction-powered inference, Angelopoulos et al. 2023.)

教训：评估器本身需要被评估，而且不能只看总体一致率 —— 要检查它在”被比较的系统之间不同的那些属性”（长度、风格、格式、语言……）上有没有偏差。第 11 讲会系统地讲 LLM judge 的验证与改进。

8.4 Documentation · 把选择写下来

既然可信是多目标的、指标有误差、选择是规范性的，那么”可信”的一个重要组成部分就是透明地报告：用了什么数据、在哪些子群体上测过、指标的置信区间、已知的局限、做了哪些取舍。两个被广泛采用的格式：

• Model Cards（Mitchell et al. 2019）：模型的用途、训练 / 评估数据、按子群体拆开的指标、伦理考量与局限。
• Datasheets for Datasets（Gebru et al. 2021）：数据集的动机、组成、采集过程、预处理、用途与维护。我们的 officer_flag 问题，在一份认真写的 datasheet 里会被”这个字段是什么时候、由谁填写的？“这个问题暴露出来。

演示 7 的成绩单就是一张极简的 model card。

9. 课程地图 · Course map

Part I   导论            L1  可信机器学习导论：准确率之外
                              │
         ┌────────────────────┼─────────────────────┬────────────────────┐
Part II  可解释性          Part III 公平          Part IV 鲁棒性        Part V 隐私
  L2 解释与 Shapley 值       L5 度量公平            L7 对抗样本与攻击      L9 隐私攻击
  L3 深度网络的归因          L6 缓解偏差            L8 防御与可证明鲁棒性  L10 差分隐私
  L4 训练数据归因
         └────────────────────┼─────────────────────┴────────────────────┘
                              │（经典方法在 LLM 上的新形式）
Part VI  可信的 LLM 应用   L11 评估 LLM 应用：RAG triad 与 LLM-as-a-judge
                          L12 Grounding、事实性与幻觉
                          L13 置信度、校准与不确定性
                              │
Part VII 智能体与安全       L14 评估智能体系统：Goal–Plan–Action
                          L15 LLM 与智能体的安全：越狱、提示注入与护栏
                          L16 以评估为驱动的优化与课程总结

依赖关系：L3 用到 L2 的公理；L4 和 L9 都用到 influence / memorization；L8 依赖 L7；L10 依赖 L9（DP 的动机就是防 MIA）；L12 的 grounding 检测用到 L2–L3 的 attribution 思想；L13 是 §5.5 的展开；L15 是 L7 在离散文本上的延续。每一讲的具体内容见 SYLLABUS.md。

10. 小结 · Summary

Six takeaways:

1. Accuracy is an average over one distribution of one kind of correctness. Every trust property relaxes one of those commitments: other distributions, conditional averages, reasons, side effects, confidence, truth.
2. Equally accurate models can differ wildly in trustworthiness (Rashomon / underspecification), so selecting by accuracy can’t settle trust — and can actively select the wrong model (our leaderboard).
3. Explanations are first a debugging tool: a one-line permutation-importance check found the leak that the test set could not.
4. Some trade-offs are mathematical (Chouldechova’s identity), some appear only past a point (privacy vs utility), some can be bought (robustness via abstention). Knowing which is which is most of the work.
5. Measuring trustworthiness is itself a measurement problem: small subgroups, selection effects, biased judges, Goodhart. Report uncertainty, keep a clean test set, evaluate your evaluators.
6. The object of trust has grown from models to applications to agents; the old pillars reappear in new forms (attribution → grounding, adversarial examples → jailbreaks, membership inference → extraction).

11. 常见误区 · Common pitfalls

• “It scored well on a held-out test set, so it will work in deployment.” A held-out set shares the training pipeline’s flaws (leaks, sampling biases). Ask how deployment data are produced.
• “We don’t use the protected attribute, so the model is fair.” Other features carry the information, and feature–outcome relationships can differ by group (§5.2).
• “Our overall ECE is tiny, so the model is calibrated for everyone.” Errors in opposite directions cancel in the average (Model B: 0.013 overall, 0.051 within group 1).
• “We added random noise and accuracy barely changed, so the model is robust.” Random noise is the average case; an adversary picks the worst case, and in high dimensions the gap is hundreds-fold (§5.3).
• “The membership inference attack only got 0.07 advantage, so there’s no leak.” Compare with the noise floor, and remember a failed attack is not a proof of privacy (only DP-style guarantees are).
• “The explanation looks sensible, so the model is right.” Explanations can be plausible and unfaithful, and a model can be manipulated to produce innocent-looking explanations (L8).
• “Our LLM judge agrees with humans 88% of the time, so we can optimize against it.” Check for bias along the attributes that differ between the systems you compare (§8.3).
• “The fairness metric improved.” Check what else moved: other fairness metrics, PPV, and whether the model is simply approving (or denying) everyone (§5.8).

12. 练习 · Exercises

练习 1（Chouldechova 恒等式）. 设某群体有 $N$ 人，base rate 为 $p$，分类器的 TPR、FPR、PPV 如 §0.4 定义。用混淆矩阵的四个计数（TP、FP、FN、TN）证明 $\text{FPR} = \frac{p}{1-p}\cdot\frac{1 - \text{PPV}}{\text{PPV}}\cdot\text{TPR}$。再说明：如果两个群体的 PPV 和 TPR 都相等、但 $p$ 不同，它们的 FPR 之比是多少？

$\text{TP} = \text{TPR}\cdot pN$，$\text{FP} = \text{FPR}\cdot(1-p)N$，$\text{PPV} = \text{TP}/(\text{TP} + \text{FP})$ ⇒ $\text{FP} = \text{TP}\cdot(1 - \text{PPV})/\text{PPV}$。代入 $\text{FPR} = \text{FP}/((1-p)N) = \frac{\text{TPR}\cdot pN}{(1-p)N}\cdot\frac{1-\text{PPV}}{\text{PPV}}$。PPV、TPR 相等时，$\text{FPR}_0/\text{FPR}_1 = \frac{p_0/(1-p_0)}{p_1/(1-p_1)}$，即两个群体 base rate 的odds 之比。以 §7.1 的 $p_0 = 0.707$、$p_1 = 0.590$ 为例，比值约 $2.41/1.44 \approx 1.68$：base rate 高的群体 FPR 必然高约 68%。

练习 2（线性模型的局部解释）. For a linear logit $z(x) = \theta_0 + \sum_j \theta_j (x_j - \mu_j)/\sigma_j$, define the contribution of feature $j$ as $c_j(x) = \theta_j(x_j - \mu_j)/\sigma_j$. (a) Show completeness: $\sum_j c_j(x) = z(x) - z(\mu)$. (b) Show that the average contribution of each feature over the training set is 0. (c) Why does (a) fail for the probability $\sigma(z)$ instead of the logit, and what does that suggest about which output space to explain?

(a) $z(\mu) = \theta_0$，所以 $z(x) - z(\mu) = \sum_j c_j(x)$。(b) 训练集上 $\frac{1}{n}\sum_i (x_{ij} - \mu_j) = 0$。(c) $\sigma$ 是非线性的：$\sigma(\theta_0 + \sum_j c_j) - \sigma(\theta_0) \ne \sum_j[\sigma(\theta_0 + c_j) - \sigma(\theta_0)]$，“每个特征单独的效应”加起来不等于总效应，特征之间通过 $\sigma$ 发生了交互。所以对逻辑回归通常在 logit 空间解释（可加、精确）；如果坚持在概率空间解释，就需要 Shapley 值这样的方法来公平地分配交互项 —— 第 2 讲的出发点。

练习 3（为什么高维输入这么脆弱）. Let $w \in \mathbb{R}^d$ have i.i.d. entries $w_j \sim \mathcal{N}(0, 1/d)$. (a) Show $\mathbb{E}\lVert w\rVert_1 = \sqrt{2d/\pi}$ and $\mathbb{E}\lVert w\rVert_2^2 = 1$. (b) For the $L_\infty$ perturbation $\delta = \epsilon\,\text{sign}(w)$, compute the change in $w^\top x$. (c) For random signs $\delta_j = \pm\epsilon$, compute the mean and standard deviation of the change. (d) Check your answer against the demo 5(b) table.

(a) $\mathbb{E}|w_j| = \sqrt{2/\pi}\cdot(1/\sqrt d)$（半正态分布的均值），乘以 $d$ 得 $\sqrt{2d/\pi}$；$\mathbb{E}w_j^2 = 1/d$，乘以 $d$ 得 1。(b) $w^\top\delta = \epsilon\lVert w\rVert_1 \approx \epsilon\sqrt{2d/\pi}$，随 $\sqrt d$ 增长。(c) 均值 0，方差 $\epsilon^2\sum_j w_j^2 = \epsilon^2\lVert w\rVert_2^2 \approx \epsilon^2$，标准差 ≈ $\epsilon$，与 $d$ 无关。(d) $\sqrt{2\cdot784/\pi} \approx 22.3$（表中 22.2），$\sqrt{2\cdot150528/\pi} \approx 309.6$（表中 309.3）。这就是 FGSM（第 7 讲）的出发点：$\delta = \epsilon\,\text{sign}(\nabla_x\ell)$ 正是对局部线性化的模型做 (b)。

练习 4（赢家的诅咒有多大）. 60 个候选模型在 100 个样本的验证集上评估，真实准确率都约为 0.75。(a) 单个模型验证准确率的标准差是多少？(b) 如果 60 个模型的验证误差相互独立，被挑中者的期望 optimism 约是多少？（提示：60 个独立标准正态的最大值的期望约为 2.32。）(c) 演示 2 实测的 optimism 只有 0.044，为什么比 (b) 小？

(a) $\sqrt{0.75\times0.25/100} \approx 0.043$。(b) 约 $0.043\times2.32 \approx 0.10$。(c) 两个原因：候选模型在同一批人身上犯错（它们用相似的特征，错误高度相关），所以验证误差远不是独立的，有效的”独立尝试次数”远少于 60；另外它们的真实准确率不相等（0.675–0.765），最大值大多落在少数真正较好的模型上，噪声带来的额外提升更小。LLM 应用里，不同 prompt 变体的输出往往也高度相关，但只要尝试次数足够多，虚高依然显著。

练习 5（修正有偏的评估器）. In stratum $s$, a judge has true-positive rate $T$ and false-positive rate $F$ (with $T > F$), and the true positive fraction is $\pi$. (a) Derive the Rogan–Gladen estimator $\hat\pi = (\hat q - F)/(T - F)$ from $q = T\pi + F(1 - \pi)$. (b) If $T$ and $F$ are known exactly and $\hat q$ is estimated from $m$ judged items, what is $\text{Var}(\hat\pi)$? (c) Use (b) to explain why the corrected estimate was more precise than human labels for System Y but less precise for System X in §8.3.

(a) 线性方程直接解出 $\pi$。(b) $\text{Var}(\hat\pi) = \frac{q(1-q)}{m\,(T-F)^2}$。(c) 修正估计的方差被 $1/(T-F)^2$ 放大。短回答层 $T - F \approx 0.86$，放大约 1.35 倍，但 judge 判断了 2000 条（$m$ 大），所以比 300 条人工标注更准；长回答层 $T - F \approx 0.44$，放大约 5 倍。而且现实中 $T$、$F$ 本身是从少量人工标注估计出来的，它们的误差也被同一个分母放大 —— 这部分在 X（81% 长回答）上占主导。

练习 6（写下你的 trust requirements）. 选一个应用：(i) 医院用 LLM 把医生手写笔记整理成结构化表格（诊断、处方、检查）；(ii) 一个能读写公司数据库、回答业务问题的 data agent。为它写一份一页的 trust requirements：列出最重要的 3 个透镜，每个透镜给出 (a) 一个可测量的指标，(b) 需要什么数据来测量，(c) 一个具体的失败场景，(d) 一个 threat model 或部署分布的描述。

1. Grounding：(a) 表格中每个”特征–值”对能在笔记原文中找到依据的比例（claim-level precision），以及笔记中提到的诊断被抽取出来的比例（recall）；(b) 一份由医生标注的笔记–表格对照集，按科室分层；(c) 模型从”排除糖尿病”里抽取出”诊断：糖尿病”（CS329T 2023 年客座讲座的例子）；(d) 部署分布：不同医生的书写习惯、缩写、科室。
2. Robustness：(a) 对语义不变的改写（标点、停用词、信息顺序——比如处方写在诊断之前）输出一致的比例；(b) 自动生成的改写版本 + 人工抽查；(c) 调换段落顺序后漏掉了一个处方；(d) 非对抗性的输入变化，不是恶意攻击者。
3. Confidence / calibration：(a) 模型对每个字段给出的置信度的 ECE，以及”低置信度转人工”策略下的覆盖率与错误率；(b) 同上的标注集；(c) 模型对字迹潦草的笔记仍然给出高置信度的错误剂量；(d) 何时必须转人工的规则。

另外应该写明不在范围内的东西（比如这个阶段不评估公平性的原因和计划），以及每个指标的样本量和置信区间。

代码相关的练习（“动手改一改”）见 code/lecture01/README.md。

13. 延伸阅读 · Further reading

本讲的主要来源

• CS329T 课程主页与各年份 syllabus（Spring 2022、Fall 2023、Fall 2024、Autumn 2025），尤其是各年的第 1 讲 slides，以及 Fall 2023 第 6 讲 “Project Design and Planning”。
• Sun et al., “TrustLLM: Trustworthiness in Large Language Models”, 2024（Fall 2024 第 1 讲阅读材料，Sections 2.2 和 3）。
• Kenthapadi, Sameki & Taly, “Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)”, 2024（Fall 2024、Autumn 2025 的阅读材料）。

按透镜

• Shortcut learning：Geirhos et al., “Shortcut Learning in Deep Neural Networks”, Nature Machine Intelligence 2020；Caruana et al., KDD 2015（哮喘与肺炎）。
• Underspecification：D’Amour et al., “Underspecification Presents Challenges for Credibility in Modern Machine Learning”, JMLR 2022；Breiman, “Statistical Modeling: The Two Cultures”, 2001。
• Fairness：Kleinberg, Mullainathan & Raghavan, “Inherent Trade-Offs in the Fair Determination of Risk Scores”, 2016；Chouldechova, “Fair Prediction with Disparate Impact”, 2017；Barocas, Hardt & Narayanan, Fairness and Machine Learning（免费在线教材）。
• Robustness：Goodfellow, Shlens & Szegedy, “Explaining and Harnessing Adversarial Examples”, ICLR 2015。
• Privacy：Yeom et al., “Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting”, CSF 2018；Shokri et al., “Membership Inference Attacks Against Machine Learning Models”, S&P 2017。
• Calibration：Guo et al., “On Calibration of Modern Neural Networks”, ICML 2017。
• LLM judges：Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena”, NeurIPS 2023；Angelopoulos et al., “Prediction-Powered Inference”, Science 2023。

Trust 与框架

• Lee & See, “Trust in Automation: Designing for Appropriate Reliance”, Human Factors 2004；Jacovi et al., “Formalizing Trust in Artificial Intelligence”, FAccT 2021。
• NIST AI Risk Management Framework 1.0（2023）；EU HLEG, Ethics Guidelines for Trustworthy AI（2019）。
• Mitchell et al., “Model Cards for Model Reporting”, FAT* 2019；Gebru et al., “Datasheets for Datasets”, CACM 2021。

Agents（Autumn 2025 第 1 讲阅读材料）

• Anthropic, “Building Effective Agents”（2024）；OpenAI, “A Practical Guide to Building Agents”（2025）；LangGraph 文档的 “Agent architectures”。

14. 下一讲预告 · Next lecture

Lecture 2：解释与 Shapley 值（Explanations & Shapley values）。

本讲的局部解释 $\theta_j\tilde x_j$ 只对线性模型成立。对一个树集成或神经网络，“收入对这次拒绝贡献了多少”该怎么定义？朴素的做法（一次改一个特征）会在特征交互时给出互相矛盾的答案。第 2 讲从合作博弈论出发：把特征看作”玩家”、把模型输出看作”收益”，Shapley 值是唯一满足 efficiency（即本讲的 completeness）、symmetry、dummy、additivity 四条公理的分配方式。我们会实现精确的 Shapley 值、QII（Datta, Sen & Zick 2016）、LIME 和 KernelSHAP，并讨论一个看似技术、实则决定一切的选择：“把特征去掉”到底是什么意思（基线 / baseline 的选择）。