[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"learning-part-4-llm-principles-advanced":3,"learning-all":13},{"id":4,"slug":5,"title":6,"description":7,"level":8,"topic":9,"url":10,"content":11,"sortOrder":12},101,"part-4-llm-principles-advanced","Part 4 · 大模型原理 · 进阶可选","从零写迷你模型、RLHF、PPO、分布式训练实操等进阶内容","INTERMEDIATE","大模型原理",null,"# Part 4 · 大模型原理 · 进阶可选\n\n\n---\n\n## Section 1 · 从零写一个迷你\"预测下一个词\"模型\n\n**Subsection 1 · 跑代码**\nVS Code 新建 `mini_lm.py`：\n```python\nimport torch\nimport torch.nn as nn\n\n# 超简单的\"语言模型\"：根据前一个词预测下一个词\ntorch.manual_seed(42)\n\n# 一个小词表：一句话 \"我 爱 吃 苹果\"\nvocab = [\"我\", \"爱\", \"吃\", \"苹果\"]\nword2idx = {w: i for i, w in enumerate(vocab)}\nidx2word = {i: w for i, w in enumerate(vocab)}\nvocab_size = len(vocab)\n\n# 用\"爱吃\"预测\"苹果\"：输入 [爱, 吃] 的索引\nX = torch.tensor([[word2idx[\"爱\"]], [word2idx[\"吃\"]]])\ny = torch.tensor([word2idx[\"吃\"], word2idx[\"苹果\"]])\n\n# 一个 Embedding + 一个线性层 = 最简\"预测下一个词\"\nemb = nn.Embedding(vocab_size, 8)\nfc = nn.Linear(8, vocab_size)\nloss_fn = nn.CrossEntropyLoss()\nopt = torch.optim.Adam(list(emb.parameters()) + list(fc.parameters()), lr=0.05)\n\n# 训练 200 轮\nfor step in range(200):\n    e = emb(X)              # (2,8)\n    logits = fc(e).squeeze(1)  # (2,4)\n    loss = loss_fn(logits, y)\n    opt.zero_grad()\n    loss.backward()\n    opt.step()\n    if step % 50 == 0:\n        print(f\"step {step} loss={loss.item():.3f}\")\n\n# 测试：输入\"爱\"，模型预测下一个词\ntest = torch.tensor([[word2idx[\"爱\"]]])\nlogits = fc(emb(test)).squeeze(1)\npred_idx = logits.argmax().item()\nprint(\"看到'爱'，模型预测下一个词是:\", idx2word[pred_idx])\n```\n- 运行\n- ✅ 预期看到：loss 从 1.x 降到接近 0，最后打印 `看到'爱'，模型预测下一个词是: 吃`\n- 💡 别小看它——**GPT 的本质就是这么回事**（只是大了一亿倍，且会看整句）。\n\n**过关**：能跑通并说出\"语言模型 = 预测下一个词\" = Section 1 完成。\n\n---\n\n## Section 2 · 把迷你模型放大一点：学整句话\n\n**Subsection 1 · 跑代码**\nVS Code 新建 `mini_lm2.py`：\n```python\nimport torch\nimport torch.nn as nn\n\n# 这次用\"窗口\"：根据前面 N 个词预测下一个\ntorch.manual_seed(0)\nvocab = [\"我\", \"爱\", \"吃\", \"苹果\", \"和\", \"香蕉\", \"。\"]\n\n# 训练句子（简化版）\nsentences = [\n    [\"我\", \"爱\", \"吃\", \"苹果\"],\n    [\"我\", \"爱\", \"吃\", \"香蕉\"],\n    [\"苹果\", \"和\", \"香蕉\", \"。\"],\n]\n# 构造输入窗口=3\nX, y = [], []\nfor s in sentences:\n    for i in range(len(s) - 3):\n        X.append([vocab.index(s[i]), vocab.index(s[i+1]), vocab.index(s[i+2])])\n        y.append(vocab.index(s[i+3]))\n\nX = torch.tensor(X)\ny = torch.tensor(y)\n\nemb = nn.Embedding(len(vocab), 16)\nfc = nn.Linear(16 * 3, len(vocab))\nloss_fn = nn.CrossEntropyLoss()\nopt = torch.optim.Adam(list(emb.parameters()) + list(fc.parameters()), lr=0.03)\n\nfor step in range(500):\n    e = emb(X).view(X.shape[0], -1)   # 把 3 个词的向量拼一起\n    logits = fc(e)\n    loss = loss_fn(logits, y)\n    opt.zero_grad(); loss.backward(); opt.step()\n    if step % 100 == 0:\n        print(f\"step {step} loss={loss.item():.3f}\")\n\n# 测试 \"我爱吃\" → ？\ntest = torch.tensor([[vocab.index(\"我\"), vocab.index(\"爱\"), vocab.index(\"吃\")]])\ne = emb(test).view(1, -1)\npred = fc(e).argmax().item()\nprint(\"看到'我爱吃'，预测下一个词:\", vocab[pred])\n```\n- ✅ 预期看到：预测\"苹果\"或\"香蕉\"。\n- 💡 这就是\"根据上下文预测\"的雏形——再叠上注意力机制，就是 Transformer 了。\n\n**过关**：能跑通，并说一句\"窗口 + 预测 = 语言模型的基本盘\" = Section 2 完成。\n\n---\n\n## Section 3 · RLHF 深度：InstructGPT 怎么来的\n\n**Subsection 1 · 看精读**\n- B 站搜 `InstructGPT 论文 解读`，看 30 分钟精读视频。\n- 打开 https:\u002F\u002Fzhuanlan.zhihu.com\u002Fp\u002F625249476 （InstructGPT 中文解读），读全文。\n\n**Subsection 2 · 画三步流程**\n备忘录画：\n```\n第1步 SFT  找人工写好\"问题→好回答\"，微调模型 → 会说人话\n第2步 奖励模型  让模型同时给多个回答，人工排序，训练一个\"打分器\"\n第3步 PPO  用打分器给模型的回答打分，通过强化学习把高分行为放大\n结果 → 模型更\"听话\"、更少乱说（对齐人类偏好）\n```\n\n**过关**：能对着这张图讲一遍 RLHF 三步 = Section 3 完成。\n\n---\n\n## Section 4 · PPO 思想梳理（不做公式，懂直觉）\n\n**Subsection 1 · 记直觉**\nB 站搜 `PPO 强化学习 通俗` 看 20 分钟。记：\n```\nPPO = 一种强化学习算法\n直觉：让模型多做\"被奖励的行为\"，但每次改一点点（不剧烈），防止越改越乱\n\"近端策略优化\" = 新的策略和旧策略别差太远\n```\n- 💡 面试问\"RLHF 里的 PPO 干嘛的\"：答\"让模型根据奖励优化行为，且更新幅度可控\"就够了。\n\n**Subsection 2 · 一句话串联**\n写：`预训练(读书) → SFT(学说话) → 奖励模型(有裁判) → PPO(被裁判训练得更听话)`\n\n**过关**：能把这条链讲顺 = Section 4 完成。\n\n---\n\n## Section 5 · 分布式训练实操认识\n\n**Subsection 1 · 记三种并行**\nB 站搜 `分布式训练 数据并行 模型并行 流水线` 看 20 分钟。记：\n```\n数据并行   一台机器存一个模型副本，各算各的数据，最后同步梯度\n模型并行   一个模型太大，拆开放在多张卡\n流水线并行 层按顺序分给多张卡，像工厂流水线\n```\n\n**Subsection 2 · 动手看你的 GPU**\ncmd 输入：`nvidia-smi`\n- ✅ 预期看到：显卡信息表（如果你的电脑有 NVIDIA 独显）。没有也没关系，说明用 CPU 学习。\n- 💡 记下你的显存大小（比如 8GB\u002F16GB），后面 Part 14\u002F17 算\"能跑多大模型\"要用。\n\n**过关**：能说出三种并行 + 知道自己的显存 = Section 5 完成。\n\n---\n\n## Section 6 · 精读一篇论文：BERT（或 ChatGLM 解读）\n\n**Subsection 1 · 精读**\n- B 站搜 `BERT 论文 通俗 解读` 看 30 分钟。\n- 然后打开 https:\u002F\u002Fzhuanlan.zhihu.com\u002Fp\u002F46652512 （BERT 论文中文笔记）读前 60%。\n\n**Subsection 2 · 写 200 字读后感**\n包含：BERT 用了什么架构（Transformer Encoder）、预训练做了什么（掩码预测 + 下一句预测）、为什么它影响深远。\n\n**过关**：读后感写完 = Section 6 完成。\n\n---\n\n## Section 7 · 复盘站 2 全部内容\n\n**Subsection 1 · 自测问答**\n不看资料，回答（写下来或打字）：\n1. 判别式 vs 生成式？\n2. Transformer 自注意力解决什么问题？\n3. 预训练→SFT→RLHF 三步各干嘛？\n4. CoT 是什么、为什么有用？\n5. 词向量从 Word2Vec 到 BERT 进化了什么？\n6. FlashAttention \u002F PagedAttention 各解决什么？\n7. 我电脑显存多少、能跑多大模型？\n\n**Subsection 2 · 修漏**\n答不出的题，回到对应 Day 再看一遍，直到 7 题全对。\n\n**过关**：7 题全对 = Section 7 完成。\n\n---\n\n## Section 8 · 站 2 总验收（全部 30 天）\n\n**Subsection 1 · 最终勾选**\n- [ ] Part 2 九项全过（Part 2 · Section 10 清单）\n- [ ] Part 3 十一项全过（Part 3 · Section 12 清单）\n- [ ] 迷你语言模型两段代码跑通（Section 1\u002F24）\n- [ ] 能画并讲清 RLHF 三步流程（Section 3\u002F26）\n- [ ] 知道三种并行 + 自己机器显存（Section 5）\n- [ ] BERT 读后感写完（Section 6）\n- [ ] 29 天七问全对（Section 7）\n\n**Subsection 2 · 写一份 500 字Part 4 总结**\n把你理解的\"大模型是怎么工作的\"从头到尾写一遍。写完 = Part 4 全部完成。\n\n> Part 4 结束。你已经有\"应用开发者里比较懂原理\"的底子了。\n> 接下来进入真正的工程主线：**Part 5 · Prompt 提示词工程**。\n\n\n\n",4,[14,21,26,32,33,39,45,51,56,62,68,75,81,87,93,99,105,111,117,123,129],{"id":15,"slug":16,"title":17,"description":18,"level":19,"topic":20,"url":10,"content":10,"sortOrder":15},1,"part-1-llm-basics","Part 1 · 大模型基础认知","小白零基础友好。这一站不写代码，只建立\"大模型到底是什么\"的骨架。","BEGINNER","大模型认知",{"id":22,"slug":23,"title":24,"description":25,"level":19,"topic":9,"url":10,"content":10,"sortOrder":22},2,"part-2-llm-principles","Part 2 · 大模型原理 · 入门实操","这一站不卷数学，只把\"理解模型所需的知识\"讲透。；站 2A 解决\"理解模型\"，站 2B 解决\"面试能答\"。；这一层是\"有余力再做\"，不做也不影响你进入站 3。",{"id":27,"slug":28,"title":29,"description":30,"level":19,"topic":9,"url":10,"content":10,"sortOrder":31},100,"part-3-llm-principles-deep","Part 3 · 大模型原理 · 面试深入","数学基础、机器学习、神经网络、词向量、思维链等面试必考原理",3,{"id":4,"slug":5,"title":6,"description":7,"level":8,"topic":9,"url":10,"content":10,"sortOrder":12},{"id":31,"slug":34,"title":35,"description":36,"level":19,"topic":37,"url":10,"content":10,"sortOrder":38},"part-5-prompt-engineering","Part 5 · Prompt 提示词工程","从这里开始，你每天都要真的调大模型 API。","提示工程",5,{"id":12,"slug":40,"title":41,"description":42,"level":19,"topic":43,"url":10,"content":10,"sortOrder":44},"part-6-llm-api","Part 6 · 大模型 API 调用","站 3 你已经调通了 API。这一站把\"调用\"做成专业水准：","API 工程",6,{"id":38,"slug":46,"title":47,"description":48,"level":8,"topic":49,"url":10,"content":10,"sortOrder":50},"part-7-rag","Part 7 · RAG 检索增强生成","这一站是\"能落地\"的关键：让模型回答**你的私有知识**。","RAG",7,{"id":44,"slug":52,"title":53,"description":54,"level":8,"topic":49,"url":10,"content":10,"sortOrder":55},"part-8-rag-optimization","Part 8 · RAG 优化与评估","站 5 你做出了\"能跑\"的 RAG。这一站把它做到\"能打\"：",8,{"id":50,"slug":57,"title":58,"description":59,"level":8,"topic":60,"url":10,"content":10,"sortOrder":61},"part-9-langchain","Part 9 · LangChain 框架","目标：用 LangChain 把\"模型、提示词、检索、记忆、工具\"串成可复用的链和 Agent。","LangChain",9,{"id":55,"slug":63,"title":64,"description":65,"level":8,"topic":66,"url":10,"content":10,"sortOrder":67},"part-10-agent","Part 10 · Agent 基础","让模型从\"会聊天\"变成\"能干活\"：自己拆任务、调工具、看结果、再行动。","Agent",10,{"id":61,"slug":69,"title":70,"description":71,"level":72,"topic":73,"url":10,"content":10,"sortOrder":74},"part-11-multi-agent","Part 11 · 多 Agent 与 Agent IDE","从\"单个 Agent\"升级到\"多 Agent 协作\"，并掌握 AutoGen \u002F LangGraph \u002F GPTs \u002F Coze \u002F Dify 五个工具。","ADVANCED","多 Agent",11,{"id":67,"slug":76,"title":77,"description":78,"level":72,"topic":79,"url":10,"content":10,"sortOrder":80},"part-12-llamaindex","Part 12 · LlamaIndex","站 5~7 你用了 LangChain 做 RAG。LlamaIndex 是另一条专攻\"数据 + LLM\"的路线。","LlamaIndex",12,{"id":74,"slug":82,"title":83,"description":84,"level":72,"topic":85,"url":10,"content":10,"sortOrder":86},"part-13-transformer","Part 13 · Transformer 深入","站 2A 建立了直觉，这一站把手写代码跑通——从\"懂概念\"到\"写得出\"。","Transformer",13,{"id":80,"slug":88,"title":89,"description":90,"level":72,"topic":91,"url":10,"content":10,"sortOrder":92},"part-14-open-models","Part 14 · 开源模型与私有化部署","目标：把主流开源模型在本地\u002F服务器跑通，理解服务化工程（显存、量化、并发）。","模型部署",14,{"id":86,"slug":94,"title":95,"description":96,"level":72,"topic":97,"url":10,"content":10,"sortOrder":98},"part-15-finetuning","Part 15 · 模型微调 Fine-Tuning","目标：理解\"什么时候必须微调、怎么选基座、怎么备数据\"，并完整跑通一次业务微调。","微调",15,{"id":92,"slug":100,"title":101,"description":102,"level":72,"topic":103,"url":10,"content":10,"sortOrder":104},"part-16-peft-lora","Part 16 · PEFT 参数高效微调","重点吃透 **LoRA**：用极少量可训练参数微调大模型（消费级显卡就能跑）。","PEFT\u002FLoRA",16,{"id":98,"slug":106,"title":107,"description":108,"level":72,"topic":109,"url":10,"content":10,"sortOrder":110},"part-17-quantization","Part 17 · 模型量化","目标：理解\"为什么量化、主流算法各自思路\"，能动手量化并评估效果。","量化",17,{"id":104,"slug":112,"title":113,"description":114,"level":72,"topic":115,"url":10,"content":10,"sortOrder":116},"part-18-data-evaluation","Part 18 · 训练数据与模型评估","数据决定模型上限，评估决定你\"能不能交付\"。","数据与评估",18,{"id":110,"slug":118,"title":119,"description":120,"level":72,"topic":121,"url":10,"content":10,"sortOrder":122},"part-19-multimodal","Part 19 · 多模态应用","让 AI 同时\"看得懂图、听得了音、说得出话\"。","多模态",19,{"id":116,"slug":124,"title":125,"description":126,"level":72,"topic":127,"url":10,"content":10,"sortOrder":128},"part-20-projects-career","Part 20 · 项目实战 + 求职备战","把 0→1 到 1→100 学到的全部收敛成**能展示、能讲、能面试**的项目。","项目与求职",20,{"id":130,"slug":131,"title":132,"description":133,"level":72,"topic":134,"url":10,"content":10,"sortOrder":135},102,"extra-2026-ai-tech","额外篇 · 2026 年 AI 应用开发必学的 5 项成熟技术","从近半年爆发的技术里，挑出已经过了尝鲜期、能真正用在项目里的 5 项：推理模型、MCP、多模态、GraphRAG、AI 编程工具链，每项都给到能跑通的代码。","AI 工程实践",21]