[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"learning-part-18-data-evaluation":3,"learning-all":13},{"id":4,"slug":5,"title":6,"description":7,"level":8,"topic":9,"url":10,"content":11,"sortOrder":12},16,"part-18-data-evaluation","Part 18 · 训练数据与模型评估","数据决定模型上限，评估决定你\"能不能交付\"。","ADVANCED","数据与评估",null,"# Part 16 · 训练数据与模型评估\n\n\n---\n\n## Section 1 · 训练数据来源\n\n**Subsection 1 · 记两类数据**\n```\n通用数据  互联网语料（网页\u002F书\u002F论文\u002F代码）→ 训练\"基础能力\"\n专业数据  领域语料（医学\u002F法律\u002F工业日志）→ 训练\"领域能力\"\n```\n- 💡 预训练用通用数据\"打底\"，微调用专业数据\"定向\"。\n\n**过关**：能说出两类数据各自作用 = Section 1 完成。\n\n---\n\n## Section 2 · 数据清洗实操\n\n**Subsection 1 · 记清洗维度**\n```\n低质过滤   去掉乱码\u002F广告\u002F重复废话\n冗余去除   去重（近义重复也要去）\n隐私消除   去掉个人信息\u002F密钥（合规必须）\n```\n\n**Subsection 2 · 跑清洗脚本**\n新建 `clean_corpus.py`：\n```python\nimport re\n\ndef clean_text(text: str) -> str:\n    # 1. 去掉 URL\n    text = re.sub(r\"https?:\u002F\u002F\\S+\", \"\", text)\n    # 2. 去掉 HTML 标签\n    text = re.sub(r\"\u003C[^>]+>\", \"\", text)\n    # 3. 去掉多余空白\n    text = re.sub(r\"\\s+\", \" \", text)\n    # 4. 过滤太短的\n    if len(text.strip()) \u003C 20:\n        return \"\"\n    # 5. 过滤乱码（含过多异常字符）\n    if sum(1 for c in text if ord(c) > 0xFFFF) > 10:\n        return \"\"\n    return text.strip()\n\nsamples = [\n    \"这是一段正常的中文内容，讲的是如何学习大模型开发。\",\n    \"\u003Cp>有HTML标签的\u003Cstrong>内容\u003C\u002Fstrong>\u003C\u002Fp>\",\n    \"https:\u002F\u002Fexample.com 这有个链接\",\n    \"短\",\n]\nfor s in samples:\n    r = clean_text(s)\n    print(f\"清洗后: {r if r else '(被过滤)'}\")\n```\n- ✅ 预期看到：URL\u002F标签被清掉，太短的被过滤。\n\n**过关**：清洗脚本跑通 = Section 2 完成。\n\n---\n\n## Section 3 · 开源数据集盘点\n\n**Subsection 1 · 记四个数据集**\n```\nPile          800GB 英文通用语料\nROOTS         欧洲 BigScience 多语言语料\nRefinedWeb    互联网清洗语料（很干净）\nSlimPajama    可商用的大规模语料\n```\n\n**Subsection 2 · 上 HuggingFace 看**\n- 打开 https:\u002F\u002Fhuggingface.co\u002Fdatasets 搜 `Chinese`，找几个中文预训练数据集（如 `shibing624\u002Fnli_zh`、`HuggingFaceFW\u002Ffineweb-edu`）。\n- 看数据集卡片（size\u002F字段\u002F用途）。\n\n**过关**：能报出 4 个数据集 = Section 3 完成。\n\n---\n\n## Section 4 · 评估体系\n\n**Subsection 1 · 记三层评估**\n```\n知识与能力   模型懂多少、能不能干（做题\u002F推理\u002F代码）\n伦理与安全   有没有毒输出\u002F偏见\u002F幻觉\n垂直领域     医学\u002F法律等专业场景准不准\n```\n- 💡 面试点：**\"不能只测准确率，要分维度评估\"**。\n\n**过关**：能说三层 = Section 4 完成。\n\n---\n\n## Section 5 · 评估指标（BLEU \u002F ROUGE）\n\n**Subsection 1 · 记指标**\n```\nBLEU   机器翻译\u002F生成：看\"生成的词和参考文本重合多少\"（越像参考越高）\nROUGE  摘要\u002F文本生成：看\"参考文本里的词有没有被生成出来\"\n准确率\u002F召回\u002FF1  分类任务三大件\n```\n- 💡 注意：这些指标对\"开放式生成\"意义有限（好答案可以不唯一），所以还要配合人工\u002FLLM 评判。\n\n**过关**：能说 BLEU\u002FROUGE 大概 = Section 5 完成。\n\n---\n\n## Section 6 · 用 HF Evaluate 跑指标\n\n**Subsection 1 · 装**\ncmd：`pip install evaluate sacrebleu`\n\n**Subsection 2 · 跑 BLEU**\n新建 `eval_bleu.py`：\n```python\nfrom evaluate import load\n\nbleu = load(\"bleu\")\n# 假设：模型生成 vs 参考答案\npredictions = [\"I love AI development.\"]\nreferences = [[\"I love AI development.\"]]   # 完全一样 → 高分\n\nresult = bleu.compute(predictions=predictions, references=references)\nprint(\"BLEU 分数:\", round(result[\"bleu\"], 3))\n\n# 换个明显不对的\npredictions2 = [\"The weather is nice.\"]\nresult2 = bleu.compute(predictions=predictions2, references=references)\nprint(\"不相关内容 BLEU:\", round(result2[\"bleu\"], 3))\n```\n- ✅ 预期看到：第一组高分，第二组很低。\n\n**过关**：能跑出 BLEU = Section 6 完成。\n\n---\n\n## Section 7 · GLUE \u002F SuperGLUE（基准榜）\n\n**Subsection 1 · 记概念**\n```\nGLUE \u002F SuperGLUE = 一整套 NLP 任务的\"考试题\"\n  （情感分类\u002F自然语言推理\u002F问答等，共几十项）\n跑完拿综合分 → 横向对比模型能力\nSuperGLUE = GLUE 的更难版本\n```\n\n**Subsection 2 · 看排行榜**\n- 打开 https:\u002F\u002Fpaperswithcode.com\u002F 搜 GLUE \u002F SuperGLUE，看排行榜（了解即可）。\n\n**过关**：能说 GLUE 是什么 = Section 7 完成。\n\n---\n\n## Section 8 · 安全\u002F伦理评估\n\n**Subsection 1 · 用安全测试集**\n- 打开 https:\u002F\u002Fgithub.com\u002Fthu-coai\u002FSafety-Prompts （清华安全测试集）看它的测试分类。\n- 用你已部署的模型跑几个安全用例（如诱导输入恶意指令），记录模型反应。\n- 写笔记：模型的\"拒答\"表现如何。\n\n**过关**：跑过安全用例 = Section 8 完成。\n\n---\n\n## Section 9 · 综合评估项目\n\n**Subsection 1 · 评估你的 RAG**\n用Part 6 的 RAGAS + 今天的指标，对你的 rag_tool 出一份完整评估报告：\n- RAGAS 三指标（忠实度\u002F相关度\u002F上下文相关度）\n- 20 个测试问题的正确率\n- 安全测试结果\n- 改进建议（Top3）\n- ✅ 预期看到：一份像样的\"评估报告\"（这就是你面试能展示的文档）。\n\n**过关**：评估报告写完 = Section 9 完成。\n\n---\n\n## Section 10 · 站 16 验收\n\n**勾选**\n- [ ] 能说通用\u002F专业数据作用\n- [ ] 清洗脚本跑通，知道三维度\n- [ ] 能报出 4 个开源数据集\n- [ ] 能说评估三层体系\n- [ ] 能说 BLEU\u002FROUGE\u002F准确率\n- [ ] BLEU 代码跑通（Section 6）\n- [ ] 知道 GLUE\u002FSuperGLUE\n- [ ] 跑过安全测试用例\n- [ ] RAG 评估报告写完（Section 9）\n\n**写 300 字Part 16 总结**：你的模型\u002FRAG 强在哪、弱在哪、怎么改进。\n\n**全勾选 = Part 18 通过** → 进入下一站 Part 19 · 多模态应用，20 天）。\n\n\n\n",18,[14,21,27,33,40,46,52,58,63,69,75,81,87,93,99,105,110,116,117,123,129],{"id":15,"slug":16,"title":17,"description":18,"level":19,"topic":20,"url":10,"content":10,"sortOrder":15},1,"part-1-llm-basics","Part 1 · 大模型基础认知","小白零基础友好。这一站不写代码，只建立\"大模型到底是什么\"的骨架。","BEGINNER","大模型认知",{"id":22,"slug":23,"title":24,"description":25,"level":19,"topic":26,"url":10,"content":10,"sortOrder":22},2,"part-2-llm-principles","Part 2 · 大模型原理 · 入门实操","这一站不卷数学，只把\"理解模型所需的知识\"讲透。；站 2A 解决\"理解模型\"，站 2B 解决\"面试能答\"。；这一层是\"有余力再做\"，不做也不影响你进入站 3。","大模型原理",{"id":28,"slug":29,"title":30,"description":31,"level":19,"topic":26,"url":10,"content":10,"sortOrder":32},100,"part-3-llm-principles-deep","Part 3 · 大模型原理 · 面试深入","数学基础、机器学习、神经网络、词向量、思维链等面试必考原理",3,{"id":34,"slug":35,"title":36,"description":37,"level":38,"topic":26,"url":10,"content":10,"sortOrder":39},101,"part-4-llm-principles-advanced","Part 4 · 大模型原理 · 进阶可选","从零写迷你模型、RLHF、PPO、分布式训练实操等进阶内容","INTERMEDIATE",4,{"id":32,"slug":41,"title":42,"description":43,"level":19,"topic":44,"url":10,"content":10,"sortOrder":45},"part-5-prompt-engineering","Part 5 · Prompt 提示词工程","从这里开始，你每天都要真的调大模型 API。","提示工程",5,{"id":39,"slug":47,"title":48,"description":49,"level":19,"topic":50,"url":10,"content":10,"sortOrder":51},"part-6-llm-api","Part 6 · 大模型 API 调用","站 3 你已经调通了 API。这一站把\"调用\"做成专业水准：","API 工程",6,{"id":45,"slug":53,"title":54,"description":55,"level":38,"topic":56,"url":10,"content":10,"sortOrder":57},"part-7-rag","Part 7 · RAG 检索增强生成","这一站是\"能落地\"的关键：让模型回答**你的私有知识**。","RAG",7,{"id":51,"slug":59,"title":60,"description":61,"level":38,"topic":56,"url":10,"content":10,"sortOrder":62},"part-8-rag-optimization","Part 8 · RAG 优化与评估","站 5 你做出了\"能跑\"的 RAG。这一站把它做到\"能打\"：",8,{"id":57,"slug":64,"title":65,"description":66,"level":38,"topic":67,"url":10,"content":10,"sortOrder":68},"part-9-langchain","Part 9 · LangChain 框架","目标：用 LangChain 把\"模型、提示词、检索、记忆、工具\"串成可复用的链和 Agent。","LangChain",9,{"id":62,"slug":70,"title":71,"description":72,"level":38,"topic":73,"url":10,"content":10,"sortOrder":74},"part-10-agent","Part 10 · Agent 基础","让模型从\"会聊天\"变成\"能干活\"：自己拆任务、调工具、看结果、再行动。","Agent",10,{"id":68,"slug":76,"title":77,"description":78,"level":8,"topic":79,"url":10,"content":10,"sortOrder":80},"part-11-multi-agent","Part 11 · 多 Agent 与 Agent IDE","从\"单个 Agent\"升级到\"多 Agent 协作\"，并掌握 AutoGen \u002F LangGraph \u002F GPTs \u002F Coze \u002F Dify 五个工具。","多 Agent",11,{"id":74,"slug":82,"title":83,"description":84,"level":8,"topic":85,"url":10,"content":10,"sortOrder":86},"part-12-llamaindex","Part 12 · LlamaIndex","站 5~7 你用了 LangChain 做 RAG。LlamaIndex 是另一条专攻\"数据 + LLM\"的路线。","LlamaIndex",12,{"id":80,"slug":88,"title":89,"description":90,"level":8,"topic":91,"url":10,"content":10,"sortOrder":92},"part-13-transformer","Part 13 · Transformer 深入","站 2A 建立了直觉，这一站把手写代码跑通——从\"懂概念\"到\"写得出\"。","Transformer",13,{"id":86,"slug":94,"title":95,"description":96,"level":8,"topic":97,"url":10,"content":10,"sortOrder":98},"part-14-open-models","Part 14 · 开源模型与私有化部署","目标：把主流开源模型在本地\u002F服务器跑通，理解服务化工程（显存、量化、并发）。","模型部署",14,{"id":92,"slug":100,"title":101,"description":102,"level":8,"topic":103,"url":10,"content":10,"sortOrder":104},"part-15-finetuning","Part 15 · 模型微调 Fine-Tuning","目标：理解\"什么时候必须微调、怎么选基座、怎么备数据\"，并完整跑通一次业务微调。","微调",15,{"id":98,"slug":106,"title":107,"description":108,"level":8,"topic":109,"url":10,"content":10,"sortOrder":4},"part-16-peft-lora","Part 16 · PEFT 参数高效微调","重点吃透 **LoRA**：用极少量可训练参数微调大模型（消费级显卡就能跑）。","PEFT\u002FLoRA",{"id":104,"slug":111,"title":112,"description":113,"level":8,"topic":114,"url":10,"content":10,"sortOrder":115},"part-17-quantization","Part 17 · 模型量化","目标：理解\"为什么量化、主流算法各自思路\"，能动手量化并评估效果。","量化",17,{"id":4,"slug":5,"title":6,"description":7,"level":8,"topic":9,"url":10,"content":10,"sortOrder":12},{"id":115,"slug":118,"title":119,"description":120,"level":8,"topic":121,"url":10,"content":10,"sortOrder":122},"part-19-multimodal","Part 19 · 多模态应用","让 AI 同时\"看得懂图、听得了音、说得出话\"。","多模态",19,{"id":12,"slug":124,"title":125,"description":126,"level":8,"topic":127,"url":10,"content":10,"sortOrder":128},"part-20-projects-career","Part 20 · 项目实战 + 求职备战","把 0→1 到 1→100 学到的全部收敛成**能展示、能讲、能面试**的项目。","项目与求职",20,{"id":130,"slug":131,"title":132,"description":133,"level":8,"topic":134,"url":10,"content":10,"sortOrder":135},102,"extra-2026-ai-tech","额外篇 · 2026 年 AI 应用开发必学的 5 项成熟技术","从近半年爆发的技术里，挑出已经过了尝鲜期、能真正用在项目里的 5 项：推理模型、MCP、多模态、GraphRAG、AI 编程工具链，每项都给到能跑通的代码。","AI 工程实践",21]