[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"learning-part-17-quantization":3,"learning-all":13},{"id":4,"slug":5,"title":6,"description":7,"level":8,"topic":9,"url":10,"content":11,"sortOrder":12},15,"part-17-quantization","Part 17 · 模型量化","目标：理解\"为什么量化、主流算法各自思路\"，能动手量化并评估效果。","ADVANCED","量化",null,"# Part 15 · 模型量化\n\n\n---\n\n## Section 1 · 为什么量化（概念）\n\n**Subsection 1 · 记概念**\n```\n量化 = 把模型的权重\u002F激活从高精度（FP32\u002FFP16）压到低精度（INT8\u002FINT4）\n目的：省显存、加速推理\n代价：精度损失（可能影响输出质量）\n```\n\n**Subsection 2 · 记精度与显存**\n```\nFP32 = 4字节\u002F参数   7B ≈ 28GB\nFP16 = 2字节\u002F参数   7B ≈ 14GB\nINT8 = 1字节\u002F参数   7B ≈ 7GB\nINT4 ≈ 0.5字节\u002F参数  7B ≈ 3.5GB\n```\n\n**过关**：能说清量化换什么、代价什么 = Section 1 完成。\n\n---\n\n## Section 2 · GPU 算力测算\n\n**Subsection 1 · 记概念**\n```\n算力单位 TFLOPS：每秒万亿次浮点运算\n推理算力需求 ≈ 2 × 参数量 × 每秒生成 token 数（FLOPs）\n例：7B 模型要 20 tokens\u002Fs → 需 ~2×7e9×20 ≈ 280 GFLOPs → 一般消费级卡轻松满足\n训练需求则大几个数量级 → 所以训练要数据中心\n```\n\n**Subsection 2 · 查自己的卡**\n`nvidia-smi` 看显存；`nvidia-smi --query-gpu=name,memory.total --format=csv` 看型号。\n- 记下你的卡能跑多大量化模型。\n\n**过关**：会算显存 + 了解算力 = Section 2 完成。\n\n---\n\n## Section 3 · PTQ 训练后量化\n\n**Subsection 1 · 记概念**\n```\nPTQ（Post-Training Quantization，训练后量化）\n= 模型训练好之后，直接量化，不重新训练\n优点：快、简单\n缺点：精度损失相对大\n```\n\n**Subsection 2 · 用 transformers 跑 PTQ 雏形**\n新建 `ptq_demo.py`：\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n\n# 加载时直接 8bit 量化（HF 内建 PTQ）\nmodel_name = \"Qwen\u002FQwen2.5-0.5B-Instruct\"\nconfig = BitsAndBytesConfig(load_in_8bit=True)\nmodel = AutoModelForCausalLM.from_pretrained(model_name, quantization_config=config, trust_remote_code=True)\ntokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)\n\nout = model.generate(tokenizer(\"你好\", return_tensors=\"pt\").input_ids, max_new_tokens=20)\nprint(tokenizer.decode(out[0], skip_special_tokens=True))\nprint(\"8bit 量化模型加载成功\")\n```\n- ✅ 预期看到：模型正常运行（内部已是 8bit）。\n\n**过关**：PTQ 概念 + 8bit 加载跑通 = Section 3 完成。\n\n---\n\n## Section 4 · QAT 量化感知训练\n\n**Subsection 1 · 记概念**\n```\nQAT（Quantization-Aware Training，量化感知训练）\n= 训练时就模拟量化误差，让模型\"适应\"低精度\n优点：效果比 PTQ 好\n缺点：要训练，成本高\n```\n- 💡 面试点：**PTQ 快但损失大，QAT 好但贵**。\n\n**过关**：能说 PTQ\u002FQAT 区别 = Section 4 完成。\n\n---\n\n## Section 5 · GPTQ（量化算法）\n\n**Subsection 1 · 记概念**\n```\nGPTQ = 专为 GPT 系模型设计的 4bit 量化\n思路：逐层量化，用\"误差补偿\"减少损失\n生态：AutoGPTQ 库，GGUF\u002FGPTQ 格式常见\n```\n\n**Subsection 2 · 装并量化**\ncmd：`pip install auto-gptq`（或 `llm-compressor`）\n用 AutoGPTQ 官方示例对一个小模型做 4bit 量化（代码见 https:\u002F\u002Fgithub.com\u002FAutoGPTQ\u002FAutoGPTQ README）。\n- ✅ 预期看到：量化后的模型能正常推理。\n\n**过关**：能说 GPTQ 思路 + 跑通示例 = Section 5 完成。\n\n---\n\n## Section 6 · AWQ（激活感知量化）\n\n**Subsection 1 · 记概念**\n```\nAWQ（Activation-aware Weight Quantization）\n= 根据\"激活值的重要性\"决定哪些权重少量化、哪些多量化\n思路：不是所有权重一样重要，重要的保留高精度\n效果：4bit 下效果通常优于 GPTQ\n```\n\n**Subsection 2 · 查资料**\n- 精读：https:\u002F\u002Fzhuanlan.zhihu.com\u002Fp\u002F681578090 （AWQ 量化技术解析）\n- 写 100 字理解笔记。\n\n**过关**：能说出 AWQ 与 GPTQ 区别 = Section 6 完成。\n\n---\n\n## Section 7 · GGUF 量化实操（llama.cpp \u002F Ollama）\n\n**Subsection 1 · 用 Ollama 体验量化档位**\n- `ollama run qwen2.5:1.5b-q4_K_M`（4bit 量化版）\n- `ollama run qwen2.5:1.5b`（默认更高精度）\n- 对比两个的回答质量与速度。\n- ✅ 预期看到：q4 版更快、显存更省，质量略降但中文通常仍可用。\n\n**Subsection 2 · 看 GGUF 档位**\n```\nq2_K\u002Fq3_K  极省，质量差\nq4_K       主流折中（推荐）\nq5_K\u002Fq6_K  更稳，略大\nf16        无量化，最大\n```\n\n**过关**：能对比不同档位 = Section 7 完成。\n\n---\n\n## Section 8 · 量化前后完整对比\n\n**Subsection 1 · 做对比实验**\n- 同一模型，分别用：FP16 \u002F INT8 \u002F INT4 跑同一批 10 个问题。\n- 记录：显存占用、生成速度、回答质量（自己打分 1~5）。\n- 整理成表格写进笔记。\n- ✅ 预期看到：量化程度越高，显存↓ 速度↑ 质量↓，找到你的\"甜点档位\"。\n\n**过关**：有完整对比数据 = Section 8 完成。\n\n---\n\n## Section 9 · 面试题\n\n**Subsection 1 · 练习**\n1. 什么是量化？为什么需要？\n2. PTQ 和 QAT 区别？\n3. GPTQ 和 AWQ 各自思路？\n4. 4bit 模型质量能接受吗？怎么验证？\n5. 一个 13B 模型 FP16\u002FINT4 各多少显存？\n- 先自己答，再让 DeepSeek 补充。\n\n**过关**：5 题能答 = Section 9 完成。\n\n---\n\n## Section 10 · 站 15 验收\n\n**勾选**\n- [ ] 能说清量化换什么、代价什么\n- [ ] 会算显存，知道自己的卡能跑什么档位\n- [ ] PTQ 概念 + 8bit 加载跑通\n- [ ] 能说 PTQ \u002F QAT 区别\n- [ ] 了解 GPTQ \u002F AWQ 思路\n- [ ] 用 GGUF 不同档位对比过（Section 7）\n- [ ] 有量化前后完整对比数据（Section 8）\n- [ ] 5 道面试题能答\n\n**写 300 字Part 15 总结**：你的甜点档位是什么，为什么。\n\n**全勾选 = Part 17 通过** → 进入下一站 Part 18 · 训练数据与模型评估，10 天）。\n\n\n\n",17,[14,21,27,33,40,46,52,58,63,69,75,81,87,93,99,104,110,111,117,123,129],{"id":15,"slug":16,"title":17,"description":18,"level":19,"topic":20,"url":10,"content":10,"sortOrder":15},1,"part-1-llm-basics","Part 1 · 大模型基础认知","小白零基础友好。这一站不写代码，只建立\"大模型到底是什么\"的骨架。","BEGINNER","大模型认知",{"id":22,"slug":23,"title":24,"description":25,"level":19,"topic":26,"url":10,"content":10,"sortOrder":22},2,"part-2-llm-principles","Part 2 · 大模型原理 · 入门实操","这一站不卷数学，只把\"理解模型所需的知识\"讲透。；站 2A 解决\"理解模型\"，站 2B 解决\"面试能答\"。；这一层是\"有余力再做\"，不做也不影响你进入站 3。","大模型原理",{"id":28,"slug":29,"title":30,"description":31,"level":19,"topic":26,"url":10,"content":10,"sortOrder":32},100,"part-3-llm-principles-deep","Part 3 · 大模型原理 · 面试深入","数学基础、机器学习、神经网络、词向量、思维链等面试必考原理",3,{"id":34,"slug":35,"title":36,"description":37,"level":38,"topic":26,"url":10,"content":10,"sortOrder":39},101,"part-4-llm-principles-advanced","Part 4 · 大模型原理 · 进阶可选","从零写迷你模型、RLHF、PPO、分布式训练实操等进阶内容","INTERMEDIATE",4,{"id":32,"slug":41,"title":42,"description":43,"level":19,"topic":44,"url":10,"content":10,"sortOrder":45},"part-5-prompt-engineering","Part 5 · Prompt 提示词工程","从这里开始，你每天都要真的调大模型 API。","提示工程",5,{"id":39,"slug":47,"title":48,"description":49,"level":19,"topic":50,"url":10,"content":10,"sortOrder":51},"part-6-llm-api","Part 6 · 大模型 API 调用","站 3 你已经调通了 API。这一站把\"调用\"做成专业水准：","API 工程",6,{"id":45,"slug":53,"title":54,"description":55,"level":38,"topic":56,"url":10,"content":10,"sortOrder":57},"part-7-rag","Part 7 · RAG 检索增强生成","这一站是\"能落地\"的关键：让模型回答**你的私有知识**。","RAG",7,{"id":51,"slug":59,"title":60,"description":61,"level":38,"topic":56,"url":10,"content":10,"sortOrder":62},"part-8-rag-optimization","Part 8 · RAG 优化与评估","站 5 你做出了\"能跑\"的 RAG。这一站把它做到\"能打\"：",8,{"id":57,"slug":64,"title":65,"description":66,"level":38,"topic":67,"url":10,"content":10,"sortOrder":68},"part-9-langchain","Part 9 · LangChain 框架","目标：用 LangChain 把\"模型、提示词、检索、记忆、工具\"串成可复用的链和 Agent。","LangChain",9,{"id":62,"slug":70,"title":71,"description":72,"level":38,"topic":73,"url":10,"content":10,"sortOrder":74},"part-10-agent","Part 10 · Agent 基础","让模型从\"会聊天\"变成\"能干活\"：自己拆任务、调工具、看结果、再行动。","Agent",10,{"id":68,"slug":76,"title":77,"description":78,"level":8,"topic":79,"url":10,"content":10,"sortOrder":80},"part-11-multi-agent","Part 11 · 多 Agent 与 Agent IDE","从\"单个 Agent\"升级到\"多 Agent 协作\"，并掌握 AutoGen \u002F LangGraph \u002F GPTs \u002F Coze \u002F Dify 五个工具。","多 Agent",11,{"id":74,"slug":82,"title":83,"description":84,"level":8,"topic":85,"url":10,"content":10,"sortOrder":86},"part-12-llamaindex","Part 12 · LlamaIndex","站 5~7 你用了 LangChain 做 RAG。LlamaIndex 是另一条专攻\"数据 + LLM\"的路线。","LlamaIndex",12,{"id":80,"slug":88,"title":89,"description":90,"level":8,"topic":91,"url":10,"content":10,"sortOrder":92},"part-13-transformer","Part 13 · Transformer 深入","站 2A 建立了直觉，这一站把手写代码跑通——从\"懂概念\"到\"写得出\"。","Transformer",13,{"id":86,"slug":94,"title":95,"description":96,"level":8,"topic":97,"url":10,"content":10,"sortOrder":98},"part-14-open-models","Part 14 · 开源模型与私有化部署","目标：把主流开源模型在本地\u002F服务器跑通，理解服务化工程（显存、量化、并发）。","模型部署",14,{"id":92,"slug":100,"title":101,"description":102,"level":8,"topic":103,"url":10,"content":10,"sortOrder":4},"part-15-finetuning","Part 15 · 模型微调 Fine-Tuning","目标：理解\"什么时候必须微调、怎么选基座、怎么备数据\"，并完整跑通一次业务微调。","微调",{"id":98,"slug":105,"title":106,"description":107,"level":8,"topic":108,"url":10,"content":10,"sortOrder":109},"part-16-peft-lora","Part 16 · PEFT 参数高效微调","重点吃透 **LoRA**：用极少量可训练参数微调大模型（消费级显卡就能跑）。","PEFT\u002FLoRA",16,{"id":4,"slug":5,"title":6,"description":7,"level":8,"topic":9,"url":10,"content":10,"sortOrder":12},{"id":109,"slug":112,"title":113,"description":114,"level":8,"topic":115,"url":10,"content":10,"sortOrder":116},"part-18-data-evaluation","Part 18 · 训练数据与模型评估","数据决定模型上限，评估决定你\"能不能交付\"。","数据与评估",18,{"id":12,"slug":118,"title":119,"description":120,"level":8,"topic":121,"url":10,"content":10,"sortOrder":122},"part-19-multimodal","Part 19 · 多模态应用","让 AI 同时\"看得懂图、听得了音、说得出话\"。","多模态",19,{"id":116,"slug":124,"title":125,"description":126,"level":8,"topic":127,"url":10,"content":10,"sortOrder":128},"part-20-projects-career","Part 20 · 项目实战 + 求职备战","把 0→1 到 1→100 学到的全部收敛成**能展示、能讲、能面试**的项目。","项目与求职",20,{"id":130,"slug":131,"title":132,"description":133,"level":8,"topic":134,"url":10,"content":10,"sortOrder":135},102,"extra-2026-ai-tech","额外篇 · 2026 年 AI 应用开发必学的 5 项成熟技术","从近半年爆发的技术里，挑出已经过了尝鲜期、能真正用在项目里的 5 项：推理模型、MCP、多模态、GraphRAG、AI 编程工具链，每项都给到能跑通的代码。","AI 工程实践",21]