Part 15 · 模型微调 Fine-Tuning

需求类型                   用啥临时改风格/格式              Prompt要答私有知识/新资料           RAG要固定输出格式/领域术语       微调要模型学会新能力/新知识       微调要低成本快速验证             先 Prompt → 不够再 RAG → 再微调

中文通用/业务   Qwen2.5 系列（首选）中文对话        ChatGLM 系列英文通用        Llama3 系列推理强          DeepSeek 系列（较大，需好显卡）小体量快速试     Qwen2.5-0.5B / 1.5B

{"messages": [{"role": "user", "content": "问题"}, {"role": "assistant", "content": "答案"}]}{"messages": [{"role": "user", "content": "另一个问题"}, {"role": "assistant", "content": "另一个答案"}]}

{"messages": [{"role": "user", "content": "你好"}, {"role": "assistant", "content": "您好，请问有什么可以帮您？"}]}{"messages": [{"role": "user", "content": "怎么退款"}, {"role": "assistant", "content": "亲，请在订单页点击申请退款，1-3个工作日到账。"}]}

import jsonpairs = [    ("你们几点发货", "我们工作日下午4点前下单当天发货。"),    ("能开发票吗", "可以，下单时填写抬头，随货寄出。"),    ("支持七天无理由吗", "支持，未拆封7天内可申请。"),    # ... 再加 10~20 条]with open("train_data.jsonl", "w", encoding="utf-8") as f:    for q, a in pairs:        f.write(json.dumps({"messages": [            {"role": "user", "content": q},            {"role": "assistant", "content": a},        ]}, ensure_ascii=False) + "\n")print("生成完成，共", len(pairs), "条")

少样本风格微调：100~500 条高质量即可领域能力提升：1000~10000 条原则：宁可少而精，不要多而脏

import json# 读入rows = []with open("train_data.jsonl", encoding="utf-8") as f:    rows = [json.loads(line) for line in f]# 1. 去重（按 user 问题去重）seen, cleaned = set(), []for r in rows:    q = r["messages"][0]["content"]    if q in seen:        continue    seen.add(q)    cleaned.append(r)# 2. 过滤：答案太短的扔掉（<5字说明质量差）cleaned = [r for r in cleaned if len(r["messages"][1]["content"]) >= 5]# 3. 写回with open("train_data_clean.jsonl", "w", encoding="utf-8") as f:    for r in cleaned:        f.write(json.dumps(r, ensure_ascii=False) + "\n")print(f"原 {len(rows)} 条 → 清洗后 {len(cleaned)} 条")

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainerfrom datasets import Datasetimport json# 读数据rows = [json.loads(l) for l in open("train_data_clean.jsonl", encoding="utf-8")]dataset = Dataset.from_list(rows)# 加载小模型（CPU 也能跑 0.5B）model_name = "Qwen/Qwen2.5-0.5B-Instruct"tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)def fmt(ex):    text = tokenizer.apply_chat_template(ex["messages"], tokenize=False)    return tokenizer(text, truncation=True, max_length=256, padding="max_length")dataset = dataset.map(fmt)args = TrainingArguments(    output_dir="hf_out",    num_train_epochs=3,    per_device_train_batch_size=1,    logging_steps=1,    save_strategy="no",    report_to=[],)trainer = Trainer(model=model, args=args, train_dataset=dataset)trainer.train()print("训练完成")

LLaMA-Factory = 一行命令/一个网页 完成 LoRA 全参微调支持：SFT / LoRA / QLoRA / DPO，支持 Qwen 等全系模型

git clone https://github.com/hiyouga/LLaMA-Factory.gitcd LLaMA-Factorypip install -e .

python src/train_bash.py \  --model_name_or_path Qwen/Qwen2.5-1.5B-Instruct \  --dataset train_data_clean.jsonl \  --template qwen \  --output_dir ./output_sft \  --num_train_epochs 3 \  --per_device_train_batch_size 1 \  --learning_rate 5e-5 \  --save_strategy no

from transformers import AutoModelForCausalLM, AutoTokenizer# 加载你微调后的模型model = AutoModelForCausalLM.from_pretrained("./output_sft", trust_remote_code=True)tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", trust_remote_code=True)text = tokenizer.apply_chat_template(    [{"role": "user", "content": "能开发票吗"}], tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt")out = model.generate(**inputs, max_new_tokens=50)print(tokenizer.decode(out[0][len(inputs["input_ids"][0]):], skip_special_tokens=True))

DeepSpeed = 微软的分布式训练框架，让大模型在有限显存上训练  ZeRO 1/2/3：把优化器状态/梯度/参数分片到多卡作用：你本地单卡跑不动的模型，用 DeepSpeed + 多卡能跑

坑1  显存 OOM → 换更小模型 / batch_size=1 / 加 gradient_accumulation坑2  中文乱码/重复 → max_length 太短或数据太脏，清洗+加长坑3  训练 loss 不降 → 学习率太大/数据太少，调小 lr 或加数据坑4  模型"忘了"原来能力（灾难性遗忘）→ 混入一些通用数据坑5  模板不匹配 → 用对 LLaMA-Factory 的 template（qwen/chatglm 等）