Stay Hungry,Stay Foolish!

LoRA

LoRA

https://arxiv.org/abs/2106.09685

An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes less feasible. Using GPT-3 175B as an example -- deploying independent instances of fine-tuned models, each with 175B parameters, is prohibitively expensive. We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times. LoRA performs on-par or better than fine-tuning in model quality on RoBERTa, DeBERTa, GPT-2, and GPT-3, despite having fewer trainable parameters, a higher training throughput, and, unlike adapters, no additional inference latency. We also provide an empirical investigation into rank-deficiency in language model adaptation, which sheds light on the efficacy of LoRA. We release a package that facilitates the integration of LoRA with PyTorch models and provide our implementations and model checkpoints for RoBERTa, DeBERTa, and GPT-2 at this https URL.

 

https://github.com/microsoft/LoRA

LoRA: Low-Rank Adaptation of Large Language Models

This repo contains the source code of the Python package loralib and several examples of how to integrate it with PyTorch models, such as those in Hugging Face. We only support PyTorch for now. See our paper for a detailed description of LoRA.

LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu*, Yelong Shen*, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen
Paper: https://arxiv.org/abs/2106.09685
Video explainer: https://www.youtube.com/watch?v=DhRoTONcyZE

Update 2/2023: LoRA is now supported by the State-of-the-art Parameter-Efficient Fine-Tuning (PEFT) library by Hugging Face.

LoRA reduces the number of trainable parameters by learning pairs of rank-decompostion matrices while freezing the original weights. This vastly reduces the storage requirement for large language models adapted to specific tasks and enables efficient task-switching during deployment all without introducing inference latency. LoRA also outperforms several other adaptation methods including adapter, prefix-tuning, and fine-tuning.

We obtain result comparable or superior to full finetuning on the GLUE benchmark using RoBERTa (Liu et al., 2019) base and large and DeBERTa (He et al., 2020) XXL 1.5B, while only training and storing a fraction of the parameters. Click the numbers below to download the RoBERTa and DeBERTa LoRA checkpoints.

  RoBERTa base
Fine-tune		RoBERTa base
LoRA		DeBERTa XXL
Fine-tune		DeBERTa XXL
LoRA
  # of Trainable Params. 125M 0.8M 1.5B 4.7M
  MNLI (m-Acc/mm-Acc) 87.6 87.5±.3/86.9±.3 91.7/91.9 91.9±.1/91.9±.2
  SST2 (Acc) 94.8 95.1±.2 97.2 96.9±.2
  MRPC (Acc) 90.2 89.7±.7 92.0 92.6±.6
  CoLA (Matthew's Corr) 63.6 63.4±1.2 72.0 72.4±1.1
  QNLI (Acc) 92.8 93.3±.3 96.0 96.0±.1
  QQP (Acc) 91.9 90.8±.1 92.7 92.9±.1
  RTE (Acc) 78.7 86.6±.7 93.9 94.9±.4
  STSB (Pearson/Spearman Corr) 91.2 91.5±.2/91.3±.2 92.9/92.6 93.0±.2/92.9±.3
  Average 86.40 87.24 91.06 91.32

Note: You still need the original pre-trained checkpoint from Hugging Face to use the LoRA checkpoints.

Fine-tuning numbers are taken from Liu et al. (2019) and He et al. (2020). We include confidence intervals on results from our experiments. Please follow the instructions in examples/NLU/ to reproduce our results.

On GPT-2, LoRA compares favorably to both full finetuning and other efficient tuning methods, such as adapter (Houlsby et al., 2019) and prefix tuning (Li and Liang, 2021). We evaluated on E2E NLG Challenge, DART, and WebNLG:

 Method# of Trainable ParamsE2E (BLEU)DART (BLEU)WebNLG (BLEU-U/S/A)
  GPT-2 M (Fine-Tune) 354.92M 68.2 46.0 30.4/63.2/47.6
  GPT-2 M (Adapter) 0.37M 66.3 42.4 45.1/54.5/50.2
  GPT-2 M (Prefix) 0.35M 69.7 45.7 44.1/63.1/54.4
  GPT-2 M (LoRA) 0.35M 70.4±.1 47.1±.2 46.7±.4/62.1±.2/55.3±.2
  GPT-2 L (Fine-Tune) 774.03M 68.5 46.5 41.7/64.6/54.2
  GPT-2 L (Adapter) 0.88M 69.1±.1 45.7±.1 49.8±.0/61.1±.0/56.0±.0
  GPT-2 L (Prefix) 0.77M 70.3 46.5 47.0/64.2/56.4
  GPT-2 L (LoRA) 0.77M 70.4±.1 47.5±.1 48.4±.3/64.0±.3/57.0±.1

Non-LoRA baselines, except for adapter on GPT-2 large, are taken from Li and Liang (2021). We include confidence intervals on results from our experiments.

Download the GPT-2 LoRA checkpoints:

Please follow the instructions in examples/NLG/ to reproduce our result.

 

大模型微调后不用merge，下次也启动也可以加载lora部分权重并运行，是吧？

是的，完全正确。

大模型微调后不需要将 LoRA 权重合并（merge）到基座模型中。在推理或下次启动时，你可以直接加载基座模型和 LoRA 适配器权重来运行。

目前主流的框架和库都完美支持这种“基座 + Adapter”的分离式加载方式：

1. 主流加载方式

  • Hugging Face PEFT： 使用 PeftModel.from_pretrained(base_model, lora_path) 即可自动挂载 LoRA 权重。
  • vLLM / SGLang： 这些高性能推理引擎原生支持动态加载 LoRA，甚至可以在同一个基座模型上同时服务多个不同的 LoRA 适配器，且无需重启。
  • Ollama / LM Studio： 也支持直接指定 GGUF 格式的 LoRA 文件配合基座模型运行。

2. 不 Merge 的优势

  • 节省存储： 一个 7B/8B 模型的 LoRA 权重通常只有几十 MB 到几百 MB，而全量模型动辄十几 GB。如果你有多个微调任务，只需存一份基座 + N 个轻量 LoRA 即可。
  • 灵活切换： 可以在运行时动态热插拔不同的 LoRA 适配器，适合多任务场景。
  • 保留基座通用能力： 随时可以卸载 LoRA 回退到原始基座模型。

3. 什么时候才需要 Merge？

只有在以下少数场景中才建议合并：

  • 部署到不支持 LoRA 的老旧/特殊推理框架
  • 需要做量化（部分量化工具要求先 merge 再量化，不过现在 AutoGPTQ/AWQ/GGUF 等工具也越来越多地支持带 LoRA 量化了）
  • 追求极致的单请求推理延迟（merge 后省去了运行时加权计算的微小开销，但差距通常很小）

⚠️ 注意事项 加载时务必确保 LoRA 权重与基座模型的版本、精度、tokenizer 完全匹配，否则会出现输出乱码或报错。保存时建议将 adapter_config.json 和 adapter_model.safetensors 放在同一目录下，方便后续加载。

总结：日常使用和绝大多数生产部署场景下，都不需要 merge，直接加载 LoRA 即可。

 

运行时动态热插拔，这是什么含义？ 一个基座模型已经运行，然后单独加载lora参数？

下面用一个完整的、可运行的代码示例来演示"基座常驻 + LoRA 热插拔"的实际效果。

🎯 场景设定

你有一个 Qwen2.5-7B 基座模型，以及两个微调好的 LoRA：

  • code-lora：擅长写代码
  • translate-lora：擅长中英翻译

目标：同一个基座，不重启，按需切换。

💻 示例代码（基于 HuggingFace PEFT）

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# ========== 1. 加载基座模型（只加载一次）==========
base_model_path = "Qwen/Qwen2.5-7B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(base_model_path)
model = AutoModelForCausalLM.from_pretrained(
    base_model_path, torch_dtype="auto", device_map="auto"
)
print("✅ 基座模型已加载到显存")

# ========== 2. 定义热插拔函数 ==========
def generate_with_lora(prompt: str, lora_path: str | None = None):
    """
    lora_path=None  → 使用纯基座
    lora_path=路径  → 动态加载/切换 LoRA
    """
    if lora_path is not None:
        # 🔥 关键：动态加载 LoRA，基座不卸载
        model.load_adapter(lora_path, adapter_name=lora_path)
        model.set_adapter(lora_path)
        print(f"🔌 已切换到 LoRA: {lora_path}")
    else:
        # 禁用所有 LoRA，回退到纯基座
        model.disable_adapter_layers()
        print("🔌 已切换回纯基座模型")

    messages = [{"role": "user", "content": prompt}]
    inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
    outputs = model.generate(inputs, max_new_tokens=256)
    response = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)
    return response.strip()

# ========== 3. 实际使用：热插拔演示 ==========

# 请求1：用代码 LoRA 写代码
print(generate_with_lora(
    "用Python实现快速排序",
    lora_path="/path/to/code-lora"
))

# 请求2：不重启，切换到翻译 LoRA
print(generate_with_lora(
    "请将'人工智能正在改变世界'翻译成英文",
    lora_path="/path/to/translate-lora"
))

# 请求3：不重启，回退到纯基座
print(generate_with_lora(
    "今天天气怎么样？",
    lora_path=None
))

# 请求4：再切回代码 LoRA（如果之前已加载过，直接从缓存激活，毫秒级）
print(generate_with_lora(
    "用Python实现归并排序",
    lora_path="/path/to/code-lora"
))

🔍 运行时的内存与行为变化

时间线          显存状态                          说明
─────────────────────────────────────────────────────────
T0             [基座 14GB]                       初始加载
T1 请求1       [基座 14GB + code-lora 80MB]      load_adapter 首次加载
T2 请求2       [基座 14GB + code-lora 80MB       set_adapter 切换
               + translate-lora 60MB]            （code-lora 保留在缓存）
T3 请求3       [基座 14GB + 两个LoRA仍在缓存]     disable_adapter，权重不删除
T4 请求4       [同上]                            set_adapter 激活已有缓存，零I/O

💡 关键点：load_adapter() 只在首次加载某个 LoRA 时有 I/O 开销；之后 set_adapter() 只是修改内部指针，是毫秒级操作。LoRA 权重会一直缓存在显存中，直到你手动调用 delete_adapter() 或显存不足被自动驱逐。

🚀 生产环境对比（vLLM 服务端热插拔）

上面的例子是单机脚本级别。在生产服务中，vLLM 的热插拔是通过 API 请求参数 实现的，连 Python 代码都不用改：

# 启动服务
vllm serve Qwen/Qwen2.5-7B-Instruct \
  --enable-lora \
  --lora-modules code=/path/to/code-lora translate=/path/to/translate-lora

# 客户端只需在请求中指定 lora 名称，服务端自动热切换
import openai
client = openai.OpenAI(base_url="http://localhost:8000/v1")

# 同一个 endpoint，不同请求用不同 LoRA
resp1 = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role":"user","content":"用Python实现快排"}],
    extra_body={"lora_request": "code"}        # ← 服务端热切换到 code-lora
)

resp2 = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role":"user","content":"翻译这句话"}],
    extra_body={"lora_request": "translate"}   # ← 服务端热切换到 translate-lora
)

resp3 = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role":"user","content":"你好"}]
    # 不传 lora_request → 纯基座
)

三个请求打到同一个服务进程、同一份基座显存，LoRA 在服务端按请求粒度自动热插拔，客户端完全无感。

📌 总结一句话

热插拔 = 基座不动，LoRA 像"插件"一样按需插入/拔出/切换，整个过程服务不停、显存不释放、无需重启。

 

 

posted @ 2026-09-18 12:47  lightsong  阅读(5)  评论(0)    收藏  举报
千山鸟飞绝，万径人踪灭