unsloth + vllm-windows示例

啥都不说了

vLLM Windows CUDA support

vllm-windows Windows wheels by SystemPanic

unsloth Windows folk （star pls）

notebook

原始(不支持windows) https://github.com/unslothai/notebooks/blob/main/nb/Qwen2.5_(3B)-GRPO.ipynb 

修改后支持windows https://github.com/fenglui/unsloth/blob/main/nb/Qwen2.5_(3B)-GRPO-windows.ipynb 

以下为主要的修改

install

conda create -n vllm python=3.12
conda activate vllm
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install https://github.com/SystemPanic/vllm-windows/releases/download/v0.8.2/vllm-0.8.2+cu124-cp312-cp312-win_amd64.whl
pip install https://github.com/SystemPanic/flashinfer-windows/releases/download/v0.2.3/flashinfer_python-0.2.3+cu124torch2.6-cp312-cp312-win_amd64.whl
pip install --upgrade pillow
pip install --upgrade pandas
pip install --upgrade triton-windows
pip install grpcio==1.71.0
pip install "unsloth[windows] @ git+https://github.com/fenglui/unsloth.git"
pip install --no-deps git+https://github.com/huggingface/transformers.git
pip install trl==0.15.2

training test

download https://github.com/unslothai/notebooks/blob/main/nb/Qwen2.5_(3B)-GRPO.ipynb and remove the installation code block

add code block at top

import os
os.environ['UNSLOTH_DISABLE_AUTO_UPDATES'] = '1'
os.environ["VLLM_USE_V1"] = "0"
os.environ["VLLM_ATTENTION_BACKEND"] = "FLASHINFER"

# Disable libuv on Windows by default
os.environ["USE_LIBUV"] = os.environ.get("USE_LIBUV", "0")

then you can run the rest of the training code.

if the vllm serve faild, add env var "USE_LIBUV" and set value "0" to your windows system var

and that's it, we can run unsloth training with vllm support with Windows.