0. 配置環境變量
HF_ENDPOINT=https://hf-mirror.com
HF_HOME=/root/autodl-tmp/paul/tools/huggingface
1. 本機安裝python 3.10, 并設置軟件源
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
pip config set global.index-url https://mirrors.huaweicloud.com/repository/pypi/simple
2. 安裝miniconda
3. 新建一個環境, 并激活
conda create -n quantization python=3.12
2. 本機安裝pytorch2.5.1+cuda12.4
pip3 install torch torchvision torchaudio
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
3. clone llamafactory源碼
git clone https://github.com/hiyouga/LLaMA-Factory
4. llamafactory本地安裝依賴
pip install -e .["vllm","gptq"]
5. 啟動webui
6. 在頁面中填入相關參數進行操作