11 2023 档案

llama-factory fine-tuning-2 (conception and technologies explanation)
摘要：train method Reward Modeling PPO training DPO training full-parameter fine-tuning all weights. partial-parameter freeze some weights and change some w 阅读全文
posted @ 2023-11-29 15:42 Daze_Lu 阅读(194) 评论(0) 推荐(0)
llama-factory fine-tuning-1
摘要：data preparation for llama-factory fine-tuning, here is the instruction for custom dataset preparation. dataset classification alpaca stanford_alpaca 阅读全文
posted @ 2023-11-29 12:48 Daze_Lu 阅读(493) 评论(0) 推荐(0)
llama-recipes fine-tuning-2
摘要：data preparation we use huggingface shibin6624/medical to fine-tuning llama2, please note that this dataset is consist of en and cn data, here we just 阅读全文
posted @ 2023-11-29 09:13 Daze_Lu 阅读(113) 评论(0) 推荐(0)
activation functions summary and comparision
摘要：written in the foreword Any nonlinear function that has good derivative properties has the potential to become an activation function. So here, we wil 阅读全文
posted @ 2023-11-28 03:33 Daze_Lu 阅读(42) 评论(0) 推荐(0)
llama-recipes fine-tuning-1
摘要：bash code cd /home/ludaze/Docker/Llama/llama conda activate llama_etuning git clone https://github.com/facebookresearch/llama.git wget https://raw.git 阅读全文
posted @ 2023-11-27 06:52 Daze_Lu 阅读(206) 评论(0) 推荐(0)
use google server
摘要：1 login in VM login in the console and then change the configuration sudo -i vim /etc/ssh/sshd_config change the following lines PermitRootLogin yes P 阅读全文
posted @ 2023-11-18 17:28 Daze_Lu 阅读(28) 评论(0) 推荐(0)
Split the model
摘要：why do we need to split the model Sometimes a model is a little large for a single GPU, we can deploy the model into several GPUS by splitting one nod 阅读全文
posted @ 2023-11-16 03:08 Daze_Lu 阅读(23) 评论(0) 推荐(0)
USE MMLU datset to test llama2
摘要：run llama2 1 llama2 repository: here dataset mmlu dataset structure RESULT command CUDA_VISIBLE_DEVICES=0 python src/evaluate.py \ --model_name_or_pat 阅读全文
posted @ 2023-11-15 15:43 Daze_Lu 阅读(460) 评论(0) 推荐(0)