指令记录
命令行环境变量设置:
https://www.cnblogs.com/fuhai0815/p/9753585.html
复制:
https://blog.csdn.net/u012964600/article/details/134926880
解压:
https://blog.csdn.net/imyang2007/article/details/7634470
tar:
https://blog.csdn.net/qq_43657810/article/details/132328941
ubuntu版本查看:
https://blog.csdn.net/whbing1471/article/details/52074390
vim多行删除:
https://blog.csdn.net/xueqinmax/article/details/100153229
xz命令?:(误区之一开始以为要下载xz,然后还想要下载yum,而且还卡住了,但是实际上没有必要 https://computingforgeeks.com/how-to-extract-xz-files-on-linux/)
https://cn.linux-console.net/?p=3682
文件查看:
https://www.cnblogs.com/LovelyAmy/articles/7986959.html
scp命令:
https://blog.csdn.net/a545812327/article/details/111313810
https://blog.csdn.net/qq_29307291/article/details/72819802
conda:
路径配置:
vim ~./bashrc
export
source ~/.bashrc
首次activate:https://blog.csdn.net/sdnuwjw/article/details/112448792
huggingface数据下载:
模型:https://blog.csdn.net/lanlinjnc/article/details/136709225
数据集:https://blog.csdn.net/sinat_29950703/article/details/143063793
加载(load_dataset):
原理:https://developer.baidu.com/article/details/2798734
用法:https://blog.csdn.net/orangerfun/article/details/131927248
& https://www.cnblogs.com/zhangxuegold/p/17531896.html
本地加载数据集的两个例子:
load_dataset("/defaultShare/archive/zhangyang/cache/huggingface/datasets/wikitext/wikitext-2-raw-v1", split='train') traindata = load_dataset( 'json', data_files={'train': '/defaultShare/archive/zhangyang/cache/huggingface/hub/datasets--allenai--c4/snapshots/1588ec454efa1a09f29cd18ddd04fe05fc8653a2/en/c4-train.00000-of-01024.json.gz'}, split='train' )
load_dataset(cache_dir = "")缓存路径下 一定是datasets集合下有数据吗?
以上示例均来自claude
load_datasets()缓存的文件怎么加载?
缓存路径
HF_HOME
git镜像(不一定有效)
git clone https://hub.fastgit.org/
浙公网安备 33010602011771号