ValueError: You can't train a model that has been loaded with `device_map='auto'` in any distributed mode.

完整问题：

ValueError: You can't train a model that has been loaded with device_map='auto' in any distributed mode. Please rerun your script specifying --num_processes=1 or by launching with python {{myscript.py}}.

从 huggingface 下载有些模型训练会出现这个问题，有些不会。

如果出现这个问题，请考虑把模型下载到本地，然后看看模型源代码是否用 device_map='auto' 加载一些 module，有的话删掉即可。