馒头and花卷

摘要： 目录概1-bit Adam1-bit SGD代码 Seide F., Fu H., Droppo J., Li G. and Yu D. 1-bit stochastic gradient descent and its application to data-parallel distribute 阅读全文

摘要： 目录概MotivationZeROZeRO-OffloadZeRO-InfiniteZeRO++代码 Rajbhandari S., Rasley J., Ruwase O. and He Y. ZeRO: Memory optimizations toward training trillion 阅读全文