当前标签：optimization

Communication Efficient Large-Scale Training with Adam's Convergence Speed

馒头and花卷 2025-02-13 21:28 阅读:10 评论:0 推荐:0

Convergence of Adam Under Relaxed Assumptions

馒头and花卷 2025-01-13 16:42 阅读:10 评论:0 推荐:0

Memory Efﬁcient Optimizers with 4-bit States

馒头and花卷 2024-12-07 13:58 阅读:30 评论:0 推荐:0

8-bit Optimizers via Block-wise Quantization

馒头and花卷 2024-12-06 11:21 阅读:116 评论:0 推荐:0

Symbolic Discovery of Optimization Algorithms

馒头and花卷 2024-11-28 14:53 阅读:17 评论:0 推荐:0

An Empirical Model of Large-Batch Training

馒头and花卷 2024-11-26 17:01 阅读:84 评论:0 推荐:0

Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

馒头and花卷 2024-09-11 15:28 阅读:162 评论:0 推荐:0

Memory-Efficient Adaptive Optimization

馒头and花卷 2024-09-10 21:25 阅读:24 评论:0 推荐:0

Adam-mini Use Fewer Learning Rates To Gain More

馒头and花卷 2024-08-28 15:58 阅读:43 评论:0 推荐:0

GaLore Memory-Efficient LLM Training by Gradient Low-Rank Projection

馒头and花卷 2024-08-27 16:05 阅读:105 评论:0 推荐:0

BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models

馒头and花卷 2024-08-27 10:12 阅读:171 评论:0 推荐:0

Why Transformers Need Adam: A Hessian Perspective

馒头and花卷 2024-08-26 17:13 阅读:89 评论:0 推荐:0

优化与收敛率小记

馒头and花卷 2024-07-18 20:19 阅读:139 评论:0 推荐:1

Regularized Stochastic Learning and Online Optimization

馒头and花卷 2024-07-16 09:27 阅读:62 评论:0 推荐:0

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients

馒头and花卷 2024-07-10 17:05 阅读:62 评论:0 推荐:0

Double Descent in Adversarial Training: An Implicit Label Noise Perspective

馒头and花卷 2022-04-14 22:26 阅读:70 评论:0 推荐:0

Exploring Memorization in Adversarial Training

馒头and花卷 2022-04-12 21:32 阅读:99 评论:2 推荐:0

DropAttack: A Masked Weight Adversarial Training Method to Improve Generalization of Neural Networks

馒头and花卷 2022-04-02 14:41 阅读:88 评论:0 推荐:0

HAT

馒头and花卷 2021-11-10 16:12 阅读:528 评论:0 推荐:0

FAT

馒头and花卷 2021-11-09 19:01 阅读:366 评论:0 推荐:0