fariver

摘要： 目录TL;DR大规模高效训练算法优化通信重叠优化其他优化容错设计(Fault Tolerance)数据收集与分析诊断测试Fast Checkpointing and RecoveryExperiment相关链接 MegaScale: Scaling Large Language Model Trai 阅读全文

摘要： 目录Efficient Memory Management for Large Language Model Serving with PagedAttentionTL;DRMotivation现状：GPU显存是瓶颈具体浪费情况MethodvLLM Framework调度与抢占其它TrickExpe 阅读全文

摘要： 目录PyTorch FSDP: Experiences on Scaling Fully Sharded Data ParallelTL;DRMethodSystem DesignModel InitializationSharding Strategies（分片策略）​​Full Sharding 阅读全文

摘要： 目录DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented ScaleTL;DR推理优化方法针对Transformer Kernel优化DeepFusionSBI-GeMM针对D 阅读全文

摘要： 目录FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTL;DRMethodFlashAttention算法详解Sparse FlashAttentionExperimentQ&A总结与思考相关链接 阅读全文

摘要： 目录Megatron: Reducing Activation Recomputation in Large Transformer ModelsTL;DRMethodSP(Sequence Parallelism、序列并行)Selective RecomputationCode && Implem 阅读全文

摘要： 目录BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationTL;DRMethod预训练DataFilt数据ImplementationExperi 阅读全文

摘要： 目录FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image GenerationTL; DR;Method数据训练过程推理过程Experiment总结与思考Contribution写作 FoundHand 阅读全文

摘要： 目录简介TL;DRMethodDatasetExperiment总结与思考 简介 LXMERT: Learning Cross-Modality Encoder Representations from Transformers 时间：2019.08（EMNLP 2019） 单位：UNC Chape 阅读全文

摘要： 目录简介TL;DRMethod核心创新点学习方式Experiment 简介 link 时间：2019.08.06 单位：Georgia Institute of Technology, Facebook AI Research, Oregon State University 相关领域：计算机视觉与 阅读全文