fariver

摘要： 快捷键配置 workbench.view.explorer -> cmd + E workbench.action.previousEditor -> shift+ h workbench.action.nextEditor -> shift + l go to definition -> ctr+ 阅读全文

摘要： 整体架构 物理模块 包含关系为：GPC > TPC > SM > CORE GPC(Graphics Processing Clusters 图形处理簇)：GPC负责处理图形渲染和计算任务。每个GPC包含多个TPC，以及与其相关的专用硬件单元和缓存。 TPC(Texture Processing C 阅读全文

摘要： 目录TL;DR大规模高效训练算法优化通信重叠优化其他优化容错设计(Fault Tolerance)数据收集与分析诊断测试Fast Checkpointing and RecoveryExperiment相关链接 MegaScale: Scaling Large Language Model Trai 阅读全文

摘要： 目录Efficient Memory Management for Large Language Model Serving with PagedAttentionTL;DRMotivation现状：GPU显存是瓶颈具体浪费情况MethodvLLM Framework调度与抢占其它TrickExpe 阅读全文

摘要： 目录PyTorch FSDP: Experiences on Scaling Fully Sharded Data ParallelTL;DRMethodSystem DesignModel InitializationSharding Strategies（分片策略）​​Full Sharding 阅读全文

摘要： 目录DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented ScaleTL;DR推理优化方法针对Transformer Kernel优化DeepFusionSBI-GeMM针对D 阅读全文

摘要： 目录FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTL;DRMethodFlashAttention算法详解Sparse FlashAttentionExperimentQ&A总结与思考相关链接 阅读全文

摘要： 目录Megatron: Reducing Activation Recomputation in Large Transformer ModelsTL;DRMethodSP(Sequence Parallelism、序列并行)Selective RecomputationCode && Implem 阅读全文

摘要： 目录BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationTL;DRMethod预训练DataFilt数据ImplementationExperi 阅读全文

摘要： 目录FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image GenerationTL; DR;Method数据训练过程推理过程Experiment总结与思考Contribution写作 FoundHand 阅读全文