Sparse Transformer实现

参考informer, sputnik等。

 

DeepSeed的Sparse Attention:

https://www.deepspeed.ai/tutorials/sparse-attention/

https://www.deepspeed.ai/news/2020/09/08/sparse-attention.html

posted @ 2021-10-12 11:37  xuyv  阅读(351)  评论(0)    收藏  举报