强化学习算法 —— TRPO —— KL散度求费雪信息矩阵时的trick —— 用10%数据估算费雪信息矩阵FIM

相关：

https://digitalassets.lib.berkeley.edu/techreports/ucb/text/EECS-2016-217.pdf

 Hence, a naïve implementation would spend more than 90% of the computational effort on these Fisher-vector
products. However, we can greatly reduce this burden by subsampling the data for the
computation of Fisher-vector product. Since the Fisher information matrix merely acts as
a metric, it can be computed on a subset of the data without severely degrading the quality of the final step. Hence, we can compute it on 10% of the data, and the total cost of
Hessian-vector products will be about the same as computing the gradient. With this optimization, the computation of a natural gradient step A-1g does not incur a significant
extra computational cost beyond computing the gradient g.

豆包AI给出解释：