XAI/MLI 可解释机器学习系列1- 开源&paper汇总

一直在关注可解释机器学习领域，因为确实在工作中有许多应用

• 模型检查，特征重要性是否符合预期和AUC一样重要
• 模型解释，比起虚无缥缈的模型指标，解释模型学到的规律更能说服业务方
• 样本解释，为什么这些用户会违约，是否有指标能提前预警？
• 决策归因，有时模型只是提取pattern的方式，最终需要给到归因/决策，例如HTE模型和XAI结合是否也是一种落地方式

18年被H2O Driverless AI 提供的可解释机器学习引擎（下图）种草后，就对这个领域产生了兴趣。不过用的越多，XAI暴露的问题就越多，比如特征的微调可能会导致整个特征解释发生翻天覆地的变化，再比如表现很好的模型会给出完全不能理解的特征解释。不过在接触因果推理后希望可以换个视角来看XAI，于是重新捡起这个系列(挖坑慎入，这是一个18年就开始挖，到现在都没有填完的坑)～

Algo & paper

开源库每个算法只提供了一个，大多是原作者或者我用过的，并不一定是start最多的，要是你知道better source欢迎留言哟～

算法	paper	GitHub
Permutation Importance	【1】	eli5
Feature Importace	计算方法有多种【2】	LGB/XGB/sklearn自带
Surrogate Model	【3】	h2o.ai
Local interpretable model_agnostic explanations（LIME）	【4】	lime
Leave one covariate out(LOCO)	【5】	h2o.ai
Individual Conditional Expectation(ICE)	[【6】][11]	[PDPbox][12]
Partial Dependence Plot(PDP)	[【7】][13]	[PDPbox][14]
shapley/SHAP	[【8】][15][【9】][16][【10】][17]	[shap][18]
DeepLift	[【11】][19]	[deeplift][20]
Layerwise Relevance Propagation（LRP）	[【12】][21]	[LRP demo][22]
Integrated Gradients	[【13】][23]	[Integrated-Gradients][24]

【1】Breiman, 2001, Random Forests
【2】方法有很多可以找xgb/lgb文档来看
【3】Osbert Bastani, Carolyn Kim, and Hamsa Bastani, 2017. Interpreting Blackbox Models via Model Extraction.
【4】Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. Why should I trust you?: Explaining the predictions of any classifier. 2016
【5】Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J. Tibshirani, and Larry Wasserman, 2016, Distribution-Free Predictive Inference For Regression
【6】Goldstein, Alex, et al, 2015, Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation.
【7】J. H. Friedman, 2001, Greedy function approximation: a gradient boosting machine
【8】Lundberg, Scott M., and Su-In Lee, 2017. A unified approach to interpreting model predictions
【9】Lundberg, Scott M., Gabriel G. Erion, and Su-In Lee, 2018. Consistent individualized feature attribution for tree ensembles.
【10】Sundararajan, Mukund, and Amir Najmi, 2019, The many Shapley values for model explanation
【11】 Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje, 2017 . Learning important features through
propagating activation differences
【12】Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller,
and Wojciech Samek, 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise
relevance propagation
【13】Mukund Sundararajan, Ankur Taly, and Qiqi Yan, 2017. Axiomatic attribution for deep networks

Tutorial

以下tutorial不同程度覆盖了上述算法，这两年的只能看paper咯。

推荐第一本，据说是LMU2019年学生研讨会的作业汇总。。。引入了因果的概念来分析在哪些情况下XAI会cheating，虽然大多是点到即止没有深入，不过指出的一些坑命中率还是很高的>_< ，有一句话记忆很深刻 可解释算法解释的是模型学到了什么，而非实际数据表现如何

1. [Limitations of Interpretable Machine Learning Methods][25]
2. [Interpretable Machine Learning, A Guide for Making Black Box Models Explainable][26].
3. [OREILLY, Ideas on interpreting machine learning][27]
4. [Kaggle, Machine Learning Explainability][28]
5. [H2O AI, An-Introduction-to-Machine-Learning-Interpretability-Second-Edition][29]
6. [MLI-source][30]
7. [h2o.ai interpretable_machine_learning_with_python][31]
8. [h2o.ai awesome-machine-learning-interpretability][32]

XAI的难度不在理解算法本身，而是算法和数据结合时，你需要知道什么时候算法会fail, 以及在模型解释不如预期的时候如何追查原因。说白了就是要在玄学中找规律。。。所以后面我们会找个数据集来试试看

持续更新中～

[11]: https://www.google.com/search?q=Goldstein ,%20Alex,%20et%20al.%20%22Peeking%20inside%20the%20black%20box:%20Visualizing%20statistical%20learning%20with%20plots%20of%20individual%20conditional%20expectation.%22%20Journal%20of%20Computational%20and%20Graphical%20Statistics%2024.1%20%282015%29&oq=Goldstein,%20Alex,%20et%20al.%20%22Peeking%20inside%20the%20black%20box:%20Visualizing%20statistical%20learning%20with%20plots%20of%20individual%20conditional%20expectation.%22%20Journal%20of%20Computational%20and%20Graphical%20Statistics%2024.1%20%282015%29&aqs=chrome..69i57j69i59l7.159j0j7&sourceid=chrome&ie=UTF-8
[12]: https://github.com/SauceCat/PDPbox
[13]: https://statweb.stanford.edu/~jhf/ftp/trebst.pdf
[14]: https://github.com/SauceCat/PDPbox
[15]: https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions
[16]: https://arxiv.org/abs/1802.03888
[17]: https://arxiv.org/pdf/1908.08474.pdf
[18]: https://github.com/slundberg/shap
[19]: https://arxiv.org/abs/1704.02685
[20]: https://github.com/kundajelab/deeplift
[21]: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140
[22]: http://heatmapping.org/tutorial/
[23]: https://arxiv.org/abs/1703.01365
[24]: https://github.com/ankurtaly/Integrated-Gradients
[25]: https://compstat-lmu.github.io/iml_methods_limitations/
[26]: https://christophm.github.io/interpretable-ml-book/
[27]: https://www.oreilly.com/radar/ideas-on-interpreting-machine-learning/
[28]: https://www.h2o.ai/wp-content/uploads/2019/08/An-Introduction-to-Machine-Learning-Interpretability-Second-Edition.pdf
[29]: https://www.google.com/search?q=H2O+AI%2C+An-Introduction-to-Machine-Learning-Interpretability-Second-Edition&oq=H2O+AI%2C+An-Introduction-to-Machine-Learning-Interpretability-Second-Edition&aqs=chrome..69i57.204j0j9&sourceid=chrome&ie=UTF-8
[30]: https://github.com/h2oai/mli-resources
[31]: https://github.com/jphall663/interpretable_machine_learning_with_python
[32]: https://github.com/jphall663/awesome-machine-learning-interpretability