MATLAB小函数：计算KL散度与JS散度

MATLAB小函数：计算KL散度与JS散度

作者：凯鲁嘎吉 - 博客园 http://www.cnblogs.com/kailugaji/

问题：给定两个向量，计算这两个向量之间的Kullback-Leibler Divergence与Jensen-Shannon Divergence。KL散度与JS散度的计算公式参考：相似性度量 - 凯鲁嘎吉 - 开发者的网上家园 

1. MATLAB程序

function [score_KL, score_JS] = KL_JS_div(vec1, vec2)
% Input: vec1: vector 1, vec2: vector 2
% Output: score_KL: KL divergence, source_JS: JS divergence
% Author: kailugaji 
% https://www.cnblogs.com/kailugaji/

% Make sure vec1 and vec2 sum to 1
if any(vec1(:))
    vec1 = vec1/sum(vec1(:));
end
 
if any(vec2(:))
    vec2 = vec2/sum(vec2(:));
end

% Compute Kullback-Leibler Divergence
score_KL = sum(sum(vec1.* log(eps + vec1./(vec2+eps))));

% Compute Jensen-Shannon Divergence
score_JS = (sum(sum(vec1.* log(eps + vec1./((vec1+vec2)./2+eps))))+sum(sum(vec2.* log(eps + vec2./((vec1+vec2)./2+eps)))))./2;

if vec1==vec2
    score_KL=0;
    score_JS=0;
end

2. 结果

>> vec1=[0.2 0.4 0.4];
>> vec2=[0.3 0.2 0.5];
>> [score_KL, score_JS] = KL_JS_div(vec1, vec2)

score_KL =

   0.106908430076661


score_JS =

   0.024807303850391

2020-09-30