如何理解感知机学习算法的对偶形式?
Talk is cheap. Show me the code.
# 对偶形式 2.3
class PLA_dual:
def __init__(self, max_iter=1000):
self.b = 0
self.lr = 0.1
self.max_iter = max_iter
self.iter = 0
def cal_w(self, X):
w = 0
for i in range(len(self.alpha)):
w += self.alpha[i]*y[i]*X[i]
return w
def gram_matrix(self, X):
return np.dot(X, X.T)
def fit(self, X, y):
N, M = X.shape
self.alpha = np.zeros(N)
gram = self.gram_matrix(X) #计算一次矩阵运算
for n in range(self.max_iter):
self.iter = n
wrong_items = 0
for i in range(N):
tmp = 0
for j in range(N):
tmp += self.alpha[j] * y[j] * gram[i,j]#迭代时采用查找表,减少计算复杂度
tmp += self.b
if y[i] * tmp <= 0:
self.alpha[i] += self.lr
self.b += self.lr * y[i]
wrong_items += 1
if wrong_items == 0:
self.w = self.cal_w(X)
print("finished at iters: {}, w: {}, b: {}".format(self.iter, self.w, self.b))
return
self.w = self.cal_w(X)
print("finished for reaching the max_iter: {}, w: {}, b: {}".format(self.max_iter, self.w, self.b))
return
假设按照感知机原始形式所有的参数更新一共需要n次,对偶形式就是把这n次分摊到i个样本中去,这样最终的参数可以展开使用每个样本点进行表示,这样在判断误分类的时候的计算就都可以展开成样本之间的点乘形式,这样就可以通过提前算好的Gram矩阵来大大降低计算量,因为仅仅计算了一次,后续全部通过查表就可以了。而反观原始形式,每次参数改变,所有的矩阵计算全部需要计算,导致计算量比对偶形式要大很多。

浙公网安备 33010602011771号