Andrew Ng课程作业第二周补充（python版）

作业目的：学习正则化逻辑回归

 

作业内容：根据芯片两次测试结果判断是否合格

 

提供的数据：ex2data2.txt（mooc可下载），数据集第一列是第一次测试，第二列是第二次测试，最后一列是是否合格

处理过程：

方法一：

• step1:读取数据

path = 'd:\jupyter\ipython-notebooks-master'  + '\data\ex2data2.txt'
data2 = pd.read_csv(path, header=None, names=['Test 1', 'Test 2', 'Accepted'])
data2.head()

• step2:画散点图

positive = data2[data2['Accepted'].isin([1])]
negative = data2[data2['Accepted'].isin([0])]

fig, ax = plt.subplots(figsize=(12,8))
ax.scatter(positive['Test 1'], positive['Test 2'], s=50, c='b', marker='o', label='Accepted')
ax.scatter(negative['Test 1'], negative['Test 2'], s=50, c='r', marker='x', label='Rejected')
ax.legend()
ax.set_xlabel('Test 1 Score')
ax.set_ylabel('Test 2 Score')

• step3:根据得到的图形分析，很难用线性直线区分两类散点，因此decision boundary可以考虑多项式组成的曲线。创建多项式【还好只有两个维度的测试结果，多项式数目有限，再多，decision boundary会非常复杂，计算量也很大。】

#最高幂次是4
degree = 5
x1 = data2['Test 1']
x2 = data2['Test 2']

data2.insert(3,'Ones',1)

for i in range(1, degree):
    for j in range(0,i):
         data2['F'+str(i)+str(j)] = np.power(x1,i-j)*np.power(x2,j)

data2.drop('Test 1', axis=1, inplace=True)
data2.drop('Test 2', axis=1, inplace=True)

data2.head()

• step4:定义带正则化因子的cost function

def costReg(theta, X, y, learningRate):
    theta = np.matrix(theta)
    X = np.matrix(X)
    y = np.matrix(y)
    first = np.multiply(-y, np.log(sigmoid(X * theta.T)))
    second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T)))
    reg = (learningRate / 2 * len(X)) * np.sum(np.power(theta[:,1:theta.shape[1]], 2))
    return np.sum(first - second) / (len(X)) + reg

• step5:梯度函数

def gradientReg(theta, X, y, learningRate):
    theta = np.matrix(theta)
    X = np.matrix(X)
    y = np.matrix(y)
    
    parameters = int(theta.ravel().shape[1])
    grad = np.zeros(parameters)
    
    error = sigmoid(X * theta.T) - y
    
    for i in range(parameters):
        term = np.multiply(error, X[:,i])
        
        #theta[0]没有正则化因子
        if (i == 0):
            grad[i] = np.sum(term) / len(X)
        else:
            grad[i] = (np.sum(term) / len(X)) + ((learningRate / len(X)) * theta[:,i])
    
    return grad

• step6:定义X，y，theta

cols = data2.shape[1]
X2 = data2.iloc[:,1:cols]
y2 = data2.iloc[:,0:1]

# convert to numpy arrays and initalize the parameter array theta
X2 = np.array(X2.values)
y2 = np.array(y2.values)
theta2 = np.zeros(11)

• 初设学习因子

learningRate = 1

• step7:得到cost function初始值和梯度初始值

costReg(theta2, X2, y2, learningRate)
gradientReg(theta2, X2, y2, learningRate)

• step8:使用TNC算法得到拟合最优的theta参数值

result2 = opt.fmin_tnc(func=costReg, x0=theta2, fprime=gradientReg, args=(X2, y2, learningRate))
result2

• step9:判定准确率

theta_min = np.matrix(result2[0])
predictions = predict(theta_min, X2)
correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y2)]
accuracy = (sum(map(int, correct)) % len(correct))
print 'accuracy = {0}%'.format(accuracy)

方法二：使用scikit-learn库

• step1:代入逻辑回归模型

# set X and y (remember from above that we moved the label to column 0)
cols = data2.shape[1]
X2 = data2.iloc[:,1:cols]
y2 = data2.iloc[:,0:1]

# convert to numpy arrays and initalize the parameter array theta
X2 = np.array(X2.values)
y2 = np.array(y2.values)

from sklearn import linear_model
model = linear_model.LogisticRegression(penalty='l2', C=1.0)
model.fit(X2, y2.ravel())

• step2:判定分类的准确率

model.score(X2,y2)

这个得到准确率没有方法一高，说明scikit-learn参数需要进行调整，如用网格搜索或者随机搜索等方法。