西⽠书课后习题——第三章3.1式3.2 f(x)=ωT x+b中，ωT和b有各⾃的意义，简单来说，ωT决定学习得到模型(直线、平⾯)的⽅向，⽽b则决定截距，当学习得到的模型恰好经过原点时，可以不考虑偏置项b。
偏置项b实质上就是体现拟合模型整体上的浮动，可以看做是其它变量留下的偏差的线性修正，因此⼀般情况下是需要考虑偏置项的。
但如果对数据集进⾏了归⼀化处理，即对⽬标变量减去均值向量，此时就不需要考虑偏置项了。
3.2对区间[a,b]上定义的函数f(x)，若它对区间中任意两点x1，x2均有f(x1+x22)≤f(x1)+f(x2)2，则称f(x)为区间[a,b]上的凸函数。
对于实数集上的函数，可通过⼆阶导数来判断：若⼆阶导数在区间上⾮负，则称为凸函数，在区间上恒⼤于零，则称为严格凸函数。
对于式3.18 y=11+e−(ωT x+b)，有dydωT=1(1+e−(ωT x+b))2e−(ωT x+b)(−x)=(−x)11+e−(ωT x+b)(1−11+e−(ωT x+b))=xy(y−1)=x(y2−y)d dωT(dydωT)=x(2y−1)(dydωT)=x2y(2y−1)(y−1)其中，y的取值范围是(0,1)，不难看出⼆阶导有正有负，所以该函数⾮凸。
3.3对率回归即Logis regression西⽠集数据如图所⽰：将好⽠这⼀列变量⽤0/1变量代替，进⾏对率回归学习，python代码如下：import numpy as npimport matplotlib.pyplot as pltimport pandas as pdfrom sklearn import model_selectionfrom sklearn.linear_model import LogisticRegressionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/watermelon3a.csv')#数据预处理X = dataset[['密度','含糖率']]Y = dataset['好⽠']good_melon = dataset[dataset['好⽠'] == 1]bad_melon = dataset[dataset['好⽠'] == 0]#画图f1 = plt.figure(1)plt.title('watermelon_3a')plt.xlabel('density')plt.ylabel('radio_sugar')plt.xlim(0,1)plt.ylim(0,1)plt.scatter(bad_melon['密度'],bad_melon['含糖率'],marker='o',color='r',s=100,label='bad')plt.scatter(good_melon['密度'],good_melon['含糖率'],marker='o',color='g',s=100,label='good')plt.legend(loc='upper right')#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0) #训练log_model = LogisticRegression()log_model.fit(X_train,Y_train)#验证Y_pred = log_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred, target_names=['Bad','Good']))print(log_model.coef_)theta1, theta2 = log_model.coef_[0][0], log_model.coef_[0][1]X_pred = np.linspace(0,1,100)line_pred = theta1 + theta2 * X_predplt.plot(X_pred, line_pred)plt.show()View Codeimport numpy as npimport matplotlib.pyplot as pltimport pandas as pdfrom sklearn import model_selectionfrom sklearn.linear_model import LogisticRegressionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/watermelon3a.csv')#数据预处理X = dataset[['密度','含糖率']]Y = dataset['好⽠']good_melon = dataset[dataset['好⽠'] == 1]bad_melon = dataset[dataset['好⽠'] == 0]#画图f1 = plt.figure(1)plt.title('watermelon_3a')plt.xlabel('density')plt.ylabel('radio_sugar')plt.xlim(0,1)plt.ylim(0,1)plt.scatter(bad_melon['密度'],bad_melon['含糖率'],marker='o',color='r',s=100,label='bad')plt.scatter(good_melon['密度'],good_melon['含糖率'],marker='o',color='g',s=100,label='good')plt.legend(loc='upper right')#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0) #训练log_model = LogisticRegression()log_model.fit(X_train,Y_train)#验证Y_pred = log_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred))print(log_model.coef_)theta1, theta2 = log_model.coef_[0][0], log_model.coef_[0][1]X_pred = np.linspace(0,1,100)line_pred = theta1 + theta2 * X_predplt.plot(X_pred, line_pred)plt.show()View Code模型效果输出(查准率、查全率、预测效果评分)：precision recall f1-score supportBad 0.75 0.60 0.67 5Good 0.60 0.75 0.67 4micro avg 0.67 0.67 0.67 9macro avg 0.68 0.68 0.67 9weighted avg 0.68 0.67 0.67 9也可以输出验证集的实际结果和预测结果：密度含糖率 Y_test Y_pred1 0.774 0.376 1 16 0.481 0.149 1 08 0.666 0.091 0 09 0.243 0.267 0 113 0.657 0.198 0 04 0.556 0.215 1 12 0.634 0.264 1 114 0.360 0.370 0 110 0.245 0.057 0 03.4⾸先附上使⽤葡萄酒品质数据做的对率回归学习代码import numpy as npimport matplotlib.pyplot as pltimport pandas as pdpd.set_option('display.max_rows',None)pd.set_option('max_colwidth',200)pd.set_option('expand_frame_repr', False)from sklearn import model_selectionfrom sklearn.linear_model import LogisticRegressionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/winequality-red_new.csv')#数据预处理dataset['quality2'] = dataset['quality'].apply(lambda x: 0 if x < 5 else 1) #新加⼊⼆分类变量是否为好酒，基于原数据中quality的值，其⼤于等于5就定义为好酒，反之坏酒X = dataset[["fixed_acidity","volatile_acidity","citric_acid","residual_sugar","chlorides","free_sulfur_dioxide","total_sulfur_dioxide","density","pH","sulphates","alcohol"]]Y = dataset["quality2"]#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0)#训练log_model = LogisticRegression()log_model.fit(X_train,Y_train)#验证Y_pred = log_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred))print(log_model.coef_)View Code其中，从UCI下载的数据集格式有问题，⽆法直接使⽤，先编写程序将格式调整完毕再使⽤数据fr = open('/home/zwt/Desktop/winequality-red.csv','r',encoding='utf-8')fw = open('/home/zwt/Desktop/winequality-red_new.csv','w',encoding='utf-8')f = fr.readlines()for line in f:line = line.replace(';',',')fw.write(line)fr.close()fw.close()View Code两种⽅法的错误率⽐较from sklearn.linear_model import LogisticRegressionfrom sklearn import model_selectionfrom sklearn.datasets import load_wine# 载⼊wine数据dataset = load_wine()#10次10折交叉验证法⽣成训练集和测试集def tenfolds():k = 0truth = 0while k < 10:kf = model_selection.KFold(n_splits=10, random_state=None, shuffle=True)for x_train_index, x_test_index in kf.split(dataset.data):x_train = dataset.data[x_train_index]y_train = dataset.target[x_train_index]x_test = dataset.data[x_test_index]y_test = dataset.target[x_test_index]# ⽤对率回归进⾏训练，拟合数据log_model = LogisticRegression()log_model.fit(x_train, y_train)# ⽤训练好的模型预测y_pred = log_model.predict(x_test)for i in range(len(x_test)): #这⾥和留⼀法不同，是因为10折交叉验证的验证集是len(dataset.target)/10，验证集的预测集也是，都是⼀个列表，是⼀串数字，⽽留⼀法是⼀个数字if y_pred[i] == y_test[i]:truth += 1k += 1# 计算精度accuracy = truth/(len(x_train)+len(x_test)) #accuracy = truth/len(dataset.target)print("⽤10次10折交叉验证对率回归的精度是：", accuracy)tenfolds()#留⼀法def leaveone():loo = model_selection.LeaveOneOut()i = 0true = 0while i < len(dataset.target):for x_train_index, x_test_index in loo.split(dataset.data):x_train = dataset.data[x_train_index]y_train = dataset.target[x_train_index]x_test = dataset.data[x_test_index]y_test = dataset.target[x_test_index]# ⽤对率回归进⾏训练，拟合数据log_model = LogisticRegression()log_model.fit(x_train, y_train)# ⽤训练好的模型预测y_pred = log_model.predict(x_test)if y_pred == y_test:true += 1i += 1# 计算精度accuracy = true / len(dataset.target)print("⽤留⼀法验证对率回归的精度是：", accuracy)leaveone()View Code3.5import numpy as npimport pandas as pdimport matplotlib.pyplot as pltfrom sklearn.discriminant_analysis import LinearDiscriminantAnalysisfrom sklearn import model_selectionfrom sklearn import metricsdataset = pd.read_csv('/home/zwt/Desktop/watermelon3a.csv')#数据预处理X = dataset[['密度','含糖率']]Y = dataset['好⽠']#分割训练集和验证集X_train,X_test,Y_train,Y_test = model_selection.train_test_split(X,Y,test_size=0.5,random_state=0)#训练LDA_model = LinearDiscriminantAnalysis()LDA_model.fit(X_train,Y_train)#验证Y_pred = LDA_model.predict(X_test)#汇总print(metrics.confusion_matrix(Y_test, Y_pred))print(metrics.classification_report(Y_test, Y_pred, target_names=['Bad','Good']))print(LDA_model.coef_)#画图good_melon = dataset[dataset['好⽠'] == 1]bad_melon = dataset[dataset['好⽠'] == 0]plt.scatter(bad_melon['密度'],bad_melon['含糖率'],marker='o',color='r',s=100,label='bad')plt.scatter(good_melon['密度'],good_melon['含糖率'],marker='o',color='g',s=100,label='good')View Codeimport numpy as npimport matplotlib.pyplot as pltdata = [[0.697, 0.460, 1],[0.774, 0.376, 1],[0.634, 0.264, 1],[0.608, 0.318, 1],[0.556, 0.215, 1],[0.403, 0.237, 1],[0.481, 0.149, 1],[0.437, 0.211, 1],[0.666, 0.091, 0],[0.243, 0.267, 0],[0.245, 0.057, 0],[0.343, 0.099, 0],[0.639, 0.161, 0],[0.657, 0.198, 0],[0.360, 0.370, 0],[0.593, 0.042, 0],[0.719, 0.103, 0]]#数据集按⽠好坏分类data = np.array([i[:-1] for i in data])X0 = np.array(data[:8])X1 = np.array(data[8:])#求正反例均值miu0 = np.mean(X0, axis=0).reshape((-1, 1))miu1 = np.mean(X1, axis=0).reshape((-1, 1))#求协⽅差cov0 = np.cov(X0, rowvar=False)cov1 = np.cov(X1, rowvar=False)#求出wS_w = np.mat(cov0 + cov1)Omiga = S_w.I * (miu0 - miu1)#画出点、直线plt.scatter(X0[:, 0], X0[:, 1], c='b', label='+', marker = '+')plt.scatter(X1[:, 0], X1[:, 1], c='r', label='-', marker = '_')plt.plot([0, 1], [0, -Omiga[0] / Omiga[1]], label='y')plt.xlabel('密度', fontproperties='SimHei', fontsize=15, color='green');plt.ylabel('含糖率', fontproperties='SimHei', fontsize=15, color='green');plt.title(r'LinearDiscriminantAnalysis', fontproperties='SimHei', fontsize=25);plt.legend()plt.show()View Code3.6对于⾮线性可分的数据，要想使⽤判别分析，⼀般思想是将其映射到更⾼维的空间上，使它在⾼维空间上线性可分进⼀步使⽤判别分析。
