Lecture 9_Simple Linear Regression 第九章 简单线性回归分析

9. Nonparametric Statistics9.1 Sign Test 符号检验1The simplest of all nonparametric methods is the sign test, which is usually used to test the significance of the difference between two means in a paired experiment.最简单的非参数检验是符号检验检验两个总体均值差的显著程度It is particularly suitable when the various pairs are observed under different conditions, a case in which the assumption of normality may not hold. However, because of its simplicity, the sign test is often used even though the populations are normally distributed. As is implied by its name in this test only the sign of the differencebetween the paired variates is used.若两个总体的均值相等，那么符号‘＋’、‘－’的概率一样。

D = sign of (X 1－X 2 )If p denotes the probability of a difference D being positive andq the probability of its being negative, we have as hypothesis p=1/2. appropriate test statistic is X , X~B (n, p)， X --- N(‘+”)we will reject 0Hin favor of1Honly if the proportion of plussigns is sufficiently less than 1/2, that is , when the value x of our random variable is small. Hence, if the computed P -value12()P P X x when p =≤=is less than or equal to the significance level α, we reject 0Hinfavor of1H .we reject0Hin favor1Hwhen the proportion of plus signs issignificantly less than or significantly greater than 1/2. This, of course, is equivalent to x being sufficiently small or sufficiently large, respectively. Therefore, if /2x n < and the computed P-value 122()P P X x when p =≤=is less than or equal to α, or if /2x n > and the computed P-value 122()P P X x when p =≥= is less than or equal to α, we reject 0Hin favor1H .Car Radial tires Belted tires D1 4.2 4.1 ＋ 2 4.7 4.9 －3 6.6 6.2 ＋4 7.0 6.9 ＋5 6.7 6.8 －6 4.5 4.4 ＋7 5.7 5.78 6.0 5.8 ＋9 7.4 6.9 ＋10 4.9 4.911 6.1 6.0 ＋12 5.2 4.9 ＋13 5.7 5.3 ＋14 6.9 6.5 ＋15 6.8 7.1 －16 4.9 4.8 ＋符号检验的利弊n 必须比较大因为对于n =5的样本，会出现永远不拒绝“总体均值相等“的假设。

linear regression知识点1.引言1.1 概述引言部分是文章的开头，用来介绍文章的背景和重要性。

在"概述"部分，我们可以对linear regression（线性回归）的基本概念和作用进行简单介绍。

概述：线性回归是机器学习领域中最简单且最常用的回归方法之一。

它是一种建立输入变量（自变量）和输出变量（因变量）之间线性关系的统计学模型。

线性回归可以帮助我们探索和理解数据，预测未知的因变量值，并在实际问题中做出决策。

线性回归的基本思想是基于已知的训练数据，通过拟合一条直线（或超平面）来近似描述输入和输出之间的关系。

这条直线可以用来做预测和回答各种问题。

线性回归的关键是通过最小化预测值与实际观测值之间的差距，找到最佳拟合直线。

线性回归不仅可以用于预测连续性数值型数据，还可以用于分类问题，例如将输出变量划分为两个或多个不同的类别。

尽管线性回归在实际问题中很常见，但它也有一些局限性，例如对于非线性关系的建模能力较弱。

为了克服这些局限性，研究人员还提出了各种改进方法。

本文将深入探讨线性回归的基本概念和原理，介绍线性回归模型的建立与求解过程，并探讨线性回归在实际应用中的场景和局限性，同时提出一些改进方法。

通过阅读本文，读者将能够全面了解线性回归的知识和应用，从而在实际问题中更好地应用和理解线性回归方法。

下面我们将详细介绍本文的结构和目的。

1.2 文章结构文章结构部分的内容可以描述整篇文章的组织和安排，可以按照以下内容进行阐述：在本篇文章中，我们将从引言、正文和结论三个部分来组织和阐述关于Linear Regression（线性回归）的知识点。

首先，在引言部分，我们将对线性回归进行概述，介绍其基本概念和原理。

同时，我们将阐明本篇文章的目的，即通过介绍线性回归的知识点，让读者对线性回归有一个全面的了解。

接着，在正文部分，我们将分为两个小节来详细讲解线性回归的知识点。

首先，我们将介绍线性回归的基本概念，包括线性回归的定义、特点以及模型表示等。

第9章 非线性回归9.1 在非线性回归线性化时，对因变量作变换应注意什么问题？答：在对非线性回归模型线性化时，对因变量作变换时不仅要注意回归函数的形式， 还要注意误差项的形式。

如：(1) 乘性误差项，模型形式为e y AK L αβε=， (2) 加性误差项，模型形式为y AK L αβε=+。

对乘法误差项模型（1）可通过两边取对数转化成线性模型，（2）不能线性化。

一般总是假定非线性模型误差项的形式就是能够使回归模型线性化的形式，为了方便通常省去误差项，仅考虑回归函数的形式。

9.2为了研究生产率与废料率之间的关系，记录了如表9.14所示的数据，请画出散点图，根据散点图的趋势拟合适当的回归模型。

表9.14 生产率x （单位/周） 135 000 废品率y （%） 5.2 6.5 6.8 8.1 10.2 10.3 13.0解：先画出散点图如下图：从散点图大致可以判断出x 和y 之间呈抛物线或指数曲线，由此采用二次方程式和指数函数进行曲线回归。

（1）二次曲线SPSS 输出结果如下：从上表可以得到回归方程为：72ˆ 5.8430.087 4.4710yx x -=-+⨯ 由x 的系数检验P 值大于0.05，得到x 的系数未通过显著性检验。

由x 2的系数检验P 值小于0.05，得到x 2的系数通过了显著性检验。

（2）指数曲线从上表可以得到回归方程为：0.0002t ˆ 4.003ye = 由参数检验P 值≈0<0.05，得到回归方程的参数都非常显著。

从R2值，σ的估计值和模型检验统计量F值、t值及拟合图综合考虑，指数拟合效果更好一些。

9.3 已知变量x与y的样本数据如表9.15，画出散点图，试用αeβ/x来拟合回归模型，假设：(1)乘性误差项，模型形式为y=αeβ/x eε(2)加性误差项，模型形式为y=αeβ/x+ε。

表9.15解：散点图：(1)乘性误差项，模型形式为y=αeβ/x eε线性化：lny=lnα+β/x +ε令y1=lny, a=lnα,x1=1/x .做y1与x1的线性回归，SPSS输出结果如下：从以上结果可以得到回归方程为：y1=-3.856+6.08x1F检验和t检验的P值≈0<0.05，得到回归方程及其参数都非常显著。

Simple Linear RegressionChapter TopicsTypes of Regression ModelsDetermining the Simple Linear RegressionEquationMeasures of VariationAssumptions of Regression and CorrelationResidual AnalysisMeasuring AutocorrelationInferences about the Slope© 2003 Prentice-Hall, Inc. Chap 10-2Chapter Topics(continued)Correlation -Measuring the Strength of theAssociationEstimation of Mean Values and Prediction ofIndividual ValuesPitfalls in Regression and Ethical Issues© 2003 Prentice-Hall, Inc. Chap 10-3Purpose of Regression AnalysisRegression Analysis is Used Primarily to Model Causality and Provide PredictionPredict the values of a dependent (response)variable based on values of at least oneindependent (explanatory) variableExplain the effect of the independent variables onthe dependent variablePositive Linear Relationship Negative Linear Relationship Relationship NOT Linear No RelationshipSimple Linear Regression ModelRelationship Between Variables is Describedby a Linear FunctionThe Change of One Variable Causes the OtherVariable to ChangeA Dependency of One Variable on the Other© 2003 Prentice-Hall, Inc. Chap 10-6Linear Regression Equationand are obtained by finding the values of and that minimizes the sum of the squared residualsprovides an estimate ofprovides and estimate of 0b 1b 0b 1b 0b β01b β1(continued)()2211ˆnni iii i Y Y e==−=∑∑Simple Linear Regression:ExampleYou wish to examine the linear dependency of the annual sales of produce stores on their sizes in square footage. Sample data for 7 stores were obtained. Find the equation of the straight line that fits the data best.Annual Store Square SalesFeet($1000)1 1,7263,6812 1,5423,395 32,8166,653 45,5559,543 51,2923,318 62,2085,563 71,3133,760Interpretation of Results:ExampleThe slope of 1.487 means that each increase of oneunit in X, we predict the average of Y to increase by an estimated 1.487 units.The equation estimates that for each increase of 1 square foot in the size of the store, the expected annual sales are predicted to increase by $1487.ˆ1636.415 1.487i iY X =+Simple Linear Regression inPHStatIn Excel, use PHStat| Regression | Simple Linear Regression …EXCEL Spreadsheet of Regression Sales on FootageMicrosoft ExcelWorksheetMeasures of Variation:The Sum of Squares SST=SSR+ SSETotal Sample Variability =ExplainedVariability+UnexplainedVariabilityLinear Regression Assumptions NormalityY values are normally distributed for each XProbability distribution of error is normal2.Homoscedasticity (Constant Variance)3.Independence of ErrorsResidual Analysis PurposesExamine linearityEvaluate violations of assumptionsGraphical Analysis of Residuals Plot residuals vs. X and timeResidual Analysis for HomoscedasticityHeteroscedasticity9HomoscedasticitySRXSRXYXXYDurbin-Watson Statistic inPHStatPHStat| Regression | Simple LinearRegression …Check the box for Durbin-Watson Statistic© 2003 Prentice-Hall, Inc. Chap 10-36Accept H 0(no autocorrelatin)Using the Durbin-WatsonStatistic: No autocorrelation (error terms are independent): There is autocorrelation (error terms are notindependent)0H 1H 042d L 4-d Ld U4-d U Reject H 0(positive autocorrelation)Inconclusive Reject H 0(negative autocorrelation)Relationship between a t Testand an F TestNull and Alternative HypothesesH 0: β1= 0(No Linear Dependency)H 1: β1≠ 0(Linear Dependency)()221,2n n t F −−=Purpose of Correlation AnalysisCorrelation Analysis is Used to MeasureStrength of Association (Linear Relationship)Between 2 Numerical VariablesOnly Strength of the Relationship is ConcernedNo Causal Effect is Implied© 2003 Prentice-Hall, Inc. Chap 10-47Purpose of Correlation Analysis(continued)Population Correlation Coefficient ρ(Rho) isUsed to Measure the Strength between theVariablesSample Correlation Coefficient r is anEstimate of ρ and is Used to Measure theStrength of the Linear Relationship in theSample Observations© 2003 Prentice-Hall, Inc. Chap 10-48Sample of Observations from Various r ValuesYX YXYXYX YXr= -1r= -.6r= 0r= .6r= 1Features of ρ and rUnit FreeRange between -1 and 1The Closer to -1, the Stronger the NegativeLinear RelationshipThe Closer to 1, the Stronger the PositiveLinear RelationshipThe Closer to 0, the Weaker the LinearRelationship© 2003 Prentice-Hall, Inc. Chap 10-50。

1. #!/usr/bin/python2. # -*- coding: utf-8 -*-3.4. """5. author ： duanxxnj@6. time : 2016-06-19-20-487.8. 这个是线性回归的示例代码9. 使用一条直线对一个二维数据拟合10.11. """12. print(__doc__)13.14.15. import matplotlib.pyplot as plt16. import numpy as np17. from sklearn import datasets, linear_model18.19. # 加载用于回归模型的数据集20. # 这个数据集中一共有442个样本，特征向量维度为1021. # 特征向量每个变量为实数，变化范围（-.2 ，.2）22. # 目标输出为实数，变化范围（25 ，346）23. diabetes = datasets.load_diabetes()24.25. # 查看数据集的基本信息26. print diabetes.data.shape27. print diabetes.data.dtype28. print diabetes.target.shape29. print diabetes.target.dtype30.31. # 为了便于画图显示32. # 仅仅使用一维数据作为训练用的X33. # 这里使用np.newaxis的目的是让行向量变成列向量34. # 这样diabetes_X每一项都代表一个样本35. diabetes_X = diabetes.data[:, np.newaxis,2]36.37. # 此时diabetes_X的shape是(442L, 1L)38. # 如果上面一行代码是：diabetes_X = diabetes.data[:, 2]39. # 则diabetes_X的shape是(442L，)，是一个行向量40. print diabetes_X.shape41.42. # 人工将输入数据划分为训练集和测试集43. # 前400个样本作为训练用，后20个样本作为测试用44. diabetes_X_train = diabetes_X[:-20]45. diabetes_X_test = diabetes_X[-20:]46. diabetes_y_train = diabetes.target[:-20]47. diabetes_y_test = diabetes.target[-20:]48.49. # 初始化一个线性回归模型50. regr = linear_model.LinearRegression()51.52. # 基于训练数据，对线性回归模型进行训练53. regr.fit(diabetes_X_train, diabetes_y_train)54.55. # 模型的参数56. print'模型参数：', regr.coef_57. print'模型截距：', regr.intercept_58.59. # 模型在测试集上的均方差（mean square error）60. print("测试集上的均方差: %.2f"61. % np.mean((regr.predict(diabetes_X_test)- diabetes_y_test)**2))62. # 模型在测试集上的得分，得分结果在0到1之间，数值越大，说明模型越好63. print('模型得分: %.2f'% regr.score(diabetes_X_test, diabetes_y_test))64.65. # 绘制模型在测试集上的效果66. plt.scatter(diabetes_X_test, diabetes_y_test, color='black')67. plt.plot(diabetes_X_test, regr.predict(diabetes_X_test), color='blue',68. linewidth=3)69.70. plt.grid()71. plt.show()。

各种线性回归模型原理线性回归是一种广泛应用于统计学和机器学习领域的方法，用于建立自变量和因变量之间线性关系的模型。

在这里，我将介绍一些常见的线性回归模型及其原理。

1. 简单线性回归模型（Simple Linear Regression）简单线性回归模型是最简单的线性回归模型，用来描述一个自变量和一个因变量之间的线性关系。

模型方程为：Y=α+βX+ε其中，Y是因变量，X是自变量，α是截距，β是斜率，ε是误差。

模型的目标是找到最优的α和β，使得模型的残差平方和最小。

这可以通过最小二乘法来实现，即求解最小化残差平方和的估计值。

2. 多元线性回归模型（Multiple Linear Regression）多元线性回归模型是简单线性回归模型的扩展，用来描述多个自变量和一个因变量之间的线性关系。

模型方程为：Y=α+β1X1+β2X2+...+βnXn+ε其中，Y是因变量，X1,X2,...,Xn是自变量，α是截距，β1,β2,...,βn是自变量的系数，ε是误差。

多元线性回归模型的参数估计同样可以通过最小二乘法来实现，找到使残差平方和最小的系数估计值。

3. 岭回归（Ridge Regression）岭回归是一种用于处理多重共线性问题的线性回归方法。

在多元线性回归中，如果自变量之间存在高度相关性，会导致参数估计不稳定性。

岭回归加入一个正则化项，通过调节正则化参数λ来调整模型的复杂度，从而降低模型的过拟合风险。

模型方程为：Y=α+β1X1+β2X2+...+βnXn+ε+λ∑βi^2其中，λ是正则化参数，∑βi^2是所有参数的平方和。

岭回归通过最小化残差平方和和正则化项之和来估计参数。

当λ=0时，岭回归变为多元线性回归，当λ→∞时，参数估计值将趋近于0。

4. Lasso回归（Lasso Regression）Lasso回归是另一种用于处理多重共线性问题的线性回归方法，与岭回归不同的是，Lasso回归使用L1正则化，可以使得一些参数估计为0，从而实现特征选择。

应用回归分析-第9章课后习题答案第9章 含定性变量的回归模型思考与练习参考答案9.1 一个学生使用含有季节定性自变量的回归模型，对春夏秋冬四个季节引入4个0-1型自变量，用SPSS 软件计算的结果中总是自动删除了其中的一个自变量，他为此感到困惑不解。

出现这种情况的原因是什么？答：假如这个含有季节定性自变量的回归模型为：t t t t kt k t t D D D X X Y μαααβββ++++++=332211110Λ其中含有k 个定量变量，记为x i 。

对春夏秋冬四个季节引入4个0-1型自变量，记为D i ，只取了6个观测值，其中春季与夏季取了两次，秋、冬各取到一次观测值，则样本设计矩阵为：⎪⎪⎪⎪⎪⎪⎪⎪⎭⎫⎝⎛=000110010110001010010010100011)(616515414313212111k k k k k k X X X X X X X X X X X X ΛΛΛΛΛΛD X,显然，(X,D)中的第1列可表示成后4列的线性组合，从而(X,D)不满秩，参数无法唯一求出。

这就是所谓的“虚拟变量陷井”，应避免。

当某自变量x j 对其余p-1个自变量的复判定系数2j R 超过一定界限时，SPSS 软件将拒绝这个自变量x j 进入回归模型。

称Tol j =1-2j R 为自变量x j 的容忍度（Tolerance ），SPSS 软件的默认容忍度为0.0001。

也就是说，当2j R ＞0.9999时，自变量x j 将被自动拒绝在回归方程之外，除非我们修改容忍度的默认值。

⎪⎪⎪⎪⎪⎭⎫⎝⎛=k βββM 10β⎪⎪⎪⎪⎪⎭⎫ ⎝⎛=4321ααααα而在这个模型中出现了完全共线性，所以SPSS软件计算的结果中总是自动删除了其中的一个定性自变量。

9.2对自变量中含有定性变量的问题，为什么不对同一属性分别建立回归模型，而采取设虚拟变量的方法建立回归模型？答：原因有两个，以例9.1说明。

Python 机器学习LinearRegression （线性回归模型）（附源码）LinearRegression （线性回归）1.线性回归简介线性回归定义：我个⼈的理解就是：线性回归算法就是⼀个使⽤线性函数作为模型框架（y =w ∗x +b ）、并通过优化算法对训练数据进⾏训练、最终得出最优（全局最优解或局部最优）参数的过程。

y ：我们需要预测的数值；w ：模型的参数（即我们需要通过训练调整的的值）x ：已知的特征值b ：模型的偏移量我们的⽬的是通过已知的x 和y ，通过训练找出合适的参数w 和b 来模拟x 与y 之间的关系，并最终通过x 来预测y 。

分类： 线性回归属于监督学习中的回归算法； 线性回归作为机器学习的⼊门级算法，很适合刚接触机器学习的新⼿。

虽然线性回归本⾝⽐较简单，但是⿇雀虽⼩，五脏俱全，其中涉及到的“线性模型”、“⽬标函数”、“梯度下降”、“迭代”、“评价准则”等思想与其他复杂的机器学习算法是相通的，深⼊理解线性回归后可以帮助你更加轻松的学习其他机器学习算法。

2.线性回归模型解析2.1线性回归模型⽰意图2.2模型的组成部件 2.2.1 假设函数（Hypothesis function ） h w (x )=b +w 0x 0+w 1x 1+···+w n x n 使⽤向量⽅式表⽰：X = x 0x 1⋮x n ,W =w 0w 1⋮w n 则有：h w (x )=W T X +b [][] 2.2.2 损失函数：（Cost function） 这⾥使⽤平⽅差作为模型的代价函数 J(w)=12m∑mi=1(h w(x(i))−y(i))2 2.2.3 ⽬标函数：（Goal function） minimize(J(w)) 2.2.4 优化算法：(optimization algorithm) 梯度下降法（Gradient descent） 关于梯度下降法这⾥不详细介绍；3.使⽤python实现线性回归算法1#-*- coding: utf-8 -*-2import numpy as np3from matplotlib import pyplot as plt456#⽣成训练使⽤数据;这⾥线性函数为 y = 1.5*x + 1.37def data_generate():8#随机⽣成100个数据9 x = np.random.randn(100)10 theta = 0.5 #误差系数11#为数据添加⼲扰12 y = 1.5*x + 1.3 + theta*np.random.randn(100)13return x,y1415class LinearRegression():16'''17线性回归类18参数：19 alpha:迭代步长20 n_iter:迭代次数21使⽤⽰例：22 lr = LinearRegression() #实例化类23 lr.fit(X_train,y_train) #训练模型24 y_predict = lr.predict(X_test) #预测训练数据25 lr.plotFigure()⽤于画出样本散点图与预测模型26'''27def__init__(self,alpha=0.02,n_iter=1000):28 self._alpha = alpha #步长29 self._n_iter = n_iter #最⼤迭代次数3031#初始化模型参数32def initialPara(self):33#初始化w，b均为034return 0,03536#训练模型37def fit(self,X_train,y_train):38#保存原始数据39 self.X_source = X_train.copy()40 self.y_source = y_train.copy()4142#获取训练样本个数43 sample_num = X_train.shape[0]44# 初始化w，w045 self._w, self._b = self.initialPara()4647#创建列表存放每次每次迭代后的损失值48 self.cost = []4950#开始训练迭代51for _ in range(self._n_iter):52 y_predict = self.predict(X_train)53 y_bias = y_train - y_predict54 self.cost.append(np.dot(y_bias,y_bias)/(2 * sample_num))55 self._w += self._alpha * np.dot(X_train.T,y_bias)/sample_num56 self._b += self._alpha * np.sum(y_bias)/sample_num5758def predict(self,X_test):59return self._w * X_test + self._b6061#画出样本散点图以及使⽤模型预测的线条62def plotFigure(self):63#样本散点图64 plt.scatter(self.X_source,self.y_source,c='r',label="samples",linewidths=0.4) 6566#模型预测图67 x1_min = self.X_source.min()68 x1_max = self.X_source.max()69 X_predict = np.arange(x1_min,x1_max,step=0.01)70 plt.legend(loc='upper left')7172 plt.plot(X_predict,self._w*X_predict+self._b)73 plt.show()7475if__name__ == '__main__':76#创建训练数据77 x_data,y_data = data_generate()7879#使⽤线性回归类⽣成模型80 lr = LinearRegression()81 lr.fit(x_data,y_data)8283#打印出参数84print(lr._w,lr._b)85#画出损失值随迭代次数的变化图86 plt.plot(lr.cost)87 plt.show()88#画出样本散点图以及模型的预测图89 lr.plotFigure()9091#预测x92 x = np.array([3])93print("The input x is{0},then the predict of y is:{1}".format(x,lr.predict(x)))线性回归代码更多线性回归的代码参考github：Processing math: 100%。

Simple Linear Regression 笔记Simple Linear RegressionCreating the Regression LineCalculating b1 & b0, creating the lineand testing its significance with a t-test.DEFINITIONS:b1 - This is the SLOPE of the regression line. Thus this is the amount that the Y variable (dependent) will change for each 1 unit change in the X variable.b0 - This is the intercept of the regression line with the y-axis. In otherwords it is the value of Y if the value of X = 0.Y-hat = b0 + b1(x) - This is the sample regression line. You must calculate b0 & b1 to create this line. Y-hat stands for the predicted value of Y, and it can be obtained by plugging an individual value of x into the equation and calculating y-hat. EXAMPLE:A firm wants to see if there is sales is explained by the number of hours overtime that their salespeople work. Using a spreadsheet containing 25 months of sales & overtime figures, the following calculations are made; SSx = 85, SSy = 997 and SSxy = 2,765, X-bar = 13 and Y-bar = 67,987, also s(b1) = 21.87. Create the regression line.(1) find b1 - One method of caluating b1 is b1 = SSxy/SSx = 2765/85 = 32.53. This is the slope of the line - for every unit change in X, y will increase by 32.53. It is a positive number, thus its a direct relationship - as X goes up, so does Y. However, if b1 = -32.53, then we would know the relationship between X & Y is an inverse relationship - as X goes up, y goes down)(2) find b0 - again the formula is on pg. 420 and is b0 = Y-bar - b1(x-bar) = 67,987 - 32.53(13) = 67,987 - 422.89 = 67,564, this is the intercept of the line and the Y-axis, and can be interpreted as the value of Y if zero hours of overtime (x=0) are worked.(3) Create Line - Y-hat = b0 + b1(x) or Y-hat = 67,564 + 32.53(x), This line quantifies the relationship between X & Y. Under the normal error model, b1 is unbiased for Beta1 with:Sy/x is the residual standard deviation:But is this Relationship "Significant"Since it is based on a sample and we wish to generalize to a population, it must be tested to see if it is "significant," meaning would the relationship we found actually exist in the population or is the result due to sampling error (our sample did not represent the true population). The specific test we use is a t-test to test to see if b1 is different from 0. Since B1 would be the slope of the regression line in the population, it makes sense to test to see if it is different from zero. If it is zero, then our slope is 0, meaning if we graphed the relationship between X & Y we would end up with a horizontal (flat) line. And if this line is flat then we know that no matter what value the X variable takes on, the Y variable's value will not change. This means there is no linear relationship between the two variables. This also means that the regression line we calculated is useless for explaining or predicting the dependent variable.TESTING B1 We use our standard five step hypothesis testing procedure.Hypotheses: H0: B1 = 0, H1: B1 not = 0Critical value: a t-value based on n-2 degrees of freedom. Also divide alpha by 2 because it is a 2-tailed test. In this case n = 25 (25 months data used) thus n-2 = 23. With alpha = .05 we have alpha/2 = .025 and then t = 2.069 (from t-table inside front cover of book).Calculated Value: The formula is on page 442 and is simply t = b1/s(b1) = 32.53/21.87 = 1.49. s(b1) is the standard error of b1 and is given in the problem)Compare: t-calc < t-crit and thus accept H0.Conclusion: B1 = 0, the population slope of our regression is a flat line, thus there is no linear relationship between sales and overtime worked, and the sample regression line we calculated is not useful for explaining or predicting sales dollars from overtime worked.CorrelationCorrelation is a measure of the degree of linear association between two variables. The value of a correlation can range from -1, thru 0, to +1. A correlation = 0 means there is no LINEAR association between the two variables, a value of -1 or +1 means there is a perfect linear association between the two variables, the difference being that -1 indicates a perfect inverse relationship and +1 a perfect positive relationship. The sample notation for a correlation is "r" while the population correlation coefficient is represented by the greek letter "Rho" (which looks like a small "p").We often want to find out if a calculated sample correlation would be "significant." Again this would mean we would test to see if Rho = 0 or not. If Rho=0 then there would be no linear relationship between the two variables in the population.AN EXAMPLE:Based on a sample of 42 days, the correlation between sales and number of sunny hours in the day is calculated for the Sunglass Hut store in Meridian Mall. The r = .56. Is this a "significant" correlation?This is a basic hypothesis test.....Hypotheses: H0: Rho = 0, H1: Rho not = 0.Critical Value: The t-test for the significance of Rho has n-2 degrees of freedom, and alpha will need to be divided by 2, thus n-2 = 40 and alpha (.05/2) = .025 ... from the table we find: 2.021.Calculated Value: The formula on page 438 is t = r / sqr root of (1-r-sqrd)/(n-2). In this case that equals .56 / the square root of (1-.56-squared)/(40) = .56/.131 = 4.27Compare: The t-calc is larger than the t-crit thus we REJECT Ho.Conclusion: Rho does not equal zero and thus there is evidence of a linear association between the two variables in the population.The F-test in RegressionEXAMPLEUsing the information given, construct the ANOVA table and determine whether there is a regression relationship between years of car ownership (Y) and salary (X). n= 47, SSR = 458 and SSE = 1281.ANOVA Table: The anova table is on page 451, and is basically the same as a one-way ANOVA table. The first thing we need is the df, and by definition the df for the regression = 1, the df for the error = n-2 or 45, and the total df = n-1 or 46. Next we need the MS calculations. MSR = SSR/df for the regression = SSR/1 = SSR or 458. MSE = SSE/n-2 = 1281/45 = 28.47. Finally, the F-calc = MSR/MSE or 458/28.47 = 16.09.Hypotheses: H0: There is no regression relationship,i.e, B1 =0. H1: There is a regression relationship, i.e, B1 is not = 0. Critical Value: F(num. df, den. df) = F(1, 45) at alpha = .05 = 4.08Calculated Value: from above ANOVA table = 16.09Compare: F-calc larger than F-crit thus REJECTConclusion: There is a regression (linear) relationship between years of car ownership and salary.The Coefficient of Determination - r-sqrdWe can also test the significance of the regression coefficient using an F-test. Since we only have one coefficient in simple linear regression, this test is analagous to the t-test. However, when we proceed to multiple regression, the F-test will be atest of ALL of the regression coefficients jointly being 0. (Note: b0 is not a coefficient and we generally do not test its significance although we could do so with a t-test just as we did ofr b1.r-sqrd is always a number between 0 and 1. The closer it is to 1.0 the better the X-Y relationship predicts or explains the variance in Y. Unfortunately there are no set values that allow you to say that is a "good" r-sqrd or "bad" r-sqrd. Such a determination is subjective and is determined by the research you are conducting. If nobody has ever explained more that 15% of the variance in some Y variable before, and you design a study that explains 25% of variance, then this might be considered good r-sqrd, even though the actual number, 25%, is not very high.EXAMPLE:What is the r-sqrd if SSR = 345 and SSE = 123?r-sqrd = SSR/SST. We don't have SST, but we know that SSR + SSE = SST, thus SST = 345 + 123 = 468, thus r-sqrd = 345/468 = .737. This means that the regression relationship between X & Y explains 73.7% of the variance in the Y variable. Under most circumstances this would be a high amount, but again we would have to know more about our research varaibles.。

线性回归LinearRegression 成本函数（cost function）也叫损失函数（loss function），⽤来定义模型与观测值的误差。

模型预测的价格与训练集数据的差异称为残差（residuals）或训练误差（test errors）。

我们可以通过残差之和最⼩化实现最佳拟合，也就是说模型预测的值与训练集的数据最接近就是最佳拟合。

对模型的拟合度进⾏评估的函数称为残差平⽅和（residual sum of squares）成本函数。

就是让所有训练数据与模型的残差的平⽅之和最⼩。

我们⽤R⽅（r-squared）评估预测的效果。

R⽅也叫确定系数（coefficient of determination），表⽰模型对现实数据拟合的程度。

计算R⽅的⽅法有⼏种。

⼀元线性回归中R⽅等于⽪尔逊积矩相关系数（Pearson product moment correlation coefficient 或Pearson's r）的平⽅。

这种⽅法计算的R⽅⼀定介于0～1之间的正数。

其他计算⽅法，包括scikit-learn中的⽅法，不是⽤⽪尔逊积矩相关系数的平⽅计算的，因此当模型拟合效果很差的时候R⽅会是负值。

SStot是⽅差平⽅和 SSres是残差的平⽅和⼀元线性回归X_test = [[8], [9], [11], [16], [12]]y_test = [[11], [8.5], [15], [18], [11]]model = LinearRegression()model.fit(X, y)model.score(X_test, y_test）score⽅法计算R⽅多元线性回归最⼩⼆乘的代码from numpy.linalg import lstsqprint(lstsq(X, y)[0])多项式回归⼀种特殊的多元线性回归⽅法，增加了指数项（x 的次数⼤于1）。

现实世界中的曲线关系都是通过增加多项式实现的，其实现⽅式和多元线性回归类似。