SPSS数据统计分析与实践主讲：周涛副教授北京师范大学资源学院2007-12-18教学网站：/Courses/SPSS第十四章：非线性回归Contents:1. 非线性回归概述2. SPSS实例3. 常用的非线性模型SPSS procedures for Regression1.The Nonlinear Regression procedure allows you tocreate powerful and flexible models fornonlinear relationships between a dependentvariable and one or more independent variables. 2.The Linear Regression procedure provides morestatistics for models that are intrinsically linear.3.The Curve Estimation procedure allows you tomore easily specify certain nonlinear models,and can be useful for quickly comparing severaldifferent types of models.Linear vs. Nonlinear models . Regression models, whether linear or nonlinear, assume that the form of the model is Y=F(X,B) +error , where Y is the dependent variable, X represents the predictors, and F is a function of X. In linear models, F is of the form:Where x j is the jth predictor, and b j is the jth regressioncoefficient. Note that for a model to be considered linear, F must be a linear function of the parameters, notnecessarily the predictors . Thus, y=bx 2+ error is a linear model. Additionally, some models in which the error is multiplicative, such as y=e bx error , are linear models under the log-transformation: ln(y) = bx + ln(error).These model are known as intrinsically linear. Nonlinear models are all other forms of F.∑==pj jj X b B X F 1),(Parameters estimation in Nonlinear Regressionz A difference from linear regression is that the solution of the normal equations usually requires an iterative numerical search procedure because analytical solutions generally cannot be found.z To make things still more difficult, multiple solutions may be possible.Basic Ideas for parameter estimationExamples for Search methodsMethods of parameter estimation (1)z解析解（Analytic solution ）z 梯度下降算法（Gradient descent algorithms)z Steepest-descentz quasi-Newtonz Levenberg-Marquardt剃度下降法的优点：速度快算法相对简单缺点：通常只能找到“Local minimum”需要提供“Gradient vector”ky y J J ∂∂=∇/Methods of parameter estimation (2) z解析解（Analytic solution）z梯度下降算法（Gradient descent algorithms)z全参数空间搜索算法（Global search methods）z优点：能搜索到全局最优参数（Global minimum）很多算法不需要提供“Gradient vector”z缺点：速度慢，需要消耗较大的计算时间z代表性算法:模拟退火（Simulated annealing）遗传算法（Genetic Algorithms）马尔可夫链蒙特卡洛法（Markov chain Monte Carlo）ExampleSPSS解决方案1.根据散点图或经验确定模型2.根据经验给出初始值和参数空间（非常重要）Examplez A retailer wants to examine the relationship between money spent on advertising and the resulting sales. To this end, they have collected past sales figures and the associated advertising costs.z This data file was previously analyzed using Linear and Quadratic models via the Curve Estimation procedure, and the the Quadratic model was found to be superior to the Linear model for this situation. However, the retailer is concerned that the Quadratic model may not beappropriate because it suggests that increasedadvertising will eventually decrease sales. Use Nonlinear Regression to fit an appropriate model.Step 1: Scatter plotThe resulting scatterplotshows that salesincrease with increasedadvertising; however,the sales return onadvertising investmentappears to decreasewith increased spending,until increasedadvertising has nofurther effect on sales.An appropriate model forthis kind of pattern is theasymptotic(]渐近线的)regression model.Step 2: Choosing ModelThe asymptotic regression model (渐近回归模型) has form:Xb eb b Y 321+=When b1>0, b2<0, and b3<0, it gives Mistcherlich's model of the "law of diminishing returns ". This model initially increases quickly with increasing values of x, but then the gains slow and finally taper off just below the value b1.6065707580859095100105246810Y =100-30*EXP(-0.5*X)Step 3: Choosing starting valuesz The Nonlinear Regression procedure requires that you supply starting values for the parameters in the model. This seems adaunting（使人畏缩的）task at first, but becomes easier with some familiarity with the model.z b1represents the upper asymptote for sales. Looking at the chart, even the largest sales values fall justs short of13, so that's areasonable starting value.z b2is the difference between the value of y when x=0 and the upper asymptote. A reasonable starting value is the minimum value of y minus b1. Looking at the chart, say that's about7-13= -6.z b3 can be roughly initially estimated by the negative of the slope between two "well separated" points on the plot. Looking at the chart there are a few points about x=2, y=8, and about x=5, y=12. Theslope between these points is (12-8)/(5-2)=1.33, thus a rough initial estimate for b3 is -1.33.Step 4: Running Nonlinear Regression(1) Define model1.Analyze ÆRegression ÆNonlinear...2.Select Detrended sales as the dependentvariable.3.Type b1 + b2*exp(b3*advert)as the modelexpression.Step 4: Running Nonlinear Regression(1) Define modelStep 4: Running Nonlinear Regression(2) Input initial values of parameters 4.Click Parameters....z Type b1as the parameter name.z Type 13as the starting valuez Click Add.z Type b2as the parameter name.z Type -6as the starting value.z Click Add.z Type b3as the parameter name.z Type -1.33as the starting value.z Click Add.Step 4: Running Nonlinear Regression(2) Input initial values of parametersStep 4: Running Nonlinear Regression(3) Constrains5.Click Constraints in the Nonlinear Regressiondialog box.z Select Define parameter constraint.z Select b1as the parameter to be constrained.z Select >= from the dropdown list.z Type 0 as the constraintz Click Addz Select b2as the parameter to be constrained.z Select <= from the dropdown listz Type0as the constraint.z Click Addz Select b3as the parameter to be constrained.z Select <= from the dropdown listz Type 0as the constraint.z Click Add(3) Constrains(3) ConstrainsClick OK in the warning. The sequential quadratic programming algorithm（顺序二次规划）will be used instead.(4) Save variables6.Click Save in the Nonlinear Regression dialog box.•Select Predicted values and Residuals.Step 5: Output and InterpretingThe parameter estimates table summarizes the model-estimated value of each parameter. Parameters in a nonlinear regression model usually do not have the same interpretation as linear regression coefficients, and often vary from model to model .Parameter Estimates12.904.61011.63614.173-11.268 1.581-14.556-7.979-.496.138-.782-.209Parameterb1b2b3Estimate Std. ErrorLower Bound Upper Bound95% Confidence IntervalStep 5: Output and InterpretingAs previously discussed, b1 represents the maximum possible sales , even if infinite advertising money were available. Its small standard error with respect to the value of the estimate suggests that you can be confident in the estimate.Parameter Estimates12.904.61011.63614.173-11.268 1.581-14.556-7.979-.496.138-.782-.209Parameterb1b2b3Estimate Std. ErrorLower Bound Upper Bound95% Confidence IntervalStep 5: Output and Interpretingb2 is the difference between maximum possible sales and sales when no advertising money is spent . Its standard error is large and confidence interval is wide compared to the value of the estimate, so there is some uncertainty here.Parameter Estimates12.904.61011.63614.173-11.268 1.581-14.556-7.979-.496.138-.782-.209Parameterb1b2b3Estimate Std. ErrorLower Bound Upper Bound95% Confidence IntervalStep 5: Output and Interpretingb3 controls the rate at which the maximum is reached , the so-called "rate constant ". Like b2, there is some uncertainty in the estimate.Parameter Estimates12.904.61011.63614.173-11.268 1.581-14.556-7.979-.496.138-.782-.209Parameterb1b2b3Estimate Std. ErrorLower Bound Upper Bound95% Confidence IntervalStep 5: Output and InterpretingThe ANOVA table provides a breakdown of the sum of squares , a measure of variability in the dependent variable, for this model.ANOVA a2748.5193916.1736.77821.3232755.2972474.52023Source Regression ResidualUncorrected Total Corrected TotalSum of Squares dfMean Squares Dependent variable: Detrended salesR squared = 1 - (Residual Sum of Squares) /(Corrected Sum of Squares) = .909.a.Step 5: Output and InterpretingThe Uncorrected Total represents the entire variability in the dependent variable, while the Corrected Total is adjusted to only reflect variability about "average" sales.ANOVA a2748.5193916.1736.77821.3232755.2972474.52023Source Regression ResidualUncorrected Total Corrected TotalSum of Squares dfMean Squares Dependent variable: Detrended salesR squared = 1 - (Residual Sum of Squares) /(Corrected Sum of Squares) = .909.a.Step 5: Output and InterpretingThe Residual sum of squares and Corrected Total are used to compute r2. An r 2value of 0.909 means that the model accounts for about 90.9% of the variability in the dependent variableANOVA a2748.5193916.1736.77821.3232755.2972474.52023Source Regression ResidualUncorrected Total Corrected TotalSum of Squares dfMean Squares Dependent variable: Detrended salesR squared = 1 - (Residual Sum of Squares) /(Corrected Sum of Squares) = .909.a. CommentComments:Some properties that exist for linear regression least squares do not hold for nonlinear regression least squares.z The residuals do not necessarily sum to zero for nonlinear least squares.z Additionally, the error sum of squares SSE and the regression sum of squares SSR do not necessarily sum to the total sum of squares SSTO.z Consequently, the coefficient of multiple determination R2=SSR/SSTO is not a meaningful descriptive statistic fornonlinear regression.Step 6: Scatter Plot of residualsz To produce a scatterplot of residuals by fit values for the Nonlinear model, from the menuschoose:GraphsÆScatter/Dot...z Select Residuals as the y variable and Predicted Values as the x variable.Step 6: Scatter Plot of residuals These residuals do not show a pattern, thus the Asymptotic model is acceptable in the sense the residuals are independent of the fit values.Example 2An internet service provider (ISP) is determining the effects of a virus on its networks. As part of this effort, they have tracked the (approximate) percentage of infected e-mail traffic on its networks over time, from the moment of discovery until the threat was contained.Use Nonlinear Regression to model the rise and decline of the infection.Scatter Plotz To produce a scatterplot of infected e-mails by time, from the menus choose:z Graphs ÆScatter/Dot...Scatter Plot•The resulting scatterplotshows a rise, leveling out,and eventual decline in theproportion of infected e-mailsover time. The shape of theplot is such that it is unlikelythat a single nonlinearequation will both provide agood fit and allow sufficientinterpretability.Closerexamination suggests that asegmented model couldperform quite well here.•The initial curve in the plothas an S-shape--there is aninitial bend before the rapidrise, followed by anotherbend as it levels off. A classicgrowth curve, the logisticequation, can be used tomodel this shape.Scatter PlotAt approximately hour 20, theproportion of infected e-mailsdrops precipitously with eachpassing hour and the rate atwhich the proportion dropsappears to decrease withtime, until the virus threat isessentially eliminated. Anappropriate model for thiskind of pattern is theasymptotic regressionmodel (渐近线回归模型).A segmented model thatuses a logistic equation forthe first 19 hours and anasymptotic regression forthe remaining hours shouldprovide a good fit andinterpretability over the entiretime period.Choosing starting values for the logistic modelThe logistic model has form:Generally, b1>0, b2>0, and b3>0. This model has an "S" shaped curve .z b1represents the upper asymptote for viral growth. Looking at the chart, even the largest values fall short of 0.65, so that's a reasonable starting value.z b2is the ratio between the value of y when x=0 and the upper asymptote . A reasonable starting value is the ratio of b1 to the minimum value of y. Looking at the chart, say that's about 0.65/0.13=5.z b3can be roughly initially estimated by the slope between two "well separated" points on the plot. Looking at the chart there are a few points about x=3, y=0.12, and about x=19, y=0.60. The slope between these points is (0.60-0.12)/(19-3)=0.03, thus a rough initial estimate for b3 is 0.03.Xb eb b Y 3211−+=Choosing starting values for the asymptotic regression modelThe asymptotic regression model has form:When a1>0, a2>0, and a3<0, this model initially decreases quickly with increasing values of x, but then it slows and finally tapers off just above the value a1.z a1represents the lower asymptote for the proportion of infected e-mails. The lowest value this can be is 0, so that's a reasonable starting value.z a2is the difference between the value of y when x=20and the lower asymptote . A reasonable starting value is the maximum value of y minus a1. Looking at the chart, say that's about 0.6-0.0=0.6.z a3 can be roughly initially estimated by the slope between two "well separated" points on the plot. Looking at the chart there are points about x=20, y=0.6, and about x=40, y=0.1. The slope between these points is (0.6-0.1)/(20-40)=-0.025, thus a rough initial estimate for a3 is -0.025.Xa ea a Y 321+=Running Nonlinear Regressionz Analyze ÆRegression ÆNonlinear...z Select Proportion of infected messages as the dependent variable.z Type (time<20)*b1/(1 + b2*exp(-b3*time)) + (time>=20)*(a1 + a2*exp(a3*(time-19)))as the model expression.z Note (time<20), (time>=20) terms;z note (time-19) term.Running Nonlinear RegressionRunning Nonlinear Regression z Setting initial values:z b1 = 0.65z b2 = 5z b3 = 0.03z a1 = 0z a2 = 0.6z a3 = -0.025Running Nonlinear Regression z Setting Constraintsz b1 >= 0z b2 >= 0z b3 >= 0z a1 >= 0z a2 >= 0z a3 <= 0Running Nonlinear Regression z Setting SaveOutputsParameter Estimates.734.127.477.9917.428 1.375 4.63810.217.184.040.103.265.091.030.030.153.661.044.572.750-.150.027-.205-.095Parameterb1b2b3a1a2a3EstimateStd. ErrorLower Bound Upper Bound95% Confidence Interval The parameter estimates table summarizes the model-estimated value of each parameter. The standard errors of the logistic model's parameter estimates are considerably larger than those of the asymptotic regression model, relative to the values of the estimates. This is due in part to the fewerobservations available to fit the logistic portion of the model;the rest is likely due to greater variation in the data during the first 20 hours.OutputsThe ANOVA table provides a breakdown of the sum ofsquares, a measure of variability in the dependent variable, for this model.The Residual sum of squares and Corrected Total are used to compute r 2. An r 2value of 0.933means that the model accounts for about 93.3% of the variability in the dependent variable.ANOVA a4.8846.814.08236.0024.966421.21241Source Regression ResidualUncorrected Total Corrected TotalSum of SquaresdfMean SquaresDependent variable: Proportion of infected messages R squared = 1 - (Residual Sum of Squares) /(Corrected Sum of Squares) = .933.a.Scatter PlotThese residualsdo not show apattern, thus themodel isacceptable inthe sense theresiduals areindependent ofthe fit values常用非线性模型: 2D Model1. PolynomialModel Group Descriptiona*x+b Polynomial First Order Polynomiala*x^2+b*x+c Polynomial Second Order Polynomial a*x^3+b*x^2+c*x+d Polynomial Third Order Polynomiala*x^4+b*x^3+...+e Polynomial Fourth Order Polynomial a*x^5+b*x^4+...+f Polynomial Fifth Order Polynomiala*x^6+b*x^5+...+g Polynomial Sixth Order Polynomiala*x^7+b*x^6+...+h Polynomial Seventh Order Polynomial a*x^8+b*x^7+...+i Polynomial Eighth Order Polynomial a*x^9+b*x^8+...+j Polynomial Ninth Order Polynomiala*x^10+b*x^9+...+k Polynomial Tenth Order Polynomial。
![非线性回归的参数设置_SPSS 统计分析从入门到精通_[共3页]](https://img.taocdn.com/s1/m/b94e9cb227d3240c8547ef16.png)
