包邮回归分析

contents chapter 1 preliminaries: matrix algebra and random vectors ........ 1 1.1 preliminary matrix algebra ............................................................ 1 1.1.1 trace and eigenvalues.......................................................... 1 1.1.2 symmetric matrices............................................................. 3 1.1.3 idempotent matrices and orthogonal projection..................... 6 1.1.4 singular value decomposition ............................................... 9 1.1.5 vector di.erentiation and generalized inverse .......................10 1.1.6 exercises ...........................................................................10 1.2 expectation and covariance...........................................................11 1.2.1 basic properties .................................................................11 1.2.2 mean and variance of quadratic forms .................................12 1.2.3 exercises ...........................................................................14 1.3 moment generating functions and independence .............................16 1.3.1 exercises ...........................................................................17 chapter 2 multivariate normal distributions.....................................18 2.1 de.nitions and fundamental results ...............................................18 2.2 distribution of quadratic forms .....................................................25 2.3 exercises......................................................................................27 chapter 3 linear regression models ...................................................29 3.1 introduction.................................................................................29 3.2 regression interpreted as conditional mean ....................................31 3.3 linear regression interpreted as linear prediction ............................33 3.4 some nonlinear regressions............................................................34 3.5 typical data structure of linear regression models ..........................35 3.6 exercises......................................................................................36 chapter 4 estimation and distribution theory ..................................38 4.1 least squares estimation (lse) .....................................................38 4.1.1 motivation: why is ls reasonable........................................38 regression analysis 4.1.2 the ls solution .................................................................40 4.1.3 exercises ...........................................................................48 4.2 properties of lse .........................................................................50 4.2.1 small sample distribution-free properties .............................51 4.2.2 properties under normally distributed errors........................55 4.2.3 asymptotic properties ........................................................57 4.2.4 exercises ...........................................................................60 4.3 estimation under linear restrictions ...............................................60 4.4 introducing further explanatory variables and related topics ...........67 4.4.1 introducing further explanatory variables ............................67 4.4.2 centering and scaling the explanatory variables ...................71 4.4.3 estimation in terms of linear prediction...............................72 4.4.4 exercises ...........................................................................73 4.5 design matrices of less than full rank.............................................74 4.5.1 an example .......................................................................74 4.5.2 estimability.......................................................................74 4.5.3 identi.ability under constraints..........................................76 4.5.4 dropping variables to change the model ..............................77 4.5.5 exercises ...........................................................................77 4.6 generalized least squares ..............................................................78 4.6.1 basic theory ......................................................................78 4.6.2 incorrect speci.cation of variance matrix.............................80 4.6.3 exercises ...........................................................................83 4.7 bayesian estimation......................................................................83 4.7.1 the basic idea....................................................................83 4.7.2 normal-noninformative structure ........................................84 4.7.3 conjugate priors ................................................................85 4.8 numerical examples......................................................................86 4.9 exercises......................................................................................90 chapter 5 testing linear hypotheses ..................................................92 5.1 linear hypotheses.........................................................................92 5.2 f -test .........................................................................................93 5.2.1 f -test................................................................................94 contents vii 5.2.2 what are actually tested ....................................................95 5.2.3 examples...........................................................................96 5.3 con.dence ellipse ....................................................................... 101 5.4 prediction and calibration...........................................................103 5.5 multiple correlation coe.cient .................................................... 105 5.5.1 variable selection ............................................................. 105 5.5.2 multiple correlation coe.cient: straight line ...................... 106 5.5.3 multiple correlation coe.cient: multiple regression ............ 108 5.5.4 partial correlation coe.cient ............................................ 110 5.5.5 adjusted multiple correlation coe.cient ............................ 111 5.6 testing linearity: goodness-of-.t test ........................................... 112 5.7 multiple comparisons..................................................................114 5.7.1 simultaneous inference ..................................................... 114 5.7.2 some classical methods for simultaneous intervals .............. 116 5.8 univariate analysis of variance .................................................... 120 5.8.1 anova model................................................................. 120 5.8.2 ancova model .............................................................. 126 5.8.3 sas procedures for anova ............................................. 127 5.9 exercises.................................................................................... 129 chapter 6 variable selection............................................................... 133 6.1 impact of variable selection ......................................................... 133 6.2 mallows’ cp ............................................................................... 137 6.3 akaike’s information criterion (aic)............................................139 6.3.1 prelimilaries: asymptotic normality of mle ...................... 140 6.3.2 kullback-leibler’s distance ............................................... 143 6.3.3 akaike’s information criterion .......................................... 144 6.3.4 aic for linear regression...................................................147 6.4 bayesian information criterion (bic) ........................................... 150 6.5 stepwise variable selection procedures.......................................... 152 6.6 some newly proposed methods .................................................... 154 6.6.1 penalized rss.................................................................. 154 6.6.2 nonnegative garrote ......................................................... 157 6.7 final remarks on variable selection .............................................. 158 regression analysis 6.8 exercises.................................................................................... 161 chapter 7 miscellaneous for linear regression ................................ 165 7.1 collinearity ................................................................................ 165 7.1.1  introduction .................................................................... 165 7.1.2  examine collinearity......................................................... 166 7.1.3 remedies.........................................................................169 *7.2 some remedies for collinearity ..................................................... 170 7.2.1  ridge regression...............................................................170 7.2.2  principal component regression ....................................... 173 7.2.3  partial least square .......................................................... 175 7.2.4  exercises ......................................................................... 176 7.3 outliers ..................................................................................... 177 7.3.1  introduction .................................................................... 177 7.3.2  single outlier ................................................................... 179 7.3.3  multiple outliers .............................................................. 181 7.3.4  relevant quantities........................................................... 183 7.3.5  remarks..........................................................................185 7.4 testing features of errors............................................................. 186 7.4.1  serial correlation and durbin-watson test ......................... 186 7.4.2  testing heteroskeasticity and related topics ....................... 187 7.5 some extensions and variants ...................................................... 191 7.5.1  box-cox model ................................................................ 191 7.5.2  modeling the variances ..................................................... 192 7.5.3  a remark.........................................................................193 chapter 8 logistic regression: modeling categorical responses ... 194 8.1 logistic regression ...................................................................... 194 8.1.1  logistic regression for dichotomous responses..................... 194 8.1.2  likelihood function for logistic regression........................... 196 8.1.3  interpreting the logistic regression.....................................198 8.2 multiple logistic regression .......................................................... 199 8.2.1  maximum likelihood estimation for multiple logistic regression ........................................................................ 200 contents ix 8.3 inference for logistic regression .................................................... 202 8.3.1 inference for simple logistic regression ............................... 202 8.3.2 inference for multiple logistic regression............................. 204 8.4 logistic regression for multinomial responses................................205 8.4.1 nominal responses baseline-category logistic regression.......206 8.4.2 ordinal responses: cumulative logistic regression................208 8.5 exercises.................................................................................... 210 bibliography ............................................................................................ 212