第4章n　多元回归估计与假设检验.ppt

1,第4章 多元回归分析:估计与假设检验 Multiple Regression Analysis,y = b0 + b1x1 + b2x2 + . . . bkxk + u Estimation and Inference,2,Parallels with Simple Regression,Yi = b0 + b1Xi1 + b2Xi2 + . . . bkXik + ui b0 is still the intercept b1 to bk all called slope parameters, also called partial regression coefficients and any coefficient bj denote the change of Y with the changes of Xj as all the other independent variables fixed. u is still the error term (or disturbance) Still minimizing the sum of squared residuals, so have k+1 first order conditions,3,Obtaining OLS Estimates,4,Obtaining OLS Estimates, cont.,The above estimated equation is called the OLS regression line or the sample regression function (SRF) the above equation is the estimated equation, is not the really equation. The really equation is population regression line which we dont know. We only estimate it. So, using a different sample, we can get another different estimated equation line. The population regression line is,5,Interpreting Multiple Regression,6,An Example (Wooldridge, p76),The determination of wage (dollars per hour), wage: Years of education, educ Years of labor market experience, exper Years with the current employer, tenure The relationship btw. wage and educ, exper, tenure: wage=b0+b1educ+b2exper+b3tenure+u The estimated equation as below: wage=-2.873+0.599educ+0.022exper+0.169tenure,7,A “Partialling Out” Interpretation,8,A “Partialling Out” Interpretation,9,“Partialling Out” continued,Previous equation implies that regressing Y on X1 and X2 gives same effect of X1 as regressing Y on residuals from a regression of X1 on X2 This means only the part of Xi1 that is uncorrelated with Xi2 are being related to Yi , so were estimating the effect of X1 on Y after X2 has been “partialled out”,10,The wage determinations,The estimated equation as below: wage=-2.873+0.599educ+0.022exper+0.169tenure Now, we first regress educ on exper and tenure to patial out the exper and tenures effects. Then we regress wage on the residuals of educ on exper and tenure. Whether we get the same result.? educ=13.575-0.0738exper+0.048tenure denote residuals resid wage=5.896+0.599resid We can see that the coefficient of resid is the same of the coefficien of the variable educ in the first estimated equation. So is in the second equation.,11,Goodness-of-Fit: R2,12,Goodness-of-Fit,13,Goodness-of-Fit (continued),How do we think about how well our sample regression line fits our sample data? Can compute the fraction of the total sum of squares (SST) that is explained by the model, call this the R-squared of regression R2 = ESS/TSS = 1 RSS/TSS,14,Goodness-of-Fit (continued),15,More about R-squared,R2 can never decrease when another independent variable is added to a regression, and usually will increase Because R2 will usually increase with the number of independent variables, it is not a good way to compare models,16,An Example,Using wage determination model to show that when we add another new independent variable will increase the value of R2.,17,Adjusted R-Squared,R2 is simply an estimate of how much variation in y is explained by X1, X2,Xk. That is, Recall that the R2 will always increase as more variables are added to the model The adjusted R2 takes into account the number of variables in a model, and may decrease,18,Adjusted R-Squared (cont),Most packages will give you both R2 and adj-R2 You can compare the fit of 2 models (with the same Y) by comparing the adj-R2 wâge=-3.391+0.644educ+0.070exper adj-R2=0.2222 wâge=-2.222+0.569educ+0.190tenure adj-R2=0.2992 You cannot use the adj-R2 to compare models with different ys (e.g. y vs. ln(Y) wâge=-3.391+0.644educ+0.070exper adj-R2=0.2222 log(wâge)=0.404+0.087educ+0.026exper adj-R2=0.3059 Because the variance of the dependent variables is different, the comparation btw them make no sense.,19,Assumptions for Unbiasedness,20,Assumptions for Unbiasedness,Population model is linear in parameters: Y = b0 + b1X1 + b2X2 + bkXk + u We can use a sample of size n, (Xi1, Xi2, Xik, Yi): i=1, 2, , n, from the population model, so that the sample model is Yi = b0 + b1Xi1 + b2Xi2 + bkXik + ui Cov(uXi)=0, E(uXi)=0 , i=1, 2, , n. E(u|X1, X2, Xk) = 0, implying that all of the explanatory variables are exogenous. E(u|X)=0, where X= (X1, X2, Xk), which will reduce to E(u)=0 if independent variables X are not random variables. None of the Xs is constant, and there are no exact linear relationships among them.,The new additional assumption.,21,About multicollinearity,It does allow the independent variables to be correlated; they just cannot be perfectly linear correlated. Student performance: colGPA=b0+b1 hsGPA+b2ACT+ b3 skipped + u Consumption function: consum=b0+b1inc+b2inc2+u But, the following is invalid: log(consum)=b0+b1log(inc)+b2log(inc2)+u In this case, we can not estimate the regression coefficients b1, b2 .,22,Unbiasedness of OLS estimation,Under the three assumptions above, we can get,23,Too Many or Too Few Variables,24,Too Many or Too Few Variables,What happens if we include variables in our specification that dont belong? There is no effect on our parameter estimate, and OLS remains unbiased What if we exclude a variable from our specification that does belong? OLS will usually be biased,25,Omitted Variable Bias,26,Omitted Variable Bias (cont),27,Omitted Variable Bias (cont),28,Omitted Variable Bias (cont),There are two cases where the estimated parameter is unbiased: If b2=0, so that X2 does not appear in the true model If tilde of d1=0, the tilde b1 is unbiased for b1,29,Summary of Direction of Bias,30,An example,The estimated equation as below: wage=-3.391+0.644educ+0.070exper the correlation between educ and exper corr(educ, exper)=-0.2295 Estimate them again without exper wage=-0.905+0.541educ,31,Omitted Variable Bias Summary,Two cases where bias is equal to zero b2 = 0, that is X2 doesnt really belong in model X1 and X2 are uncorrelated in the sample If correlation between X2 , X1 and X2 , Y is the same direction, bias will be positive If correlation between X2 , X1 and X2 , Y is the opposite direction, bias will be negative,32,The More General Case,When there are multiple regressors in the estimated model, to derive the omitted variable bias is more difficult. Because correlation btw a single explanatory variable and the error generally results in all OLS estimators being biased.,33,Variance of the OLS Estimators,34,Variance of the OLS Estimators,Now we know that the sampling distribution of our estimate is centered around the true parameter Want to think about how spread out this distribution is Much easier to think about this variance under an additional assumption, so Assume Var(u|X1, X2, Xk) = s2 , or Var(u)= s2 (Homoskedasticity),35,Variance of OLS (cont),Yi = b0 + b1Xi1 + b2Xi2 + . . . bkXik + ui Let X stand for (X1, X2,Xk) Assuming that Var(u|X) = s2 also implies that Var(Y| X) = s2, we can rewrite them as Var(u)= s2 and Var(Y)= s2 if X are not random variables.,36,Variance of OLS (cont),VIF,37,Variance of OLS (cont),38,Components of OLS Variances,The error variance: a larger s2 implies a larger variance for the OLS estimators The total sample variation: a larger TSSj implies a smaller variance for the estimators Linear relationships among the independent variables: a larger Rj2 implies a larger variance for the estimators If Rj21, then the variances of the estimated parameters will become infinity. Rj21 means there are multicollinearity among the independent variables.,39,Error Variance Estimate (cont),df = n (k + 1), or df = n k 1 df (i.e. degrees of freedom) is the (number of observations) (number of estimated parameters) Attention: the difference btw sd() and se(),40,Assumption for Serial Correlation,There is no serial correlation (auto correlation) between any ui and uj. Cov(ui, uj) = 0, i j. The 3 assumptions for unbiasedness, plus the homoskedasticity assumption and this no serial correlation assuption are known as the Gauss-Markov assumptions.,41,The Gauss-Markov Theorem,Given our 5 Gauss-Markov Assumptions it can be shown that OLS is “BLUE” Best: smallest variance Linear Unbiased Estimator Thus, if the assumptions hold, use OLS,42,The Multiple Regression Model in Matrix Form,43,The Multiple Regression Model in Matrix Form,44,The Multiple Regression Model in Matrix Form, cont.,45,The Multiple Regression Model in Matrix Form, cont.,46,The Multiple Regression Model in Matrix Form, cont.,47,Multiple Regression Analysis: Inference,48,Assumptions of the Classical Linear Model (CLM),So far, we know that given the Gauss-Markov assumptions, OLS is BLUE, In order to do classical hypothesis testing, we need to add another assumption (beyond the Gauss-Markov assumptions) Assume that u is independent of X1, X2, Xk and u is normally distributed with zero mean and variance s2: u Normal(0,s2),49,CLM Assumptions (cont.),Under CLM, We can summarize the population assumptions of CLM as follows Y|X Normal(b0 + b1x1 + bkxk, s2) While for now we just assume normality, clear that sometimes not the case Large samples will let us drop normality,50,.,.,x1,x2,The homoskedastic normal distribution with a single explanatory variable,E(y|x) = b0 + b1x,y,f(y|x),Normal distributions,51,Normal Sampling Distributions,52,The t Test,53,The t Test (cont),Knowing the sampling distribution for the standardized estimator allows us to carry out hypothesis tests Start with a null hypothesis For example, H0: bj=0 If accept null, then accept that Xj has no effect on Y, controlling for other Xs,54,The t Test (cont),55,t Test: One-Sided Alternatives,Besides our null, H0, we need an alternative hypothesis, H1, and a significance level H1 may be one-sided, or two-sided H1: bj 0 and H1: bj 0 are one-sided H1: bj 0 is a two-sided alternative If we want to have only a 5% probability of rejecting H0 if it is really true, then we say our significance level is 5%,56,One-Sided Alternatives (cont),Having picked a significance level, a, we look up the (1 a)th percentile in a t distribution with n k 1 df and call this c, the critical value We can reject the null hypothesis if the t statistic is greater than the critical value If the t statistic is less than the critical value then we fail to reject the null,57,yi = b0 + b1xi1 + + bkxik + ui H0: bj = 0 H1: bj 0,One-Sided Alternatives (cont),58,An Example: Hourly Wage Equation,Wage determination: (wooldridge, p123) log(wâge)=0.284 + 0.092educ + 0.0041exper + 0.022tenure (0.104) (0.007) (0.0017) (0.003) n=526 R2=0.316 Whether the return to exper, controlling for educ and tenure, is zero in the population, against the alternative that it is positive. H0: bexper= 0 vs. H1:bexper 0 The t statistic is t =0.0041/0.00172.41 The degree of freedom: df=n-k-1=526-3-1=522 The critical value of 5% is 1.645 And the t statistic is larger than the critical value, ie., 2.411.645 That is, we will reject the null hypothesis and bexper is really positive.,59,Another example: Student Performance and School Size,Whether the school size has effect on student performance? math10, math test scores, reveal the student performance totcomp, average annual teacher compensation staff, the number of staff per one thousand students enroll, student enrollment, reveal the school size. The Model Equation math10=b0+b1totcomp+b2staff+b3enroll+u H0: b3=0, H1:b3 -1.645, so we cant reject the null hypothesis.,60,One-sided vs Two-sided,Because the t distribution is symmetric, testing H1: bj c, then we fail to reject the null For a two-sided test, we set the critical value based on a/2 and reject H0: bj = 0 if the absolute value of the t statistic c,61,yi = b0 + b1Xi1 + + bkXik + ui H0: bj = 0 H1: bj 0,c,0,a/2,(1 - a),-c,a/2,Two-Sided Alternatives,reject,reject,fail to reject,62,Summary for H0: bj = 0,Unless otherwise stated, the alternative is assumed to be two-sided If we reject the null, we typically say “Xj is statistically significant at the 100a % level” If we fail to reject the null, we typically say “Xj is statistically insignificant at the 100a % level”,63,An Example: Determinants of College GPA (wooldridge, p128),Variables: colGPA, college GPA skipped, the average number of lectures missed per week ACT, achievement test score hsGPA, high school GPA The estimated model olGPA = 1.39 + 0.412 hsGPA + 0.015 ACT 0.083 skipped (0.33) (0.094) (0.011) (0.026) n = 141, R2 = 0.234 H0: bskipped= 0, H1: bskipped 0 fd: n-k-1=137, the critical value t137=1.96 The t statistic is |-0.083/0.026|=3.19 t137=1.96, so we will reject the null hypothesis and the bskipped is signanificantly beyond zero.,64,Testing other hypotheses,A more general form of the t statistic recognizes that we may want to test something like H0: bj = aj In this case, the appropriate t statistic is,65,An Example: Campus Crime and Enrollment (wooldridge, p129),Variables crime, the annual number of crimes on college campuses enroll, student enrollment, reveal the size of college. The regression model log(crime) = b0 + b1log(enroll) + u Whether b1 = 1, that is H0: b1 = 1, H1: b1 1 log(crime) = -6.63 + 1.27 log(enroll) (1.03) (0.11) n = 97 R2 = 0.585 df: n-k-1=95, the critical value at 5% is t95=1.645 The t-statistic is (1.27-1)/0.112.45t95=1.645 So we reject the null hypothesis and the evidence prove that b1 1.,66,Confidence Intervals,Another way to use classical statistical testing is to construct a confidence interval using the same critical value as was used for a two-sided test A 100(1 - a) % confidence interval is defined as,67,Computing p-values for t tests,An alternative to the classical approach is to ask, “what is the smallest significance level at which the null would be rejected?” So, compute the t statistic, and then look up what percentile it is in the appropriate t distribution this is the p-value p-value is the probability we would observe the t statistic we did, if the null were true,68,Stata and p-values, t tests, etc.,Most computer packages will compute the p-value for you, assuming a two-sided test If you really want a one-sided alternative, just divide the two-sided p-value by 2 Stata provides the t statistic, p-value, and 95% confidence interval for H0: bj = 0 for you, in columns labeled “t”, “P |t|” and “95% Conf. Interval”, respectively,69,Testing a Linear Combination,Suppose instead of testing whether b1 is equal to a constant, you want to test if it is equal to another parameter, that is H0 : b1 = b2, or b1 - b2=0 Use same basic procedure for forming a t statistic,70,Testing Linear Combination (cont),71,Testing a Linear Combo (cont),So, to use formula, need s12, which standard output does not have Many packages will have an option to get it, or will just perform the test for you In Stata, after reg Y X1 X2 Xk you would type test X1 = X2 to get a p-value for the test More generally, you can alway