Assess Significance of Regression Coefficients Using t-statistic
This example shows how to test for the significance of the regression coefficients using t-statistic.
Load the sample data and fit the linear regression model.
load hald mdl = fitlm(ingredients,heat)
mdl = Linear regression model: y ~ 1 + x1 + x2 + x3 + x4 Estimated Coefficients: Estimate SE tStat pValue ________ _______ ________ ________ (Intercept) 62.405 70.071 0.8906 0.39913 x1 1.5511 0.74477 2.0827 0.070822 x2 0.51017 0.72379 0.70486 0.5009 x3 0.10191 0.75471 0.13503 0.89592 x4 -0.14406 0.70905 -0.20317 0.84407 Number of observations: 13, Error degrees of freedom: 8 Root Mean Squared Error: 2.45 R-squared: 0.982, Adjusted R-Squared 0.974 F-statistic vs. constant model: 111, p-value = 4.76e-07
You can see that for each coefficient, tStat = Estimate/SE. The -values for the hypotheses tests are in the pValue column. Each -statistic tests for the significance of each term given other terms in the model. According to these results, none of the coefficients seem significant at the 5% significance level, although the R-squared value for the model is really high at 0.97. This often indicates possible multicollinearity among the predictor variables.
Use stepwise regression to decide which variables to include in the model.
load hald mdl = stepwiselm(ingredients,heat)
1. Adding x4, FStat = 22.7985, pValue = 0.000576232 2. Adding x1, FStat = 108.2239, pValue = 1.105281e-06 mdl = Linear regression model: y ~ 1 + x1 + x4 Estimated Coefficients: Estimate SE tStat pValue ________ ________ _______ __________ (Intercept) 103.1 2.124 48.54 3.3243e-13 x1 1.44 0.13842 10.403 1.1053e-06 x4 -0.61395 0.048645 -12.621 1.8149e-07 Number of observations: 13, Error degrees of freedom: 10 Root Mean Squared Error: 2.73 R-squared: 0.972, Adjusted R-Squared 0.967 F-statistic vs. constant model: 177, p-value = 1.58e-08
In this example, stepwiselm starts with the constant model (default) and uses forward selection to incrementally add x4 and x1. Each predictor variable in the final model is significant given the other one is in the model. The algorithm stops when adding none of the other predictor variables significantly improves in the model. For details on stepwise regression, see stepwiselm.