Multiple Least-Square Regression Example 1
Dependent Variable, Y: CEMENT MIX
X1, X2, X3, X4 are four ingredients in mix
Obs Y X1 X2 X3 X4 X1X2 X1X3 X1X4
1 78.5 7 26 6 60 182 42 420
2 74.3 1 29 15 52 29 15 52
3 104.3 11 56 8 47 616 88 517
4 87.6 11 31 8 47 341 88 517
5 95.9 7 52 6 33 364 42 231
6 109.2 11 55 9 22 605 99 242
7 102.7 3 71 17 6 213 51 18
8 72.5 1 31 22 44 31 22 44
9 93.1 2 54 18 22 108 36 44
10 115.9 21 47 4 26 987 84 546
11 83.8 1 40 23 34 40 23 34
12 113.3 11 66 9 12 726 99 132
13 109.4 10 68 8 12 680 80 120
14 . 10 70 6 14 700 60 140
15 . 15 75 4 12 1125 60 180
Multiple Least-Square Regression Example 2
General linear model & related residual plots
The GLM Procedure
Number of Observations Read 15
Number of Observations Used 13
Multiple Least-Square Regression Example 3
General linear model & related residual plots
The GLM Procedure
Dependent Variable: Y
Sum of
Source DF Squares Mean Square F Value Pr > F
Model 4 2670.365050 667.591263 117.64 <.0001
Error 8 45.398026 5.674753
Corrected Total 12 2715.763077
R-Square Coeff Var Root MSE Y Mean
0.983284 2.496434 2.382174 95.42308
Source DF Type I SS Mean Square F Value Pr > F
X1 1 1450.076328 1450.076328 255.53 <.0001
X2 1 1207.782266 1207.782266 212.83 <.0001
X3 1 9.793869 9.793869 1.73 0.2254
X4 1 2.712588 2.712588 0.48 0.5089
Source DF Type III SS Mean Square F Value Pr > F
X1 1 281.3174755 281.3174755 49.57 0.0001
X2 1 163.2592552 163.2592552 28.77 0.0007
X3 1 3.1459775 3.1459775 0.55 0.4778
X4 1 2.7125876 2.7125876 0.48 0.5089
Standard
Parameter Estimate Error t Value Pr > |t|
Intercept 55.28234314 11.01724868 5.02 0.0010
X1 1.62976040 0.23147223 7.04 0.0001
X2 0.58798438 0.10962267 5.36 0.0007
X3 0.16734770 0.22475814 0.74 0.4778
X4 -0.07179770 0.10384657 -0.69 0.5089
Multiple Least-Square Regression Example 4
General linear model & related residual plots
The GLM Procedure
Observation Observed Predicted Residual
1 78.50000000 78.67448398 -0.17448398
2 74.30000000 72.74038564 1.55961436
3 104.30000000 104.10112263 0.19887737
4 87.60000000 89.40151301 -1.80151301
5 95.90000000 95.90061594 -0.00061594
6 109.20000000 105.47542849 3.72457151
7 102.70000000 104.33264028 -1.63264028
8 72.50000000 75.66216990 -3.16216990
9 93.10000000 91.72572981 1.37427019
10 115.90000000 115.94522809 -0.04522809
11 83.80000000 81.83935408 1.96064592
12 113.30000000 112.66123374 0.63876626
13 109.40000000 112.04009442 -2.64009442
14 * . 112.73777239 .
15 * . 123.63539631 .
95% Confidence Limits for
Observation Mean Predicted Value
1 74.94837105 82.40059691
2 69.75726883 75.72350246
3 98.63416098 109.56808428
4 86.62444420 92.17858182
5 92.66902393 99.13220794
6 103.43385896 107.51699802
7 100.97222811 107.69305245
8 72.24829215 79.07604766
9 89.28384995 94.16760966
10 111.31872937 120.57172681
11 78.44902289 85.22968528
12 110.12969158 115.19277590
13 109.27047786 114.80971097
14 * 109.51433239 115.96121239
15 * 120.01610747 127.25468514
* Observation was not used in this analysis
Sum of Residuals -0.0000000
Sum of Squared Residuals 45.3980265
Sum of Squared Residuals - Error SS -0.0000000
PRESS Statistic 521.1471688
First Order Autocorrelation -0.1359120
Durbin-Watson D 2.1176203
Multiple Least-Square Regression Example 5
General linear model & related residual plots
Plot of Y*YHAT. Symbol used is 'P'.
Plot of RESID*YHAT. Symbol used is 'R'.
Y |
|
120 +
| P
| P
| P P
|
| P
100 +
| P
| P
|
| P
| P
80 + P
|
| P P
|
|
|
60 +
|
|
|
|
|
40 +
|
|
|
|
|
20 +
|
|
|
|
| R R
0 + R R R R R R R
| R R R
|
|
|
|
-20 +
|
--+-----------+-----------+-----------+-----------+-----------+-----------+-
70 80 90 100 110 120 130
YHAT
NOTE: 4 obs had missing values. 2 obs hidden.
Forward Regression 6
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Forward Selection: Step 1
Variable X1X2 Entered: R-Square = 0.7884 and C(p) = 61.2741
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 1 2141.20142 2141.20142 40.99 <.0001
Error 11 574.56165 52.23288
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 79.34871 3.21263 31864 610.04 <.0001
X1X2 0.04246 0.00663 2141.20142 40.99 <.0001
Bounds on condition number: 1, 1
--------------------------------------------------------------------------------
Forward Selection: Step 2
Variable X2 Entered: R-Square = 0.9677 and C(p) = 3.7308
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 2 2628.02762 1314.01381 149.77 <.0001
Error 10 87.73545 8.77355
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 60.98339 2.79502 4176.65198 476.05 <.0001
X2 0.47545 0.06383 486.82620 55.49 <.0001
X1X2 0.03049 0.00316 818.60089 93.30 <.0001
Forward Regression 7
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Forward Selection: Step 2
Bounds on condition number: 1.3492, 5.397
--------------------------------------------------------------------------------
Forward Selection: Step 3
Variable X1 Entered: R-Square = 0.9790 and C(p) = 1.9915
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 3 2658.60032 886.20011 139.53 <.0001
Error 9 57.16276 6.35142
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 53.70675 4.08112 1099.93944 173.18 <.0001
X1 1.27458 0.58095 30.57270 4.81 0.0559
X2 0.63581 0.09106 309.66576 48.76 <.0001
X1X2 0.00420 0.01228 0.74172 0.12 0.7404
Bounds on condition number: 28.215, 162.22
--------------------------------------------------------------------------------
Forward Selection: Step 4
Variable X4 Entered: R-Square = 0.9826 and C(p) = 2.7694
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 2668.59260 667.14815 113.15 <.0001
Error 8 47.17048 5.89631
Corrected Total 12 2715.76308
Forward Regression 8
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Forward Selection: Step 4
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 63.33232 8.37464 337.20901 57.19 <.0001
X1 1.21827 0.56142 27.76487 4.71 0.0618
X2 0.51063 0.13017 90.73245 15.39 0.0044
X4 -0.11727 0.09008 9.99228 1.69 0.2292
X1X2 0.00574 0.01189 1.37353 0.23 0.6423
Bounds on condition number: 28.498, 255.42
--------------------------------------------------------------------------------
No other variable met the 0.5000 significance level for entry into the model.
Summary of Forward Selection
Variable Number Partial Model
Step Entered Vars In R-Square R-Square C(p) F Value Pr > F
1 X1X2 1 0.7884 0.7884 61.2741 40.99 <.0001
2 X2 2 0.1793 0.9677 3.7308 55.49 <.0001
3 X1 3 0.0113 0.9790 1.9915 4.81 0.0559
4 X4 4 0.0037 0.9826 2.7694 1.69 0.2292
Backward Regression 9
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Backward Elimination: Step 0
All Variables Entered: R-Square = 0.9849 and C(p) = 8.0000
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 7 2674.88306 382.12615 46.74 0.0003
Error 5 40.88001 8.17600
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 2.89001 88.10760 0.00880 0.00 0.9751
X1 5.71832 6.95186 5.53192 0.68 0.4482
X2 1.18331 1.03510 10.68509 1.31 0.3047
X3 0.39744 0.47725 5.67012 0.69 0.4429
X4 0.55666 1.02214 2.42494 0.30 0.6094
X1X2 -0.05315 0.09235 2.70798 0.33 0.5899
X1X3 0.05274 0.08404 3.22026 0.39 0.5578
X1X4 -0.05635 0.08923 3.26127 0.40 0.5554
Bounds on condition number: 2454.4, 35266
--------------------------------------------------------------------------------
Backward Elimination: Step 1
Variable X4 Removed: R-Square = 0.9841 and C(p) = 6.2966
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 6 2672.45812 445.40969 61.71 <.0001
Error 6 43.30495 7.21749
Corrected Total 12 2715.76308
Backward Regression 10
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Backward Elimination: Step 1
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 50.73167 6.36354 458.71950 63.56 0.0002
X1 1.96787 0.89299 35.04961 4.86 0.0697
X2 0.62280 0.10352 261.21478 36.19 0.0010
X3 0.17604 0.23487 4.05439 0.56 0.4819
X1X2 -0.00368 0.01565 0.39898 0.06 0.8219
X1X3 0.01972 0.05468 0.93910 0.13 0.7307
X1X4 -0.00815 0.01053 4.31668 0.60 0.4687
Bounds on condition number: 45.877, 641.33
--------------------------------------------------------------------------------
Backward Elimination: Step 2
Variable X1X2 Removed: R-Square = 0.9839 and C(p) = 4.3454
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 5 2672.05915 534.41183 85.60 <.0001
Error 7 43.70393 6.24342
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 51.21222 5.60497 521.22187 83.48 <.0001
X1 1.77343 0.31331 200.02920 32.04 0.0008
X2 0.60850 0.07792 380.71800 60.98 0.0001
X3 0.18419 0.21606 4.53733 0.73 0.4221
X1X3 0.01750 0.05009 0.76201 0.12 0.7371
X1X4 -0.00680 0.00823 4.26633 0.68 0.4357
Bounds on condition number: 6.5286, 116.17
--------------------------------------------------------------------------------
Backward Elimination: Step 3
Variable X1X3 Removed: R-Square = 0.9836 and C(p) = 2.4386
Backward Regression 11
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Backward Elimination: Step 3
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 2671.29713 667.82428 120.15 <.0001
Error 8 44.46594 5.55824
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 50.94043 5.23729 525.83473 94.60 <.0001
X1 1.82417 0.26194 269.56531 48.50 0.0001
X2 0.62373 0.06094 582.27711 104.76 <.0001
X3 0.18721 0.20369 4.69521 0.84 0.3849
X1X4 -0.00609 0.00752 3.64467 0.66 0.4415
Bounds on condition number: 5.1557, 63.591
--------------------------------------------------------------------------------
Backward Elimination: Step 4
Variable X1X4 Removed: R-Square = 0.9823 and C(p) = 0.8844
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 3 2667.65246 889.21749 166.34 <.0001
Error 9 48.11061 5.34562
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 48.19363 3.91330 810.75653 151.67 <.0001
X1 1.69589 0.20458 367.33211 68.72 <.0001
X2 0.65691 0.04423 1178.96145 220.55 <.0001
X3 0.25002 0.18471 9.79387 1.83 0.2089
Bounds on condition number: 3.2511, 22.37
--------------------------------------------------------------------------------
Backward Elimination: Step 5
Backward Regression 12
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Backward Elimination: Step 5
Variable X3 Removed: R-Square = 0.9787 and C(p) = 0.0822
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 2 2657.85859 1328.92930 229.50 <.0001
Error 10 57.90448 5.79045
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 52.57735 2.28617 3062.60416 528.91 <.0001
X1 1.46831 0.12130 848.43186 146.52 <.0001
X2 0.66225 0.04585 1207.78227 208.58 <.0001
Bounds on condition number: 1.0551, 4.2205
--------------------------------------------------------------------------------
All variables left in the model are significant at the 0.1000 level.
Summary of Backward Elimination
Variable Number Partial Model
Step Removed Vars In R-Square R-Square C(p) F Value Pr > F
1 X4 6 0.0009 0.9841 6.2966 0.30 0.6094
2 X1X2 5 0.0001 0.9839 4.3454 0.06 0.8219
3 X1X3 4 0.0003 0.9836 2.4386 0.12 0.7371
4 X1X4 3 0.0013 0.9823 0.8844 0.66 0.4415
5 X3 2 0.0036 0.9787 0.0822 1.83 0.2089
Stepwise Regression 13
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Stepwise Selection: Step 1
Variable X1X2 Entered: R-Square = 0.7884 and C(p) = 61.2741
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 1 2141.20142 2141.20142 40.99 <.0001
Error 11 574.56165 52.23288
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 79.34871 3.21263 31864 610.04 <.0001
X1X2 0.04246 0.00663 2141.20142 40.99 <.0001
Bounds on condition number: 1, 1
--------------------------------------------------------------------------------
Stepwise Selection: Step 2
Variable X2 Entered: R-Square = 0.9677 and C(p) = 3.7308
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 2 2628.02762 1314.01381 149.77 <.0001
Error 10 87.73545 8.77355
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 60.98339 2.79502 4176.65198 476.05 <.0001
X2 0.47545 0.06383 486.82620 55.49 <.0001
X1X2 0.03049 0.00316 818.60089 93.30 <.0001
Stepwise Regression 14
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Stepwise Selection: Step 2
Bounds on condition number: 1.3492, 5.397
--------------------------------------------------------------------------------
Stepwise Selection: Step 3
Variable X1 Entered: R-Square = 0.9790 and C(p) = 1.9915
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 3 2658.60032 886.20011 139.53 <.0001
Error 9 57.16276 6.35142
Corrected Total 12 2715.76308
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 53.70675 4.08112 1099.93944 173.18 <.0001
X1 1.27458 0.58095 30.57270 4.81 0.0559
X2 0.63581 0.09106 309.66576 48.76 <.0001
X1X2 0.00420 0.01228 0.74172 0.12 0.7404
Bounds on condition number: 28.215, 162.22
--------------------------------------------------------------------------------
Stepwise Selection: Step 4
Variable X1X2 Removed: R-Square = 0.9787 and C(p) = 0.0822
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 2 2657.85859 1328.92930 229.50 <.0001
Error 10 57.90448 5.79045
Corrected Total 12 2715.76308
Stepwise Regression 15
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Stepwise Selection: Step 4
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
Intercept 52.57735 2.28617 3062.60416 528.91 <.0001
X1 1.46831 0.12130 848.43186 146.52 <.0001
X2 0.66225 0.04585 1207.78227 208.58 <.0001
Bounds on condition number: 1.0551, 4.2205
--------------------------------------------------------------------------------
All variables left in the model are significant at the 0.1500 level.
No other variable met the 0.1500 significance level for entry into the model.
Summary of Stepwise Selection
Variable Variable Number Partial Model
Step Entered Removed Vars In R-Square R-Square C(p) F Value Pr > F
1 X1X2 1 0.7884 0.7884 61.2741 40.99 <.0001
2 X2 2 0.1793 0.9677 3.7308 55.49 <.0001
3 X1 3 0.0113 0.9790 1.9915 4.81 0.0559
4 X1X2 2 0.0003 0.9787 0.0822 0.12 0.7404
R-square Regression 16
The REG Procedure
Model: MODEL1
Dependent Variable: Y
R-Square Selection Method
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Number in
Model R-Square Variables in Model
1 0.7884 X1X2
1 0.6663 X2
1 0.6570 X1X3
1 0.5339 X1
1 0.5296 X4
1 0.2859 X3
1 0.0674 X1X4
----------------------------------------------------
2 0.9787 X1 X2
2 0.9677 X2 X1X2
2 0.9491 X4 X1X2
2 0.9157 X1 X4
2 0.9045 X4 X1X4
2 0.8870 X2 X1X3
2 0.8775 X2 X1X4
2 0.8707 X4 X1X3
2 0.8649 X1 X1X2
2 0.8634 X1X2 X1X4
2 0.8553 X3 X4
2 0.8470 X2 X3
2 0.8249 X3 X1X2
2 0.7961 X1X2 X1X3
2 0.7668 X1 X1X4
2 0.7132 X1X3 X1X4
2 0.6699 X1 X1X3
2 0.6663 X2 X4
2 0.6574 X3 X1X3
2 0.5482 X1 X3
2 0.3374 X3 X1X4
----------------------------------------------------
3 0.9823 X1 X2 X3
3 0.9821 X1 X2 X4
3 0.9819 X1 X2 X1X4
3 0.9790 X1 X2 X1X2
3 0.9787 X1 X2 X1X3
3 0.9724 X2 X4 X1X2
3 0.9691 X2 X1X2 X1X4
3 0.9690 X2 X1X2 X1X3
3 0.9679 X2 X3 X1X2
3 0.9607 X4 X1X2 X1X4
3 0.9545 X4 X1X2 X1X3
3 0.9517 X3 X4 X1X2
3 0.9492 X1 X4 X1X2
R-square Regression 17
The REG Procedure
Model: MODEL1
Dependent Variable: Y
R-Square Selection Method
Number in
Model R-Square Variables in Model
3 0.9381 X2 X4 X1X4
3 0.9306 X1 X4 X1X3
3 0.9238 X1 X4 X1X4
3 0.9232 X1 X3 X4
3 0.9209 X4 X1X3 X1X4
3 0.9200 X3 X4 X1X4
3 0.9113 X3 X4 X1X3
3 0.9069 X2 X1X3 X1X4
3 0.9044 X2 X3 X1X3
3 0.8950 X2 X4 X1X3
3 0.8851 X1X2 X1X3 X1X4
3 0.8844 X2 X3 X1X4
3 0.8797 X2 X3 X4
3 0.8774 X1 X1X2 X1X3
3 0.8694 X1 X1X2 X1X4
3 0.8674 X1 X3 X1X2
3 0.8639 X3 X1X2 X1X4
3 0.8429 X1 X1X3 X1X4
3 0.8364 X3 X1X2 X1X3
3 0.7692 X1 X3 X1X4
3 0.7403 X3 X1X3 X1X4
3 0.6841 X1 X3 X1X3
----------------------------------------------------
4 0.9836 X1 X2 X3 X1X4
4 0.9833 X1 X2 X3 X4
4 0.9827 X2 X4 X1X2 X1X4
4 0.9826 X1 X2 X4 X1X2
4 0.9824 X1 X2 X4 X1X4
4 0.9824 X1 X2 X3 X1X2
4 0.9823 X1 X2 X3 X1X3
4 0.9823 X1 X2 X4 X1X3
4 0.9822 X1 X2 X1X3 X1X4
4 0.9821 X1 X2 X1X2 X1X4
4 0.9806 X1 X4 X1X2 X1X4
4 0.9790 X1 X2 X1X2 X1X3
4 0.9746 X2 X4 X1X2 X1X3
4 0.9728 X2 X3 X4 X1X2
4 0.9707 X2 X3 X1X2 X1X4
4 0.9696 X2 X1X2 X1X3 X1X4
4 0.9695 X2 X3 X1X2 X1X3
4 0.9616 X4 X1X2 X1X3 X1X4
4 0.9609 X3 X4 X1X2 X1X4
4 0.9561 X3 X4 X1X2 X1X3
4 0.9550 X1 X4 X1X2 X1X3
4 0.9522 X1 X3 X4 X1X2
4 0.9459 X2 X4 X1X3 X1X4
4 0.9448 X2 X3 X4 X1X4
4 0.9352 X1 X3 X4 X1X3
4 0.9334 X1 X4 X1X3 X1X4
R-square Regression 18
The REG Procedure
Model: MODEL1
Dependent Variable: Y
R-Square Selection Method
Number in
Model R-Square Variables in Model
4 0.9310 X3 X4 X1X3 X1X4
4 0.9294 X1 X3 X4 X1X4
4 0.9262 X2 X3 X4 X1X3
4 0.9103 X2 X3 X1X3 X1X4
4 0.8875 X1 X1X2 X1X3 X1X4
4 0.8854 X3 X1X2 X1X3 X1X4
4 0.8807 X1 X3 X1X2 X1X3
4 0.8700 X1 X3 X1X2 X1X4
4 0.8437 X1 X3 X1X3 X1X4
----------------------------------------------------
5 0.9839 X1 X2 X3 X1X3 X1X4
5 0.9837 X1 X2 X3 X1X2 X1X4
5 0.9837 X1 X2 X3 X4 X1X4
5 0.9836 X1 X2 X3 X4 X1X2
5 0.9834 X1 X2 X3 X4 X1X3
5 0.9829 X2 X3 X4 X1X2 X1X4
5 0.9828 X1 X2 X4 X1X2 X1X3
5 0.9828 X1 X2 X4 X1X2 X1X4
5 0.9827 X1 X2 X4 X1X3 X1X4
5 0.9827 X2 X4 X1X2 X1X3 X1X4
5 0.9826 X1 X2 X1X2 X1X3 X1X4
5 0.9825 X1 X2 X3 X1X2 X1X3
5 0.9809 X1 X3 X4 X1X2 X1X4
5 0.9808 X1 X4 X1X2 X1X3 X1X4
5 0.9748 X2 X3 X4 X1X2 X1X3
5 0.9711 X2 X3 X1X2 X1X3 X1X4
5 0.9618 X3 X4 X1X2 X1X3 X1X4
5 0.9571 X1 X3 X4 X1X2 X1X3
5 0.9506 X2 X3 X4 X1X3 X1X4
5 0.9374 X1 X3 X4 X1X3 X1X4
5 0.8879 X1 X3 X1X2 X1X3 X1X4
----------------------------------------------------
6 0.9841 X1 X2 X3 X1X2 X1X3 X1X4
6 0.9839 X1 X2 X3 X4 X1X3 X1X4
6 0.9838 X1 X2 X3 X4 X1X2 X1X4
6 0.9837 X1 X2 X3 X4 X1X2 X1X3
6 0.9829 X2 X3 X4 X1X2 X1X3 X1X4
6 0.9829 X1 X2 X4 X1X2 X1X3 X1X4
6 0.9810 X1 X3 X4 X1X2 X1X3 X1X4
----------------------------------------------------
7 0.9849 X1 X2 X3 X4 X1X2 X1X3 X1X4
Adjusted R-square Regression 19
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Adjusted R-Square Selection Method
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Number in Adjusted
Model R-Square R-Square Variables in Model
3 0.9764 0.9823 X1 X2 X3
3 0.9762 0.9821 X1 X2 X4
3 0.9759 0.9819 X1 X2 X1X4
4 0.9754 0.9836 X1 X2 X3 X1X4
4 0.9749 0.9833 X1 X2 X3 X4
2 0.9744 0.9787 X1 X2
4 0.9740 0.9827 X2 X4 X1X2 X1X4
4 0.9739 0.9826 X1 X2 X4 X1X2
4 0.9736 0.9824 X1 X2 X4 X1X4
4 0.9736 0.9824 X1 X2 X3 X1X2
4 0.9735 0.9823 X1 X2 X3 X1X3
4 0.9735 0.9823 X1 X2 X4 X1X3
4 0.9734 0.9822 X1 X2 X1X3 X1X4
4 0.9732 0.9821 X1 X2 X1X2 X1X4
5 0.9724 0.9839 X1 X2 X3 X1X3 X1X4
5 0.9721 0.9837 X1 X2 X3 X1X2 X1X4
5 0.9720 0.9837 X1 X2 X3 X4 X1X4
3 0.9719 0.9790 X1 X2 X1X2
5 0.9718 0.9836 X1 X2 X3 X4 X1X2
5 0.9716 0.9834 X1 X2 X3 X4 X1X3
3 0.9716 0.9787 X1 X2 X1X3
4 0.9710 0.9806 X1 X4 X1X2 X1X4
5 0.9707 0.9829 X2 X3 X4 X1X2 X1X4
5 0.9706 0.9828 X1 X2 X4 X1X2 X1X3
5 0.9705 0.9828 X1 X2 X4 X1X2 X1X4
5 0.9704 0.9827 X1 X2 X4 X1X3 X1X4
5 0.9704 0.9827 X2 X4 X1X2 X1X3 X1X4
5 0.9701 0.9826 X1 X2 X1X2 X1X3 X1X4
5 0.9699 0.9825 X1 X2 X3 X1X2 X1X3
4 0.9684 0.9790 X1 X2 X1X2 X1X3
6 0.9681 0.9841 X1 X2 X3 X1X2 X1X3 X1X4
6 0.9679 0.9839 X1 X2 X3 X4 X1X3 X1X4
6 0.9675 0.9838 X1 X2 X3 X4 X1X2 X1X4
6 0.9675 0.9837 X1 X2 X3 X4 X1X2 X1X3
5 0.9672 0.9809 X1 X3 X4 X1X2 X1X4
5 0.9671 0.9808 X1 X4 X1X2 X1X3 X1X4
6 0.9658 0.9829 X2 X3 X4 X1X2 X1X3 X1X4
6 0.9657 0.9829 X1 X2 X4 X1X2 X1X3 X1X4
7 0.9639 0.9849 X1 X2 X3 X4 X1X2 X1X3 X1X4
3 0.9632 0.9724 X2 X4 X1X2
6 0.9620 0.9810 X1 X3 X4 X1X2 X1X3 X1X4
4 0.9618 0.9746 X2 X4 X1X2 X1X3
2 0.9612 0.9677 X2 X1X2
Adjusted R-square Regression 20
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Adjusted R-Square Selection Method
Number in Adjusted
Model R-Square R-Square Variables in Model
4 0.9592 0.9728 X2 X3 X4 X1X2
3 0.9588 0.9691 X2 X1X2 X1X4
3 0.9587 0.9690 X2 X1X2 X1X3
3 0.9572 0.9679 X2 X3 X1X2
5 0.9568 0.9748 X2 X3 X4 X1X2 X1X3
4 0.9561 0.9707 X2 X3 X1X2 X1X4
4 0.9544 0.9696 X2 X1X2 X1X3 X1X4
4 0.9542 0.9695 X2 X3 X1X2 X1X3
5 0.9505 0.9711 X2 X3 X1X2 X1X3 X1X4
3 0.9475 0.9607 X4 X1X2 X1X4
4 0.9424 0.9616 X4 X1X2 X1X3 X1X4
4 0.9413 0.9609 X3 X4 X1X2 X1X4
3 0.9394 0.9545 X4 X1X2 X1X3
2 0.9389 0.9491 X4 X1X2
3 0.9355 0.9517 X3 X4 X1X2
5 0.9346 0.9618 X3 X4 X1X2 X1X3 X1X4
4 0.9342 0.9561 X3 X4 X1X2 X1X3
4 0.9325 0.9550 X1 X4 X1X2 X1X3
3 0.9323 0.9492 X1 X4 X1X2
4 0.9283 0.9522 X1 X3 X4 X1X2
5 0.9264 0.9571 X1 X3 X4 X1X2 X1X3
4 0.9188 0.9459 X2 X4 X1X3 X1X4
3 0.9174 0.9381 X2 X4 X1X4
4 0.9172 0.9448 X2 X3 X4 X1X4
5 0.9154 0.9506 X2 X3 X4 X1X3 X1X4
3 0.9074 0.9306 X1 X4 X1X3
4 0.9027 0.9352 X1 X3 X4 X1X3
4 0.9001 0.9334 X1 X4 X1X3 X1X4
2 0.8989 0.9157 X1 X4
3 0.8984 0.9238 X1 X4 X1X4
3 0.8976 0.9232 X1 X3 X4
4 0.8965 0.9310 X3 X4 X1X3 X1X4
3 0.8945 0.9209 X4 X1X3 X1X4
4 0.8941 0.9294 X1 X3 X4 X1X4
3 0.8934 0.9200 X3 X4 X1X4
5 0.8927 0.9374 X1 X3 X4 X1X3 X1X4
4 0.8894 0.9262 X2 X3 X4 X1X3
2 0.8853 0.9045 X4 X1X4
3 0.8817 0.9113 X3 X4 X1X3
3 0.8759 0.9069 X2 X1X3 X1X4
3 0.8725 0.9044 X2 X3 X1X3
4 0.8654 0.9103 X2 X3 X1X3 X1X4
2 0.8644 0.8870 X2 X1X3
3 0.8600 0.8950 X2 X4 X1X3
2 0.8531 0.8775 X2 X1X4
3 0.8468 0.8851 X1X2 X1X3 X1X4
3 0.8458 0.8844 X2 X3 X1X4
2 0.8449 0.8707 X4 X1X3
3 0.8396 0.8797 X2 X3 X4
Adjusted R-square Regression 21
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Adjusted R-Square Selection Method
Number in Adjusted
Model R-Square R-Square Variables in Model
2 0.8379 0.8649 X1 X1X2
3 0.8365 0.8774 X1 X1X2 X1X3
2 0.8360 0.8634 X1X2 X1X4
4 0.8312 0.8875 X1 X1X2 X1X3 X1X4
4 0.8281 0.8854 X3 X1X2 X1X3 X1X4
2 0.8264 0.8553 X3 X4
3 0.8258 0.8694 X1 X1X2 X1X4
3 0.8233 0.8674 X1 X3 X1X2
4 0.8210 0.8807 X1 X3 X1X2 X1X3
3 0.8186 0.8639 X3 X1X2 X1X4
2 0.8164 0.8470 X2 X3
5 0.8078 0.8879 X1 X3 X1X2 X1X3 X1X4
4 0.8051 0.8700 X1 X3 X1X2 X1X4
3 0.7905 0.8429 X1 X1X3 X1X4
2 0.7898 0.8249 X3 X1X2
3 0.7818 0.8364 X3 X1X2 X1X3
1 0.7692 0.7884 X1X2
4 0.7656 0.8437 X1 X3 X1X3 X1X4
2 0.7553 0.7961 X1X2 X1X3
2 0.7201 0.7668 X1 X1X4
3 0.6923 0.7692 X1 X3 X1X4
2 0.6559 0.7132 X1X3 X1X4
3 0.6537 0.7403 X3 X1X3 X1X4
1 0.6359 0.6663 X2
1 0.6259 0.6570 X1X3
2 0.6038 0.6699 X1 X1X3
2 0.5996 0.6663 X2 X4
2 0.5889 0.6574 X3 X1X3
3 0.5788 0.6841 X1 X3 X1X3
1 0.4916 0.5339 X1
1 0.4868 0.5296 X4
2 0.4578 0.5482 X1 X3
1 0.2210 0.2859 X3
2 0.2049 0.3374 X3 X1X4
1 -.0174 0.0674 X1X4
Mallows. Cp statistics Regression 22
The REG Procedure
Model: MODEL1
Dependent Variable: Y
C(p) Selection Method
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Number in
Model C(p) R-Square Variables in Model
2 0.0822 0.9787 X1 X2
3 0.8844 0.9823 X1 X2 X3
3 0.9374 0.9821 X1 X2 X4
3 1.0129 0.9819 X1 X2 X1X4
3 1.9915 0.9790 X1 X2 X1X2
3 2.0779 0.9787 X1 X2 X1X3
4 2.4386 0.9836 X1 X2 X3 X1X4
4 2.5526 0.9833 X1 X2 X3 X4
4 2.7478 0.9827 X2 X4 X1X2 X1X4
4 2.7694 0.9826 X1 X2 X4 X1X2
4 2.8352 0.9824 X1 X2 X4 X1X4
4 2.8410 0.9824 X1 X2 X3 X1X2
4 2.8672 0.9823 X1 X2 X3 X1X3
4 2.8711 0.9823 X1 X2 X4 X1X3
4 2.9003 0.9822 X1 X2 X1X3 X1X4
4 2.9409 0.9821 X1 X2 X1X2 X1X4
4 3.4275 0.9806 X1 X4 X1X2 X1X4
2 3.7308 0.9677 X2 X1X2
4 3.9876 0.9790 X1 X2 X1X2 X1X3
3 4.1653 0.9724 X2 X4 X1X2
5 4.3454 0.9839 X1 X2 X3 X1X3 X1X4
5 4.4115 0.9837 X1 X2 X3 X1X2 X1X4
5 4.4231 0.9837 X1 X2 X3 X4 X1X4
5 4.4552 0.9836 X1 X2 X3 X4 X1X2
5 4.4986 0.9834 X1 X2 X3 X4 X1X3
5 4.6800 0.9829 X2 X3 X4 X1X2 X1X4
5 4.7013 0.9828 X1 X2 X4 X1X2 X1X3
5 4.7217 0.9828 X1 X2 X4 X1X2 X1X4
5 4.7316 0.9827 X1 X2 X4 X1X3 X1X4
5 4.7435 0.9827 X2 X4 X1X2 X1X3 X1X4
5 4.7925 0.9826 X1 X2 X1X2 X1X3 X1X4
5 4.8246 0.9825 X1 X2 X3 X1X2 X1X3
3 5.2655 0.9691 X2 X1X2 X1X4
3 5.2965 0.9690 X2 X1X2 X1X3
5 5.3582 0.9809 X1 X3 X4 X1X2 X1X4
5 5.3770 0.9808 X1 X4 X1X2 X1X3 X1X4
4 5.4498 0.9746 X2 X4 X1X2 X1X3
3 5.6542 0.9679 X2 X3 X1X2
4 6.0340 0.9728 X2 X3 X4 X1X2
6 6.2966 0.9841 X1 X2 X3 X1X2 X1X3 X1X4
6 6.3312 0.9839 X1 X2 X3 X4 X1X3 X1X4
6 6.3939 0.9838 X1 X2 X3 X4 X1X2 X1X4
6 6.3989 0.9837 X1 X2 X3 X4 X1X2 X1X3
Mallows. Cp statistics Regression 23
The REG Procedure
Model: MODEL1
Dependent Variable: Y
C(p) Selection Method
Number in
Model C(p) R-Square Variables in Model
6 6.6766 0.9829 X2 X3 X4 X1X2 X1X3 X1X4
6 6.6935 0.9829 X1 X2 X4 X1X2 X1X3 X1X4
4 6.7158 0.9707 X2 X3 X1X2 X1X4
4 7.0967 0.9696 X2 X1X2 X1X3 X1X4
4 7.1416 0.9695 X2 X3 X1X2 X1X3
6 7.3069 0.9810 X1 X3 X4 X1X2 X1X3 X1X4
5 7.3790 0.9748 X2 X3 X4 X1X2 X1X3
7 8.0000 0.9849 X1 X2 X3 X4 X1X2 X1X3 X1X4
3 8.0704 0.9607 X4 X1X2 X1X4
5 8.5835 0.9711 X2 X3 X1X2 X1X3 X1X4
4 9.7541 0.9616 X4 X1X2 X1X3 X1X4
2 9.8988 0.9491 X4 X1X2
4 9.9879 0.9609 X3 X4 X1X2 X1X4
3 10.1005 0.9545 X4 X1X2 X1X3
3 11.0583 0.9517 X3 X4 X1X2
4 11.5737 0.9561 X3 X4 X1X2 X1X3
5 11.6746 0.9618 X3 X4 X1X2 X1X3 X1X4
3 11.8668 0.9492 X1 X4 X1X2
4 11.9555 0.9550 X1 X4 X1X2 X1X3
4 12.8843 0.9522 X1 X3 X4 X1X2
5 13.2583 0.9571 X1 X3 X4 X1X2 X1X3
4 14.9833 0.9459 X2 X4 X1X3 X1X4
4 15.3359 0.9448 X2 X3 X4 X1X4
5 15.4005 0.9506 X2 X3 X4 X1X3 X1X4
3 15.5731 0.9381 X2 X4 X1X4
3 18.0609 0.9306 X1 X4 X1X3
4 18.5357 0.9352 X1 X3 X4 X1X3
4 19.1239 0.9334 X1 X4 X1X3 X1X4
5 19.7923 0.9374 X1 X3 X4 X1X3 X1X4
4 19.9162 0.9310 X3 X4 X1X3 X1X4
3 20.3025 0.9238 X1 X4 X1X4
4 20.4424 0.9294 X1 X3 X4 X1X4
3 20.5207 0.9232 X1 X3 X4
2 20.9909 0.9157 X1 X4
3 21.2845 0.9209 X4 X1X3 X1X4
4 21.4970 0.9262 X2 X3 X4 X1X3
3 21.5581 0.9200 X3 X4 X1X4
3 24.4715 0.9113 X3 X4 X1X3
2 24.7370 0.9045 X4 X1X4
3 25.9259 0.9069 X2 X1X3 X1X4
3 26.7706 0.9044 X2 X3 X1X3
4 26.8108 0.9103 X2 X3 X1X3 X1X4
3 29.8851 0.8950 X2 X4 X1X3
2 30.5371 0.8870 X2 X1X3
3 33.1779 0.8851 X1X2 X1X3 X1X4
3 33.4089 0.8844 X2 X3 X1X4
2 33.6739 0.8775 X2 X1X4
4 34.3770 0.8875 X1 X1X2 X1X3 X1X4
3 34.9603 0.8797 X2 X3 X4
Mallows. Cp statistics Regression 24
The REG Procedure
Model: MODEL1
Dependent Variable: Y
C(p) Selection Method
Number in
Model C(p) R-Square Variables in Model
4 35.0728 0.8854 X3 X1X2 X1X3 X1X4
3 35.7390 0.8774 X1 X1X2 X1X3
2 35.9367 0.8707 X4 X1X3
5 36.2456 0.8879 X1 X3 X1X2 X1X3 X1X4
4 36.6364 0.8807 X1 X3 X1X2 X1X3
2 37.8665 0.8649 X1 X1X2
2 38.3831 0.8634 X1X2 X1X4
3 38.3963 0.8694 X1 X1X2 X1X4
3 39.0297 0.8674 X1 X3 X1X2
4 40.1691 0.8700 X1 X3 X1X2 X1X4
3 40.1992 0.8639 X3 X1X2 X1X4
2 41.0539 0.8553 X3 X4
2 43.8124 0.8470 X2 X3
3 47.1803 0.8429 X1 X1X3 X1X4
4 48.9107 0.8437 X1 X3 X1X3 X1X4
3 49.3578 0.8364 X3 X1X2 X1X3
2 51.1709 0.8249 X3 X1X2
2 60.7444 0.7961 X1X2 X1X3
1 61.2741 0.7884 X1X2
2 70.4766 0.7668 X1 X1X4
3 71.6564 0.7692 X1 X3 X1X4
3 81.2685 0.7403 X3 X1X3 X1X4
2 88.2488 0.7132 X1X3 X1X4
3 99.9179 0.6841 X1 X3 X1X3
1 101.8532 0.6663 X2
2 102.6611 0.6699 X1 X1X3
2 103.8384 0.6663 X2 X4
1 104.9217 0.6570 X1X3
2 106.7916 0.6574 X3 X1X3
2 143.0821 0.5482 X1 X3
1 145.8051 0.5339 X1
1 147.2482 0.5296 X4
2 213.0825 0.3374 X3 X1X4
1 228.2064 0.2859 X3
1 300.7848 0.0674 X1X4
Stepwise Regression with No Int. and other Optional Statistics 25
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
Stepwise Selection: Step 1
Variable X2 Entered: R-Square = 0.9614 and C(p) = 675.0274
NOTE: No intercept in model. R-Square is redefined.
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 1 116413 116413 298.80 <.0001
Error 12 4675.14075 389.59506
Uncorrected Total 13 121088
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
X2 1.87679 0.10857 116413 298.80 <.0001
Bounds on condition number: 1, 1
--------------------------------------------------------------------------------
Stepwise Selection: Step 2
Variable X4 Entered: R-Square = 0.9895 and C(p) = 177.8678
NOTE: No intercept in model. R-Square is redefined.
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 2 119815 59907 517.47 <.0001
Error 11 1273.46733 115.76976
Uncorrected Total 13 121088
Stepwise Regression with No Int. and other Optional Statistics 26
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Stepwise Selection: Step 2
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
X2 1.53355 0.08667 36242 313.05 <.0001
X4 0.65759 0.12131 3401.67341 29.38 0.0002
Bounds on condition number: 2.1446, 8.5786
--------------------------------------------------------------------------------
Stepwise Selection: Step 3
Variable X1 Entered: R-Square = 0.9962 and C(p) = 59.6483
NOTE: No intercept in model. R-Square is redefined.
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 3 120634 40211 885.33 <.0001
Error 10 454.19509 45.41951
Uncorrected Total 13 121088
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
X1 1.44826 0.34100 819.27225 18.04 0.0017
X2 1.35129 0.06920 17319 381.31 <.0001
X4 0.59354 0.07747 2666.31355 58.70 <.0001
Bounds on condition number: 3.4846, 25.889
--------------------------------------------------------------------------------
Stepwise Selection: Step 4
Variable X3 Entered: R-Square = 0.9984 and C(p) = 22.6279
NOTE: No intercept in model. R-Square is redefined.
Stepwise Regression with No Int. and other Optional Statistics 27
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Stepwise Selection: Step 4
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 120900 30225 1444.80 <.0001
Error 9 188.27873 20.91986
Uncorrected Total 13 121088
Parameter Standard
Variable Estimate Error Type II SS F Value Pr > F
X1 2.36137 0.34518 979.01317 46.80 <.0001
X2 1.08834 0.08744 3241.02438 154.93 <.0001
X3 1.01494 0.28467 265.91636 12.71 0.0061
X4 0.41313 0.07297 670.58841 32.06 0.0003
Bounds on condition number: 12.079, 126.97
--------------------------------------------------------------------------------
All variables left in the model are significant at the 0.1500 level.
No other variable met the 0.1500 significance level for entry into the model.
NOTE: No intercept in model. R-Square is redefined.
Summary of Stepwise Selection
Variable Variable Number Partial Model
Step Entered Removed Vars In R-Square R-Square C(p) F Value Pr > F
1 X2 1 0.9614 0.9614 675.027 298.80 <.0001
2 X4 2 0.0281 0.9895 177.868 29.38 0.0002
3 X1 3 0.0068 0.9962 59.6483 18.04 0.0017
4 X3 4 0.0022 0.9984 22.6279 12.71 0.0061
Stepwise Regression with No Int. and other Optional Statistics 28
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Number of Observations Read 15
Number of Observations Used 13
Number of Observations with Missing Values 2
NOTE: No intercept in model. R-Square is redefined.
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 4 120900 30225 1444.80 <.0001
Error 9 188.27873 20.91986
Uncorrected Total 13 121088
Root MSE 4.57382 R-Square 0.9984
Dependent Mean 95.42308 Adj R-Sq 0.9978
Coeff Var 4.79320
Parameter Estimates
Parameter Standard
Variable DF Estimate Error t Value Pr > |t| Type I SS
X1 1 2.36137 0.34518 6.84 <.0001 88359
X2 1 1.08834 0.08744 12.45 <.0001 29608
X3 1 1.01494 0.28467 3.57 0.0061 2261.64150
X4 1 0.41313 0.07297 5.66 0.0003 670.58841
Parameter Estimates
Standardized
Variable DF Type II SS Estimate 95% Confidence Limits
X1 1 979.01317 0.22902 1.58051 3.14222
X2 1 3241.02438 0.56859 0.89054 1.28614
X3 1 265.91636 0.13967 0.37097 1.65892
X4 1 670.58841 0.15421 0.24807 0.57820
Covariance of Estimates
Variable X1 X2 X3 X4
X1 0.1191509082 -0.025629347 0.0729084214 -0.015328061
X2 -0.025629347 0.0076454602 -0.020996144 0.0026420658
X3 0.0729084214 -0.020996144 0.0810396971 -0.014404936
X4 -0.015328061 0.0026420658 -0.014404936 0.0053245835
Stepwise Regression with No Int. and other Optional Statistics 29
The REG Procedure
Model: MODEL1
Dependent Variable: Y
Correlation of Estimates
Variable X1 X2 X3 X4
X1 1.0000 -0.8492 0.7420 -0.6085
X2 -0.8492 1.0000 -0.8435 0.4141
X3 0.7420 -0.8435 1.0000 -0.6935
X4 -0.6085 0.4141 -0.6935 1.0000