Four Training Sets

Fitting a simple model with just an intercept

This simulation generates 1000 training sets each with 40 observations which are subsequently fit with a simple model that includes only an intercept. The black line is the true function, the red line is the average of the 1000 fitted simple models consisting of only an intercept. The dashed pink lines are the each of the 1000 fitted models.

\(\widehat{\text{Bias}}(f_{\hat{\beta}}(x_0 = 8)) = f(x_0=8) - f_{\bar{\beta}}(x_0 = 8) = 0.9893582 - 0.2996897 = 0.6896686\).

\(\widehat{\text{Var}}(f_{\hat{\beta}}(x_0 = 8)) = \text{Var}(\hat{f}) = 0.0170668.\)

\(\widehat{\text{MSE}} = \widehat{\text{Bias}}^2 + \widehat{\text{Var}} + \text{Var}(\varepsilon)= 0.4756428 + 0.0170668 + 0.25= 0.7427095\)

The average training MSE for this model is 0.6547499, and the average testing MSE for this model is 0.6678183.

Fitting a straight line model

This simulation generates 1000 training sets each with 40 observations which are subsequently fit with a simple straight line model. The black line is the true function, the red line is the average of the 1000 fitted models. The dashed pink lines are the each of the 1000 fitted models.

\(\widehat{\text{Bias}}(f_{\hat{\beta}}(x_0 = 8)) = f(x_0=8) - f_{\bar{\beta}}(x_0 = 8) = 0.9893582 - 0.8058583 = 0.1835\).

\(\widehat{\text{Var}}(f_{\hat{\beta}}(x_0 = 8)) = \text{Var}(\hat{f}) = 0.0205799.\)

\(\widehat{\text{MSE}} = \widehat{\text{Bias}}^2 + \widehat{\text{Var}} + \text{Var}(\varepsilon) = 0.0336722 + 0.0205799 + 0.25 = 0.3042521\)

The average training MSE for this model is 0.329643, and the average testing MSE for this model is 0.3551832.

Fitting a second order polynomial model

This simulation generates 1000 training sets each with 40 observations which are subsequently fit with a second order polynomial model. The black line is the true function, the red line is the average of the 1000 fitted models. The dashed pink lines are the each of the 1000 fitted models.

\(\widehat{\text{Bias}}(f_{\hat{\beta}}(x_0 = 8)) = f(x_0=8) - f_{\bar{\beta}}(x_0 = 8) = 0.9893582 - 0.8733845 = 0.1159737\).

\(\widehat{\text{Var}}(f_{\hat{\beta}}(x_0 = 8)) = \text{Var}(\hat{f}) = 0.0135464.\)

\(\widehat{\text{MSE}} = \widehat{\text{Bias}}^2 + \widehat{\text{Var}} + \text{Var}(\varepsilon) = 0.0134499 + 0.0135464 + 0.25 = 0.2769963\)

The average training MSE for this model is 0.2434497, and the average testing MSE for this model is 0.2810794.

Fitting a third order polynomial model

This simulation generates 1000 training sets each with 40 observations which are subsequently fit with third order polynomial model. The black line is the true function, the red line is the average of the 1000 fitted models. The dashed pink lines are the each of the 1000 fitted models.

\(\widehat{\text{Bias}}(f_{\hat{\beta}}(x_0 = 8)) = f(x_0=8) - f_{\bar{\beta}}(x_0 = 8) = 0.9893582 - 1.017832 = -0.0284737\).

\(\widehat{\text{Var}}(f_{\hat{\beta}}(x_0 = 8)) = \text{Var}(\hat{f}) = 0.0234942.\)

\(\widehat{\text{MSE}} = \widehat{\text{Bias}}^2 + \widehat{\text{Var}} + \text{Var}(\varepsilon) = 8.1075227\times 10^{-4} + 0.0234942 + 0.25 = 0.274305\)

The average training MSE for this model is 0.2266852, and the average testing MSE for this model is 0.2739072.

Fitting a fifth order polynomial model

This simulation generates 1000 training sets each with 40 observations which are subsequently fit with a fifth order polynomial model. The black line is the true function, the red line is the average of the 1000 fitted models. The dashed pink lines are the each of the 1000 fitted models.

\(\widehat{\text{Bias}}(f_{\hat{\beta}}(x_0 = 8)) = f(x_0=8) - f_{\bar{\beta}}(x_0 = 8) = 0.9893582 - 0.9873069 = 0.0020514\).

\(\widehat{\text{Var}}(f_{\hat{\beta}}(x_0 = 8)) = \text{Var}(\hat{f}) = 0.0302238.\)

\(\widehat{\text{MSE}} = \widehat{\text{Bias}}^2 + \widehat{\text{Var}} + \text{Var}(\varepsilon) = 4.2080486\times 10^{-6} + 0.0302238 + 0.25 = 0.280228\)

The average training MSE for this model is 0.2140156, and the average testing MSE for this model is 0.2949161.

Fitting a tenth order polynomial model

This simulation generates 1000 training sets each with 40 observations which are subsequently fit with a tenth order polynomial model. The black line is the true function, the red line is the average of the 1000 fitted models. The dashed pink lines are the each of the 1000 fitted models.

\(\widehat{\text{Bias}}(f_{\hat{\beta}}(x_0 = 8)) = f(x_0=8) - f_{\bar{\beta}}(x_0 = 8) = 0.9893582 - 0.9956441 = -0.0062858\).

\(\widehat{\text{Var}}(f_{\hat{\beta}}(x_0 = 8)) = \text{Var}(\hat{f}) = 0.0741528.\)

\(\widehat{\text{MSE}} = \widehat{\text{Bias}}^2 + \widehat{\text{Var}} + \text{Var}(\varepsilon) = 3.9511863\times 10^{-5} + 0.0741528 + 0.25 = 0.3241923\)

The average training MSE for this model is 0.1820414, and the average testing MSE for this model is 0.3195581.

Bias Variance Tradeoff