Three models of fitting. Consider trying to fit some dataset |D|= n that’s roughly quadratic with… a linear model: underfit, high bias (i.e. “model imposes bias of linearity on data”) a nth order polynomial: overfit, high variance (i.e. “a small perturbation of data brings lots of change”) Its important to pay attention if you are having high bias of high variance—solutions of each is different from each other. intuition of overfitting See overfit diagnosing bias variance tradeoff problems Let’s consider: Reference Training Test Judgment 2% 2% 2% High Bias 10% 2.5% 10% High Variance 10% 10% 20% Both High Bias and Variance