18-11-13 Powerful Concepts in Machine Learning

Category: Idea Lists (Upon Request)

Read the original document

<!-- gdoc-inlined -->

Understand their implications for thought. Their implications for the nature of information itself. The nature of knowledge, and the limitations of knowledge.

1. Bias-Variance Tradeoff
   1. Overfitting
   2. Controlling complexity
      1. Model simplicity (restriction methods)
      2. Selection methods (over features)
      3. Regularization
2. Curse of dimensionality
3. Ensemble Modeling
4. Occam’s Razor (Formalized)
5. Training vs. Generalization Error
6. Interpolation vs. Extrapolation
7. Smoothness
8. VC Dimension
9. Solomonoff Induction
10. Variance Maximization
11. Optimizing for Volatility vs Expected Value
12. Bayes Rule
13. Bayes Error
14. Exploration-Exploitation
15. Manifolds as Data Representation

I would like to make this stuff runnable.

Source: Original Google Doc