Textbook: Hastie, Trevor, Tibshirani, Robert, Friedman, Jerome: The Elements of Statistical Learning, 2009. 2nd edition, Springer.
Curriculum:
- Chapter 1: Introduction
- Chapter 2: Overview of Supervised Learning
- 2.1 Introduction
- 2.2 Variable Types and Terminology
- 2.3 Two Simple Approaches to Prediction: Least Squares and Nearest Neighbors
- 2.4 Statistical Decision Theory
- 2.5 Local Methods in High Dimensions
- 2.6 Statistical Models, Supervised Learning and Function Approximation
- 2.7 Structured Regression Models
- 2.8 Classes of Restricted Estimators
- 2.9 Model Selection and the Bias–Variance Tradeoff
- Chapter 3: Linear Methods for Regression
- 3.1 Introduction
- 3.2 Linear Regression Models and Least Squares
- 3.3 Subset Selection
- 3.4 Shrinkage Methods
- 3.5 Methods Using Derived Input Directions
- 3.6 Discussion: A Comparison of the Selection and Shrinkage Methods
- 3.8 More on the Lasso and Related Path Algorithms
- 3.9 Computational Considerations
- Chapter 4: Linear Methods for Classification
- 4.1 Introduction
- 4.2 Linear Regression of an Indicator Matrix
- 4.3 Linear Discriminant Analysis
- 4.4 Logistic Regression
- Chapter 7: Model Assessment and Selection
- 7.1 Introduction
- 7.2 Bias, Variance and Model Complexity
- 7.3 The Bias–Variance Decomposition
- 7.4 Optimism of the Training Error Rate
- 7.5 Estimates of In-Sample Prediction Error
- 7.6 The Effective Number of Parameters
- 7.7 The Bayesian Approach and BIC
- 7.10 Cross-Validation
- 7.11 Bootstrap Methods
- Chapter 9: Additive Models, Trees, and Related Methods
- 9.1 Generalized Additive Models
- 9.2 Tree-Based Methods
- Chapter 10: Boosting and Additive Trees
- 10.1 Boosting Methods
- 10.2 Boosting Fits an Additive Model
- 10.3 Forward Stagewise Additive Modeling
- 10.4 Exponential Loss and AdaBoost
- 10.5 Why Exponential Loss?
- 10.6 Loss Functions and Robustness
- 10.9 Boosting Trees
- 10.10 Numerical Optimization via Gradient Boosting