STK4030 - Autumn 2017

Lectures

Lecture 1:
- Ch. 1 - Introduction
- Ch. 2 - Overview of Supervised Learning
  - 2.1 Introduction
  - 2.2 Variable Types and Terminology
  - 2.3 Two Simple Approaches to Prediction: Least Squares and Nearest Neighbors
  - 2.4 Statistical Decision Theory
  - 2.5 Local Methods in High Dimensions
Lecture 2:
- Ch. 2 - Overview of Supervised Learning
  - 2.6 Statistical Models, Supervised Learning and Function Approximation
  - 2.7 Structured Regression Models
  - 2.8 Classes of Restricted Estimators (only 2.8.1)
  - 2.9 Model Selection and the Bias–Variance Tradeoff
- Ch. 3 - Linear Methods for Regression
  - 3.1 Introduction
  - 3.2 Linear Regression Models and Least Squares (including 3.2.1: Example: Prostate Cancer)
Lecture 3:
- code to solve exercise 2.8
- Ch. 3 - Linear Methods for Regression
  - 3.2 Linear Regression Models and Least Squares (from Gauss-Markov Theorem)
  - 3.3 Subset selection (including 3.3.4 Prostate Cancer Data Example)
Lecture 4:
- Ch. 3 - Linear Methods for Regression
  - addition to 3.3 Subset selection
  - 3.4 Shrinkage methods
- Ch. 18 - High-Dimensional Problems: p>>N
  - 18.4 Linear classifier with L_1 regularization (until 18.4.1, excluded)
Lecture 5:
- Ch. 3 - Linear Methods for Regression
  - 3.5 Methods Using Derived Input Directions
  - 3.6 A Comparison of the Selection and Shrinkage Methods
  - 3.8 More on the Lasso and Related Path Algorithms (excluding 3.8.2, 3.8.3 and 3.8.6)
  - 3.9 Computational Considerations
Lecture 6:
- code to reproduce Table 3.3
- Ch. 4 - Linear Methods for Classification
  - 4.1 Introduction
  - 4.2 Linear Regression of an Indicator Matrix
  - 4.3 Linear Discriminant Analysis
Lecture 7:
- Ch 4 - Linear Methods for Classification
  - 4.3 Linear Discriminant Analysis
  - 4.4 Logistic Regression
Lecture 8:
- code for the exercise
- Ch. 7 - Model Assessment and Selection
  - 7.1 Introduction
  - 7.2 Bias, Variance and Model Complexity
  - 7.3 The Bias-Variance Decomposition
Lecture 9:
- Ch. 7 - Model Assessment and Selection
  - 7.4 Optimism of the Training Error Rate
  - 7.5 Estimates of In-Sample Prediction Error
  - 7.6 The Effective Number of Parameters
  - 7.7 The Bayesian Approach and BIC
  - 7.10 Cross-Validation (including 7.10.2)
Lecture 10:
- Ch. 7 - Model Assessment and Selection
  - 7.11 Bootstrap Methods
- Ch. 9 - Additive Models, Trees, and Related Methods
  - 9.1 Generalized Additive Models
- Ch. 5 - Basis Expansions and Regularization
  - 5.2 Piecewise Polynomials and Splines
- Ch. 6 - Kernal Smoothing Methods
  - 6.1 One-Dimensional Kernel Smoothers
Lecture 11:
- code to solve exercise 7.9
- Ch. 8 - Model Inference and Averaging
  - 8.2.2 Maximum Likelihood Inference
- Ch. 10 - Boosting and Additive Tree
  - 10.1 Boosting Methods
  - 10.2 Boosting Fits an Additive Model
  - 10.3 Forward Stagewise Additive Modeling
  - 10.4 Exponential Loss and AdaBoost
  - 10.5 Why Exponential Loss
Lecture 12: (NB: the topics below will be treated by considering L2Boost instead of trees)
- code to solve exercise 10.4
- Ch. 10 - Boosting and Additive Tree
  - Using boosting in R
  - 10.14 Illustration
  - 10.13 Intrerpretation
  - 10.12 Regularization
  - 10.11 Right-Sized Trees for Boosting
  - 10.10 Numerical Optimization via Gradient Boosting
Lecture 13:
- statistical learning: L2Boost and likelihood-based boosting
- Ch. 16 - Boosting and Regularization Paths
  - 16.2 Boosting and Regularization Paths
- Ch. 18 - High-Dimensional Problems: p>>N
  - 18.1 When p is Much Bigger than N
- overview on course topics
Lecture 14:
- correction of the project part
- exercises with previous exams

Published Aug. 21, 2017 1:18 PM - Last modified Feb. 7, 2020 4:14 PM