Week 3
- Lecture 1:
- Ch. 2.9: Intro to stochastic processes (excluding example 2.53)
- Start ch. 4.1: Introduction to Markov chains
- Lecture 2:
- Finish ch. 4.1
- Ch. 4.2: Chapman-Kolmogorov equations (excluding pages 193-194, including the Remark on page 194)
- Lecture 3:
- Ch. 4.3: Classification of states (excluding the last part of 4.3 from the last 1/3 of page 199, from random-walk in 2 dimensions)
- Ch. 4.4: Long-run proportions and limiting probabilities
- Lecture 4:
- Finish ch. 4.4 (excluding examples 4.24, 4.25, 4.26)
- Ch. 4.5.1: The gambler's ruin problem
- Lecture 5:
- Ch. 4.6: Mean time spent in transient states
- Ch. 4.7: Branching processes
- Lecture 6:
- Ch. 4.8: Time reversible Markov chains (until example 4.35)
- Ch. 4.9: Markov Chain Monte Carlo Methods (until example 4.39), with R examples
- Lecture 7:
- Ch. 5.2.1: The exponential distribution (excluding example 5.1)
- Ch. 5.2.2: Properties of the exponential distribution (excluding example 5.5)
- Lecture 8:
- Ch. 5.2.3: Further properties of the exponential distribution (excluding examples 5.7, 5.9 and 5.10)
- Ch. 5.2.4: Convolutions of the exponential random variables (excluding example 5.11)
- Lecture 9:
- Ch. 5.3.1: Counting Processes
- Ch. 5.3.2: Definition of the Poisson Process
- Ch. 5.3.3: Interarrival and Waiting Time Distributions
- Lecture 10:
- Ch. 5.3.4: Further Properties of Poisson Processes (until example 5.16)
- Ch. 5.3.5: Conditional Distribution of the Arrival Times
- Lecture 11:
- Finish Ch. 5.3.5 (excluding examples 5.19, 5.20, 5.21, 5.22)
- Start Ch. 5.4.1: Nonhomogeneous Poisson Process
- Lecture 12:
- Finish Ch. 5.4.1
- Ch. 5.4.2: Compound Poisson Processes
- Lecture 13:
- Ch. 6.1: Introduction to Continuous-time Markov chains
- Ch. 6.2: Continuous-time Markov chains
- Ch. 6.3: Birth and Death Processes
- Lecture 14:
- Finish Ch. 6.3
- Ch. 6.4: The Transition Probability Function Pij(t)
- Lecture 15:
- Finish Ch. 6.4
- Lecture 16:
- Ch. 6.5: Limiting Probabilities (excluding Example 6.16)
- Ch. 6.8: Uniformization
- Lecture 17:
- Ch. 6.9: Computing the Transition Probabilities
- Ch. 7.1: Renewal Process
- Ch. 7.2: Distribution of N(t)
- Lecture 18:
- Ch. 10.1: Brownian Motion
- Ch. 10.2: Hitting Times, Maximum Variable, and the Gambler's Ruin Problem
- Ch. 10.3: Variations on Brownian Motion