Week 3
- Wednesday 20 January:
- Ch. 2.9: Intro to stochastic processes (excluding example 2.53)
- Start on ch. 4.1: Intro to Markov Chains
- Thursdag 21 January:
- Finish ch. 4.1
- Ch. 4.2: Start Chapman-Kolmogorov equations, including the Remark on page 194
- Friday 22 January:
- Finish Ch. 4.2 (excluding pages 193-194 until the Remark on page 194)
Week 4
- Wednesday 27 January:
- Start ch. 4.3: Classification of states (excluding the last part of 4.3 from the last 1/3 of page 199, from random-walk in 2 dimensions)
- Thursday 28 January:
- Finish ch 4.3
- Start ch. 4.4: Long-Run proportions and Limiting Probabilities
Week 5
- Wednesday 3 February: Cont. ch 4.4: Long-Run proportions and Limiting Probabilities
- Thursday 4 February: Cont. ch 4.4: Long-Run proportions and Limiting Probabilities
Week 6
- Wednesday 10 February:
- Finish ch 4.4: Long-Run proportions and Limiting Probabilities
- Thursday 11 February:
- Ch 4.5.1: The gambler's ruin problem
- Ch 4.6: Mean time spent in transient states
Week 7
- Wednesday 17 February:
- Finish ch 4.6: Mean time spent in transient states
- Start ch 4.7: Branching processes
- Thursday 18 February:
- ??Finish ch 4.7: Branching processes
- Ch 4.8: Time reversible Markov Chains, until Example 4.35
- Start ch 4.9: Markov Chain Monte Carlo Methods
Week 8
- Wednesday 23 and Thursday 25 February:
- Finish ch 4.9: Markov Chain Monte Carlo Methods, until example 4.39
- Ch 5.1
- Start ch 5.2: The exponential distribution
Week 9
- Wednesday 2 March:
- Continued ch 5.2
- Thursday 3 March:
- Finish ch 5.2
- Start ch 5.3: The Poisson process
Week 10
- Wednesday 9 March:
- Finished 5.3.2: Definition of the Poisson process. NB: We used the Laplace transform to prove Theorem 5.1. A definition of the Laplace transform can be seen in the remark on page 64 in the textbook.
- Thursday 10 March:
- 5.3.3, 5.3.4 and start 5.3.5
Week 13
- Wednesday 30 March:
- Ch 5.3.5 continued
- Thursday 31 March:
- Finishes ch 5.3.5
- Ch 5.4.1: Nonhomogeneous Poisson process
- Started 5.4.2: Compound Poisson process
Week 14
- Wednesday 6 April:
- Finish ch 5.4.2
- Thursday 7 April:
- Started ch 6: Continuous-time Markov Chains
Week 15
- Wednesday 13 and Thursday 14 April:
- Finish 6.3: Birth and death processes
- Ch 6.4: The transition probability function
Week 16
- Wednesday 20 and Thursday 21 April: (note: I made a mistake in the Thursday lecture when deducing the reduced version of condition (6.21) for the M/M/s queuing model. This has been corrected in the published notes)
- Finished ch 6.4
- Ch 6.5: Limiting probabilities
- Started Ch 6.8: Uniformization
Week 17
- Wednesday 27 April:
- Finished ch 6.8
- Ch 6.9: Computing the transition probabilities
- Thursday 28 April:
- Ch 7.1-7.2: Introduction to renewal processes
- Started ch 10.1: Brownian motion
- Friday 29 April:
- Finished ch 10.1
Week 18
- Wednesday 4 May: This will be the final ordinary lecture!
- Ch 10.2: Hitting times, maximum variable, and the gambler's ruin problem
- Ch 10.3: Variations on Brownian motion
Week 19
- No lectures, only exercises and tutorial
Week 20
- No teaching
Week 21
- Thursday 26 May: I will go through some exam exercises:
- Exam 2004: Exercises 2 a)-c) and 3. NB: Note that in this exam pji(t) corresponds to what we know as pij(t), and similarily qij corresponds to what we know as qij. This also means that the matrices P(t), the infintesimal generator matrix Q (R in our textbook) and the matrix H in the exam text must be transposed to correspond to our notation. For exercise 2 c, Q*pi=0 equals RT*pi=0 in our notation, and hence is just (6.18) written in matrix form
- Exam 2013, Exercise 1