Plans and status for the lectures in STK2130 Spring 2013

Below are given an overview over the status so far and the plans for the future lectures.
All references are to chapter and sections in the textbook (Ross: Introduction to Probability models, 10th edition).

Status of lectures so far:

• Week 3:
  
  • Introduction to the course
    
  • Stochastic processes, Section 2.9 (excluding Example 2.53)
  • Started on Markov chains, Chapter 4
    • 4.1 Intro
    • 4.2 Chapman-Kolmogorov equations (except Examples 4.10, 4.11 and 4.13)
    • A couple of simple R-scripts for Example 4.8 and Example 4.9
• Week 4:
  
  • Markov chains, Chapter 4
    • 4.3 Classification of states (excluding last part 4.3 from random walk in 2 dimensions, page 209-214)
    • 4.4 Limiting probabilities until pg. 216
    • R-scripts used in the lectures for one-dimensional random walks, two-dimensional random walks and for the probability of visiting a state
• Week 5:
  
  • Remainder of Section 4.4 Limiting propabilities
  • Here is a recipe for calculating the stationary distribution using matrices

• Week 6:
  
  • Section 4.5.1 (4.5.2 and 4.5.3 are not part of curriculum)
  • Section 4.6
  • Section 4.7
• Week 7, Monday 11/2:
  
  • I discussed Law of large number to some more details than out textbook (but this comments will not turn up on the exam).
  • I also discussed the law of large number with irreducible and ergodic Markov chains, and tested this out based on some simulations. This correspond to results in Ross' book in Section 4.4, particularly that the limit of the proportion of time the MC visits a state converge to the limit probabilities and Proposition 4.3.
  • Finally simulation and Monte Carlo approximation to numerically evaluate intergrals were discussed again using simulations.
  • We started on Section 4.9 and introduced the Hastings-Metropolis algorithm.

• Week 8
  
  • MCMC-lecture
  • Exponential distribution (Section 5.2) (not all examples and results, roughly we covered Pg. 292-295 until Example 5.4, pg. 299-300 about hazard rates and 5.2.3 "Further properties .." until Example 5.7)

• Week 9
  
  • 5.3 Poisson processes until 5.3.4 "Further properties of the Poisson Process"

• Week 10:
  
  • Poisson processes, chapter 5 cont.: 5.3.4 "Further properties of the Poisson Process"until Example 5.17, 5.3.5 "Conditional distribution of the arrival time" until Proposition 5.3
  • Start on 5.4.1 Non-homogeneous Poisson processes

• Week 11:
  
  • The remaining part of chapter 5: Proposition 5.3, more on Non-homogeneous Poisson-processes, 5.4.2 Compound Poisson processes.
  • Chapter 7, Section 7.1 and 7.2: Renewal processes.

• Week 14 (Thursday 4/4)-Week 14:
  
  • Start on Chapter 6: Continuous time Markov chains
  • We covered Pg. 370-377 (plus a few other things that we will get back to)

• Week 15 (Lecture only on Thursday 11/4):
  
  • Continuation Chapter 6: Remainder of 6.3

• Week 16 (15-18/4):
  
  • Continuation Chapter 6
    • Section 6.4

• Week 17 (22-25/4):
  
  • Sections 6.5 and 6.8
  • Here is a script for computing P(t)=exp(Rt)