Notes on Math 632: Introduction to Stochastic Processes @ University of Wisconsin-Madison

Your comments and criticism are greatly welcomed.

# Resources

Textbook: Essentials of stochastic processes by Rick Durrett

Homeworks: HW01 HW02 HW03 HW04 HW05 HW06 HW07 HW08

# Lecture Notes (PDF version)

Week 1 | |
---|---|

9/6 | Review, Introduction to Stochastic Processes |

Week 2 | |

9/11 | Introduction to Markov Chain |

9/13 | Simple Random Walk, P^n, Gambler’s Ruin |

Week 3 | |

9/18 | T, Strong Markov Property, T_y, ρ_yy, Recurrence |

9/20 | Recurrence, Closed, Irreducible, Communication |

Week 4 | |

9/24 | Theorems Related to Recurrence |

9/27 | Stationary Distribution/Measure, Renewal Chain |

Week 5 | |

10/2 | Positive/Null Recurrent, Limit Behavior |

10/4 | Periodicity, Limiting Behavior |

Week 6 | |

10/9 | Convergence Theorem |

10/11 | Midterm 1 |

Week 7 | |

10/16 | Doubly Stochastic, Detailed Balance |

10/18 | Exit Distribution |

Week 8 | |

10/23 | Exit Time |

10/25 | Probability Review for Poisson Process |

Week 9 | |

10/30 | Introduction to Poisson Process |

11/1 | Compound Poisson Process |

Week 10 | |

11/6 | Thinning, Superposition, and Conditioning |

11/8 | Poisson Process Comprehensive Problems |

Week 11 | |

11/13 | More Exercises on Poisson Process |

11/15 | Midterm 2 |

Week 12 | |

11/20 | Introduction to Renewal Process |

11/22 | Thanksgiving |

Week 13 | |

11/27 | Renewal Process, Age and Residual Life |

11/29 | Continuous Time Markov Processes |

Week 14 | |

12/4 | M/M/s Queue, Kolmogorov Equations |

12/6 | Properties of CTMC |

Week 15 | |

12/11 | CTMC Exercises |

Note: You can download a PDF version of the notes here