Syllabus

Topics Covered

Modes of convergence for random variables
Big-O, little-o notation
MDPs

Renewal processes (RPs)
Renewal theorems
Alternative renewal processes (ARPs)
Regenerative processes
Renewal-reward process
Semi-Markov processes (SMPs)
Phase-type distributions

Random walks
Duality and Wiener-Hopf Factorization
M/M/1, G/M/1 queues

Discrete time martingales
Exponential martingales
Filtrations
Optional stopping

Brownian motion
Donsker's theorem
Stochastic differential equations (SDEs) and partial differential equations (PDEs)

Filtering
Sequential Hypothesis Testing
Markov Chain Monte Carlo
Gibbs sampler
Gaussian processes
Probability inequalities

Projects

The following Wikipedia pages were created or improved as part of students' course projects: Stochastic Programming, Optimal Stopping, Quasi-Monte Carlo, Branching Process, Jackson Network, Uncertainty Quantification, Karhunen-Loeve Theorem