Module: Markov Chains

Learning Outcomes

Markov Chains

• You understand Understand the definition of a Markov chain, the Markov property, and the stationary property.
• You know how to draw the transition probability graph and write the transition probability matrix.
• You understand recurrent and transient states in a Markov chain.
• You know how to compute the steady-state distribution of a Markov chain.

Readings