Markov chain

cosmos 20th April 2017 at 4:36pm
Markov process

Markov process with a discrete state space. More generally it's a chain (a Directed network, where every node has out-degree and in-degree 1, forming a chain) of Random variables that satisfy a Markov property.

Markov Chains explained visually. Nice simulator online

Can have:

Definition of Markov Chain

Random walk on directed graph

Ergodic theorem for Markov chains

Has applications in theory of Stochastic processes, and in Machine learning. In particular through a Hidden Markov model

See also Finite state channel

Order of a Markov chain. See here

Markov subchains A subchain of a Markov chain is also a Markov chain

Regular Markov chain here here

See book Markov chains by Norris

Presentation