Operations Research/Markov Models

A little bit of history Knowing the background sometimes helps to understand a topic deeply. That's why I'm starting this chapter with a bit of historical reference. In the early 1900s, many probability studies were involved with independent events, like coin tosses, where one event is not related to another. But the real world is different! You can't just assume, "Let's imagine the cat is square-shaped!" Similarly, one event can be interconnected to another. Around 1906, a Russian mathematician, Andrey Andreyevich Markov, approached solving this problem. Later, in 1913, he gave a famous example by analyzing vowel and consonant patterns in Pushkin’s Eugene Onegin. He studied the sequence of vowels and consonants and showed that the probability of the next letter type could depend on the current letter type. This was important because it proved that probability could model patterns with dependency, not only completely random, independent events. There are two different events: (i) totally independent events and (ii) totally dependent events. But Markov chose the middle ground instead of tackling the entirely dependent event chain. He introduced a process. The next event depends on the present event, but not on the entire past history. Jump into theory