What’s Markov Chain
A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.
Transition Matrix
Transition Matrix is also called stochastic matrix. It describes a Markov chain over a finite state space with cardinality
If the probability of moving from to in one time step is , the stochastic matrix is given by using as the -th row and -th column element, e.g.,
Since the total of transition probability from a state to all other states must be 1, so,
Example
An RPG with a 33% blitz rate. But if the first two times you don’t blitz, the third time you’re bound to blitz. So what is the actual hit rate?
Simulation Code:
import numpy as np
import random
from rich.progress import track
def active():
x = random.randint(0, 2)
if x == 0:
return 1
else:
return 0
if __name__ == '__main__':
record_list = []
count = 0
for i in track(range(int(10e6)), description="Simulating..."):
if count == 2:
record_list.append(1)
count = 0
else:
if not active():
record_list.append(0)
count += 1
else:
record_list.append(1)
count = 0
countActive = record_list.count(1)
countInactive = record_list.count(0)
probility = countActive / (countActive + countInactive)
print(f"Probability of being active: {probility}")