Description
Book SynopsisA fascinating and instructive guide to Markov chains for experienced users and newcomers alike This unique guide to Markov chains approaches the subject along the four convergent lines of mathematics, implementation, simulation, and experimentation.
Table of ContentsAbstract ix
Preface xi
Acknowledgments xiii
About the CompanionWebsite xv
1 Historical Notes 1
1.1 Introduction 1
1.2 On theWings of Dependent Variables 2
1.3 From Bernoulli to Markov 5
2 FromObservation to Simulation 9
2.1 Introduction 9
2.2 Stochastic Matrices 9
2.3 Transition Probabilities 11
2.4 The Simulation of a Two-State Markov Chain 14
3 Building the Stochastic Matrix 25
3.1 Introduction 25
3.2 Building a Stochastic Matrix from Events 25
3.3 Building a Stochastic Matrix from Percentages 32
4 Predictions Using Two-State Markov Chains 37
4.1 Introduction 37
4.2 Performing the Predictions by Using the Stochastic Matrix 37
4.3 The Steady State of a Markov Chain 46
4.4 The Long-Run Distribution of a Markov Chain 55
5 Predictions Using n-State Markov Chains 61
5.1 Introduction 61
5.2 Predictions by Using the Three-State Markov Chain 61
5.3 Predictions by Using the Four-State Markov Chain 71
5.4 Predictions by Using n-State Markov Chains 80
5.5 Markov Chain Modeling on Measurements 84
6 AbsorbingMarkov Chains 93
6.1 Introduction 93
6.2 The Absorbing State 93
7 The Average Time Spent in Each State 99
7.1 Introduction 99
7.2 The Proportion of Balls in the System 99
7.3 The Average Time Spent in A Particular State 100
7.4 Exemplification of the Average Time and Proportions 101
8 Discussions on Different Configurations of Chains 107
8.1 Introduction 107
8.2 Examples of Two-State Diagrams 113
8.3 Examples of Three-State Diagrams 115
8.4 Examples of Four-State Diagrams 117
8.5 Examples of State Diagrams Divided into Classes 123
8.6 Examples of State Diagrams with Absorbing States 127
8.7 The Gambler’s Ruin 128
9 The Simulation of an n-State Markov Chain 131
9.1 Introduction 131
9.2 The Simulation of Behavior 131
9.3 Simulation of Different Chain Configurations 145
A Supporting Algorithms in PHP 165
B Supporting Algorithms in Javascript 193
C Syntax Equivalence between Languages 223
Glossary 225
References 227
Index 231
