Markov Processes for Stochastic Modeling
- This format is currently out of stock.
This book presents an algebraic development of the theory of countable state space Markov chains with discrete and continuous time parameters.
Table of Contents
Introduction: Stochastic processes; the Markov property; some examples; transition probabilities; the strong Markov property; exercises. Discrete-time Markov chains: First time passages; classification of states; recurrent Markov chains; finite Markov chains; time-reversable Markov chains; the rate of convergence to stationary; absorbing Markov chanins and their applications; Lossy Markov chains; exercises. Monotone Markov chains: Preliminaries; distribution classes of interest; stochastic ordering relations; monotone Markov chains; unimodality of transition probabilities; first-passage-time distributions; bounds for quasi-stationary distributions; renewal processes in discrete time; comparability of Markov chains; exercises. Continuous-time Markov chains: transition probability functions; finite Markov chains in continuous time; uniformization; more on finite Markov chains; absorbing Markov chains in continuous time; calculation of transition probability functions; stochastic monotonicity; semi-Markov processes; exercises. Birth-death processes: Boundary classifications; birth-death polynomials; finite birth-death processes; the Karlin-McGregor representation theorem; asymptotics of birth-death polynomials; quasi-stationary distributions; the decay parameter; the M/M/1 queue; exercises. Appendix A Review of matrix theory: Nonnegative matrices; ML-matrices; infinite matrices. Appendix B Generating functions and Laplace transforms: Generating functions; Laplace transforms. Appendix C Total positivity: TP functions; the variation-diminishing property.