本书是图灵原版数学统计学系列之一，主要内容有随机变量、条件概率及条件期望、离散及连续马尔可夫链、指数分布、泊松过程、布朗运动及平稳过程、更新理论及排队论等；也包括了随机过程在物理、生物、运筹、网络、遗传、经济、保险、金融及可靠性中的应用。特别是有关随机模拟的内容，给随机系统运行的模拟计算提供了有力的工具。除正文外，本书有约700道习题，其中带星号的习题还提供了解答。

本书可作为概率论与统计、计算机科学、保险学、物理学、社会科学、生命科学、管理科学与工程学等专业随机过程基础课教材。

本书叙述深入浅出，涉及面广。主要内容有随机变量、条件概率及条件期望、离散及连续马尔可夫链、指数分布、泊松过程、布朗运动及平稳过程、更新理论及排队论等；也包括了随机过程在物理、生物、运筹、网络、遗传、经济、保险、金融及可靠性中的应用。特别是有关随机模拟的内容，给随机系统运行的模拟计算提供了有力的工具。除正文外，本书有约700道习题，其中带星号的习题还提供了解答。

本书可作为概率论与统计、计算机科学、保险学、物理学、社会科学、生命科学、管理科学与工程学等专业随机过程基础课教材。

1. Introduction to Probability Theory 1

1.1. Introduction 1

1.2. Sample Space and Events 1

1.3. Probabilities Defined on Events 4

1.4. Conditional Probabilities 7

1.5. Independent Events 10

1.6. Bayes' Formula 12

Exercises 15

References 21

2. Random Variables 23

2.1. Random Variables 23

2.2. Discrete Random Variables 27

2.2.1. The Bernoulli Random Variable 28

2.2.2. The Binomial Random Variable 29

2.2.3. The Geometric Random Variable 31

2.2.4. The Poisson Random Variable 32

2.3. Continuous Random Variables 34

2.3.1. The Uniform Random Variable 35

2.3.2. Exponential Random Variables 36

2.3.3. Gamma Random Variables 37

2.3.4. Normal Random Variables 37

2.4. Expectation of a Random Variable 38

2.4.1. The Discrete Case 38

2.4.2. The Continuous Case 41

2.4.3. Expectation of a Function of a Random Variable 43

2.5. Jointly Distributed Random Variables 47

2.5.1. Joint Distribution Functions 47

2.5.2. Independent Random Variables 51

2.5.3. Covariance and Variance of Sums of Random Variables 53

2.5.4. Joint Probability Distribution of Functions of Random Variables 61

2.6. Moment Generating Functions 64

2.6.1. The Joint Distribution of the Sample Mean and Sample Variance from a Normal Population 74

2.7. Limit Theorems 77

2.8. Stochastic Processes 83

Exercises 85

References 96

3. Conditional Probability and Conditional Expectation 97

3.1. Introduction 97

3.2. The Discrete Case 97

3.3. The Continuous Case 102

3.4. Computing Expectations by Conditioning 105

3.4.1. Computing Variances by Conditioning 117

3.5. Computing Probabilities by Conditioning 120

3.6. Some Applications 137

3.6.1. A List Model 137

3.6.2. A Random Graph 139

3.6.3. Uniform Priors, Polya's Urn Model, and Bose-Einstein Statistics 147

3.6.4. Mean Time for Patterns 151

3.6.5. The k-Record Values of Discrete Random Variables 155

3.7. An Identity for Compound Random Variables 158

3.7.1. Poisson Compounding Distribution 161

3.7.2. Binomial Compounding Distribution 163

3.7.3. A Compounding Distribution Related to the Negative Binomial 164

Exercises 165

4. Markov Chains 185

4.1. Introduction 185

4.2. Chapman-Kolmogorov Equations 189

4.3. Classification of States 193

4.4. Limiting Probabilities 204

4.5. Some Applications 217

4.5.1. The Gambler's Ruin Problem 217

4.5.2. A Model for Algorithmic Efficiency 221

4.5.3. Using a Random Walk to Analyze a Probabilistic Algorithm for the Satisfiability Problem 224

4.6. Mean Time Spent in Transient States 230

4.7. Branching Processes 233

4.8. Time Reversible Markov Chains 236

4.9. Markov Chain Monte Carlo Methods 247

4.10. Markov Decision Processes 252

4.11. Hidden Markov Chains 256

4.11.1. Predicting the States 261

Exercises 263

References 280

5. The Exponential Distribution and the Poisson Process 281

5.1. Introduction 281

5.2. The Exponential Distribution 282

5.2.1. Definition 282

5.2.2. Properties of the Exponential Distribution 284

5.2.3. Further Properties of the Exponential Distribution 291

5.2.4. Convolutions of Exponential Random Variables 298

5.3. The Poisson Process 302

5.3.1. Counting Processes 302

5.3.2. Definition of the Poisson Process 304

5.3.3. Interarrival and Waiting Time Distributions 307

5.3.4. Further Properties of Poisson Processes 310

5.3.5. Conditional Distribution of the Arrival Times 316

5.3.6. Estimating Software Reliability 328

5.4. Generalizations of the Poisson Process 330

5.4.1. Nonhomogeneous PoissonProcess 330

5.4.2. Compound Poisson Process 337

5.4.3. Conditional or Mixed Poisson Processes 343

Exercises 346

References 364

6. Continuous-Time Markov Chains 365

6.1. Introduction 365

6.2. Continuous-Time Markov Chains 366

6.3. Birth and Death Processes 368

6.4. The Transition Probability Function eij (t) 375

6.5. Limiting Probabilities 384

6.6. Time Reversibility 392

6.7. Uniformization 401

6.8. Computing the Transition Probabilities 404

Exercises 407

References 415

7. Renewal Theory and Its Applications 417

7.1. Introduction 417

7.2. Distribution of N(t) 419

7.3. Limit Theorems and Their Applications 423

7.4. RenewalReward Processes 433

7.5. Regenerative Processes 442

7.5.1. Alternating Renewal Processes 445

7.6. Semi-Markov Processes 452

7.7. The Inspection Paradox 455

7.8. Computing the Renewal Function 458

7.9. Applications to Patterns 461

7.9.1. Patterns of Discrete Random Variables 462

7.9.2. The Expected Time to a Maximal Run of Distinct Values 469

7.9.3. Increasing Runs of Continuous Random Variables 471

7.10. The Insurance Ruin Problem 473

Exercises 479

References 492

8. Queueing Theory 493

8.1. Introduction 493

8.2. Preliminaries 494

8.2.1. Cost Equations 495

8.2.2. Steady-State Probabilities 496

8.3. Exponential Models 499

8.3.1. A Single-Server Exponential Queueing System 499

8.3.2. A Single-Server Exponential Queueing System Having Finite Capacity 508

8.3.3. A Shoeshine Shop 511

8.3.4. A Queueing System with Bulk Service 514

8.4. Network of Queues 517

8.4.1. Open Systems 517

8.4.2. Closed Systems 522

8.5. The System M/G/1 528

8.5.1. Preliminaries: Work and Another Cost Identity 528

8.5.2. Application of Work to M/G/1 529

8.5.3. Busy Periods 530

8.6. Variations on the M/G/1 531

8.6.1. The M/G/1 with Random-Sized Batch Arrivals 531

8.6.2. Priority Queues 533

8.6.3. An M/G/10ptimizationExample 536

8.6.4. The M/G/1 Queue with Server Breakdown 540

8.7. The Model G/M/1 543

8.7.1. The G/M/1 Busy and Idle Periods 548

8.8. A Finite Source Model 549

8.9. Multiserver Queues 552

8.9.1. Erlang's Loss System 553

8.9.2. The M/M/k Queue 554

8.9.3. The G/M/k Queue 554

8.9.4. The M/G/k Queue 556

Exercises 558

References 570

9. Reliability Theory 571

9.1. Introduction 571

9.2. Structure Functions 571

9.2.1. Minimal Path and Minimal Cut Sets 574

9.3. Reliability of Systems of Independent Components 578

9.4. Bounds on the Reliability Function 583

9.4.1. Method of Inclusion and Exclusion 584

9.4.2. Second Method for Obtaining Bounds on r(p) 593

9.5. System Life as a Function of Component Lives 595

9.6. Expected System Lifetime 604

9.6.1. An Upper Bound.on the Expected Life of a Parallel System 608

9.7. Systems with Repair 610

9.7.1. A Series Model with Suspended Animation 615

Exercises 617

References 624

10. Brownian Motion and Stationary Processes 625

10.1. Brownian Motion 625

10.2. Hitting Times, Maximum Variable, and the Gambler's Ruin Problem 629

10.3. Variations on Brownian Motion 631

10.3.1. Brownian Motion with Drift 631

10.3.2. Geometric Brownian Motion 631

10.4. Pricing Stock Options 632

10.4.1. An Example in Options Pricing 632

10.4.2. The Arbitrage Theorem 635

10.4.3. The Black-Scholes Option Pricing Formula 638

10.5. White Noise 644

10.6. GaussianProcesses 646

10.7. Stationary and Weakly Stationary Processes 649

10.8. Harmonic Analysis of Weakly Stationary Processes 654

Exercises 657

References 662

11. Simulation 663

11.1. Introduction 663

11.2. General Techniques for Simulating Continuous Random Variables 668

11.2.1. The Inverse Transformation Method 668

11.2.2. The Rejection Method 669

11.2.3. The Hazard Rate Method 673

11.3. Special Techniques for Simulating Continuous Random Variables 677

11.3.1. The Normal Distribution 677

11.3.2. The Gamma Distribution 680

11.3.3. The Chi-Squared Distribution 681

11.3.4. The Beta (n,m) Distribution 681

11.3.5. The Exponential Distribution The Von Neumann Algorithm 682

11.4. Simulating from Discrete Distributions 685

11.4.1. The Alias Method 688

11.5. Stochastic Processes 692

11.5.1. Simulating a Nonhomogeneous Poisson Process 693

11.5.2. Simulating a Two-Dimensional Poisson Process 700

11.6. Variance Reduction Techniques 703

11.6.1. Use of Antithetic Variables 704

11.6.2. Variance Reduction by Conditioning 708

11.6.3. Control Variates 712

11.6.4. Importance Sampling 714

11.7. Determining the Number of Runs 720

11.8. Coupling from the Past 720

Exercises 723

References 731

Appendix: Solutions to Starred Exercises 733

Index 775