FOREWORD
PREFACE
Patrick Billingsley 1925- 2011
Chapter 1
PROBABILITY
1. BOREL'S NORMAL NUMBER THEOREM, 1
The Unit Interval--The Weak Law of Large
Numbers--The Strong Law of Large Numbers--Strong Law
Versus Weak-- Length--The Measure Theory of Diophantine
Approximation*
2. PROBABILITY MEASURES, 18
Spaces --Assigning Probabilities--Classes of Sets--Probability
Measures--Lebesgue Measure on the Unit Interval--Sequence
Space* - Constructing σ-Fields*
3. EXISTENCE AND EXTENSION, 39
Construction of the Extension--Uniqueness and the π-λ
Theorem--Monotone Classes--Lebesgue Measure on the Unit
Interval- Completeness-- Nonmeasurable Sets--Two
Impossibility Theorems*
4. DENUMERABLE PROBABILITIES, 53
General Formulas-- Limit Sets-Independent
Events--Subfields--The Borel-Cantelelli
Lemmas--The Zero-One Law
5. SIMPLE RANDOM VARIABLES, 72
Definition-- Convergence of Random
Variables--Independence--Existence of Independent
Sequences-- Expected Value--Inequalities
6. THE LAW OF LARGE NUMBERS, 90
The Strong Law--The Weak Law--Bernstein's
Theorem--A Refinement of the Second BoreI-Cantelli
Lemma
7. GAMBLING SYSTEMS, 98
Gambler's Ruin--Selection Systems--Gambling Policies--Bold
Play*--Timid Play*
8. MARKOVCHAINS, 117
Definitions-- Higher-Order Transitions --An Existence
Theorem--Transience and Persistence--Another Criterion for
Persistence--Stationary Distributions-- Exponential
Convergence*--Optimal Stopping*
9. LARGE DEVIATIONS AND THE LAW
OF THE ITERATED LOGARITHM, 154
Moment Generating Functions--Large Deviations -- Chernoff's
Theorem*--The Law of the Iterated Logarithm
Chapter 2
MEASURE 167
10. GENERAL MEASURES, 167
Classes of Sets-- Conventions Involving
∞ -- Measures-- Uniqueness
11. OUTER MEASURE, 174
Outer Measure--Extension--An Approximation Theorem
12. MEASURES IN EUCLIDEAN SPACE, 181
Lebesgue Measure--Regularity--Specifying Measures on the
Line--Specifying Measures in Rk-strange Euclidean Sets*
13. MEASURABLE FUNCTIONS AND MAPPINGS, 192
Measurable Mappings-- Mappings into Rk- Limits and
Measurability--Transformations of Measures
14. DISTRIBUTION FUNCTIONS, 198
Distribution Functions--Exponential Distributions--Weak
Convergence-- Convergence of Types* -- Extremal
Distributions*
Chapter 3
INTEGRATION 211
15. THE INTEGRAL, 211
Definition -- Nonnegative Functions-- Uniqueness
16. PROPERTIES OF THE INTEGRAL, 218
Equalities and Inequalities--Integration to the Limit--Integration
over Sets-- Densities-- Change of Variable-- Uniform
Integrability-- Complex Functions
17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234
The Lebesgue Integral on the Line--The Riemann
Integral--The Fundamental Theorem of Calculus--Change of
Variable--The Lebesgue Integral in Rk--Stieltjes Integrals
18. PRODUCT MEASURE AND FUBINI'S THEOREM, 245
Product Spaces-- Product Measure-- Fubini's
Theorem--Integration by Parts-- Products of Higher Order
19. THE Lp SPACES*, 256
Definitions-- Completeness and Separability-- Conjugate
Spaces--Weak Compactness--Some Decision
Theory--The Space L2-An Estimation Problem
Chapter 4
RANDOM VARIABLES AND EXPECTED VALUES 271
20. RANDOM VARIABLES AND DISTRIBUTIONS, 271
Random Variables and Vectors--
Subfields-- Distributions -- Multidimensional
Distributions--Independence--Sequences of Random
Variables--Convolution--Convergence in
Probability--The Glivenko-Cantelli Theorem*
21. EXPECTED VALUES, 291
Expected Value as Integral--Expected Values
and Limits-- Expected Values and
Distributions-- Moments--Inequalities--Joint
Integrals--Independence and Expected Value-- Moment
Generating Functions
22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300
The Strong Law of Large Numbers--The Weak Law
and Moment Generating Functions--Kolmogorov's Zero-One
Law-- Maximal Inequalities-- Convergence of Random
Series--Random Taylor Series*
23. THE POISSON PROCESS, 316
Characterization of the Exponential Distribution--The Poisson
Process--The Poisson Approximation--Other Characterizations
of the Poisson Process--Stochastic
Processes
24. THE ERGODIC THEOREM*, 330
Measure-Preserving Transformations-- Ergodicity-- Ergodicity of
Rotations--Proof of the Ergodic Theorem--The
Continued-Fraction Transformation-- Diophantine
Approximation
Chapter 5
CONVERGENCE OF DISTRIBUTIONS 349
25. WEAK CONVERGENCE, 349
Definitions-- Uniform Distribution Modulo 1 * --Convergence
in Distribution--Convergence in Probability--Fundamental
Theorems--Helly's Theorem--Integration to the Limit
26. CHARACTERISTIC FUNCTIONS, 365
Definition -- Moments and Derivatives--
Independence--Inversion and the Uniqueness Theorem--The
Continuity Theorem-- Fourier Series*
27. THE CENTRAL LIMIT THEOREM, 380
Identically Distributed Summands--The Lindeberg
and Lyapounov Theorems--Dependent Variables*
28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394
Vague Convergence--The Possible Limits--Characterizing
the Limit
29. LIMIT THEOREMS IN Rk, 402
The Basic Theorems-- Characteristic Functions-- Normal
Distributions in Rk--The Central Limit Theorem
30. THE METHOD OF MOMENTS*, 412
The Moment Problem--Moment Generating Functions--Central
Limit Theorem by Moments--Application
to Sampling Theory--Application to Number Theory
Chapter 6
DERIVATIVES AND CONDITIONAL PROBABILITY 425
31. DERIVATIVES ON THE LINE*, 425
The Fundamental Theorem of Calculus--Derivatives
of Integrals--Singular Functions--Integrals
of Derivatives--Functions of Bounded Variation
32. THE RADON-NIKODYM THEOREM, 446
Additive Set Functions--The Hahn Decomposition--Absolute
Continuity and Singularity--The Main Theorem
33. CONDITIONAL PROBABILITY, 454
The Discrete Case--The General Case--Properties
of Conditional Probability-- Difficulties and
Curiosities-- Conditional Probability Distributions
34. CONDITIONAL EXPECTATION, 472
Definition-- Properties of Conditional Expectation--Conditional
Distributions and Expectations-- Sufficient
Subfields* -- Minimum-Variance Estimation*
35. MARTINGALES, 487
Definition -- Su bmartingales-- Gambling -- Functions
of Martingales-- Stopping Times-- Inequalities-- Convergence
Theorems--Applications: Derivatives-- Likelihood
Ratios-- Reversed Martingales--Applications: de Finetti's
Theorem--Bayes Estimation--A Central Limit Theorem*
Chapter 7
STOCHASTIC PROCESSES 513
36. KOLMOGOROV'S EXISTENCE THEOREM, 513
Stochastic Processes-- Finite-Dimensional
Distributions-- Product Spaces-- Kolmogorov's Existence
Theorem--The Inadequacy of RT-A Return to Ergodic
Theory--The Hewitt-Savage Theorem*
37. BROWNIAN MOTION, 530
Definition --Continuity of Paths-- Measurable
Processes--Irregularity of Brownian Motion Paths--The Strong
Markov Property--The Reflection Principle--Skorohod
Embedding --I nvariance*
38. NONDENUMERABLE PROBABILITIES, 558
Introduction -- Definitions-- Existence
Theorems--Consequences of Separability*
APPENDIX
NOTES ON THE PROBLEMS
BIBLIOGRAPHY
INDEX