Open Lecture Notes
Version 1.70 (November 2026)
Preface
These notes provide a broad introduction to measure-theoretic probability and stochastic processes. They were written for a two-quarter course for first-year PhD students in applied mathematics at the University of Washington, but they should also be useful to graduate students in statistics, engineering, economics and finance, and to anyone studying the subject on their own. Readers are assumed to be familiar with undergraduate probability, real analysis and linear algebra; some exposure to ordinary and partial differential equations is helpful.
The first half of the notes covers probability spaces, conditional expectation, generating and characteristic functions, Markov chains in discrete and continuous time, and convergence of random variables. The second half covers Brownian motion, stochastic calculus, stochastic differential equations and their connection to partial differential equations, jump diffusions, stochastic control, optimal stopping and Monte Carlo methods. Most chapters end with exercises, and solutions are collected at the end of the notes.
The focus of these notes is on the applied aspects of the subject. Mathematical statements (definitions, theorems, propositions, lemmas and corollaries) are intended to be rigorously true. Proofs, however, are sometimes only sketched, or make simplifying assumptions that avoid technical complications; where this is the case, we say so. Each chapter begins with a list of references, which the interested reader can consult for complete proofs. Some examples and exercises are adapted from these references.
An open and evolving set of notes
These notes are released under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0). The LaTeX source is available at
https://github.com/mattlorig/intro-probability-stochastic-processes .
Readers are encouraged to report errors, suggest improvements and contribute new material by opening an issue or a pull request; see the file CONTRIBUTING.md in the repository. Text in blue marks material that is still under development. Information on how to cite these notes (including their arXiv identifier and DOI) can be found in the repository.
Acknowledgments
The author of these notes wishes to express his sincere thanks to Weston Barger and Yu-Chen Cheng for checking and writing solutions to the exercises as well as making corrections and improvements to the text, and to the contributors listed on GitHub.
Donations
If you find these notes useful and would like to make a donation to help me develop them further you can donate ETH or ERC20 tokens to the address below.
ETH address: 0x3D6042D6a6f39735bbAd43ad1Aa676113D55BaAC
Review of probability
The notes from this chapter are taken primarily from (Shreve, 2004, Chapter 1) and (Grimmett and Stirzaker, 2001, Chapters 1–5).
Events as sets
Definition 1.1.1. The set of all possible outcomes of an experiment is called the sample space . We denote the sample space as Ω \Omega Ω .
We will typically denote by ω \omega ω a generic element of Ω \Omega Ω .
Definition 1.1.2. An event is a subset of the sample space. We usually denote events by capital roman letters A, B, C, . . ..
Example 1.1.3 (Toss a coin once). When one tosses a coin, there are two possible outcomes: heads (H) and tails (T). Thus, we have Ω = { H , T } \Omega= \{\mathrm{H}, \mathrm{T}\} Ω = { H , T } . One element of Ω \Omega Ω is, e.g., ω = H \omega= \mathrm{H} ω = H . Possible event: “toss a head.” A = H A = \mathrm{H} A = H .
Example 1.1.4 (Toss two distinguishable coins). Ω = { ( H H ) , ( H T ) , ( T H ) , ( T T ) } \Omega= \{(\mathrm{HH}),(\mathrm{HT}),(\mathrm{TH}),(\mathrm{TT})\} Ω = {( HH ) , ( HT ) , ( TH ) , ( TT )} . One element of Ω \Omega Ω is, e.g., ω = ( H T ) \omega= (\mathrm{HT}) ω = ( HT ) . Possible event: ‘’second toss a tail.” A = { ( H T ) , ( T T ) } A = \{(\mathrm{HT}),(\mathrm{TT})\} A = {( HT ) , ( TT )} .
Example 1.1.5 (Toss two indistinguishable coins). Ω = { { H H } , { H T } , { T T } } \Omega= \{\{\mathrm{HH}\},\{\mathrm{HT}\},\{\mathrm{TT}\}\} Ω = {{ HH } , { HT } , { TT }} . One element of Ω \Omega Ω is, e.g., { H H } \{\mathrm{HH}\} { HH } . Possible event: ‘’coins match.” A = { { H H } , { T T } } A = \{\{\mathrm{HH}\},\{\mathrm{TT}\}\} A = {{ HH } , { TT }} .
Example 1.1.6 (Roll a die). Ω = { 1 , 2 , 3 , 4 , 5 , 6 } \Omega= \{1, 2, 3, 4, 5, 6\} Ω = { 1 , 2 , 3 , 4 , 5 , 6 } . One element of Ω \Omega Ω is, e.g., ω = 2 \omega= 2 ω = 2 . Possible event: “roll an odd number.” A = { 1 , 3 , 5 } A = \{1, 3, 5\} A = { 1 , 3 , 5 } .
Here, we use ( ⋅ ) (\cdot) ( ⋅ ) to denote an ordered sequence and we use { ⋅ } \{\cdot\} { ⋅ } to denote an unordered set. Thus, ( H T ) ≠ ( T H ) (\mathrm{HT}) \ne(\mathrm{TH}) ( HT ) = ( TH ) but { H T } = { T H } \{\mathrm{HT}\} = \{\mathrm{TH}\} { HT } = { TH } .
If A and B are subsets of Ω \Omega Ω , we can reasonably concern ourselves with events such as “not A” (A c A^{c} A c ), “A or B” (A ∪ B A \cup B A ∪ B ), “A and B” (A ∩ B A \cap B A ∩ B ), etc. A σ \sigma σ -algebra is a mathematical way to describe all possible sets of interest for a given sample space Ω \Omega Ω .
Definition 1.1.7. A collection F \mathcal{F} F of subsets of Ω \Omega Ω is called a σ \sigma σ -algebra if it satisfies
contains the empty set: ∅ ∈ F \varnothing\in\mathcal{F} ∅ ∈ F ;
is closed under countable unions: A 1 , A 2 , A 3 , … ∈ F ⇒ ⋃ i A i ∈ F A_1, A_2, A_3, \ldots\in\mathcal{F} \Rightarrow\bigcup_i A_i \in\mathcal{F} A 1 , A 2 , A 3 , … ∈ F ⇒ ⋃ i A i ∈ F ;
is closed under complements: A ∈ F ⇒ A c ∈ F A \in\mathcal{F} \Rightarrow A^c \in\mathcal{F} A ∈ F ⇒ A c ∈ F ;
Note that if F \mathcal{F} F is a σ \sigma σ -algebra then Ω ∈ F \Omega\in\mathcal{F} Ω ∈ F by items 1 and 3. Note also that F \mathcal{F} F is closed under countable intersections since
A 1 , A 2 , A 3 , … ∈ F ⇒ A 1 c , A 2 c , A 3 c , … ∈ F ⇒ ⋃ i A i c ∈ F ⇒ ( ⋃ i A i c ) c ∈ F ⇒ ⋂ i A i ∈ F . \begin{aligned}
A_1,A_2,A_3,\ldots&\in\mathcal{F} &&\Rightarrow A_1^c,A_2^c,A_3^c,\ldots\in\mathcal{F}\\
&&&\Rightarrow\bigcup_i A_i^c \in\mathcal{F}\\
&&&\Rightarrow\left(\bigcup_i A_i^c\right)^c \in\mathcal{F}\\
&&&\Rightarrow\bigcap_i A_i \in\mathcal{F}.
\end{aligned} A 1 , A 2 , A 3 , … ∈ F ⇒ A 1 c , A 2 c , A 3 c , … ∈ F ⇒ i ⋃ A i c ∈ F ⇒ ( i ⋃ A i c ) c ∈ F ⇒ i ⋂ A i ∈ F .
Alternatively, one can define a σ \sigma σ -algebra F \mathcal{F} F as a set of subsets of Ω \Omega Ω that contains at least the empty set ∅ \varnothing ∅ and is closed under countable set operations (though, not necessarily closed under uncountable set operators).
Example 1.1.8 (Trivial σ \sigma σ -algebra). The set of subsets F 0 : = { ∅ , Ω } \mathcal{F}_0 := \{\varnothing,\Omega\} F 0 := { ∅ , Ω } of Ω \Omega Ω is commonly referred to as the trivial σ \sigma σ -algebra.
Example 1.1.9. If A A A is a subset of Ω \Omega Ω then F A : = { ∅ , Ω , A , A c } \mathcal{F}_A := \{\varnothing,\Omega,A,A^c\} F A := { ∅ , Ω , A , A c } is a σ \sigma σ -algebra.
Example 1.1.10. The power set of Ω \Omega Ω , written 2 Ω 2^\Omega 2 Ω is the collection of all subsets of Ω \Omega Ω . The power set F = 2 Ω \mathcal{F} = 2^\Omega F = 2 Ω is a σ \sigma σ -algebra.
Definition 1.1.11. Let G \mathcal{G} G be a collection of subsets of Ω \Omega Ω . The σ \sigma σ -algebra generated by G \mathcal{G} G , written σ ( G ) \sigma(\mathcal{G}) σ ( G ) , is the smallest σ \sigma σ -algebra that contains G \mathcal{G} G .
By “smallest” σ \sigma σ -algebra we mean the σ \sigma σ -algebra with the fewest sets. One can show (although we will not do so in these notes) that σ ( G ) \sigma(\mathcal{G}) σ ( G ) is equal to the intersection of all σ \sigma σ -algebras that contain G \mathcal{G} G .
Example 1.1.12. The collection of sets G = { ∅ , A , Ω } \mathcal{G} = \{\emptyset, A, \Omega\} G = { ∅ , A , Ω } is not a σ \sigma σ -algebra because it does not contain A c A^{c} A c . However, we could create a σ \sigma σ -algebra from G \mathcal{G} G by simply adding the set A c A^{c} A c . Thus, we have σ ( G ) = { ∅ , Ω , A , A c } \sigma(\mathcal{G}) = \{\emptyset, \Omega, A, A^{c}\} σ ( G ) = { ∅ , Ω , A , A c } .
Definition 1.1.13. The pair ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) where Ω \Omega Ω is a sample space and F \mathcal{F} F is a σ \sigma σ -algebra of subsets of Ω \Omega Ω is called a measurable space .
Probability
So far, we have not yet talked about probabilities at all – only outcomes of a random experiment (elements ω ∈ Ω \omega\in\Omega ω ∈ Ω ) and events (subsets A ⊆ Ω A \subseteq\Omega A ⊆ Ω ). A probability measure assigns probabilities to events.
Definition 1.2.1. A probability measure defined on ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) is a function P : F → [ 0 , 1 ] \mathbb{P} : \mathcal{F} \to[0,1] P : F → [ 0 , 1 ] that satisfies
P ( Ω ) = 1 \mathbb{P}(\Omega) = 1 P ( Ω ) = 1 ;
if A i ∩ A j = ∅ A_i \cap A_j = \emptyset A i ∩ A j = ∅ for i ≠ j i \ne j i = j then P ( ⋃ i A i ) = ∑ i P ( A i ) \mathbb{P}(\bigcup_i A_i) = \sum_i \mathbb{P}(A_i) P ( ⋃ i A i ) = ∑ i P ( A i ) . (countable additivity)
As we shall see in § 4.3.2 , it is very important to recognize that Item 2 holds only for countable unions. For an uncountable union it is not true that P ( ⋃ α A α ) = ∑ α P ( A α ) \mathbb{P}(\bigcup_\alpha A_\alpha) = \sum_\alpha\mathbb{P}(A_\alpha) P ( ⋃ α A α ) = ∑ α P ( A α ) .
How can we see the well-known result P ( A c ) = 1 − P ( A ) \mathbb{P}(A^c) = 1 - \mathbb{P}(A) P ( A c ) = 1 − P ( A ) from the above definition? Simply note that
1 = P ( Ω ) = P ( A ∪ A c ) = P ( A ) + P ( A c ) . 1 = \mathbb{P}(\Omega) = \mathbb{P}(A \cup A^c) = \mathbb{P}(A) + \mathbb{P}(A^c). 1 = P ( Ω ) = P ( A ∪ A c ) = P ( A ) + P ( A c ) .
A probability measure P \mathbb{P} P does not need to correspond to empirically observed probabilities! For example, from experience, we know that if we toss a fair coin we have P ( H ) = P ( T ) = 1 / 2 \mathbb{P}(\mathrm{H}) = \mathbb{P}(\mathrm{T}) = 1/2 P ( H ) = P ( T ) = 1/2 . However, we can always define a measure P ~ \widetilde{\mathbb{P}} P that assigns different probabilities P ~ ( H ) = p \widetilde{\mathbb{P}}(\mathrm{H}) = p P ( H ) = p and P ~ ( T ) = 1 − p \widetilde{\mathbb{P}}(\mathrm{T}) = 1 - p P ( T ) = 1 − p . As long as p ∈ [ 0 , 1 ] p \in[0,1] p ∈ [ 0 , 1 ] the measure P ~ \widetilde{\mathbb{P}} P is a probability measure on ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) where Ω = { H , T } \Omega= \{\mathrm{H}, \mathrm{T}\} Ω = { H , T } and F = { ∅ , Ω , H , T } \mathcal{F} = \{\emptyset, \Omega, \mathrm{H}, \mathrm{T}\} F = { ∅ , Ω , H , T } .
The triple ( Ω , F , P ) (\Omega, \mathcal{F}, \mathbb{P}) ( Ω , F , P ) is often referred to as a probability space or probability triple . To review, the sample space Ω \Omega Ω is the collection of all possible outcomes of an experiment. The σ \sigma σ -algebra F \mathcal{F} F is all sets of interest of an experiment. And the probability measure P \mathbb{P} P assigns probabilities to these sets.
When a sample space is countable Ω = { ω 1 , ω 2 , … } \Omega= \{\omega_1, \omega_2, \ldots\} Ω = { ω 1 , ω 2 , … } , we can always take the σ \sigma σ -algebra as the power set F = 2 Ω \mathcal{F} = 2^\Omega F = 2 Ω and construct a probability measure P \mathbb{P} P on ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) by specifying the probabilities of each individual outcome P ( ω i ) = p i \mathbb{P}(\omega_i) = p_i P ( ω i ) = p i . However, when the sample space Ω \Omega Ω is uncountable, choosing an appropriate σ \sigma σ -algebra F \mathcal{F} F , and constructing a probability measure P \mathbb{P} P on ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) is a more delicate procedure.
Infinite probability spaces
In this section, we construct a probability measure on two uncountable sample spaces.
Uniform Lebesgue measure on (0,1)
In our first example, we will construct a mathematical model for choosing a number at random on the open interval ( 0 , 1 ) (0,1) ( 0 , 1 ) . Thus, we take Ω = ( 0 , 1 ) \Omega=(0,1) Ω = ( 0 , 1 ) . Note that this sample space is uncountable. A particular outcome is, e.g., ω = 0.5 \omega=0.5 ω = 0.5 or ω = 1 / 2 \omega=1/\sqrt{2} ω = 1/ 2 . Clearly, the number of possible outcomes is infinite. We would like to construct a probability P \mathbb{P} P so that all numbers in the interval ( 0 , 1 ) (0,1) ( 0 , 1 ) are equally likely to be chosen. It makes sense, then, to choose
P ( { ω : ω ∈ ( a , b ) } ) = μ ( ( a , b ) ) : = b − a , 0 < a ≤ b < 1 , (1) \mathbb{P}(\{\omega:\omega\in(a,b)\})=\mu((a,b)):=b-a,\qquad0<a\leq b<1, \tag*{(1)} P ({ ω : ω ∈ ( a , b )}) = μ (( a , b )) := b − a , 0 < a ≤ b < 1 , (1)
where we have defined the Lebesgue measure μ \mu μ . Equation (1) tells us how to determine the probability that ω \omega ω falls within an open interval. But, in fact, (1) tells us more than that. If P \mathbb{P} P is to be a probability measure, then it must satisfy the countable additivity property given in Definition 1.2.1 . Thus, we also know, for example, that
P ( { ω : ω ∈ ( a , b ) ∪ ( c , d ) } ) = P ( { ω : ω ∈ ( a , b ) } ) + P ( { ω : ω ∈ ( c , d ) } ) = ( b − a ) + ( d − c ) , 0 < a < b < c < d < 1. \begin{aligned}
\mathbb{P}(\{\omega:\omega\in(a,b)\cup(c,d)\})&=\mathbb{P}(\{\omega:\omega\in(a,b)\})+\mathbb{P}(\{\omega:\omega\in(c,d)\})\\
&=(b-a)+(d-c),\qquad0<a<b<c<d<1.
\end{aligned} P ({ ω : ω ∈ ( a , b ) ∪ ( c , d )}) = P ({ ω : ω ∈ ( a , b )}) + P ({ ω : ω ∈ ( c , d )}) = ( b − a ) + ( d − c ) , 0 < a < b < c < d < 1.
It is natural to ask, what are all of the sub-sets of ( 0 , 1 ) (0,1) ( 0 , 1 ) whose probabilities are uniquely determined by (1) ? Surprisingly, the answer is not the power set 2 ( 0 , 1 ) 2^{(0,1)} 2 ( 0 , 1 ) . It turns out that the power set 2 ( 0 , 1 ) 2^{(0,1)} 2 ( 0 , 1 ) has sets whose probabilities are not determined by (1) . The sets whose probabilities are uniquely determined by (1) and Definition 1.2.1 are the sets in σ \sigma σ -algebra generated by the open intervals
B ( ( 0 , 1 ) ) : = σ ( O ) ) , where O : = { A ⊆ ( 0 , 1 ) : A = ( a , b ) , 0 ≤ a < b ≤ 1 } . \mathcal{B}((0,1)):=\sigma(\mathcal{O)),
\qquad\text{where}\qquad
\mathcal{O}:=\{A\subseteq(0,1):A=(a,b),\ 0\leq a<b\leq1\}.} B (( 0 , 1 )) := σ ( O )) , where O := { A ⊆ ( 0 , 1 ) : A = ( a , b ) , 0 ≤ a < b ≤ 1 } .
We call B ( ( 0 , 1 ) ) \mathcal{B}((0,1)) B (( 0 , 1 )) the Borel σ \sigma σ -algebra on ( 0 , 1 ) (0,1) ( 0 , 1 ) . Thus, the appropriate sample space for our experiment is ( Ω , F , P ) = ( ( 0 , 1 ) , B ( ( 0 , 1 ) ) , μ ) (\Omega,\mathcal{F},\mathbb{P})=((0,1),\mathcal{B}((0,1)),\mu) ( Ω , F , P ) = (( 0 , 1 ) , B (( 0 , 1 )) , μ ) .
We can generalize the notion of Borel sets to all topological spaces.
Definition 1.3.1. Let Ω \Omega Ω be some topological space and let O ( Ω ) \mathcal{O}(\Omega) O ( Ω ) be the set of open sets in Ω \Omega Ω . We define we define the Borel σ \sigma σ -algebra on Ω \Omega Ω , denoted B ( Ω ) \mathcal{B}(\Omega) B ( Ω ) by B ( Ω ) : = σ ( O ( Ω ) ) \mathcal{B}(\Omega):=\sigma(\mathcal{O}(\Omega)) B ( Ω ) := σ ( O ( Ω )) .
Infinite sequence of coin tosses
In this section we consider and infinite sequence of coin tosses. We define
Ω : = the set of infinite sequences of Hs and Ts . \Omega:=\text{the set of infinite sequences of Hs and Ts}. Ω := the set of infinite sequences of Hs and Ts .
Note that this set is uncountable because there is a one-to-one correspondence between Ω \Omega Ω and the set of reals in [ 0 , 1 ] [0,1] [ 0 , 1 ] . We will denote a generic element of Ω \Omega Ω as follows:
ω = ω 1 ω 2 ω 3 ⋯ \omega= \omega_1\omega_2\omega_3\cdots ω = ω 1 ω 2 ω 3 ⋯
where ω i \omega_i ω i is the result of the i i i th coin toss. We want to construct a σ \sigma σ -algebra for this experiment.
Let us define some σ \sigma σ -algebras. First, consider the trivial σ \sigma σ -algebra
F 0 = { ∅ , Ω } . \mathcal{F}_0 = \{\varnothing,\Omega\}. F 0 = { ∅ , Ω } .
Given no information, I can tell if ω \omega ω is in the sets in F 0 \mathcal{F}_0 F 0 because we know ω ∈ Ω \omega\in\Omega ω ∈ Ω and ω ∉ ∅ \omega\notin\varnothing ω ∈ / ∅ . Next, define two sets
A H = { ω ∈ Ω : ω 1 = H } , A T = { ω ∈ Ω : ω 1 = T } . A_{\mathrm{H}} = \{\omega\in\Omega: \omega_1 = \mathrm{H}\}, \qquad A_{\mathrm{T}} = \{\omega\in\Omega: \omega_1 = \mathrm{T}\}. A H = { ω ∈ Ω : ω 1 = H } , A T = { ω ∈ Ω : ω 1 = T } .
Noting that A H = A T c A_{\mathrm{H}} = A_{\mathrm{T}}^{c} A H = A T c we see that
F 1 : = { ∅ , Ω , A H , A T } , \mathcal{F}_1 := \{\varnothing,\Omega,A_{\mathrm{H}},A_{\mathrm{T}}\}, F 1 := { ∅ , Ω , A H , A T } ,
satisfies the conditions of σ \sigma σ -algebra. Given ω 1 \omega_1 ω 1 it is possible to say whether or not ω \omega ω is in each of the sets in F 1 \mathcal{F}_1 F 1 . For example, if ω 1 = H \omega_1 = \mathrm{H} ω 1 = H then ω ∈ A H \omega\in A_{\mathrm{H}} ω ∈ A H and ω ∈ Ω \omega\in\Omega ω ∈ Ω , but ω ∉ A T \omega\notin A_{\mathrm{T}} ω ∈ / A T and ω ∉ ∅ \omega\notin\varnothing ω ∈ / ∅ . Next define four sets
A H H : = { ω ∈ Ω : ω 1 = H , ω 2 = H } , A H T : = { ω ∈ Ω : ω 1 = H , ω 2 = T } , A T T : = { ω ∈ Ω : ω 1 = T , ω 2 = T } , A T H : = { ω ∈ Ω : ω 1 = T , ω 2 = H } . \begin{aligned}
A_{\mathrm{HH}} &:= \{\omega\in\Omega: \omega_1 = \mathrm{H}, \omega_2 = \mathrm{H}\}, &
A_{\mathrm{HT}} &:= \{\omega\in\Omega: \omega_1 = \mathrm{H}, \omega_2 = \mathrm{T}\},\\
A_{\mathrm{TT}} &:= \{\omega\in\Omega: \omega_1 = \mathrm{T}, \omega_2 = \mathrm{T}\}, &
A_{\mathrm{TH}} &:= \{\omega\in\Omega: \omega_1 = \mathrm{T}, \omega_2 = \mathrm{H}\}.
\end{aligned} A HH A TT := { ω ∈ Ω : ω 1 = H , ω 2 = H } , := { ω ∈ Ω : ω 1 = T , ω 2 = T } , A HT A TH := { ω ∈ Ω : ω 1 = H , ω 2 = T } , := { ω ∈ Ω : ω 1 = T , ω 2 = H } .
We wish to construct a σ \sigma σ -algebra that contains these sets and the sets in F 1 \mathcal{F}_1 F 1 . The smallest such σ \sigma σ -algebra is
F 2 = { ∅ , Ω , A H , A T , A H H , A H T , A T T , A T H , A H H c , A H T c , A T T c , A T H c , A H H ∪ A T H , A H H ∪ A T T , A H T ∪ A T H , A H T ∪ A T T } . \mathcal{F}_2 =
\left\{
\begin{gathered}
\varnothing,\Omega,A_{\mathrm{H}},A_{\mathrm{T}},A_{\mathrm{HH}},A_{\mathrm{HT}},A_{\mathrm{TT}},A_{\mathrm{TH}},A_{\mathrm{HH}}^{c},A_{\mathrm{HT}}^{c},A_{\mathrm{TT}}^{c},A_{\mathrm{TH}}^{c},\\
A_{\mathrm{HH}}\cup A_{\mathrm{TH}},A_{\mathrm{HH}}\cup A_{\mathrm{TT}},A_{\mathrm{HT}}\cup A_{\mathrm{TH}},A_{\mathrm{HT}}\cup A_{\mathrm{TT}}
\end{gathered}
\right\}. F 2 = { ∅ , Ω , A H , A T , A HH , A HT , A TT , A TH , A HH c , A HT c , A TT c , A TH c , A HH ∪ A TH , A HH ∪ A TT , A HT ∪ A TH , A HT ∪ A TT } .
Given ω 1 \omega_1 ω 1 and ω 2 \omega_2 ω 2 , we can say if ω \omega ω belongs to each of the sets in F 2 \mathcal{F}_2 F 2 . Continuing in this way, we can define a σ \sigma σ -algebra F n \mathcal{F}_n F n for every n ∈ N n \in\mathbb{N} n ∈ N . Finally, we take
F : = σ ( F ∞ ) , F ∞ = ⋃ n F n . \mathcal{F} := \sigma(\mathcal{F}_{\infty}), \qquad\mathcal{F}_{\infty} = \bigcup_{n}\mathcal{F}_n. F := σ ( F ∞ ) , F ∞ = n ⋃ F n .
One might ask if we could have simply taken F = F ∞ \mathcal{F} = \mathcal{F}_{\infty} F = F ∞ ? Well, F ∞ \mathcal{F}_{\infty} F ∞ contains every set that can be described in terms of finitely many coin tosses. However, we may be interested in sets such as “sequences for which x x x percent of coin tosses are heads,” and these sets are not in F ∞ \mathcal{F}_{\infty} F ∞ . It turns out such sets are in F \mathcal{F} F .
Now, we want to construct a probability measure on F \mathcal{F} F . Let us assume the coin tosses are independent (a term we will describe rigorously later on) and that the probability of a head is p p p . Setting q = 1 − p q=1-p q = 1 − p , it should be obvious that
P ( ∅ ) = 0 , P ( Ω ) = 1 , P ( A H ) = p , P ( A T ) = q , \mathbb{P}(\varnothing)=0,\qquad\mathbb{P}(\Omega)=1,\qquad\mathbb{P}(A_{\mathrm{H}})=p,\qquad\mathbb{P}(A_{\mathrm{T}})=q, P ( ∅ ) = 0 , P ( Ω ) = 1 , P ( A H ) = p , P ( A T ) = q ,
P ( A H H ) = p 2 , P ( A H T ) = p q , P ( A T H ) = p q , P ( A T T ) = q 2 , … \mathbb{P}(A_{\mathrm{HH}})=p^2,\qquad\mathbb{P}(A_{\mathrm{HT}})=pq,\qquad\mathbb{P}(A_{\mathrm{TH}})=pq,\qquad\mathbb{P}(A_{\mathrm{TT}})=q^2,\ldots P ( A HH ) = p 2 , P ( A HT ) = pq , P ( A TH ) = pq , P ( A TT ) = q 2 , …
Continuing in this way, we can define P ( A ) \mathbb{P}(A) P ( A ) for every A ∈ F ∞ A\in\mathcal{F}_{\infty} A ∈ F ∞ . What about the sets that are in F \mathcal{F} F but not in F ∞ \mathcal{F}_{\infty} F ∞ ? It turns out that once we have defined P \mathbb{P} P for sets in F ∞ \mathcal{F}_{\infty} F ∞ there is only one way to assign probabilities to those sets that are in F \mathcal{F} F but not in F ∞ \mathcal{F}_{\infty} F ∞ . We refer the interested reader to Carathéodory’s Extension Theorem for details.
Now, let us define
A = { ω : lim n → ∞ # H in first n coin tosses n = 1 2 } . A=\left\{\omega:\lim_{n\to\infty}\frac{\#\mathrm{H}\text{ in first }n\text{ coin tosses}}{n}=\frac{1}{2}\right\}. A = { ω : n → ∞ lim n # H in first n coin tosses = 2 1 } .
The strong law of large numbers (SLLN) tells us that P ( A ) = 1 \mathbb{P}(A)=1 P ( A ) = 1 if p = 1 / 2 p=1/2 p = 1/2 and P ( A ) = 0 \mathbb{P}(A)=0 P ( A ) = 0 if p ≠ 1 / 2 p\ne1/2 p = 1/2 (if you have not yet seen the SLLN, you should be able to see this from intuition). Now it should be clear why uncountable additivity does not hold for probability measures. The probability of any given sequence of infinite coin tosses is zero: P ( ω ) = 0 \mathbb{P}(\omega)=0 P ( ω ) = 0 . If we were to attempt to compute P ( A ) \mathbb{P}(A) P ( A ) by adding up the probabilities P ( ω ) \mathbb{P}(\omega) P ( ω ) of all elements ω ∈ A \omega\in A ω ∈ A we would find
∑ ω ∈ A P ( ω ) = ∑ ω ∈ A 0 = 0 ≠ 1 = P ( A ) , (when p = 1 / 2 ) . \sum_{\omega\in A}\mathbb{P}(\omega)=\sum_{\omega\in A}0=0\ne1=\mathbb{P}(A),\qquad\text{(when }p=1/2\text{)}. ω ∈ A ∑ P ( ω ) = ω ∈ A ∑ 0 = 0 = 1 = P ( A ) , (when p = 1/2 ) .
Thus, uncountable additivity clearly does not hold .
We finish this example (we will come back to it!) with the following definition
Definition 1.3.3. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space. If a set A ∈ F A\in\mathcal{F} A ∈ F satisfies P ( A ) = 1 \mathbb{P}(A)=1 P ( A ) = 1 , we say that the event A A A occurs P \mathbb{P} P almost surely (written, P \mathbb{P} P -a.s.).
Note in the example above that, when p = 1 / 2 p=1/2 p = 1/2 we have P ( A ) = 1 \mathbb{P}(A)=1 P ( A ) = 1 and thus A A A occurs almost surely. But it is important to recognize that A ≠ Ω A\ne\Omega A = Ω and A c ≠ ∅ A^{c}\ne\varnothing A c = ∅ . The elements of A c A^{c} A c are part of the sample space Ω \Omega Ω , but they have zero probability of occurring.
Random variables and distributions
A random variable maps the outcome of an experiment to R \mathbb{R} R . We capture this idea with the following definition.
Random Variables and Distributions
Definition 1.4.1. A random variable defined on ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) is a function X : Ω → R X : \Omega\to\mathbb{R} X : Ω → R with the property that
{ X ∈ A } : = { ω ∈ Ω : X ( ω ) ∈ A } ∈ F , \{X \in A\} := \{\omega\in\Omega: X(\omega) \in A\} \in\mathcal{F}, { X ∈ A } := { ω ∈ Ω : X ( ω ) ∈ A } ∈ F , for all A ∈ B ( R ) A \in\mathcal{B}(\mathbb{R}) A ∈ B ( R ) .
Observe all random variables must be defined on a measurable space ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) , as these appear in the definition. Note, however, that the probability measure P \mathbb{P} P does not appear in the definition. Random variables are defined independent of a probability measure P \mathbb{P} P .
What does Definition 1.4.1 mean? Recall that a probability measure P \mathbb{P} P defined on ( Ω , F ) (\Omega, \mathcal{F}) ( Ω , F ) maps F → [ 0 , 1 ] \mathcal{F} \to[0, 1] F → [ 0 , 1 ] . In order for us to answer the question: “what is the probability that X ∈ A X \in A X ∈ A ?” we need for the set { X ∈ A } ∈ F \{X \in A\} \in\mathcal{F} { X ∈ A } ∈ F . And this is precisely what Definition 1.4.1 requires. Why do we only consider sets A ∈ B ( R ) A \in\mathcal{B}(\mathbb{R}) A ∈ B ( R ) rather than any set A ⊂ R A \subset\mathbb{R} A ⊂ R ? The answer is rather technical and, frankly, not worth exploring at the moment.
A word on notation: the standard convention is to use capital Roman letters (typically, X X X , Y Y Y , Z Z Z ) for random variables and lower case Roman letters (x x x , y y y , z z z ) for real numbers.
Let us look at some random variables.
Example 1.4.2 (Discrete time model for stock prices). Consider the infinite sequence of coin tosses in § 4.3.2 . We Define a sequence of random variables ( S n ) n ≥ 0 (S_n)_{n \ge0} ( S n ) n ≥ 0 via
S 0 ( ω ) = 1 , S n + 1 ( ω ) = { u S n if ω n + 1 = H d S n if ω n + 1 = T (2) S_0(\omega) = 1,\qquad S_{n+1}(\omega) =
\begin{cases}
uS_n & \text{if } \omega_{n+1} = \mathrm{H}\\
dS_n & \text{if } \omega_{n+1} = \mathrm{T}
\end{cases} \tag*{(2)} S 0 ( ω ) = 1 , S n + 1 ( ω ) = { u S n d S n if ω n + 1 = H if ω n + 1 = T (2) Here, S n S_n S n represents the value of a stock at time n n n . Note that P ( S 1 = u ) = P ( A H ) = p \mathbb{P}(S_1 = u) = \mathbb{P}(A_{\mathrm{H}}) = p P ( S 1 = u ) = P ( A H ) = p . Likewise P ( S 2 = u d ) = P ( A H T ∪ A T H ) = 2 p q \mathbb{P}(S_2 = ud) = \mathbb{P}(A_{\mathrm{HT}} \cup A_{\mathrm{TH}}) = 2pq P ( S 2 = u d ) = P ( A HT ∪ A TH ) = 2 pq . More generally, one can show that
P ( S n = u k d n − k ) = ( n k ) p k q n − k . (3) \mathbb{P}(S_n = u^k d^{n-k}) = \binom{n}{k}p^kq^{n-k}. \tag*{(3)} P ( S n = u k d n − k ) = ( k n ) p k q n − k . (3) Note if we had simply defined the random variables ( S n ) n ≥ 1 (S_n)_{n \ge1} ( S n ) n ≥ 1 as having probabilities given by (3) we would have no information about how, e.g., S n S_n S n relates to S n − 1 S_{n-1} S n − 1 . From the above construction (2) , however, we know that if S n = u n S_n = u^n S n = u n then S n − 1 = u n − 1 S_{n-1} = u^{n-1} S n − 1 = u n − 1 . Thus, the structure of a given probability space, not just the probabilities of events, is very important.
Example 1.4.3. Let ( Ω , F ) = ( ( 0 , 1 ) , B ( ( 0 , 1 ) ) ) (\Omega, \mathcal{F}) = ((0, 1), \mathcal{B}((0, 1))) ( Ω , F ) = (( 0 , 1 ) , B (( 0 , 1 ))) . Define random variables X ( ω ) = ω X(\omega) = \omega X ( ω ) = ω and Y ( ω ) = 1 − ω Y(\omega) = 1 - \omega Y ( ω ) = 1 − ω . Clearly, we have X = 1 − Y X = 1 - Y X = 1 − Y . Now, suppose we define P ( d ω ) : = P ( [ ω , ω + d ω ] ) = d ω \mathbb{P}(\mathrm{d}\omega) := \mathbb{P}([\omega, \omega+ \mathrm{d}\omega]) = \mathrm{d}\omega P ( d ω ) := P ([ ω , ω + d ω ]) = d ω (i.e., the Lebesgue measure). Then X X X and Y Y Y have the same distribution. For x ∈ [ 0 , 1 ] x \in[0, 1] x ∈ [ 0 , 1 ] we have
P ( X ≤ x ) = P ( ω ≤ x ) = ∫ 0 x P ( d ω ) = ∫ 0 x d ω = x , \mathbb{P}(X \le x) = \mathbb{P}(\omega\le x) = \int_0^x \mathbb{P}(\mathrm{d}\omega) = \int_0^x \mathrm{d}\omega= x, P ( X ≤ x ) = P ( ω ≤ x ) = ∫ 0 x P ( d ω ) = ∫ 0 x d ω = x ,
P ( Y ≤ x ) = P ( 1 − ω ≤ x ) = ∫ 1 − x 1 P ( d ω ) = ∫ 1 − x 1 d ω = x . \mathbb{P}(Y \le x) = \mathbb{P}(1-\omega\le x) = \int_{1-x}^{1} \mathbb{P}(\mathrm{d}\omega) = \int_{1-x}^{1} \mathrm{d}\omega= x. P ( Y ≤ x ) = P ( 1 − ω ≤ x ) = ∫ 1 − x 1 P ( d ω ) = ∫ 1 − x 1 d ω = x .
However, if we define a new probability measure via P ~ ( d ω ) = 2 ω d ω \widetilde{\mathbb{P}}(\mathrm{d}\omega) = 2\omega\,\mathrm{d}\omega P ( d ω ) = 2 ω d ω then X X X and Y Y Y have different distributions. For x ∈ [ 0 , 1 ] x \in[0,1] x ∈ [ 0 , 1 ] we have
P ~ ( X ≤ x ) = P ~ ( ω ≤ x ) = ∫ 0 x P ~ ( d ω ) = ∫ 0 x 2 ω d ω = x 2 , \widetilde{\mathbb{P}}(X \le x) = \widetilde{\mathbb{P}}(\omega\le x) = \int_{0}^{x} \widetilde{\mathbb{P}}(\mathrm{d}\omega) = \int_{0}^{x} 2\omega\,\mathrm{d}\omega= x^2, P ( X ≤ x ) = P ( ω ≤ x ) = ∫ 0 x P ( d ω ) = ∫ 0 x 2 ω d ω = x 2 ,
P ~ ( Y ≤ x ) = P ~ ( 1 − ω ≤ x ) = ∫ 1 − x 1 P ~ ( d ω ) = ∫ 1 − x 1 2 ω d ω = 1 − ( 1 − x ) 2 . \widetilde{\mathbb{P}}(Y \le x) = \widetilde{\mathbb{P}}(1-\omega\le x) = \int_{1-x}^{1} \widetilde{\mathbb{P}}(\mathrm{d}\omega) = \int_{1-x}^{1} 2\omega\,\mathrm{d}\omega= 1-(1-x)^2. P ( Y ≤ x ) = P ( 1 − ω ≤ x ) = ∫ 1 − x 1 P ( d ω ) = ∫ 1 − x 1 2 ω d ω = 1 − ( 1 − x ) 2 .
Thus, the statistical properties of random variables depend not only on the structure of the probability space on which they live, but also on the probability measure.
The distribution of a random variable X X X is most easily described through its cumulative distribution function.
Definition 1.4.4. The distribution function F X : R → [ 0 , 1 ] F_X : \mathbb{R} \to[0,1] F X : R → [ 0 , 1 ] of a random variable X X X defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) is given by
F X ( x ) : = P ( X ≤ x ) . F_X(x) := \mathbb{P}(X \le x). F X ( x ) := P ( X ≤ x ) .
Observe that, while a random variable X X X is defined with respect to ( Ω , F ) (\Omega,\mathcal{F}) ( Ω , F ) (with no reference to P \mathbb{P} P ), the distribution F X F_X F X is specific to a probability measure P \mathbb{P} P .
Note that we put the random variable X X X in the subscript of F X F_X F X to remind us that F X F_X F X is the distribution function corresponding to the random variable X X X (and not, e.g., Y Y Y ). It is a good idea to do this.
We give here some obvious properties of F X F_X F X .
We have the following limits: lim x → − ∞ F X ( x ) = 0 \lim_{x \to-\infty} F_X(x) = 0 lim x → − ∞ F X ( x ) = 0 and lim x → ∞ F X ( x ) = 1 \lim_{x \to\infty} F_X(x) = 1 lim x → ∞ F X ( x ) = 1 .
F X F_X F X is non-decreasing: x < y x < y x < y implies F X ( x ) ≤ F X ( y ) F_X(x) \le F_X(y) F X ( x ) ≤ F X ( y ) .
F X F_X F X is right-continuous and has left-limits: F X ( x + ) : = lim h ↘ 0 F X ( x + h ) = F X ( x ) F_X(x+) := \lim_{h \searrow0} F_X(x+h) = F_X(x) F X ( x + ) := lim h ↘ 0 F X ( x + h ) = F X ( x ) .
Many (but not all) random variables fall in to one of two categories: discrete and continuous. We describe these two categories below.
Definition 1.4.5. A random variable X X X is called discrete if it takes values in some countable set A : = { x 1 , x 2 , … } ⊂ R A := \{x_1,x_2,\ldots\} \subset\mathbb{R} A := { x 1 , x 2 , … } ⊂ R . We associate with a discrete random variable a probability mass function f X : A → R f_X : A \to\mathbb{R} f X : A → R , defined by f X ( x i ) : = P ( X = x i ) f_X(x_i) := \mathbb{P}(X = x_i) f X ( x i ) := P ( X = x i ) .
Definition 1.4.6. A random variable X X X is called continuous if its distribution function F X F_X F X can be written as
F X ( x ) = ∫ − ∞ x d u f X ( u ) , x ∈ R , F_X(x) = \int_{-\infty}^{x} \mathrm{d}u\,f_X(u), \qquad x \in\mathbb{R}, F X ( x ) = ∫ − ∞ x d u f X ( u ) , x ∈ R , for some f X : R → [ 0 , ∞ ) f_X : \mathbb{R} \to[0,\infty) f X : R → [ 0 , ∞ ) called the probability density function .
It may help to think of the density function f X f_X f X as f X ( x ) d x = P ( X ∈ d x ) f_X(x)\mathrm{d}x = \mathbb{P}(X \in\mathrm{d}x) f X ( x ) d x = P ( X ∈ d x ) .
Note that for a continuous random variable X X X we have f X = F X ′ f_X = F'_X f X = F X ′ .
If X X X is either discrete or continuous, it is easy to compute P ( X ∈ A ) \mathbb{P}(X \in A) P ( X ∈ A ) for any A ∈ B ( R ) A \in\mathcal{B}(\mathbb{R}) A ∈ B ( R ) . We have
discrete : P ( X ∈ A ) = ∑ { i : x i ∈ A } f X ( x i ) , continuous : P ( X ∈ A ) = ∫ A d x f X ( x ) . \begin{aligned}
\text{discrete}: \qquad\mathbb{P}(X \in A) &= \sum_{\{i:x_i\in A\}} f_X(x_i),\\
\text{continuous}: \qquad\mathbb{P}(X \in A) &= \int_A \mathrm{d}x\, f_X(x).
\end{aligned} discrete : P ( X ∈ A ) continuous : P ( X ∈ A ) = { i : x i ∈ A } ∑ f X ( x i ) , = ∫ A d x f X ( x ) .
Examples of discrete random variables
The following discrete random variables frequently arise in applications in nature and social sciences.
Example 1.4.8. If X X X is distributed as a Bernoulli random variable with parameter p ∈ [ 0 , 1 ] p\in[0,1] p ∈ [ 0 , 1 ] , written X ∼ Ber ( p ) X\sim\operatorname{Ber}(p) X ∼ Ber ( p ) , then
X ∈ { 0 , 1 } , f X ( k ) = { 1 − p k = 0 , p k = 1. X\in\{0,1\},\qquad f_X(k)=
\begin{cases}
1-p & k=0,\\
p & k=1.
\end{cases} X ∈ { 0 , 1 } , f X ( k ) = { 1 − p p k = 0 , k = 1.
Example 1.4.9. If X X X is distributed as a Binomial random variable with parameters n ∈ N n\in\mathbb{N} n ∈ N and p ∈ [ 0 , 1 ] p\in[0,1] p ∈ [ 0 , 1 ] , written X ∼ Bin ( n , p ) X\sim\operatorname{Bin}(n,p) X ∼ Bin ( n , p ) , then
X ∈ { 0 , 1 , 2 , … , n } , f X ( k ) = ( n k ) p k ( 1 − p ) n − k . X\in\{0,1,2,\ldots,n\},\qquad f_X(k)=\binom{n}{k}p^k(1-p)^{n-k}. X ∈ { 0 , 1 , 2 , … , n } , f X ( k ) = ( k n ) p k ( 1 − p ) n − k . Note that if X i ∼ Ber ( p ) X_i\sim\operatorname{Ber}(p) X i ∼ Ber ( p ) and independent of each other then Y : = ∑ i = 1 n X i ∼ Bin ( n , p ) Y:=\sum_{i=1}^{n}X_i\sim\operatorname{Bin}(n,p) Y := ∑ i = 1 n X i ∼ Bin ( n , p ) .
Example 1.4.10. If X X X is distributed as a Geometric random variable with parameter p ∈ [ 0 , 1 ] p\in[0,1] p ∈ [ 0 , 1 ] , written X ∼ Geo ( p ) X\sim\operatorname{Geo}(p) X ∼ Geo ( p ) , then
X ∈ N , f X ( k ) = p ( 1 − p ) k − 1 . X\in\mathbb{N},\qquad f_X(k)=p(1-p)^{k-1}. X ∈ N , f X ( k ) = p ( 1 − p ) k − 1 . Note that X i ∼ Ber ( p ) X_i\sim\operatorname{Ber}(p) X i ∼ Ber ( p ) i = 1 , 2 , … i=1,2,\ldots i = 1 , 2 , … are independent of each other then Y : = inf { i : X i = 1 } ∼ Geo ( p ) Y:=\inf\{i:X_i=1\}\sim\operatorname{Geo}(p) Y := inf { i : X i = 1 } ∼ Geo ( p ) .
Example 1.4.11. If X is distributed as a Poisson random variable with parameter λ ∈ R + \lambda\in\mathbb{R}_{+} λ ∈ R + , written X ∼ Poi ( λ ) X \sim\operatorname{Poi}(\lambda) X ∼ Poi ( λ ) , then
X ∈ { 0 } ∪ N , f X ( k ) = λ k k ! e − λ . X \in\{0\} \cup\mathbb{N}, \qquad f_X(k) = \frac{\lambda^k}{k!}e^{-\lambda}. X ∈ { 0 } ∪ N , f X ( k ) = k ! λ k e − λ .
Examples of continuous random variables
Before introducing some common continuous random variables, let us introduce a useful function.
Definition 1.4.12. Let A be a set in some topological space Ω \Omega Ω (e.g., Ω = R d \Omega= \mathbb{R}^d Ω = R d ). The indicator function 1 A : Ω → { 0 , 1 } \mathbf{1}_A : \Omega\to\{0,1\} 1 A : Ω → { 0 , 1 } is defined as follows
1 A ( x ) : = { 1 if x ∈ A , 0 if x ∉ A . \mathbf{1}_A(x) :=
\begin{cases}
1 & \text{if } x \in A,\\
0 & \text{if } x \notin A.
\end{cases} 1 A ( x ) := { 1 0 if x ∈ A , if x ∈ / A .
Notation: We will sometimes write 1 A ( x ) = 1 { x ∈ A } \mathbf{1}_A(x) = \mathbf{1}_{\{x\in A\}} 1 A ( x ) = 1 { x ∈ A } .
We now introduce some continuous random variables that frequently arise in applications.
Example 1.4.13. If X is distributed as a Uniform random variable on the interval [ a , b ] ⊂ R [a,b] \subset\mathbb{R} [ a , b ] ⊂ R , written X ∼ U ( [ a , b ] ) X \sim\mathcal{U}([a,b]) X ∼ U ([ a , b ]) , then
X ∈ [ a , b ] , f X ( x ) = 1 [ a , b ] ( x ) 1 b − a . X \in[a,b], \qquad f_X(x) = \mathbf{1}_{[a,b]}(x)\frac{1}{b-a}. X ∈ [ a , b ] , f X ( x ) = 1 [ a , b ] ( x ) b − a 1 .
Example 1.4.14. If X is distributed as a Exponential random variable with mean λ > 0 \lambda> 0 λ > 0 , written X ∼ E ( λ ) X \sim\mathcal{E}(\lambda) X ∼ E ( λ ) , then
X ∈ [ 0 , ∞ ) , f X ( x ) = 1 [ 0 , ∞ ) ( x ) λ e − λ x . X \in[0,\infty), \qquad f_X(x) = \mathbf{1}_{[0,\infty)}(x)\lambda e^{-\lambda x}. X ∈ [ 0 , ∞ ) , f X ( x ) = 1 [ 0 , ∞ ) ( x ) λ e − λ x . Note that f X ( x ) = 0 f_X(x) = 0 f X ( x ) = 0 if x < 0 x < 0 x < 0 due to the presence of the indicator function.
Example 1.4.15. If X is distributed as a Gaussian or Normal random variable with mean μ ∈ R \mu\in\mathbb{R} μ ∈ R and variance σ 2 > 0 \sigma^2 > 0 σ 2 > 0 (we will give a meaning for “mean” and “variance” below), written X ∼ N ( μ , σ 2 ) X \sim\mathcal{N}(\mu,\sigma^2) X ∼ N ( μ , σ 2 ) , then
X ∈ R , f X ( x ) = 1 2 π σ 2 exp ( − ( x − μ ) 2 2 σ 2 ) . X \in\mathbb{R}, \qquad f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right). X ∈ R , f X ( x ) = 2 π σ 2 1 exp ( − 2 σ 2 ( x − μ ) 2 ) . A random variable Z ∼ N ( 0 , 1 ) Z \sim\mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) is referred to as standard normal .
Stochastic Processes
Intuitively, we think of a stochastic process as a process that evolves randomly in time. Now that we understand what a random variable is, we can define rigorously what we mean when we say stochastic process .
Definition 1.5.1. A Stochastic process is a collection of random variables X = ( X t ) t ∈ T \mathrm{X}=(\mathrm{X}_t)_{t\in\mathbb{T}} X = ( X t ) t ∈ T where T \mathbb{T} T is some index set. If the index set T \mathbb{T} T is countable (e.g., T = N 0 \mathbb{T}=\mathbb{N}_0 T = N 0 ) we say that X \mathrm{X} X is a discrete time process. If the index set T \mathbb{T} T is uncountable (e.g., T = R + \mathbb{T}=\mathbb{R}_+ T = R + ) we say that X \mathrm{X} X is a continuous time process. The State Space S \mathrm{S} S of a stochastic process X \mathrm{X} X is union of the state spaces of ( X t ) t ∈ T (\mathrm{X}_t)_{t\in\mathbb{T}} ( X t ) t ∈ T .
We can think of a stochastic process X : T × Ω → R \mathrm{X}:\mathbb{T}\times\Omega\to\mathbb{R} X : T × Ω → R in (at least) two ways. First, for any t ∈ T t\in\mathbb{T} t ∈ T we have that X t : Ω → R \mathrm{X}_t:\Omega\to\mathbb{R} X t : Ω → R is random variable. Second, for any ω ∈ Ω \omega\in\Omega ω ∈ Ω , we have that X ⋅ ( ω ) : T → R \mathrm{X}_{\cdot}(\omega):\mathbb{T}\to\mathbb{R} X ⋅ ( ω ) : T → R is a function of time. Both interpretations can be useful.
Expectation
When we think of averaging we think of weighting outcomes by their probabilities. The mathematical way to encode this is via the expectation.
Definition 1.6.1. Let X \mathrm{X} X be a random variable defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . The expectation of X \mathrm{X} X , written E X \mathbb{E}\mathrm{X} E X , is defined as
E X : = ∫ Ω X ( ω ) P ( d ω ) , \mathbb{E}\mathrm{X}:=\int_{\Omega}\mathrm{X}(\omega)\mathbb{P}(d\omega), E X := ∫ Ω X ( ω ) P ( d ω ) , where the integral is understood in the Lebesgue sense.
Integration in the Lebesgue sense
For those who have not previously encountered Lebesgue integration, we now give a brief (very brief! ) overview of this concept.
Definition 1.6.2. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Let A ∈ F A\in\mathcal{F} A ∈ F . The indicator random variable , denoted 1 A \mathbb{1}_A 1 A , is defined by
1 A ( ω ) : = { 1 ω ∈ A , 0 ω ∉ A . \mathbb{1}_A(\omega):=
\begin{cases}
1 & \omega\in A,\\
0 & \omega\notin A.
\end{cases} 1 A ( ω ) := { 1 0 ω ∈ A , ω ∈ / A .
Observe that 1 A ∼ Ber ( p ) \mathbb{1}_A \sim\operatorname{Ber}(p) 1 A ∼ Ber ( p ) with p = P ( A ) p = \mathbb{P}(A) p = P ( A ) . For disjoint sets A and B we have
1 A ∪ B = 1 A + 1 B , A ∩ B = ∅ . \mathbb{1}_{A\cup B} = \mathbb{1}_A + \mathbb{1}_B, \qquad A \cap B = \varnothing. 1 A ∪ B = 1 A + 1 B , A ∩ B = ∅ .
And, for any two sets A and B we have
1 A ∩ B = 1 A 1 B . \mathbb{1}_{A\cap B} = \mathbb{1}_A\mathbb{1}_B. 1 A ∩ B = 1 A 1 B .
Definition 1.6.3. A collection of non-empty sets ( A i ) (A_i) ( A i ) is said to be a partition of Ω \Omega Ω if A i ∩ A j = ∅ A_i \cap A_j = \varnothing A i ∩ A j = ∅ for all i ≠ j i \ne j i = j and ⋃ i A i = Ω \bigcup_i A_i = \Omega ⋃ i A i = Ω .
Definition 1.6.4. Let ( A i ) (A_i) ( A i ) be a finite partition of Ω \Omega Ω . A non-negative random variable X, defined on a probability space ( Ω , F , P ) (\Omega, \mathcal{F},\mathbb{P}) ( Ω , F , P ) , which is of the form
X ( ω ) = ∑ i = 1 n x i 1 A i ( ω ) , x i ≥ 0 , A i ∈ F , X(\omega) = \sum_{i=1}^{n} x_i\mathbb{1}_{A_i}(\omega), \qquad x_i \ge0, \qquad A_i \in\mathcal{F}, X ( ω ) = i = 1 ∑ n x i 1 A i ( ω ) , x i ≥ 0 , A i ∈ F ,
is called simple .
Let X be a simple random variable. We define the expectation of X as follows
E X : = ∑ i = 1 n x i P ( A i ) . (if X is simple) \mathbb{E}X := \sum_{i=1}^{n} x_i\mathbb{P}(A_i). \qquad\text{(if X is simple)} E X := i = 1 ∑ n x i P ( A i ) . (if X is simple)
Note that, from this definition, we have
E 1 A = P ( A ) . \mathbb{E}\mathbb{1}_A = \mathbb{P}(A). E 1 A = P ( A ) .
Thus, we can always represent probabilities of sets as expectations of indicator random variables.
Now, consider a non-negative random variable X, which is not necessarily simple. Let ( X n ) n ≥ 0 (X_n)_{n\ge0} ( X n ) n ≥ 0 be an increasing sequence of simple random variables that converges almost surely to X. That is
X i ≤ X i + 1 , lim i → ∞ X i → X , P -a.s. X_i \le X_{i+1}, \qquad\lim_{i\to\infty} X_i \to X, \qquad\mathbb{P}\text{-a.s.} X i ≤ X i + 1 , i → ∞ lim X i → X , P -a.s.
We define the expectation of a non-negative random variable X as the following limit
E X : = lim i → ∞ E X i , (if X is non-negative) (4) \mathbb{E}X := \lim_{i\to\infty} \mathbb{E}X_i, \qquad\text{(if X is non-negative)} \tag*{(4)} E X := i → ∞ lim E X i , (if X is non-negative) (4)
where each of the expectations on the right-hand side are well-defined because all of the X i X_i X i are simple by construction. Finally, consider a general random variable X that could take either positive or negative values. Define
X + = max { X , 0 } , X − = max { − X , 0 } . X^+ = \max\{X, 0\}, \qquad X^- = \max\{-X, 0\}. X + = max { X , 0 } , X − = max { − X , 0 } .
Note that X + X^{+} X + and X − X^{-} X − are non-negative and X = X + − X − X = X^{+} - X^{-} X = X + − X − . With this in mind, we define
E X : = E X + − E X − , \mathbb{E}X := \mathbb{E}X^{+} - \mathbb{E}X^{-}, E X := E X + − E X − ,
where the expectations of X + X^{+} X + and X − X^{-} X − are defined via (4) .
Definition 1.6.1 of E X \mathbb{E}X E X makes sense if E ∣ X ∣ < ∞ \mathbb{E}|X| < \infty E ∣ X ∣ < ∞ or if E X ± = ∞ \mathbb{E}X^{\pm} = \infty E X ± = ∞ and E X ∓ < ∞ \mathbb{E}X^{\mp} < \infty E X ∓ < ∞ . In the latter case, we have E X = ± ∞ \mathbb{E}X = \pm\infty E X = ± ∞ . If both E X + = ∞ \mathbb{E}X^{+} = \infty E X + = ∞ and E X − = ∞ \mathbb{E}X^{-} = \infty E X − = ∞ , then we find ourselves in an ∞ − ∞ \infty-\infty ∞ − ∞ situation and, in this case, E X \mathbb{E}X E X is undefined.
Computing expectations
If X X X is either discrete or continuous Definition 1.6.1 reduces to the formulas one learns as an undergraduate.
discrete : E X = ∑ i x i f X ( x i ) , continuous : E X = ∫ R d x x f X ( x ) . \begin{aligned}
\text{discrete}: \quad&\mathbb{E}X = \sum_i x_i f_X(x_i),\\
\text{continuous}: \quad&\mathbb{E}X = \int_{\mathbb{R}} \mathrm{d}x\,x f_X(x).
\end{aligned} discrete : continuous : E X = i ∑ x i f X ( x i ) , E X = ∫ R d x x f X ( x ) .
In the discrete case, the sum runs over all possible values of x x x . More generally, we can express the expected value of X X X as
E X = ∫ R x F X ( d x ) : = lim ∥ Π ∥ → 0 ∑ i ( x i + x i + 1 2 ) ( F X ( x i + 1 ) − F X ( x i ) ) , (5) \mathbb{E}X = \int_{\mathbb{R}} x\,F_X(\mathrm{d}x) := \lim_{\lVert\Pi\rVert\to0}\sum_i \left(\frac{x_i+x_{i+1}}{2}\right)\left(F_X(x_{i+1})-F_X(x_i)\right), \tag*{(5)} E X = ∫ R x F X ( d x ) := ∥ Π ∥ → 0 lim i ∑ ( 2 x i + x i + 1 ) ( F X ( x i + 1 ) − F X ( x i ) ) , (5)
where Π \Pi Π is a partition of R \mathbb{R} R , meaning
Π = { x i , x 2 , … , x n } , x i < x i + 1 ∀ i , ∥ Π ∥ : = sup i ( x i + 1 − x i ) . \Pi= \{x_i,x_2,\ldots,x_n\}, \qquad x_i < x_{i+1}\ \forall i, \qquad\lVert\Pi\rVert:= \sup_i(x_{i+1}-x_i). Π = { x i , x 2 , … , x n } , x i < x i + 1 ∀ i , ∥ Π ∥ := i sup ( x i + 1 − x i ) .
The expression on the right-hand side of (5) is known as a Stieltjes integral . The advantage of using the Stieltjes integral ∫ x F X ( d x ) \int xF_X(\mathrm{d}x) ∫ x F X ( d x ) to compute an expectation is that every random variable X X X – whether it be discrete or continuous – has a distribution F X F_X F X . Thus, by using the Stieltjes integral, we avoid having to treat discrete and continuous cases separately.
Note that E \mathbb{E} E is a linear operator. If X X X and Y Y Y are random variables and a a a and b b b are constants, then
E ( a X + b Y ) = a E X + b E Y . \mathbb{E}(aX+bY) = a\mathbb{E}X+b\mathbb{E}Y. E ( a X + bY ) = a E X + b E Y .
How does one compute E g ( X ) \mathbb{E}g(X) E g ( X ) where g : R → R g:\mathbb{R}\to\mathbb{R} g : R → R ? Although we have not stated it explicitly, it should be obvious that if X X X is a random variable, then Y : = g ( X ) Y := g(X) Y := g ( X ) is also a random variable. 1 Thus, we have
E Y = E g ( X ) = ∫ Ω g ( X ( ω ) ) P ( d ω ) , \mathbb{E}Y = \mathbb{E}g(X) = \int_{\Omega} g(X(\omega))\mathbb{P}(\mathrm{d}\omega), E Y = E g ( X ) = ∫ Ω g ( X ( ω )) P ( d ω ) ,
which in the discrete and continuous cases become
discrete: E g ( X ) = ∑ i g ( x i ) f X ( x i ) , continuous: E g ( X ) = ∫ R d x g ( x ) f X ( x ) . \begin{aligned}
\text{discrete:}\quad&\mathbb{E}g(X)=\sum_i g(x_i)f_X(x_i),\\
\text{continuous:}\quad&\mathbb{E}g(X)=\int_{\mathbb{R}}\mathrm{d}x\,g(x)f_X(x).
\end{aligned} discrete: continuous: E g ( X ) = i ∑ g ( x i ) f X ( x i ) , E g ( X ) = ∫ R d x g ( x ) f X ( x ) .
Change of measure
Consider two probability measures P \mathbb{P} P and P ~ \widetilde{\mathbb{P}} P defined on a measurable space ( Ω , F ) (\Omega,\mathcal{F}) ( Ω , F ) . What is the relation between P \mathbb{P} P and P ~ \widetilde{\mathbb{P}} P ? The following theorem answers this question.
Theorem 1.7.1. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let Z Z Z be a random variable satisfying Z ≥ 0 Z \ge0 Z ≥ 0 P \mathbb{P} P -a.s. and E Z = 1 \mathbb{E}Z=1 E Z = 1 . Define a P ~ : F → [ 0 , 1 ] \widetilde{\mathbb{P}}:\mathcal{F}\to[0,1] P : F → [ 0 , 1 ] by
P ~ ( A ) : = E Z 1 A . (6) \widetilde{\mathbb{P}}(A):=\mathbb{E}Z1_A. \tag*{(6)} P ( A ) := E Z 1 A . (6) Then P ~ \widetilde{\mathbb{P}} P is a probability measure on ( Ω , F ) (\Omega,\mathcal{F}) ( Ω , F ) . Denote by E ~ \widetilde{\mathbb{E}} E the expectation taken with respect to P ~ \widetilde{\mathbb{P}} P . Then
E ~ X = E Z X , and if Z > 0 , then E X = E ~ 1 Z X . (7) \widetilde{\mathbb{E}}X=\mathbb{E}ZX,\qquad\text{and if } Z>0,\qquad\text{then}\qquad\mathbb{E}X=\widetilde{\mathbb{E}}\frac{1}{Z}X. \tag*{(7)} E X = E Z X , and if Z > 0 , then E X = E Z 1 X . (7) where X X X is a random variable defined on ( Ω , F ) (\Omega,\mathcal{F}) ( Ω , F ) .
Definition 1.7.2. We call the random variable Z Z Z in Theorem 1.7.1 the Radon-Nikodým derivative of P ~ \widetilde{\mathbb{P}} P with respect to P \mathbb{P} P .
Proof of Theorem 1.7.1 . First, we show that P ~ \widetilde{\mathbb{P}} P satisfies the properties of a probability measure given in Definition 1.2.1 . Let us check that P ~ ( Ω ) = 1 \widetilde{\mathbb{P}}(\Omega)=1 P ( Ω ) = 1 . We have
P ~ ( Ω ) = E Z 1 Ω = E Z = 1. \widetilde{\mathbb{P}}(\Omega)=\mathbb{E}Z1_{\Omega}=\mathbb{E}Z=1. P ( Ω ) = E Z 1 Ω = E Z = 1.
Next, we check that P ~ \widetilde{\mathbb{P}} P satisfies countable additivity. Let ( A i ) i ≥ 0 (A_i)_{i\ge0} ( A i ) i ≥ 0 be a sequence of disjoint sets. We have
P ~ ( ⋃ i A i ) = E 1 ⋃ i A i Z = ∑ i E 1 A i Z = ∑ i P ~ ( A i ) . \widetilde{\mathbb{P}}\left(\bigcup_i A_i\right)=\mathbb{E}1_{\bigcup_i A_i}Z=\sum_i\mathbb{E}1_{A_i}Z=\sum_i\widetilde{\mathbb{P}}(A_i). P ( i ⋃ A i ) = E 1 ⋃ i A i Z = i ∑ E 1 A i Z = i ∑ P ( A i ) .
Note that interchanging the sum with the expectation E ∑ i → ∑ i E \mathbb{E}\sum_i\to\sum_i\mathbb{E} E ∑ i → ∑ i E is allowed by Tonelli’s Theorem. Finally, to show equation (7) holds, it is enough to check that it holds for simple random variables X = ∑ i x i 1 A i X=\sum_i x_i1_{A_i} X = ∑ i x i 1 A i . We have
E ~ X = E ~ ∑ i x i 1 A i = ∑ i x i E Z 1 A i = ∑ i x i P ~ ( A i ) . \widetilde{\mathbb{E}}X=\widetilde{\mathbb{E}}\sum_i x_i1_{A_i}=\sum_i x_i\mathbb{E}Z1_{A_i}=\sum_i x_i\widetilde{\mathbb{P}}(A_i). E X = E i ∑ x i 1 A i = i ∑ x i E Z 1 A i = i ∑ x i P ( A i ) .
which agrees with the definition of Expectation for simple random variables. Finally, if Z > 0 Z > 0 Z > 0 , we have
E ~ 1 Z X = E 1 Z X = E X . \widetilde{\mathbb{E}}\frac{1}{Z}X = \mathbb{E}\frac{1}{Z}X = \mathbb{E}X. E Z 1 X = E Z 1 X = E X .
Definition 1.7.3. A probability measure P ~ \widetilde{\mathbb{P}} P defined on ( Ω , F ) (\Omega,\mathcal{F}) ( Ω , F ) is absolutely continuous with respect to another probability measure P \mathbb{P} P (equivalently, P \mathbb{P} P dominates P ~ \widetilde{\mathbb{P}} P ), written P ≫ P ~ \mathbb{P} \gg\widetilde{\mathbb{P}} P ≫ P , if
P ( A ) = 0 ⇒ P ~ ( A ) = 0. \mathbb{P}(A) = 0 \qquad\Rightarrow\qquad\widetilde{\mathbb{P}}(A) = 0. P ( A ) = 0 ⇒ P ( A ) = 0. In order for there to exist a Radon-Nikodým derivative Z = d P ~ d P Z = \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}} Z = d P d P we must have P ≫ P ~ \mathbb{P} \gg\widetilde{\mathbb{P}} P ≫ P . The reason for this is that the relationship between P \mathbb{P} P and P ~ \widetilde{\mathbb{P}} P is multiplicative P ~ ( d ω ) = Z ( ω ) P ( d ω ) \widetilde{\mathbb{P}}(\mathrm{d}\omega) = Z(\omega)\mathbb{P}(\mathrm{d}\omega) P ( d ω ) = Z ( ω ) P ( d ω ) . If P ( A ) = 0 \mathbb{P}(A) = 0 P ( A ) = 0 , then there is no random variable Z Z Z that would result in P ~ ( A ) > 0 \widetilde{\mathbb{P}}(A) > 0 P ( A ) > 0 . To see this more formally, note that
P ( A ) = E 1 A , P ~ ( A ) = E Z 1 A . (8) \mathbb{P}(A) = \mathbb{E}\mathbf{1}_{A}, \qquad\widetilde{\mathbb{P}}(A) = \mathbb{E}Z\mathbf{1}_{A}. \tag*{(8)} P ( A ) = E 1 A , P ( A ) = E Z 1 A . (8) Now, suppose that Z Z Z exists such that (8) holds and P ≫̸ P ~ \mathbb{P} \not\gg\widetilde{\mathbb{P}} P ≫ P . Then there exists some set A A A such that P ( A ) = 0 \mathbb{P}(A) = 0 P ( A ) = 0 and P ~ ( A ) > 0 \widetilde{\mathbb{P}}(A) > 0 P ( A ) > 0 . But, this is impossible because if P ( A ) = 0 \mathbb{P}(A) = 0 P ( A ) = 0 then Z 1 A = 0 Z\mathbf{1}_{A} = 0 Z 1 A = 0 P \mathbb{P} P -a.s., and thus P ~ ( A ) = E Z 1 A = 0 \widetilde{\mathbb{P}}(A) = \mathbb{E}Z\mathbf{1}_{A} = 0 P ( A ) = E Z 1 A = 0 . Thus, if (8) holds, we must have P ≫ P ~ \mathbb{P} \gg\widetilde{\mathbb{P}} P ≫ P .
Definition 1.7.4. Two probability measures P \mathbb{P} P and P ~ \widetilde{\mathbb{P}} P on ( Ω , F ) (\Omega,\mathcal{F}) ( Ω , F ) are equivalent , written P ∼ P ~ \mathbb{P} \sim\widetilde{\mathbb{P}} P ∼ P , if
P ( A ) = 0 ⟺ P ~ ( A ) = 0. \mathbb{P}(A) = 0 \qquad\Longleftrightarrow\qquad\widetilde{\mathbb{P}}(A) = 0. P ( A ) = 0 ⟺ P ( A ) = 0. Two probability measures are equivalent P ∼ P ~ \mathbb{P} \sim\widetilde{\mathbb{P}} P ∼ P if and only if the Radon-Nikodým Derivative that relates them is strictly positive Z > 0 Z > 0 Z > 0 . Equivalent measures agree on which events will happen with probability zero (and thus, they agree on which events will happen with probability one).
Example 1.7.5. Let us return to Example 1.4.3 . We set ( Ω , F ) = ( ( 0 , 1 ) , B ( ( 0 , 1 ) ) ) (\Omega,\mathcal{F}) = ((0,1),\mathcal{B}((0,1))) ( Ω , F ) = (( 0 , 1 ) , B (( 0 , 1 ))) . On this measure space, we define two probability measures P ( d ω ) = d ω \mathbb{P}(\mathrm{d}\omega) = \mathrm{d}\omega P ( d ω ) = d ω and P ~ ( d ω ) = 2 ω d ω \widetilde{\mathbb{P}}(\mathrm{d}\omega) = 2\omega\mathrm{d}\omega P ( d ω ) = 2 ω d ω . Note that we have
P ~ ( A ) = E ~ 1 A = ∫ Ω 1 A ( ω ) P ~ ( d ω ) = ∫ Ω 1 A ( ω ) 2 ω d ω = ∫ Ω 1 A 2 ω P ( d ω ) = E 1 A Z , Z ( ω ) : = 2 ω . \widetilde{\mathbb{P}}(A) = \widetilde{\mathbb{E}}\mathbf{1}_{A} = \int_{\Omega}\mathbf{1}_{A}(\omega)\widetilde{\mathbb{P}}(\mathrm{d}\omega) = \int_{\Omega}\mathbf{1}_{A}(\omega)2\omega\mathrm{d}\omega= \int_{\Omega}\mathbf{1}_{A}2\omega\mathbb{P}(\mathrm{d}\omega) = \mathbb{E}\mathbf{1}_{A}Z, \qquad Z(\omega) := 2\omega. P ( A ) = E 1 A = ∫ Ω 1 A ( ω ) P ( d ω ) = ∫ Ω 1 A ( ω ) 2 ω d ω = ∫ Ω 1 A 2 ω P ( d ω ) = E 1 A Z , Z ( ω ) := 2 ω .
One can easily check that E Z = 1 \mathbb{E}Z = 1 E Z = 1 and Z > 0 Z > 0 Z > 0 . Defining P ~ \widetilde{\mathbb{P}} P by (6) , one can easily check that (7) holds true.
It is quite common to use the notation
Z ( ω ) = d P ~ d P ( ω ) , P ~ ( d ω ) = d P ~ d P ( ω ) P ( d ω ) . Z(\omega) = \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}(\omega), \qquad\widetilde{\mathbb{P}}(\mathrm{d}\omega) = \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}(\omega)\mathbb{P}(\mathrm{d}\omega). Z ( ω ) = d P d P ( ω ) , P ( d ω ) = d P d P ( ω ) P ( d ω ) .
as a reminder of how the Radon-Nikodým Derivative Z Z Z relates P ~ \widetilde{\mathbb{P}} P to P \mathbb{P} P . For a finite probability space, it is true that Z ( ω ) = P ~ ( ω ) / P ( ω ) Z(\omega)=\widetilde{\mathbb{P}}(\omega)/\mathbb{P}(\omega) Z ( ω ) = P ( ω ) / P ( ω ) . However, for an infinite probability space, it makes no sense in general to define Z ( ω ) = P ~ ( ω ) / P ( ω ) Z(\omega)=\widetilde{\mathbb{P}}(\omega)/\mathbb{P}(\omega) Z ( ω ) = P ( ω ) / P ( ω ) since it may be that P ( ω ) = 0 \mathbb{P}(\omega)=0 P ( ω ) = 0 . Nevertheless, the heuristic Z ( ω ) = P ~ ( ω ) / P ( ω ) Z(\omega)=\widetilde{\mathbb{P}}(\omega)/\mathbb{P}(\omega) Z ( ω ) = P ( ω ) / P ( ω ) gives the correct intuition. In particular, for the special case of an infinite probability space in which P ( d ω ) = p ( ω ) d ω \mathbb{P}(\mathrm{d}\omega)=p(\omega)\mathrm{d}\omega P ( d ω ) = p ( ω ) d ω and P ~ ( d ω ) = p ~ ( ω ) d ω \widetilde{\mathbb{P}}(\mathrm{d}\omega)=\widetilde{p}(\omega)\mathrm{d}\omega P ( d ω ) = p ( ω ) d ω and P ≫ P ~ \mathbb{P}\gg\widetilde{\mathbb{P}} P ≫ P , we have Z ( ω ) = p ~ ( ω ) / p ( ω ) Z(\omega)=\widetilde{p}(\omega)/p(\omega) Z ( ω ) = p ( ω ) / p ( ω ) .
Example 1.7.6 (Change of measure Normal random variable). On ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) let X ∼ N ( 0 , 1 ) X\sim\mathcal{N}(0,1) X ∼ N ( 0 , 1 ) and define Y = X + θ Y=X+\theta Y = X + θ . Clearly, we have Y ∼ N ( θ , 1 ) Y\sim\mathcal{N}(\theta,1) Y ∼ N ( θ , 1 ) . Now, define a random variable Z Z Z by
Z = e − θ X − 1 2 θ 2 . Z=e^{-\theta X-\frac{1}{2}\theta^2}. Z = e − θ X − 2 1 θ 2 . Clearly Z > 0 Z>0 Z > 0 . We also have E Z = 1 \mathbb{E}Z=1 E Z = 1 . To see this, simply compute
E Z = ∫ R d x e − θ x − 1 2 θ 2 f X ( x ) = ∫ R d x e − θ x − 1 2 θ 2 1 2 π e − x 2 / 2 = ∫ R d x 1 2 π e − ( x + θ ) 2 / 2 = 1. \mathbb{E}Z=\int_{\mathbb{R}}\mathrm{d}x\,e^{-\theta x-\frac{1}{2}\theta^2}f_X(x)
=\int_{\mathbb{R}}\mathrm{d}x\,e^{-\theta x-\frac{1}{2}\theta^2}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}
=\int_{\mathbb{R}}\mathrm{d}x\,\frac{1}{\sqrt{2\pi}}e^{-(x+\theta)^2/2}=1. E Z = ∫ R d x e − θ x − 2 1 θ 2 f X ( x ) = ∫ R d x e − θ x − 2 1 θ 2 2 π 1 e − x 2 /2 = ∫ R d x 2 π 1 e − ( x + θ ) 2 /2 = 1. Since Z > 0 Z>0 Z > 0 and E Z = 1 \mathbb{E}Z=1 E Z = 1 , we can define a new probability measure P ~ \widetilde{\mathbb{P}} P with Z = d P ~ / d P Z=\mathrm{d}\widetilde{\mathbb{P}}/\mathrm{d}\mathbb{P} Z = d P / d P as the Radon-Nikodým derivative. Let us compute the distribution of Y Y Y under P ~ \widetilde{\mathbb{P}} P . We have
P ~ ( Y ≤ b ) = E Z 1 { Y ≤ b } = E e − θ X − 1 2 θ 2 1 { X ≤ b − θ } = ∫ − ∞ b − θ d x e − θ x − 1 2 θ 2 f X ( x ) = ∫ − ∞ b − θ d x 1 2 π e − ( x + θ ) 2 / 2 = ∫ − ∞ b d z 1 2 π e − z 2 / 2 . \widetilde{\mathbb{P}}(Y\le b)=\mathbb{E}Z\mathbf{1}_{\{Y\le b\}}=\mathbb{E}e^{-\theta X-\frac{1}{2}\theta^2}\mathbf{1}_{\{X\le b-\theta\}}
=\int_{-\infty}^{b-\theta}\mathrm{d}x\,e^{-\theta x-\frac{1}{2}\theta^2}f_X(x)
=\int_{-\infty}^{b-\theta}\mathrm{d}x\,\frac{1}{\sqrt{2\pi}}e^{-(x+\theta)^2/2}
=\int_{-\infty}^{b}\mathrm{d}z\,\frac{1}{\sqrt{2\pi}}e^{-z^2/2}. P ( Y ≤ b ) = E Z 1 { Y ≤ b } = E e − θ X − 2 1 θ 2 1 { X ≤ b − θ } = ∫ − ∞ b − θ d x e − θ x − 2 1 θ 2 f X ( x ) = ∫ − ∞ b − θ d x 2 π 1 e − ( x + θ ) 2 /2 = ∫ − ∞ b d z 2 π 1 e − z 2 /2 . Thus, under P ~ \widetilde{\mathbb{P}} P we see that Y ∼ N ( 0 , 1 ) Y\sim\mathcal{N}(0,1) Y ∼ N ( 0 , 1 ) . The Radon-Nikodým derivative Z Z Z changes the mean of Y Y Y from θ \theta θ to 0 0 0 , but it does not affect the variance of Y Y Y .
Example 1.7.7 (Change of measure Exponential random variable). On ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) , let X ∼ E ( λ ) X\sim\mathcal{E}(\lambda) X ∼ E ( λ ) and define
Z = μ λ e − ( μ − λ ) X . Z=\frac{\mu}{\lambda}e^{-(\mu-\lambda)X}. Z = λ μ e − ( μ − λ ) X . Clearly, we have Z ≥ 0 Z\ge0 Z ≥ 0 . We also have
E Z = E μ λ e − ( μ − λ ) X = ∫ 0 ∞ d x μ λ e − ( μ − λ ) x f X ( x ) \mathbb{E}Z=\mathbb{E}\frac{\mu}{\lambda}e^{-(\mu-\lambda)X}=\int_{0}^{\infty}\mathrm{d}x\,\frac{\mu}{\lambda}e^{-(\mu-\lambda)x}f_X(x) E Z = E λ μ e − ( μ − λ ) X = ∫ 0 ∞ d x λ μ e − ( μ − λ ) x f X ( x )
= ∫ 0 ∞ d x μ λ e − ( μ − λ ) x λ e − λ x = ∫ 0 ∞ d x μ e − μ x = 1. = \int_{0}^{\infty} \mathrm{d}x\,\frac{\mu}{\lambda}e^{-(\mu-\lambda)x}\lambda e^{-\lambda x} = \int_{0}^{\infty} \mathrm{d}x\,\mu e^{-\mu x} = 1. = ∫ 0 ∞ d x λ μ e − ( μ − λ ) x λ e − λ x = ∫ 0 ∞ d x μ e − μx = 1.
Thus, we can define a new probability measure P ~ \widetilde{\mathbb{P}} P with Z = d P ~ / d P Z = \mathrm{d}\widetilde{\mathbb{P}}/\mathrm{d}\mathbb{P} Z = d P / d P as the Radon-Nikodým derivative.
Let us compute the distribution of X X X under P ~ \widetilde{\mathbb{P}} P . We have
P ~ ( X ≤ b ) = E Z 1 { X ≤ b } = E μ λ e − ( μ − λ ) X 1 { X ≤ b } = ∫ 0 b μ λ e − ( μ − λ ) x f X ( x ) = ∫ 0 b d x μ λ e − ( μ − λ ) x λ e − λ x = ∫ 0 b d x μ e − μ x \begin{aligned}
\widetilde{\mathbb{P}}(X \le b) = \mathbb{E}Z1_{\{X\le b\}} = \mathbb{E}\frac{\mu}{\lambda}e^{-(\mu-\lambda)X}1_{\{X\le b\}} \\
&= \int_{0}^{b}\frac{\mu}{\lambda}e^{-(\mu-\lambda)x}f_X(x) = \int_{0}^{b}\mathrm{d}x\,\frac{\mu}{\lambda}e^{-(\mu-\lambda)x}\lambda e^{-\lambda x} = \int_{0}^{b}\mathrm{d}x\,\mu e^{-\mu x}
\end{aligned} P ( X ≤ b ) = E Z 1 { X ≤ b } = E λ μ e − ( μ − λ ) X 1 { X ≤ b } = ∫ 0 b λ μ e − ( μ − λ ) x f X ( x ) = ∫ 0 b d x λ μ e − ( μ − λ ) x λ e − λ x = ∫ 0 b d x μ e − μx
Thus, under P ~ \widetilde{\mathbb{P}} P , we have X ∼ E ( μ ) X \sim\mathcal{E}(\mu) X ∼ E ( μ ) .
Exercises
Exercise 1.1. Let F \mathcal{F} F be a σ \sigma σ -algebra of Ω \Omega Ω . Suppose B ∈ F B \in\mathcal{F} B ∈ F . Show that G : = { A ∩ B : A ∈ F } \mathcal{G} := \{A \cap B : A \in\mathcal{F}\} G := { A ∩ B : A ∈ F } is a σ \sigma σ -algebra of B B B .
Exercise 1.2. Let F \mathcal{F} F and G \mathcal{G} G be σ \sigma σ -algebras of Ω \Omega Ω . (a) Show that F ∩ G \mathcal{F} \cap\mathcal{G} F ∩ G is a σ \sigma σ -algebra of Ω \Omega Ω . (b) Show that F ∪ G \mathcal{F} \cup\mathcal{G} F ∪ G is not necessarily a σ \sigma σ -algebra of Ω \Omega Ω .
Exercise 1.3. Describe the probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) for the following three experiments: (a) a biased coin is tossed three times; (b) two balls are drawn without replacement from an urn which originally contained two blue and two red balls; (c) a biased coin is tossed repeatedly until a head turns up.
Exercise 1.4. Suppose X X X is a continuous random variable with distribution F X F_X F X . Let g g g be a strictly increasing continuous function. Define Y = g ( X ) Y = g(X) Y = g ( X ) . (a) What if F Y F_Y F Y , the distribution of Y Y Y ? (b) What is f Y f_Y f Y , the density of Y Y Y ?
Exercise 1.5. Suppose X X X is a continuous random variable with distribution F X F_X F X . Find F Y F_Y F Y where Y Y Y is given by (a) X 2 X^2 X 2 (b) ∣ X ∣ \sqrt{|X|} ∣ X ∣ (c) sin X \sin X sin X (d) F X ( X ) F_X(X) F X ( X ) .
Exercise 1.6. Suppose X X X is a continuous random variable defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Let f f f be the density of X X X under P \mathbb{P} P and assume f > 0 f > 0 f > 0 . Let g g g be the density function of a random variable. Define Z : = g ( X ) / f ( X ) Z := g(X)/f(X) Z := g ( X ) / f ( X ) . (a) Show that Z ≡ d P ~ / d P Z \equiv\mathrm{d}\widetilde{\mathbb{P}}/\mathrm{d}\mathbb{P} Z ≡ d P / d P defines a Radon-Nikodým derivative. (b) What is the density of X X X under P ~ \widetilde{\mathbb{P}} P ?
Exercise 1.7. Let X X X be uniformly distributed on [ 0 , 1 ] [0,1] [ 0 , 1 ] . For what function g g g is the random variable g ( X ) g(X) g ( X ) exponentially distributed with parameter 1 (i.e. g ( X ) ∼ E ( 1 ) g(X) \sim\mathcal{E}(1) g ( X ) ∼ E ( 1 ) )?
The notes from this chapter are taken primarily from (Shreve, 2004, Chapter 2).
Let us return to the coin-toss example of § 4.3.2 . If we are given no information about ω \omega ω what can we say about ω \omega ω ? In other words, what are the subsets of Ω \Omega Ω for which we can say: “ω \omega ω is in this set” or “ω \omega ω is not in this set”? The answer is ∅ \varnothing ∅ and Ω \Omega Ω , which, together, form the trivial σ \sigma σ -algebra F 0 = { ∅ , Ω } \mathcal{F}_0 = \{\varnothing, \Omega\} F 0 = { ∅ , Ω } .
Now suppose we are given the value of ω 1 \omega_1 ω 1 . What are the subsets of Ω \Omega Ω for which we can say: “ω \omega ω is in this set” or “ω \omega ω is not in this set”? The answer is the sets in F 0 \mathcal{F}_0 F 0 as well as A H A_{\mathrm{H}} A H and A T A_{\mathrm{T}} A T . Together, these sets form the σ \sigma σ -algebra F 1 = { ∅ , Ω , A H , A T } \mathcal{F}_1 = \{\varnothing, \Omega, A_{\mathrm{H}}, A_{\mathrm{T}}\} F 1 = { ∅ , Ω , A H , A T } . We say the sets in F 1 \mathcal{F}_1 F 1 are resolved by the first coin toss.
Now suppose we are given the value of ω 1 \omega_1 ω 1 and ω 2 \omega_2 ω 2 . What are the subsets of Ω \Omega Ω for which we can say: “ω \omega ω is in this set” or “ω \omega ω is not in this set”? The answer is the sets in F 2 \mathcal{F}_2 F 2 , given by
F 2 = { ∅ , Ω , A H , A T , A H H , A H T , A T T , A T H , A H H c , A H T c , A T T c , A T H c , A H H ∪ A T H , A H H ∪ A T T , A H T ∪ A T H , A H T ∪ A T T } . \mathcal{F}_2 = \left\{
\begin{gathered}
\varnothing, \Omega, A_{\mathrm{H}}, A_{\mathrm{T}}, A_{\mathrm{HH}}, A_{\mathrm{HT}}, A_{\mathrm{TT}}, A_{\mathrm{TH}}, A_{\mathrm{HH}}^{c}, A_{\mathrm{HT}}^{c}, A_{\mathrm{TT}}^{c}, A_{\mathrm{TH}}^{c},\\
A_{\mathrm{HH}} \cup A_{\mathrm{TH}}, A_{\mathrm{HH}} \cup A_{\mathrm{TT}}, A_{\mathrm{HT}} \cup A_{\mathrm{TH}}, A_{\mathrm{HT}} \cup A_{\mathrm{TT}}
\end{gathered}
\right\}. F 2 = { ∅ , Ω , A H , A T , A HH , A HT , A TT , A TH , A HH c , A HT c , A TT c , A TH c , A HH ∪ A TH , A HH ∪ A TT , A HT ∪ A TH , A HT ∪ A TT } .
The sets in F 2 \mathcal{F}_2 F 2 are resolved by the first two coin tosses.
Continuing in this way, for each n ∈ N n \in\mathbb{N} n ∈ N we can define F n \mathcal{F}_n F n as the σ \sigma σ -algebra containing the sets that are resolved by the first n n n coin tosses. Note that if a set A ∈ F n A \in\mathcal{F}_n A ∈ F n then A ∈ F n + 1 A \in\mathcal{F}_{n+1} A ∈ F n + 1 . Thus, F n ⊂ F n + 1 \mathcal{F}_n \subset\mathcal{F}_{n+1} F n ⊂ F n + 1 . In other words, F n + 1 \mathcal{F}_{n+1} F n + 1 contains more “information” than F n \mathcal{F}_n F n . This kind of structure is encapsulated in the following definition.
Definition 2.1.1. Let Ω \Omega Ω be a nonempty set. Let T T T be a fixed positive number (possibly infinity), and assume that for each t ∈ [ 0 , T ] t \in[0,T] t ∈ [ 0 , T ] there is a σ \sigma σ -algebra F t \mathcal{F}_t F t . Assume further that if $0 \le s \le t \le T$, then
F s ⊆ F t \mathcal{F}_s \subseteq\mathcal{F}_t F s ⊆ F t . Then we call the sequence of σ \sigma σ -algebras F = ( F t ) t ∈ [ 0 , T ] \mathbb{F} = (\mathcal{F}_t)_{t\in[0,T]} F = ( F t ) t ∈ [ 0 , T ] a continuous time filtration .
A discrete time filtration is a sequence of σ \sigma σ -algebras F = ( F n ) n ∈ N 0 \mathbb{F} = (\mathcal{F}_n)_{n\in\mathbb{N}_0} F = ( F n ) n ∈ N 0 that satisfies F n ⊆ F n + 1 \mathcal{F}_n \subseteq\mathcal{F}_{n+1} F n ⊆ F n + 1 for all n n n .
Example 2.1.2. Let Ω = C 0 [ 0 , T ] \Omega= \mathrm{C}_0[0,T] Ω = C 0 [ 0 , T ] , the set of continuous functions defined on [ 0 , T ] [0,T] [ 0 , T ] , starting from zero. We denote by ω = ( ω t ) t ∈ [ 0 , T ] \omega= (\omega_t)_{t\in[0,T]} ω = ( ω t ) t ∈ [ 0 , T ] an element of Ω \Omega Ω . Let F t \mathcal{F}_t F t be the σ \sigma σ -algebra generated by observing ω \omega ω over the interval [ 0 , t ] [0,t] [ 0 , t ] . Mathematically, we write this as
F t : = σ ( ω s , 0 ≤ s ≤ t ) . \mathcal{F}_t := \sigma(\omega_s, 0 \le s \le t). F t := σ ( ω s , 0 ≤ s ≤ t ) . It should be obvious that the sequence of σ \sigma σ -algebras F = ( F t ) t ∈ [ 0 , T ] \mathbb{F} = (\mathcal{F}_t)_{t\in[0,T]} F = ( F t ) t ∈ [ 0 , T ] forms a filtration. Below, we define two sets, one of which is in F t \mathcal{F}_t F t , one of which is not.
A : = { ω : sup 0 ≤ s ≤ t ω s ≤ 1 } ∈ F t , B : = { ω : ω T ≤ 1 } ∉ F t . A := \left\{\omega: \sup_{0\le s\le t}\omega_s \le1\right\} \in\mathcal{F}_t, \qquad B := \{\omega: \omega_T \le1\} \notin\mathcal{F}_t. A := { ω : 0 ≤ s ≤ t sup ω s ≤ 1 } ∈ F t , B := { ω : ω T ≤ 1 } ∈ / F t . The set A A A is an element of F t \mathcal{F}_t F t because, given the path of ω \omega ω over the interval [ 0 , t ] [0,t] [ 0 , t ] one can answer the question: is the maximum of ω \omega ω over the interval [ 0 , t ] [0,t] [ 0 , t ] less than 1? The set B B B is not an element of F t \mathcal{F}_t F t because one needs to know ω T \omega_T ω T in order to answer the question: is ω T ≤ 1 \omega_T \le1 ω T ≤ 1 ?
In the above example we generated a sequence of σ \sigma σ -algebras by observing directly an element ω ∈ Ω \omega\in\Omega ω ∈ Ω . Suppose that, instead of observing ω \omega ω we can observe only a random variable X ( ω ) X(\omega) X ( ω ) . We can use this information to generate a σ \sigma σ -algebra as well.
Definition 2.1.3. Let X X X be a random variable defined on a nonempty sample space Ω \Omega Ω . The σ \sigma σ -algebra generated by X X X , denoted σ ( X ) \sigma(X) σ ( X ) , is the collection of all subsets of Ω \Omega Ω of the form { X ∈ A } \{X \in A\} { X ∈ A } where A ∈ B ( R ) A \in\mathcal{B}(\mathbb{R}) A ∈ B ( R ) .
Example 2.1.4. Let us return to Example 1.4.2 . What is σ ( S 2 ) \sigma(S_2) σ ( S 2 ) ? From the definition, we need to ask, which sets are of the form { S 2 ∈ A } \{S_2 \in A\} { S 2 ∈ A } ? Since S 2 S_2 S 2 can only take three values, u 2 u^2 u 2 , u d ud u d and d 2 d^2 d 2 we check the following sets
{ S 2 = u 2 } = A H H , { S 2 = u d } = A H T ∪ A T H , { S 2 = d 2 } = A T T . \{S_2 = u^2\} = A_{\mathrm{HH}}, \qquad\{S_2 = ud\} = A_{\mathrm{HT}} \cup A_{\mathrm{TH}}, \qquad\{S_2 = d^2\} = A_{\mathrm{TT}}. { S 2 = u 2 } = A HH , { S 2 = u d } = A HT ∪ A TH , { S 2 = d 2 } = A TT . We add to these sets the sets that are necessary to form a σ \sigma σ -algebra (i.e., ∅ \varnothing ∅ , Ω \Omega Ω and unions and complements of the above sets) to obtain
σ ( S 2 ) = σ ( { A H H , A T T , A H T ∪ A T H } ) . \sigma(S_2) = \sigma(\{A_{\mathrm{HH}}, A_{\mathrm{TT}}, A_{\mathrm{HT}} \cup A_{\mathrm{TH}}\}). σ ( S 2 ) = σ ({ A HH , A TT , A HT ∪ A TH }) . Note, that σ ( S 2 ) ⊂ F 2 \sigma(S_2) \subset\mathcal{F}_2 σ ( S 2 ) ⊂ F 2 since A H T , A T H ∈ F 2 A_{\mathrm{HT}}, A_{\mathrm{TH}} \in\mathcal{F}_2 A HT , A TH ∈ F 2 but A H T , A T H ∉ σ ( S 2 ) A_{\mathrm{HT}}, A_{\mathrm{TH}} \notin\sigma(S_2) A HT , A TH ∈ / σ ( S 2 ) . The reason is that, if S 2 = u d S_2 = ud S 2 = u d we cannot say if ω 1 = T \omega_1 = \mathrm{T} ω 1 = T or ω 1 = H \omega_1 = \mathrm{H} ω 1 = H .
Definition 2.1.5. Let X X X be a random variable defined on a nonempty sample space Ω \Omega Ω . Let G \mathcal{G} G be a σ \sigma σ -algebra of subsets of Ω \Omega Ω . If σ ( X ) ⊂ G \sigma(X) \subset\mathcal{G} σ ( X ) ⊂ G we say that X X X is G \mathcal{G} G -measurable, and we write X ∈ G X \in\mathcal{G} X ∈ G .
A random variable X X X is G \mathcal{G} G -measurable if and only if the information in G \mathcal{G} G is sufficient to determine the value of X X X . Obviously, if X ∈ G X \in\mathcal{G} X ∈ G then g ( X ) ∈ G g(X) \in\mathcal{G} g ( X ) ∈ G (assuming is g g g is a measurable map from ( R , B ( R ) ) (\mathbb{R}, \mathcal{B}(\mathbb{R})) ( R , B ( R )) to ( R , B ( R ) ) (\mathbb{R}, \mathcal{B}(\mathbb{R})) ( R , B ( R )) ).
Eventually, we will want to consider stochastic processes X = ( X t ) t ∈ [ 0 , T ] X = (X_t)_{t\in[0,T]} X = ( X t ) t ∈ [ 0 , T ] and we will want to know at each time t t t if X t X_t X t is measurable with respect to σ \sigma σ -algebra F t \mathcal{F}_t F t .
Definition 2.1.6. Let Ω \Omega Ω be a nonempty sample space equipped with a filtration F = ( F t ) t ∈ [ 0 , T ] \mathbb{F} = (\mathcal{F}_t)_{t\in[0,T]} F = ( F t ) t ∈ [ 0 , T ] . Let X = ( X t ) t ∈ [ 0 , T ] X = (X_t)_{t\in[0,T]} X = ( X t ) t ∈ [ 0 , T ] be a collection of random variables indexed by t ∈ [ 0 , T ] t \in[0,T] t ∈ [ 0 , T ] . We say this collection of random variables is F \mathbb{F} F -adapted if X t ∈ F t X_t \in\mathcal{F}_t X t ∈ F t for all t ∈ [ 0 , T ] t \in[0,T] t ∈ [ 0 , T ] .
Independence
When X ∈ G X \in\mathcal{G} X ∈ G this means that the information in G \mathcal{G} G is sufficient to determine the value of X X X . On the other extreme, if X X X is independent (a term we will define soon) of G \mathcal{G} G this means that the information in G \mathcal{G} G tells us nothing about the value of X X X .
Definition 2.2.1. Let ( Ω , F , P ) (\Omega, \mathcal{F}, \mathbb{P}) ( Ω , F , P ) be a probability space. We say that two sets two sets A A A and B B B in F \mathcal{F} F are independent , written A ⊥ ⊥ B A \perp\!\!\!\perp B A ⊥ ⊥ B , if P ( A ∩ B ) = P ( A ) ⋅ P ( B ) \mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot\mathbb{P}(B) P ( A ∩ B ) = P ( A ) ⋅ P ( B ) .
Example 2.2.2. Let us return to the coin-toss example of § 4.3.2 . Consider two sets and their intersection
{ ω 1 = H } = A H , { ω 2 = H } = A H H ∪ A T H { ω 1 = H } ∩ { ω 2 = H } = A H H \{\omega_1 = \mathrm{H}\} = A_{\mathrm{H}}, \qquad\{\omega_2 = \mathrm{H}\} = A_{\mathrm{HH}} \cup A_{\mathrm{TH}} \qquad\{\omega_1 = \mathrm{H}\} \cap\{\omega_2 = \mathrm{H}\} = A_{\mathrm{HH}} { ω 1 = H } = A H , { ω 2 = H } = A HH ∪ A TH { ω 1 = H } ∩ { ω 2 = H } = A HH Since the coin tosses are independent, we should have { ω 1 = H } ⊥ ⊥ { ω 2 = H } \{\omega_1 = \mathrm{H}\} \perp\!\!\!\perp\{\omega_2 = \mathrm{H}\} { ω 1 = H } ⊥ ⊥ { ω 2 = H } Let us verify that these events are independent according to Definition 2.2.1 . We have
P ( A H ) = p , P ( A H H ∪ A T H ) = p 2 + q p = p , P ( A H H ) = p 2 . \mathbb{P}(A_{\mathrm{H}}) = p, \qquad\mathbb{P}(A_{\mathrm{HH}} \cup A_{\mathrm{TH}}) = p^2 + qp = p, \qquad\mathbb{P}(A_{\mathrm{HH}}) = p^2. P ( A H ) = p , P ( A HH ∪ A TH ) = p 2 + q p = p , P ( A HH ) = p 2 . Thus, P ( A H ) ⋅ P ( A H H ∪ A T H ) = P ( A H H ) \mathbb{P}(A_{\mathrm{H}}) \cdot\mathbb{P}(A_{\mathrm{HH}} \cup A_{\mathrm{TH}}) = \mathbb{P}(A_{\mathrm{HH}}) P ( A H ) ⋅ P ( A HH ∪ A TH ) = P ( A HH ) , as expected.
Example 2.2.3. Can a set be independent of itself? Surprisingly, the answer is “yes.” Suppose A ⊥ ⊥ A A \perp\!\!\!\perp A A ⊥ ⊥ A . Then, by the definition of independent sets, we have P ( A ∩ A ) = P ( A ) ⋅ P ( A ) \mathbb{P}(A\cap A) = \mathbb{P}(A)\cdot\mathbb{P}(A) P ( A ∩ A ) = P ( A ) ⋅ P ( A ) . We also have P ( A ∩ A ) = P ( A ) \mathbb{P}(A\cap A) = \mathbb{P}(A) P ( A ∩ A ) = P ( A ) . Combining these equations, we obtain P ( A ) ⋅ P ( A ) = P ( A ) \mathbb{P}(A) \cdot\mathbb{P}(A) = \mathbb{P}(A) P ( A ) ⋅ P ( A ) = P ( A ) . This equation has two solutions P ( A ) = 1 \mathbb{P}(A) = 1 P ( A ) = 1 and P ( A ) = 0 \mathbb{P}(A) = 0 P ( A ) = 0 . Thus, a set is independent of itself if the probability of that set is zero or one.
Having defined independent sets, we can now extend to independent σ \sigma σ -algebras and random variables.
Definition 2.2.4. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space, and let G \mathcal{G} G and H \mathcal{H} H be sub-σ \sigma σ -algebras of F \mathcal{F} F (i.e., G , H ⊆ F \mathcal{G},\mathcal{H} \subseteq\mathcal{F} G , H ⊆ F ). We say these two σ \sigma σ -algebras are independent , written G ⊥ ⊥ H \mathcal{G} \perp\!\!\!\perp\mathcal{H} G ⊥ ⊥ H , if
P ( A ∩ B ) = P ( A ) ⋅ P ( B ) , ∀ A ∈ G , ∀ B ∈ H . \mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot\mathbb{P}(B), \qquad\forall A \in\mathcal{G}, \qquad\forall B \in\mathcal{H}. P ( A ∩ B ) = P ( A ) ⋅ P ( B ) , ∀ A ∈ G , ∀ B ∈ H . Let X X X and Y Y Y be random variables on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . We say these two random variables are independent , written X ⊥ ⊥ Y X \perp\!\!\!\perp Y X ⊥ ⊥ Y , if σ ( X ) ⊥ ⊥ σ ( Y ) \sigma(X) \perp\!\!\!\perp\sigma(Y) σ ( X ) ⊥ ⊥ σ ( Y ) . Lastly, we say the random variable X X X is independent of the σ \sigma σ -algebra G \mathcal{G} G , written X ⊥ ⊥ G X \perp\!\!\!\perp\mathcal{G} X ⊥ ⊥ G , if σ ( X ) ⊥ ⊥ G \sigma(X) \perp\!\!\!\perp\mathcal{G} σ ( X ) ⊥ ⊥ G .
Recall from Definition 2.1.3 2.1.3 2.1.3 that σ ( X ) \sigma(X) σ ( X ) contains all sets of the form { X ∈ A } \{X \in A\} { X ∈ A } , where A ∈ B ( R ) A \in\mathcal{B}(\mathbb{R}) A ∈ B ( R ) . Combining this with Definition 2.2.4 2.2.4 2.2.4 we see that
X ⊥ ⊥ Y ⟺ P ( X ∈ A , Y ∈ B ) = P ( X ∈ A ) ⋅ P ( Y ∈ B ) , ∀ A , B ∈ B ( R ) . (9) X \perp\!\!\!\perp Y \quad\Longleftrightarrow\quad\mathbb{P}(X \in A, Y \in B) = \mathbb{P}(X \in A) \cdot\mathbb{P}(Y \in B), \qquad\forall A,B \in\mathcal{B}(\mathbb{R}). \tag*{(9)} X ⊥ ⊥ Y ⟺ P ( X ∈ A , Y ∈ B ) = P ( X ∈ A ) ⋅ P ( Y ∈ B ) , ∀ A , B ∈ B ( R ) . (9)
It follows from ( 2.1 ) (2.1) ( 2.1 ) that
X ⊥ ⊥ Y ⟹ E X Y = E X ⋅ E Y . X \perp\!\!\!\perp Y \quad\Longrightarrow\quad\mathbb{E}XY = \mathbb{E}X \cdot\mathbb{E}Y. X ⊥ ⊥ Y ⟹ E X Y = E X ⋅ E Y .
Note that E X Y = E X ⋅ E Y \mathbb{E}XY = \mathbb{E}X \cdot\mathbb{E}Y E X Y = E X ⋅ E Y does not imply X ⊥ ⊥ Y X \perp\!\!\!\perp Y X ⊥ ⊥ Y .
The above notion of independence is called pairwise independence . If X ⊥ ⊥ Y X \perp\!\!\!\perp Y X ⊥ ⊥ Y and Y ⊥ ⊥ Z Y \perp\!\!\!\perp Z Y ⊥ ⊥ Z , this notion of independence not imply X ⊥ ⊥ Z X \perp\!\!\!\perp Z X ⊥ ⊥ Z (for example, what if Z = X Z = X Z = X ?). Thus, at times, we may need a stronger notion of independence.
Definition 2.2.5. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space, and let G 1 , G 2 , … , G n \mathcal{G}_1,\mathcal{G}_2,\ldots,\mathcal{G}_n G 1 , G 2 , … , G n be sub-σ \sigma σ -algebras of F \mathcal{F} F . We say the sequence of σ \sigma σ -algebras are independent , if
P ( ⋂ i = 1 n A i ) = ∏ i = 1 n P ( A i ) , ∀ A 1 ∈ G 1 , ∀ A 2 ∈ G 2 , … , ∀ A n ∈ G n . \mathbb{P}\left(\bigcap_{i=1}^{n} A_i\right) = \prod_{i=1}^{n} \mathbb{P}(A_i), \qquad\forall A_1 \in\mathcal{G}_1, \forall A_2 \in\mathcal{G}_2,\ldots,\forall A_n \in\mathcal{G}_n. P ( i = 1 ⋂ n A i ) = i = 1 ∏ n P ( A i ) , ∀ A 1 ∈ G 1 , ∀ A 2 ∈ G 2 , … , ∀ A n ∈ G n . Let X 1 , X 2 , … , X n X_1,X_2,\ldots,X_n X 1 , X 2 , … , X n be a sequence of random variables on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . We say the sequence of random variables are independent if the σ \sigma σ -algebras σ ( X 1 ) , σ ( X 2 ) , … , σ ( X n ) \sigma(X_1),\sigma(X_2),\ldots,\sigma(X_n) σ ( X 1 ) , σ ( X 2 ) , … , σ ( X n ) are independent.
As with a pair of random variables, a sequence of random variables ( X i ) i ≥ 1 (X_i)_{i\geq1} ( X i ) i ≥ 1 is independent if and only if
P ( ⋂ i = 1 n { X i ∈ A i } ) = ∏ i = 1 n P ( X i ∈ A i ) , ∀ A 1 ∈ B ( R ) , ∀ A 2 ∈ B ( R ) , … , ∀ A n ∈ B ( R ) . \mathbb{P}\left(\bigcap_{i=1}^{n}\{X_i \in A_i\}\right) = \prod_{i=1}^{n} \mathbb{P}(X_i \in A_i), \qquad\forall A_1 \in\mathcal{B}(\mathbb{R}),\forall A_2 \in\mathcal{B}(\mathbb{R}),\ldots,\forall A_n \in\mathcal{B}(\mathbb{R}). P ( i = 1 ⋂ n { X i ∈ A i } ) = i = 1 ∏ n P ( X i ∈ A i ) , ∀ A 1 ∈ B ( R ) , ∀ A 2 ∈ B ( R ) , … , ∀ A n ∈ B ( R ) . We will often say that a sequence of random variables ( X i ) i ≥ 0 (X_i)_{i\geq0} ( X i ) i ≥ 0 is independent and identically distributed (iid), by which me mean all X i X_i X i have the same distribution and ( X i ) 1 ≤ i ≤ n (X_i)_{1\leq i\leq n} ( X i ) 1 ≤ i ≤ n are independent for every n ∈ N n \in\mathbb{N} n ∈ N .
Example 2.2.6. Let us return to the coin-toss example of § 4.3.2 . Let us define a sequence of random variables ( X i ) i ≥ 1 (X_i)_{i\geq1} ( X i ) i ≥ 1 via
X 1 ( ω ) = { 1 ω 1 = H , 0 ω 1 = T , X 2 ( ω ) = { 1 ω 2 = H , 0 ω 2 = T , X i ( ω ) = { X 1 i odd , X 2 i even . X_1(\omega)=
\begin{cases}
1 & \omega_1=\mathrm{H},\\
0 & \omega_1=\mathrm{T},
\end{cases}
\qquad
X_2(\omega)=
\begin{cases}
1 & \omega_2=\mathrm{H},\\
0 & \omega_2=\mathrm{T},
\end{cases}
\qquad
X_i(\omega)=
\begin{cases}
X_1 & i\ \text{odd},\\
X_2 & i\ \text{even}.
\end{cases} X 1 ( ω ) = { 1 0 ω 1 = H , ω 1 = T , X 2 ( ω ) = { 1 0 ω 2 = H , ω 2 = T , X i ( ω ) = { X 1 X 2 i odd , i even . Clearly, since the coin tosses are independent we have X 1 ⊥ ⊥ X 2 X_1 \perp\!\!\!\perp X_2 X 1 ⊥ ⊥ X 2 and X i ⊥ ⊥ X j X_i \perp\!\!\!\perp X_j X i ⊥ ⊥ X j if i i i is even and j j j is odd. But, the sequence ( X i ) 1 ≤ i ≤ n (X_i)_{1\leq i\leq n} ( X i ) 1 ≤ i ≤ n is not independent for any n ≥ 3 n\geq3 n ≥ 3 since X i = X i + 2 n X_i=X_{i+2n} X i = X i + 2 n for all n ∈ N n\in\mathbb{N} n ∈ N .
It is not easy to verify if two random variables X X X and Y Y Y are independent using Expression (2.1), since the equation must be verified for all Borel sets A , B ∈ B ( R ) A,B\in\mathcal{B}(\mathbb{R}) A , B ∈ B ( R ) . In fact, there is an easier way to check independence.
Definition 2.2.7. The joint distribution function F X , Y : R 2 → [ 0 , 1 ] F_{X,Y}:\mathbb{R}^2\to[0,1] F X , Y : R 2 → [ 0 , 1 ] of two random variables X X X and Y Y Y defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) is given by
F X , Y ( x , y ) : = P ( X ≤ x , Y ≤ y ) . F_{X,Y}(x,y):=\mathbb{P}(X\leq x,Y\leq y). F X , Y ( x , y ) := P ( X ≤ x , Y ≤ y ) .
Again, we have two special cases for jointly discrete and jointly continuous random variables.
Definition 2.2.8. Two random variables X X X and Y Y Y are called jointly discrete if the pair ( X , Y ) (X,Y) ( X , Y ) takes values in some countable set A = { x 1 , x 2 , … } × { y 1 , y 2 , … } ⊂ R 2 A=\{x_1,x_2,\ldots\}\times\{y_1,y_2,\ldots\}\subset\mathbb{R}^2 A = { x 1 , x 2 , … } × { y 1 , y 2 , … } ⊂ R 2 . We associate is a discrete random variable a probability mass function f X , Y : A → R f_{X,Y}:A\to\mathbb{R} f X , Y : A → R , defined by f X , Y ( x i , y j ) : = P ( X = x i , Y = y j ) f_{X,Y}(x_i,y_j):=\mathbb{P}(X=x_i,Y=y_j) f X , Y ( x i , y j ) := P ( X = x i , Y = y j ) .
Definition 2.2.9. A pair of random variables X X X and Y Y Y is called jointly continuous if its joint distribution function F X , Y F_{X,Y} F X , Y can be written as
F X , Y ( x , y ) = ∫ − ∞ x ∫ − ∞ y d u d v f X , Y ( u , v ) , ( x , y ) ∈ R 2 , F_{X,Y}(x,y)=\int_{-\infty}^{x}\int_{-\infty}^{y} \mathrm{d}u\mathrm{d}v\,f_{X,Y}(u,v),\qquad(x,y)\in\mathbb{R}^2, F X , Y ( x , y ) = ∫ − ∞ x ∫ − ∞ y d u d v f X , Y ( u , v ) , ( x , y ) ∈ R 2 , for some f X , Y : R 2 → [ 0 , ∞ ) f_{X,Y}:\mathbb{R}^2\to[0,\infty) f X , Y : R 2 → [ 0 , ∞ ) called the joint probability density function .
As in the one-dimensional case, it may help to think of the joint density function f X , Y f_{X,Y} f X , Y as f X , Y ( x , y ) d x d y = P ( X ∈ d x , Y ∈ d y ) f_{X,Y}(x,y)\mathrm{d}x\mathrm{d}y=\mathbb{P}(X\in\mathrm{d}x,Y\in\mathrm{d}y) f X , Y ( x , y ) d x d y = P ( X ∈ d x , Y ∈ d y ) .
Note that for jointly continuous random variables X X X and Y Y Y we have f X , Y ( x , y ) = ∂ x ∂ y F X ( x , y ) f_{X,Y}(x,y)=\partial_x\partial_yF_X(x,y) f X , Y ( x , y ) = ∂ x ∂ y F X ( x , y ) .
If the pair ( X , Y ) (X,Y) ( X , Y ) is either jointly discrete or jointly continuous, it is easy to compute P ( ( X , Y ) ∈ A ) \mathbb{P}((X,Y)\in A) P (( X , Y ) ∈ A ) for any A ∈ B ( R 2 ) A\in\mathcal{B}(\mathbb{R}^2) A ∈ B ( R 2 ) . We have
discrete: P ( ( X , Y ) ∈ A ) = ∑ { i , j : ( x i , y j ) ∈ A } f X , Y ( x i , y j ) , \text{discrete:}\qquad\mathbb{P}((X,Y)\in A)=\sum_{\{i,j:(x_i,y_j)\in A\}}f_{X,Y}(x_i,y_j), discrete: P (( X , Y ) ∈ A ) = { i , j : ( x i , y j ) ∈ A } ∑ f X , Y ( x i , y j ) ,
continuous:
P ( ( X , Y ) ∈ A ) = ∫ A d x d y f X , Y ( x , y ) . \mathbb{P}((X,Y)\in A)=\int_A dx\,dy\,f_{X,Y}(x,y). P (( X , Y ) ∈ A ) = ∫ A d x d y f X , Y ( x , y ) .
To recover the marginal distribution F X F_X F X from F X , Y F_{X,Y} F X , Y , simply note that
F X ( x ) = P ( X ≤ x ) = P ( X ≤ x , Y ≤ ∞ ) = F X , Y ( x , ∞ ) . F_X(x)=\mathbb{P}(X\le x)=\mathbb{P}(X\le x,Y\le\infty)=F_{X,Y}(x,\infty). F X ( x ) = P ( X ≤ x ) = P ( X ≤ x , Y ≤ ∞ ) = F X , Y ( x , ∞ ) .
It follows that for the discrete and continuous cases, we have, respectively
f X ( x i ) = ∑ j f X , Y ( x i , y j ) , f_X(x_i)=\sum_j f_{X,Y}(x_i,y_j), f X ( x i ) = j ∑ f X , Y ( x i , y j ) ,
f X ( x ) = ∫ R d y f X , Y ( x , y ) . f_X(x)=\int_{\mathbb{R}}dy\,f_{X,Y}(x,y). f X ( x ) = ∫ R d y f X , Y ( x , y ) .
The following theorem gives some easy-to-check conditions for independence.
Theorem 2.2.10. Let X X X and Y Y Y be random variables defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . The following conditions are equivalent (that is, if one of them holds, all of them hold)
X ⊥ ⊥ Y X \perp\!\!\!\perp Y X ⊥ ⊥ Y .
F X , Y ( x , y ) = F X ( x ) F Y ( y ) F_{X,Y}(x,y)=F_X(x)F_Y(y) F X , Y ( x , y ) = F X ( x ) F Y ( y ) for every ( x , y ) ∈ R 2 (x,y)\in\mathbb{R}^2 ( x , y ) ∈ R 2 .
Discrete case: f X , Y ( x , y ) = f X ( x ) f Y ( y ) f_{X,Y}(x,y)=f_X(x)f_Y(y) f X , Y ( x , y ) = f X ( x ) f Y ( y ) for every ( x , y ) ∈ R 2 (x,y)\in\mathbb{R}^2 ( x , y ) ∈ R 2 . Continuous case: f X , Y ( x , y ) = f X ( x ) f Y ( y ) f_{X,Y}(x,y)=f_X(x)f_Y(y) f X , Y ( x , y ) = f X ( x ) f Y ( y ) for ‘almost’ every ( x , y ) ∈ R 2 (x,y)\in\mathbb{R}^2 ( x , y ) ∈ R 2 .
E e i u X + i v Y = E e i u X ⋅ E e i v Y \mathbb{E}e^{iuX+ivY}=\mathbb{E}e^{iuX}\cdot\mathbb{E}e^{ivY} E e i u X + i v Y = E e i u X ⋅ E e i v Y for all ( u , v ) ∈ R 2 (u,v)\in\mathbb{R}^2 ( u , v ) ∈ R 2 .
Together with expectation, the most important statistical properties of a random variable (or pair) are the variance and co-variance.
Definition 2.2.11. The variance of a random variable X X X , written V X VX V X is defined by
V X = E ( X − E X ) 2 = E X 2 − ( E X ) 2 , VX=\mathbb{E}(X-\mathbb{E}X)^2=\mathbb{E}X^2-(\mathbb{E}X)^2, V X = E ( X − E X ) 2 = E X 2 − ( E X ) 2 , whenever the expectation exists.
Definition 2.2.12. The co-variance of two random variables X X X and Y Y Y , written CoV [ X , Y ] \operatorname{CoV}[X,Y] CoV [ X , Y ] is defined by
CoV [ X , Y ] = E ( X − E X ) ( Y − E Y ) = E X Y − E X ⋅ E Y , \operatorname{CoV}[X,Y]=\mathbb{E}(X-\mathbb{E}X)(Y-\mathbb{E}Y)=\mathbb{E}XY-\mathbb{E}X\cdot\mathbb{E}Y, CoV [ X , Y ] = E ( X − E X ) ( Y − E Y ) = E X Y − E X ⋅ E Y , whenever the expectation exists.
Note that CoV [ X , X ] = V X \operatorname{CoV}[X,X]=VX CoV [ X , X ] = V X .
Note V ⋅ V\cdot V ⋅ is not a linear operator, since
V [ a X + b Y ] = a 2 V X + b 2 V Y + 2 a b CoV [ X , Y ] . V[aX+bY]=a^2VX+b^2VY+2ab\operatorname{CoV}[X,Y]. V [ a X + bY ] = a 2 V X + b 2 V Y + 2 ab CoV [ X , Y ] .
where a a a and b b b are constants.
Definition 2.2.13. We say two random variables X X X and Y Y Y are uncorrelated if CoV [ X , Y ] = 0 \operatorname{CoV}[X,Y]=0 CoV [ X , Y ] = 0 .
Note that X ⊥ ⊥ Y X \perp\!\!\!\perp Y X ⊥ ⊥ Y implies X X X and Y Y Y are uncorrelated. However, the converse is not true.
Conditional expectation
Let X X X be a random variable defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let G \mathcal{G} G be a sub-σ \sigma σ algebra of F \mathcal{F} F . When X ∈ G X \in\mathcal{G} X ∈ G this means that the information in G \mathcal{G} G is sufficient to determine the value of X X X . When X ⊥ ⊥ G X \perp\!\!\perp\mathcal{G} X ⊥ ⊥ G , this means that the information in G \mathcal{G} G gives us no information at all about X X X . Usually, however, the information in G \mathcal{G} G gives us some information about X X X , but not enough to determine X X X exactly. And this brings us to the notion of conditioning.
Presumably, you have run across the following formula for the conditional probability of a set A A A given B B B
P ( A ∣ B ) = P ( A ∩ B ) P ( B ) , P ( B ) > 0. \mathbb{P}(A\mid B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}, \qquad\mathbb{P}(B)>0. P ( A ∣ B ) = P ( B ) P ( A ∩ B ) , P ( B ) > 0.
When ( X , Y ) (X,Y) ( X , Y ) have are jointly discrete or jointly continuous, this readily leads to conditional probability mass function
discrete : f X ∣ Y ( x i , y j ) : = P ( X = x i ∣ Y = y j ) = P ( X = x i ∩ Y = y j ) P ( Y = y j ) = f X , Y ( x i , y j ) f Y ( y j ) , \text{discrete}: \qquad f_{X\mid Y}(x_i,y_j):=\mathbb{P}(X=x_i\mid Y=y_j)=\frac{\mathbb{P}(X=x_i\cap Y=y_j)}{\mathbb{P}(Y=y_j)}=\frac{f_{X,Y}(x_i,y_j)}{f_Y(y_j)}, discrete : f X ∣ Y ( x i , y j ) := P ( X = x i ∣ Y = y j ) = P ( Y = y j ) P ( X = x i ∩ Y = y j ) = f Y ( y j ) f X , Y ( x i , y j ) ,
continuous : f X ∣ Y ( x , y ) d x : = P ( X ∈ d x ∣ Y = y ) = P ( X ∈ d x ∩ Y ∈ d y ) P ( Y ∈ d y ) = f X , Y ( x , y ) f Y ( y ) d x . \text{continuous}: \qquad f_{X\mid Y}(x,y)\,\mathrm{d}x:=\mathbb{P}(X\in\mathrm{d}x\mid Y=y)=\frac{\mathbb{P}(X\in\mathrm{d}x\cap Y\in\mathrm{d}y)}{\mathbb{P}(Y\in\mathrm{d}y)}=\frac{f_{X,Y}(x,y)}{f_Y(y)}\,\mathrm{d}x. continuous : f X ∣ Y ( x , y ) d x := P ( X ∈ d x ∣ Y = y ) = P ( Y ∈ d y ) P ( X ∈ d x ∩ Y ∈ d y ) = f Y ( y ) f X , Y ( x , y ) d x .
And from this, we can define E [ X ∣ Y = y ] \mathbb{E}[X\mid Y=y] E [ X ∣ Y = y ] , the conditional expectation of X X X given Y = y Y=y Y = y
discrete : E [ X ∣ Y = y j ] : = ∑ i x i f X ∣ Y ( x i , y j ) , \text{discrete}: \qquad\mathbb{E}[X\mid Y=y_j]:=\sum_i x_i f_{X\mid Y}(x_i,y_j), discrete : E [ X ∣ Y = y j ] := i ∑ x i f X ∣ Y ( x i , y j ) ,
continuous : E [ X ∣ Y = y ] : = ∫ R d x x f X ∣ Y ( x , y ) \text{continuous}: \qquad\mathbb{E}[X\mid Y=y]:=\int_{\mathbb{R}}\mathrm{d}x\,x f_{X\mid Y}(x,y) continuous : E [ X ∣ Y = y ] := ∫ R d x x f X ∣ Y ( x , y )
Note that E [ X ∣ Y = y ] \mathbb{E}[X\mid Y=y] E [ X ∣ Y = y ] is simply a function of y y y – there is nothing random about it.
Unfortunately, there are cases for which the pair ( X , Y ) (X,Y) ( X , Y ) are neither jointly discrete nor jointly continuous. And, for these cases we need a more general notion of conditional expectation. Here we will make two conceptual leaps:
We will condition with respect to a σ \sigma σ -algebra rather than conditioning on an event.
The conditional expectation will be a random variable.
We will just hop in with our new definition of conditional expectation and then we will see, through an example, that his new definition makes sense.
Definition 2.3.1. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space, let G \mathcal{G} G be a sub-σ \sigma σ -algebra of F \mathcal{F} F , and let X X X be a random variable that is either nonnegative or integrable. The conditional expectation of X X X given G \mathcal{G} G , denoted E [ X ∣ G ] \mathbb{E}[X\mid\mathcal{G}] E [ X ∣ G ] , is any random variable that satisfies
Measurability: E [ X ∣ G ] ∈ G \mathbb{E}[X \mid\mathcal{G}] \in\mathcal{G} E [ X ∣ G ] ∈ G .
Partial averaging: E [ 1 A E [ X ∣ G ] ] = E [ 1 A X ] \mathbb{E}[\mathbf{1}_A\mathbb{E}[X \mid\mathcal{G}]] = \mathbb{E}[\mathbf{1}_A X] E [ 1 A E [ X ∣ G ]] = E [ 1 A X ] for all A ∈ G A \in\mathcal{G} A ∈ G . Alternatively, E [ Z E [ X ∣ G ] ] = E [ Z X ] \mathbb{E}[Z\mathbb{E}[X \mid\mathcal{G}]] = \mathbb{E}[ZX] E [ Z E [ X ∣ G ]] = E [ Z X ] for all bounded random variables Z ∈ G Z \in\mathcal{G} Z ∈ G .
When G = σ ( Y ) \mathcal{G} = \sigma(Y) G = σ ( Y ) we shall often use the short-hand notation E [ X ∣ Y ] : = E [ X ∣ σ ( Y ) ] \mathbb{E}[X \mid Y] := \mathbb{E}[X \mid\sigma(Y)] E [ X ∣ Y ] := E [ X ∣ σ ( Y )] .
Conditional probabilities are defined from conditional expectations using
P ( A ∣ G ) = E [ 1 A ∣ G ] . \mathbb{P}(A \mid\mathcal{G}) = \mathbb{E}[\mathbf{1}_A \mid\mathcal{G}]. P ( A ∣ G ) = E [ 1 A ∣ G ] .
Admittedly, Definition Definition 2.3.1 is rather abstract (and, for the purposes of computation, useless). In fact, it is not at all clear from Definition 2.3.1 that E [ X ∣ G ] \mathbb{E}[X \mid\mathcal{G}] E [ X ∣ G ] even exists! It does exist, though we will not prove this here.
Conditional expectation has an interesting L 2 L^2 L 2 interpretation. Consider a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Let G \mathcal{G} G be a sub-σ \sigma σ -algebra of F \mathcal{F} F (i.e., G ⊆ F \mathcal{G} \subseteq\mathcal{F} G ⊆ F ). Define
L 2 ( Ω , F , P ) : = { X : Ω → R s.t. X − 1 ( B ) ∈ F ∀ B ∈ B ( R ) and E ∣ X ∣ 2 < ∞ } , L^2(\Omega,\mathcal{F},\mathbb{P}) := \{X:\Omega\to\mathbb{R}\ \text{s.t.}\ X^{-1}(B) \in\mathcal{F}\ \forall B \in\mathcal{B}(\mathbb{R})\ \text{and}\ \mathbb{E}|X|^2 < \infty\}, L 2 ( Ω , F , P ) := { X : Ω → R s.t. X − 1 ( B ) ∈ F ∀ B ∈ B ( R ) and E ∣ X ∣ 2 < ∞ } ,
and likewise for L 2 ( Ω , G , P ) L^2(\Omega,\mathcal{G},\mathbb{P}) L 2 ( Ω , G , P ) . Clearly, since G ⊆ F \mathcal{G} \subseteq\mathcal{F} G ⊆ F we have L 2 ( Ω , G , P ) ⊆ L 2 ( Ω , F , P ) L^2(\Omega,\mathcal{G},\mathbb{P}) \subseteq L^2(\Omega,\mathcal{F},\mathbb{P}) L 2 ( Ω , G , P ) ⊆ L 2 ( Ω , F , P ) . Next, define an inner product
⟨ X , Y ⟩ : = E X Y , X , Y ∈ L 2 ( Ω , F , P ) . \langle X,Y\rangle:= \mathbb{E}XY,\qquad X,Y \in L^2(\Omega,\mathcal{F},\mathbb{P}). ⟨ X , Y ⟩ := E X Y , X , Y ∈ L 2 ( Ω , F , P ) .
From the definition of conditional expectation, we have
0 = ⟨ X − E [ X ∣ G ] , Z ⟩ = E ( X − E [ X ∣ G ] ) Z , ∀ Z ∈ G . 0 = \langle X-\mathbb{E}[X \mid\mathcal{G}],Z\rangle= \mathbb{E}(X-\mathbb{E}[X \mid\mathcal{G}])Z,\qquad\forall Z \in\mathcal{G}. 0 = ⟨ X − E [ X ∣ G ] , Z ⟩ = E ( X − E [ X ∣ G ]) Z , ∀ Z ∈ G .
Thus, E [ X ∣ G ] \mathbb{E}[X \mid\mathcal{G}] E [ X ∣ G ] is the projection of X ∈ L 2 ( Ω , F , P ) X \in L^2(\Omega,\mathcal{F},\mathbb{P}) X ∈ L 2 ( Ω , F , P ) onto the subspace L 2 ( Ω , G , P ) L^2(\Omega,\mathcal{G},\mathbb{P}) L 2 ( Ω , G , P ) .
When conditioning on the σ \sigma σ -algebra generated by a random variable, it is easiest to use the following formula
E [ X ∣ Y ] = ψ ( Y ) , ψ ( y ) : = E [ X ∣ Y = y ] . (10) \mathbb{E}[X \mid Y] = \psi(Y),\qquad\psi(y) := \mathbb{E}[X \mid Y=y]. \tag*{(10)} E [ X ∣ Y ] = ψ ( Y ) , ψ ( y ) := E [ X ∣ Y = y ] . (10)
The following example should help to build some intuition for conditional expectation.
Example 2.3.2. Let Ω = { a , b , c , d , e , f } \Omega= \{a,b,c,d,e,f\} Ω = { a , b , c , d , e , f } , F = 2 Ω \mathcal{F} = 2^\Omega F = 2 Ω and P ( ω ) = ( 1 / 6 ) \mathbb{P}(\omega) = (1/6) P ( ω ) = ( 1/6 ) for ω = a , b , … , f \omega= a,b,\ldots,f ω = a , b , … , f . Define two random variables X X X and Y Y Y on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) as follows
( ω X ( ω ) Y ( ω ) ) = ( a b c d e f 1 3 3 3 5 7 2 2 1 1 7 7 ) . \begin{pmatrix}
\omega\\
X(\omega) \\
Y(\omega)
\end{pmatrix}
=
\begin{pmatrix}
a & b & c & d & e & f \\
1 & 3 & 3 & 3 & 5 & 7 \\
2 & 2 & 1 & 1 & 7 & 7
\end{pmatrix}. ω X ( ω ) Y ( ω ) = a 1 2 b 3 2 c 3 1 d 3 1 e 5 7 f 7 7 .
Let G = σ ( Y ) \mathcal{G}=\sigma(Y) G = σ ( Y ) . Next, let us compute E [ X ∣ Y ] \mathbb{E}[X\mid Y] E [ X ∣ Y ] using (10) and check if it agrees with Definition 2.3.1. We have
( ω Y ( ω ) E [ X ∣ Y ] ( ω ) ) = ( a b c d e f 2 2 1 1 7 7 2 2 3 3 6 6 ) . \begin{pmatrix}
\omega\\
Y(\omega) \\
\mathbb{E}[X\mid Y](\omega)
\end{pmatrix}
=
\begin{pmatrix}
a & b & c & d & e & f \\
2 & 2 & 1 & 1 & 7 & 7 \\
2 & 2 & 3 & 3 & 6 & 6
\end{pmatrix}. ω Y ( ω ) E [ X ∣ Y ] ( ω ) = a 2 2 b 2 2 c 1 3 d 1 3 e 7 6 f 7 6 .
Le us check measurability: is E [ X ∣ Y ] ∈ σ ( Y ) \mathbb{E}[X\mid Y]\in\sigma(Y) E [ X ∣ Y ] ∈ σ ( Y ) ? In other words, is σ ( E [ X ∣ Y ] ) ⊆ σ ( Y ) \sigma(\mathbb{E}[X\mid Y])\subseteq\sigma(Y) σ ( E [ X ∣ Y ]) ⊆ σ ( Y ) ? We have
σ ( Y ) = σ ( { a , b } , { c , d } , { e , f } ) , \sigma(Y)=\sigma(\{a,b\},\{c,d\},\{e,f\}), σ ( Y ) = σ ({ a , b } , { c , d } , { e , f }) ,
σ ( E [ X ∣ Y ] ) = σ ( { a , b } , { c , d } , { e , f } ) . \sigma(\mathbb{E}[X\mid Y])=\sigma(\{a,b\},\{c,d\},\{e,f\}). σ ( E [ X ∣ Y ]) = σ ({ a , b } , { c , d } , { e , f }) .
So, yes, E [ X ∣ Y ] ∈ σ ( Y ) \mathbb{E}[X\mid Y]\in\sigma(Y) E [ X ∣ Y ] ∈ σ ( Y ) . Another way to think of measurability is to simply ask: given the value of Y Y Y , can one determine the value of E [ X ∣ Y ] \mathbb{E}[X\mid Y] E [ X ∣ Y ] ? Clearly, in this case the answer is “yes.” Next, let us check the partial averaging property: does E [ 1 A E [ X ∣ Y ] ] = E [ 1 A X ] \mathbb{E}[\mathbf{1}_A\mathbb{E}[X\mid Y]]=\mathbb{E}[\mathbf{1}_A X] E [ 1 A E [ X ∣ Y ]] = E [ 1 A X ] for all A ∈ σ ( Y ) A\in\sigma(Y) A ∈ σ ( Y ) . Rather than check this for every A ∈ σ ( Y ) A\in\sigma(Y) A ∈ σ ( Y ) , let us just check that this holds for the sets { a , b } \{a,b\} { a , b } , { c , d } \{c,d\} { c , d } and { e , f } \{e,f\} { e , f } . We have
A = { a , b } , E [ 1 A E [ X ∣ Y ] ] = P ( a ) 2 + P ( b ) 2 = 4 / 6 , E [ 1 A X ] = P ( a ) 1 + P ( b ) 3 = 4 / 6 , A=\{a,b\},\qquad\mathbb{E}[\mathbf{1}_A\mathbb{E}[X\mid Y]]=\mathbb{P}(a)2+\mathbb{P}(b)2=4/6,\qquad\mathbb{E}[\mathbf{1}_A X]=\mathbb{P}(a)1+\mathbb{P}(b)3=4/6, A = { a , b } , E [ 1 A E [ X ∣ Y ]] = P ( a ) 2 + P ( b ) 2 = 4/6 , E [ 1 A X ] = P ( a ) 1 + P ( b ) 3 = 4/6 ,
A = { c , d } , E [ 1 A E [ X ∣ Y ] ] = P ( c ) 3 + P ( d ) 3 = 6 / 6 , E [ 1 A X ] = P ( c ) 3 + P ( d ) 3 = 6 / 6 , A=\{c,d\},\qquad\mathbb{E}[\mathbf{1}_A\mathbb{E}[X\mid Y]]=\mathbb{P}(c)3+\mathbb{P}(d)3=6/6,\qquad\mathbb{E}[\mathbf{1}_A X]=\mathbb{P}(c)3+\mathbb{P}(d)3=6/6, A = { c , d } , E [ 1 A E [ X ∣ Y ]] = P ( c ) 3 + P ( d ) 3 = 6/6 , E [ 1 A X ] = P ( c ) 3 + P ( d ) 3 = 6/6 ,
A = { e , f } , E [ 1 A E [ X ∣ Y ] ] = P ( e ) 6 + P ( f ) 6 = 12 / 6 , E [ 1 A X ] = P ( e ) 5 + P ( f ) 7 = 12 / 6. A=\{e,f\},\qquad\mathbb{E}[\mathbf{1}_A\mathbb{E}[X\mid Y]]=\mathbb{P}(e)6+\mathbb{P}(f)6=12/6,\qquad\mathbb{E}[\mathbf{1}_A X]=\mathbb{P}(e)5+\mathbb{P}(f)7=12/6. A = { e , f } , E [ 1 A E [ X ∣ Y ]] = P ( e ) 6 + P ( f ) 6 = 12/6 , E [ 1 A X ] = P ( e ) 5 + P ( f ) 7 = 12/6.
The following properties are arguably more important to remember than the definition of conditional expectation. Memorize them!
Theorem 2.3.3. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space and let G \mathcal{G} G be a sub-σ \sigma σ -algebra of F \mathcal{F} F . Conditional expectations satisfy the following properties.
Linearity: E [ a X + b Y ∣ G ] = a E [ X ∣ G ] + b E [ Y ∣ G ] \mathbb{E}[aX+bY\mid\mathcal{G}]=a\mathbb{E}[X\mid\mathcal{G}]+b\mathbb{E}[Y\mid\mathcal{G}] E [ a X + bY ∣ G ] = a E [ X ∣ G ] + b E [ Y ∣ G ] .
Taking out what is known: if X ∈ G X\in\mathcal{G} X ∈ G and E ∣ X Y ∣ < ∞ \mathbb{E}|XY|<\infty E ∣ X Y ∣ < ∞ then E [ X Y ∣ G ] = X E [ Y ∣ G ] \mathbb{E}[XY\mid\mathcal{G}]=X\mathbb{E}[Y\mid\mathcal{G}] E [ X Y ∣ G ] = X E [ Y ∣ G ] .
Iterated conditioning: if H \mathcal{H} H is a sub-σ \sigma σ -algebra of G \mathcal{G} G then E [ E [ X ∣ G ] ∣ H ] = E [ X ∣ H ] \mathbb{E}[\mathbb{E}[X\mid\mathcal{G}]\mid\mathcal{H}]=\mathbb{E}[X\mid\mathcal{H}] E [ E [ X ∣ G ] ∣ H ] = E [ X ∣ H ] .
Independence: if X ⊥ ⊥ G X\perp\!\!\!\perp\mathcal{G} X ⊥ ⊥ G then E [ X ∣ G ] = E X \mathbb{E}[X\mid\mathcal{G}]=\mathbb{E}X E [ X ∣ G ] = E X .
Theorem 2.3.3 can be proved directly from Definition 2.3.1, though we will not do so here. In addition to the above properties, the following Theorem, which we state without proof is often useful:
Theorem 2.3.4 (Jensen’s inequality). Let X X X be a random variable defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let G \mathcal{G} G be a sub-σ \sigma σ -algebra of F \mathcal{F} F . Suppose φ : R → R \varphi:\mathbb{R}\to\mathbb{R} φ : R → R is a convex function and that E ∣ X ∣ < ∞ \mathbb{E}|X|<\infty E ∣ X ∣ < ∞ and E ∣ φ ( X ) ∣ < ∞ \mathbb{E}|\varphi(X)|<\infty E ∣ φ ( X ) ∣ < ∞ . Then we have
φ ( E [ X ∣ G ] ) ≤ E [ φ ( X ) ∣ G ] , P –a.s. (11) \varphi(\mathbb{E}[X\mid\mathcal{G}])\leq\mathbb{E}[\varphi(X)\mid\mathcal{G}],\qquad\mathbb{P}\text{--a.s.} \tag*{(11)} φ ( E [ X ∣ G ]) ≤ E [ φ ( X ) ∣ G ] , P –a.s. (11)
In order to keep straight which direction the inequality in (11) goes, it is helpful to remember that φ ( x ) = x 2 \varphi(x)=x^2 φ ( x ) = x 2 is a convex function and the conditional variance of a function satisfies
V [ X ∣ G ] = E [ X 2 ∣ G ] − E [ X ∣ G ] 2 ≥ 0. \mathbb{V}[X\mid\mathcal{G}]=\mathbb{E}[X^2\mid\mathcal{G}]-\mathbb{E}[X\mid\mathcal{G}]^2\ge0. V [ X ∣ G ] = E [ X 2 ∣ G ] − E [ X ∣ G ] 2 ≥ 0.
Now that we have defined conditional expectation and established some of its key properties, we can define “Markov process” and “martingale” – two seemingly similar, but very distinct concepts.
Definition 2.3.5. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space, let T T T be a fixed positive number, and let F = ( F t ) t ∈ [ 0 , T ] \mathbb{F}=(\mathcal{F}_t)_{t\in[0,T]} F = ( F t ) t ∈ [ 0 , T ] be a filtration of sub-σ \sigma σ -algebras of F \mathcal{F} F . Consider an F \mathbb{F} F -adapted stochastic process M = ( M t ) t ∈ [ 0 , T ] M=(M_t)_{t\in[0,T]} M = ( M t ) t ∈ [ 0 , T ] . Suppose E ∣ M t ∣ < ∞ \mathbb{E}|M_t|<\infty E ∣ M t ∣ < ∞ for all t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] . Then, we say that M M M is
a ∗ m a r t i n g a l e ∗ if E [ M t ∣ F s ] = M s , ∀ 0 ≤ s ≤ t ≤ T , a ∗ s u b - m a r t i n g a l e ∗ if E [ M t ∣ F s ] ≥ M s , ∀ 0 ≤ s ≤ t ≤ T , a ∗ s u p e r - m a r t i n g a l e ∗ if E [ M t ∣ F s ] ≤ M s , ∀ 0 ≤ s ≤ t ≤ T . \begin{aligned}
\text{a } *martingale* \text{ if}\quad&\mathbb{E}[M_t\mid\mathcal{F}_s]=M_s,\qquad&&\forall\,0\le s\le t\le T,\\
\text{a } *sub\text{-}martingale* \text{ if}\quad&\mathbb{E}[M_t\mid\mathcal{F}_s]\ge M_s,\qquad&&\forall\,0\le s\le t\le T,\\
\text{a } *super\text{-}martingale* \text{ if}\quad&\mathbb{E}[M_t\mid\mathcal{F}_s]\le M_s,\qquad&&\forall\,0\le s\le t\le T.
\end{aligned} a ∗ ma r t in g a l e ∗ if a ∗ s u b - ma r t in g a l e ∗ if a ∗ s u p er - ma r t in g a l e ∗ if E [ M t ∣ F s ] = M s , E [ M t ∣ F s ] ≥ M s , E [ M t ∣ F s ] ≤ M s , ∀ 0 ≤ s ≤ t ≤ T , ∀ 0 ≤ s ≤ t ≤ T , ∀ 0 ≤ s ≤ t ≤ T .
We have given above the definition of a continuous time martingale (resp. sub-, super-). We can also define discrete-time martingales by making the obvious modifications. In fact, the condition for a process M M M to be a discrete-time martingale is somewhat simpler
E [ M n + 1 ∣ F n ] = M n , ∀ n ∈ N 0 . (12) \mathbb{E}[M_{n+1}\mid\mathcal{F}_n]=M_n,\qquad\forall\,n\in\mathbb{N}_0. \tag*{(12)} E [ M n + 1 ∣ F n ] = M n , ∀ n ∈ N 0 . (12)
(discrete-time martingale condition)
Note that if (12) holds then we also have E [ M n + m ∣ F n ] = M n \mathbb{E}[M_{n+m}\mid\mathcal{F}_n]=M_n E [ M n + m ∣ F n ] = M n for all n , m ∈ N 0 n,m\in\mathbb{N}_0 n , m ∈ N 0 because
E [ M n + m ∣ F n ] = E [ E [ M n + m ∣ F n + m − 1 ] ∣ F n ] = E [ M n + m − 1 ∣ F n ] = E [ E [ M n + m − 1 ∣ F n + m − 2 ] ∣ F n ] = E [ M n + m − 2 ∣ F n ] = ⋯ = E [ E [ M n + 1 ∣ F n ] ∣ F n ] = E [ M n ∣ F n ] = M n , \begin{aligned}
\mathbb{E}[M_{n+m}\mid\mathcal{F}_n]
&=\mathbb{E}[\mathbb{E}[M_{n+m}\mid\mathcal{F}_{n+m-1}]\mid\mathcal{F}_n]
=\mathbb{E}[M_{n+m-1}\mid\mathcal{F}_n]\\
&=\mathbb{E}[\mathbb{E}[M_{n+m-1}\mid\mathcal{F}_{n+m-2}]\mid\mathcal{F}_n]
=\mathbb{E}[M_{n+m-2}\mid\mathcal{F}_n]\\
&=\cdots\\
&=\mathbb{E}[\mathbb{E}[M_{n+1}\mid\mathcal{F}_n]\mid\mathcal{F}_n]
=\mathbb{E}[M_n\mid\mathcal{F}_n]=M_n,
\end{aligned} E [ M n + m ∣ F n ] = E [ E [ M n + m ∣ F n + m − 1 ] ∣ F n ] = E [ M n + m − 1 ∣ F n ] = E [ E [ M n + m − 1 ∣ F n + m − 2 ] ∣ F n ] = E [ M n + m − 2 ∣ F n ] = ⋯ = E [ E [ M n + 1 ∣ F n ] ∣ F n ] = E [ M n ∣ F n ] = M n ,
where we have used the tower property of conditional expectation as well as taking out what is known.
Admittedly, the definition of sub- and super-martingales seems backwards; sub-martingales tend to rise in expectation, whereas a super-martingales tend to fall in expectation.
Note: when we say that a process M M M is a martingale (or sub- or super-martingale) this is with respect to a fixed probability measure and filtration. If P \mathbb{P} P and P ~ \widetilde{\mathbb{P}} P are two probability measures and F \mathbb{F} F and G \mathbb{G} G are two filtrations, it is entirely possible that a process M M M may be a martingale with respect to ( P , F ) (\mathbb{P},\mathbb{F}) ( P , F ) and may not be a martingale with respect to ( P ~ , F ) (\widetilde{\mathbb{P}},\mathbb{F}) ( P , F ) , ( P , G ) (\mathbb{P},\mathbb{G}) ( P , G ) or ( P ~ , G ) (\widetilde{\mathbb{P}},\mathbb{G}) ( P , G ) .
Definition 2.3.6. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space, let T T T be a fixed positive number, and let F = ( F t ) t ∈ [ 0 , T ] \mathbb{F}=(\mathcal{F}_t)_{t\in[0,T]} F = ( F t ) t ∈ [ 0 , T ] be a filtration of sub-σ \sigma σ -algebras of F \mathcal{F} F . Consider an F \mathbb{F} F -adapted stochastic process X = ( X t ) t ∈ [ 0 , T ] X=(X_t)_{t\in[0,T]} X = ( X t ) t ∈ [ 0 , T ] . Assume that for all 0 ≤ s ≤ t ≤ T 0\le s\le t\le T 0 ≤ s ≤ t ≤ T and for every nonnegative, Borel-measurable function f f f , there is another Borel-measurable function g g g (which depends on s s s , t t t , and f f f ) such that
E [ f ( X t ) ∣ F s ] = g ( X s ) . \mathbb{E}[f(X_t)\mid\mathcal{F}_s]=g(X_s). E [ f ( X t ) ∣ F s ] = g ( X s ) . Then we say X X X is a Markov process or simply “X X X is Markov.”
Identifying g ( X s ) ≡ E [ f ( X t ) ∣ X s ] g(X_s)\equiv\mathbb{E}[f(X_t)\mid X_s] g ( X s ) ≡ E [ f ( X t ) ∣ X s ] we can write the Markov property as follows
E [ f ( X t ) ∣ F s ] = E [ f ( X t ) ∣ X s ] \mathbb{E}[f(X_t)\mid\mathcal{F}_s]=\mathbb{E}[f(X_t)\mid X_s] E [ f ( X t ) ∣ F s ] = E [ f ( X t ) ∣ X s ] A Markov process is a process for which the following holds: given the present (i.e., X s X_s X s ), the future (i.e., X t X_t X t , t ≥ s t\ge s t ≥ s ) is independent of the past (i.e., F s \mathcal{F}_s F s ). What this means in practice is that
P ( X t ∈ A ∣ F s ) = P ( X t ∈ A ∣ X s ) , ∀ s ≤ t , ∀ A ∈ B ( R ) . (13) \mathbb{P}(X_t\in A\mid\mathcal{F}_s)=\mathbb{P}(X_t\in A\mid X_s),\qquad\forall s\le t,\qquad\forall A\in\mathcal{B}(\mathbb{R}). \tag*{(13)} P ( X t ∈ A ∣ F s ) = P ( X t ∈ A ∣ X s ) , ∀ s ≤ t , ∀ A ∈ B ( R ) . (13) (if X X X is Markov)
If X t X_t X t is a discrete or continuous random variable for every t t t then we have a transition kernel , written as P \mathsf{P} P in the discrete case and Γ \Gamma Γ in the continuous case.
discrete : P ( s , x ; t , y ) : = P ( X t = y ∣ X s = x ) , continuous : Γ ( s , x ; t , y ) d y = P ( X t ∈ d y ∣ X s = x ) . \begin{aligned}
\text{discrete}:&\qquad\mathsf{P}(s,x;t,y):=\mathbb{P}(X_t=y\mid X_s=x),\\
\text{continuous}:&\qquad\Gamma(s,x;t,y)\,dy=\mathbb{P}(X_t\in dy\mid X_s=x).
\end{aligned} discrete : continuous : P ( s , x ; t , y ) := P ( X t = y ∣ X s = x ) , Γ ( s , x ; t , y ) d y = P ( X t ∈ d y ∣ X s = x ) . If you can write the transition kernel of a process explicitly, then you have essentially proved that the process is Markov.
Note that any process that has independent increments is Markov since, if X t − X s ⊥ ⊥ X s X_t-X_s\perp\!\!\!\perp X_s X t − X s ⊥ ⊥ X s for t ≥ s t\ge s t ≥ s , then
P ( X t ∈ A ∣ F s ) = P ( X t − X s + X s ∈ A ∣ X s ) , \mathbb{P}(X_t\in A\mid\mathcal{F}_s)=\mathbb{P}(X_t-X_s+X_s\in A\mid X_s), P ( X t ∈ A ∣ F s ) = P ( X t − X s + X s ∈ A ∣ X s ) , Where F s \mathcal{F}_s F s is the filtration generated by observing X X X up to time s s s .
Markov processes and Martingales are entirely separate concepts . A process X X X can be both a martingale and a Markov process, it can be a martingale but not a Markov process, it can be a Markov process but not a martingale, and it can be neither a Markov process nor a martingale. We illustrate the difference with an example.
Example 2.3.7. Let us return to the stock price Example 1.4.2. Let us show that S = ( S n ) 0 ≤ n S=(S_n)_{0\le n} S = ( S n ) 0 ≤ n is a Markov process. Recall that F m \mathcal{F}_m F m is the σ \sigma σ -algebra generated by observing ω 1 , ω 2 , … , ω m \omega_1,\omega_2,\ldots,\omega_m ω 1 , ω 2 , … , ω m . Observe that S m ∈ F m S_m\in\mathcal{F}_m S m ∈ F m . Next, note that
P ( S n + m = S m u k d n − k ∣ S m ) = ( n k ) p k q n − k . \mathbb{P}(S_{n+m}=S_m u^k d^{n-k}\mid S_m)=\binom{n}{k}p^kq^{n-k}. P ( S n + m = S m u k d n − k ∣ S m ) = ( k n ) p k q n − k . :::`
Since we have written the transition kernel explicitly, we have established that S S S is Markov. Let us also find the function g g g in Definition 2.3.6. For any f : R → R f : \mathbb{R} \to\mathbb{R} f : R → R we have
E [ f ( S n + m ) ∣ F m ] = ∑ k = 0 n f ( S m u k d n − k ) ⋅ ( n k ) p k q n − k = : g ( S m ) . \mathbb{E}[f(S_{n+m}) \mid\mathcal{F}_m] = \sum_{k=0}^{n} f(S_m u^k d^{n-k}) \cdot\binom{n}{k} p^k q^{n-k} =: g(S_m). E [ f ( S n + m ) ∣ F m ] = k = 0 ∑ n f ( S m u k d n − k ) ⋅ ( k n ) p k q n − k =: g ( S m ) .
Thus, we have found g g g . Now, to see if S S S is a martingale note that
E [ S n + 1 ∣ F n ] = E [ S n + 1 ∣ S n ] = p ⋅ u S n + q ⋅ d S n = ( p ⋅ u + q ⋅ d ) S n . \mathbb{E}[S_{n+1} \mid\mathcal{F}_n] = \mathbb{E}[S_{n+1} \mid S_n] = p \cdot u S_n + q \cdot d S_n = (p \cdot u + q \cdot d)S_n. E [ S n + 1 ∣ F n ] = E [ S n + 1 ∣ S n ] = p ⋅ u S n + q ⋅ d S n = ( p ⋅ u + q ⋅ d ) S n .
Thus, if ( p ⋅ u + q ⋅ d ) = 1 (p \cdot u + q \cdot d) = 1 ( p ⋅ u + q ⋅ d ) = 1 , then E [ S n + 1 ∣ F n ] = S n \mathbb{E}[S_{n+1} \mid\mathcal{F}_n] = S_n E [ S n + 1 ∣ F n ] = S n . Let us assume that ( p ⋅ u + q ⋅ d ) = 1 (p \cdot u + q \cdot d) = 1 ( p ⋅ u + q ⋅ d ) = 1 . Then we have
E [ S n + m ∣ F n ] = E [ E [ S n + m ∣ F n + m − 1 ] ∣ F n ] = E [ S n + m − 1 ∣ F n ] = E [ E [ S n + m − 1 ∣ F n + m − 2 ] ∣ F n ] = E [ S n + m − 2 ∣ F n ] = ⋯ = E [ E [ S n + 1 ∣ F n ] ∣ F n ] = E [ S n ∣ F n ] = S n , \begin{aligned}
\mathbb{E}[S_{n+m} \mid\mathcal{F}_n] &= \mathbb{E}[\mathbb{E}[S_{n+m} \mid\mathcal{F}_{n+m-1}] \mid\mathcal{F}_n] = \mathbb{E}[S_{n+m-1} \mid\mathcal{F}_n] \\
&= \mathbb{E}[\mathbb{E}[S_{n+m-1} \mid\mathcal{F}_{n+m-2}] \mid\mathcal{F}_n] = \mathbb{E}[S_{n+m-2} \mid\mathcal{F}_n] \\
&= \cdots\\
&= \mathbb{E}[\mathbb{E}[S_{n+1} \mid\mathcal{F}_n] \mid\mathcal{F}_n] = \mathbb{E}[S_n \mid\mathcal{F}_n] = S_n,
\end{aligned} E [ S n + m ∣ F n ] = E [ E [ S n + m ∣ F n + m − 1 ] ∣ F n ] = E [ S n + m − 1 ∣ F n ] = E [ E [ S n + m − 1 ∣ F n + m − 2 ] ∣ F n ] = E [ S n + m − 2 ∣ F n ] = ⋯ = E [ E [ S n + 1 ∣ F n ] ∣ F n ] = E [ S n ∣ F n ] = S n ,
Therefore, the process S S S is a martingale. Similarly, if ( p ⋅ u + q ⋅ d ) ≥ 1 (p \cdot u + q \cdot d) \ge1 ( p ⋅ u + q ⋅ d ) ≥ 1 , then S S S is a sub-martingale and if ( p ⋅ u + q ⋅ d ) ≤ 1 (p \cdot u + q \cdot d) \le1 ( p ⋅ u + q ⋅ d ) ≤ 1 , then S S S is a super-martingale.
Stopping times
Roughly speaking, a Stopping time is a particular kind of random time τ \tau τ marks the time at which a stochastic process X = ( X t ) t ≥ 0 X = (X_t)_{t \ge0} X = ( X t ) t ≥ 0 exhibits a particular kind of behavior. A stopping time is often defined by a stopping rule , e.g., when the process X X X does this , then stop . Mathematically, a stopping time is defined as follows:
Definition 2.4.1. Fix a probability space ( Ω , F , P ) (\Omega, \mathcal{F}, \mathbb{P}) ( Ω , F , P ) and a filtration F = ( F t ) t ≥ 0 \mathbb{F} = (\mathcal{F}_t)_{t \ge0} F = ( F t ) t ≥ 0 . A random time τ : Ω → [ 0 , ∞ ] \tau: \Omega\to[0,\infty] τ : Ω → [ 0 , ∞ ] is called a F \mathbb{F} F -stopping time if it satisfies
{ τ ≤ t } ∈ F t , ∀ t ∈ [ 0 , ∞ ) . (14) \{\tau\le t\} \in\mathcal{F}_t, \qquad\forall t \in[0,\infty). \tag*{(14)} { τ ≤ t } ∈ F t , ∀ t ∈ [ 0 , ∞ ) . (14)
Above, we have focused on the continuous-time setting. If we are working with a discrete-time filtration F = ( F n ) n ∈ N 0 \mathbb{F} = (\mathcal{F}_n)_{n \in\mathbb{N}_0} F = ( F n ) n ∈ N 0 , then a stopping time is a random time τ : Ω → N 0 ∪ { ∞ } \tau: \Omega\to\mathbb{N}_0 \cup\{\infty\} τ : Ω → N 0 ∪ { ∞ } that satisfies
{ τ ≤ n } ∈ F n , ∀ n ∈ N 0 . \{\tau\le n\} \in\mathcal{F}_n, \qquad\forall n \in\mathbb{N}_0. { τ ≤ n } ∈ F n , ∀ n ∈ N 0 .
Observe that, stopping times, like martingales, are defined with respect to a specific filtration F \mathbb{F} F . Also note that a stopping time may be infinite. The meaning of a stopping time should be fairly clear from (14) . If τ \tau τ is an F \mathbb{F} F -stopping time, then for any t ≥ 0 t \ge0 t ≥ 0 we should be able to say whether or not τ \tau τ has occurred given the information in F t \mathcal{F}_t F t . Below, we give two examples of random times, one of which is a stopping time, one of which is not.
Stopping times
Example 2.4.2. Let X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 be a continuous time stochastic process on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 be the filtration generated by observing the path of X X X , that is, F t = σ ( X s , 0 ≤ s ≤ t ) \mathcal{F}_t=\sigma(X_s,0\leq s\leq t) F t = σ ( X s , 0 ≤ s ≤ t ) . Define
τ : = inf { t ≥ 0 : X t = a } , τ ‾ = sup { t ≥ 0 : X t = a } , a ∈ R . \tau:=\inf\{t\geq0:X_t=a\},\qquad\overline{\tau}=\sup\{t\geq0:X_t=a\},\qquad a\in\mathbb{R}. τ := inf { t ≥ 0 : X t = a } , τ = sup { t ≥ 0 : X t = a } , a ∈ R .
The first random time τ \tau τ is clearly a F \mathbb{F} F -stopping time. At any time t ≥ 0 t\geq0 t ≥ 0 , using the information in F t \mathcal{F}_t F t , we can clearly answer the question: has X X X hit a a a ? On the other hand, the second random time τ ‾ \overline{\tau} τ is not a stopping time. At time t t t we will not be able to answer the question: when is the last time that X X X will hit a a a ?
The following theorem, will prove to be incredibly useful.
Theorem 2.4.3. Suppose a process M = ( M t ) t ≥ 0 M=(M_t)_{t\geq0} M = ( M t ) t ≥ 0 with right-continuous sample paths is a martingale with respect to a filtration F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 . Suppose further that τ \tau τ is an F \mathbb{F} F -stopping time. Then the stopped process M τ : = ( M t ∧ τ ) t ≥ 0 M^\tau:=(M_{t\wedge\tau})_{t\geq0} M τ := ( M t ∧ τ ) t ≥ 0 is a martingale. That is, for any 0 ≤ t ≤ T < ∞ 0\leq t\leq T<\infty 0 ≤ t ≤ T < ∞ , we have
E [ M T ∧ τ ∣ F t ] = M t ∧ τ . \mathbb{E}[M_{T\wedge\tau}\mid\mathcal{F}_t]=M_{t\wedge\tau}. E [ M T ∧ τ ∣ F t ] = M t ∧ τ . In particular , M 0 = E [ M t ∧ τ ] M_0=\mathbb{E}[M_{t\wedge\tau}] M 0 = E [ M t ∧ τ ] for all t ≥ 0 t\geq0 t ≥ 0 .
We will not prove Theorem 2.4.3 rigorously. However, we will provide a bit of intuition for why the theorem is true. For any t < τ t<\tau t < τ we have M τ ∧ t = M t M_{\tau\wedge t}=M_t M τ ∧ t = M t and M M M is a martingale. Likewise, for any t ≥ τ t\geq\tau t ≥ τ we have M t τ = M τ M^\tau_t=M_\tau M t τ = M τ , which is a constant (and thus trivially a martingale). As the process M τ M^\tau M τ is a martingale both prior to and after τ \tau τ , it is reasonable to expect that M τ M^\tau M τ is in fact a martingale.
Just as we can construct a σ \sigma σ -algebra F t \mathcal{F}_t F t of information up to a fixed time t t t , we can construct a σ \sigma σ -algebra F τ \mathcal{F}_\tau F τ of information up to a stopping time τ \tau τ .
Definition 2.4.4. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and a filtration F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 . Suppose τ \tau τ is an F \mathbb{F} F -stopping time. Then we define the σ \sigma σ -algebra F τ \mathcal{F}_\tau F τ at the stopping time τ \tau τ as follows
F τ : = { A ∈ F ∞ : A ∩ { τ ≤ t } ∈ F t ∀ t ≥ 0 } . \mathcal{F}_\tau:=\{A\in\mathcal{F}_\infty:A\cap\{\tau\leq t\}\in\mathcal{F}_t\ \forall t\geq0\}. F τ := { A ∈ F ∞ : A ∩ { τ ≤ t } ∈ F t ∀ t ≥ 0 } .
The idea underlying Definition (2.4.4) is that if a set A A A is observable at time τ \tau τ then for any time t t t , its restriction to the set { τ ≤ t } \{\tau\leq t\} { τ ≤ t } should be in F t \mathcal{F}_t F t . The restriction to sets in A ∈ F ∞ A\in\mathcal{F}_\infty A ∈ F ∞ is to take account of the possibility that the stopping time can be infinite and ensures that A ∩ { τ ≤ ∞ } ∈ F ∞ A\cap\{\tau\leq\infty\}\in\mathcal{F}_\infty A ∩ { τ ≤ ∞ } ∈ F ∞ . If A ∉ F ∞ A\notin\mathcal{F}_\infty A ∈ / F ∞ then we could have A ∩ { τ ≤ ∞ } ∉ F ∞ A\cap\{\tau\leq\infty\}\notin\mathcal{F}_\infty A ∩ { τ ≤ ∞ } ∈ / F ∞ . From the above definition, a random variable X X X is F τ \mathcal{F}_\tau F τ -measurable if and only if 1 { τ ≤ t } X ∈ F t \mathbf{1}_{\{\tau\leq t\}}X\in\mathcal{F}_t 1 { τ ≤ t } X ∈ F t for all t ∈ [ 0 , ∞ ] t\in[0,\infty] t ∈ [ 0 , ∞ ] .
Now, suppose a process M = ( M t ) t ≥ 0 M=(M_t)_{t\geq0} M = ( M t ) t ≥ 0 is a martingale with respect to a filtration F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 . Suppose further that M M M has a well-defined limit M ∞ : = lim t → ∞ M t M_{\infty} := \lim_{t \to\infty} M_t M ∞ := lim t → ∞ M t . As M M M is a martingale we have (by definition) that M t 1 = E [ M t 2 ∣ F t 1 ] M_{t_1} = \mathbb{E}[M_{t_2} \mid\mathcal{F}_{t_1}] M t 1 = E [ M t 2 ∣ F t 1 ] for any 0 ≤ t 1 ≤ t 2 ≤ ∞ 0 \leq t_1 \leq t_2 \leq\infty 0 ≤ t 1 ≤ t 2 ≤ ∞ . Now, consider two F \mathbb{F} F -stopping times τ 1 \tau_1 τ 1 and τ 2 \tau_2 τ 2 satisfying 0 ≤ τ 1 ≤ τ 2 ≤ ∞ 0 \leq\tau_1 \leq\tau_2 \leq\infty 0 ≤ τ 1 ≤ τ 2 ≤ ∞ . One may ask: is it true that M τ 1 = E [ M τ 2 ∣ F τ 1 ] M_{\tau_1} = \mathbb{E}[M_{\tau_2} \mid\mathcal{F}_{\tau_1}] M τ 1 = E [ M τ 2 ∣ F τ 1 ] ? Unfortunately, the answer to this question is “no, not in general.” However, under certain conditions, provided in the following theorem, we will have M τ 1 = E [ M τ 2 ∣ F τ 1 ] M_{\tau_1} = \mathbb{E}[M_{\tau_2} \mid\mathcal{F}_{\tau_1}] M τ 1 = E [ M τ 2 ∣ F τ 1 ] .
Theorem 2.4.5 (Doob’s Optional Stopping). Let M = ( M t ) t ≥ 0 M = (M_t)_{t \geq0} M = ( M t ) t ≥ 0 be a martingale with right-continuous sample paths with respect to a filtration F = ( F t ) t ≥ 0 \mathbb{F} = (\mathcal{F}_t)_{t \geq0} F = ( F t ) t ≥ 0 . Suppose that for any ε > 0 \varepsilon> 0 ε > 0 there exists a constant K ε ∈ [ 0 , ∞ ) K_{\varepsilon} \in[0,\infty) K ε ∈ [ 0 , ∞ ) such that
sup t ≥ 0 E 1 { ∣ M t ∣ > K ε } ∣ M t ∣ < ε . (15) \sup_{t \geq0} \mathbb{E}\mathbf{1}_{\{|M_t| > K_{\varepsilon}\}} |M_t| < \varepsilon. \tag*{(15)} t ≥ 0 sup E 1 { ∣ M t ∣ > K ε } ∣ M t ∣ < ε . (15) Suppose that τ 1 \tau_1 τ 1 and τ 2 \tau_2 τ 2 are F \mathbb{F} F -stopping times and that 0 ≤ τ 1 ≤ τ 2 0 \leq\tau_1 \leq\tau_2 0 ≤ τ 1 ≤ τ 2 . Then the limit M ∞ : = lim t → ∞ M t M_{\infty} := \lim_{t \to\infty} M_t M ∞ := lim t → ∞ M t exists almost surely (so that M τ i M_{\tau_i} M τ i is well-defined on { τ i = ∞ } \{\tau_i = \infty\} { τ i = ∞ } ) and we have
M τ 1 = E [ M τ 2 ∣ F τ 1 ] . (16) M_{\tau_1} = \mathbb{E}[M_{\tau_2} \mid\mathcal{F}_{\tau_1}]. \tag*{(16)} M τ 1 = E [ M τ 2 ∣ F τ 1 ] . (16) In particular, we have M 0 = E M τ i M_0 = \mathbb{E}M_{\tau_i} M 0 = E M τ i for i = 1 , 2 i = 1, 2 i = 1 , 2 .
We will not prove Theorem 2.4.5 . Rather, we will demonstrate its usefulness through an example.
Example 2.4.6. Consider a symmetric random walk
S n = ∑ i = 1 n X i , S 0 = x ∈ Z , S_n = \sum_{i=1}^{n} X_i, \qquad S_0 = x \in\mathbb{Z}, S n = i = 1 ∑ n X i , S 0 = x ∈ Z , where the ( X i ) (X_i) ( X i ) are i.i.d. random variables with P ( X i = 1 ) = P ( X i = − 1 ) = 1 / 2 \mathbb{P}(X_i = 1) = \mathbb{P}(X_i = -1) = 1/2 P ( X i = 1 ) = P ( X i = − 1 ) = 1/2 . Let F = ( F n ) n ∈ N 0 \mathbb{F} = (\mathcal{F}_n)_{n \in\mathbb{N}_0} F = ( F n ) n ∈ N 0 where F n : = σ ( S i , 0 ≤ i ≤ n ) \mathcal{F}_n := \sigma(S_i, 0 \leq i \leq n) F n := σ ( S i , 0 ≤ i ≤ n ) . For any a ∈ Z a \in\mathbb{Z} a ∈ Z define the first hitting time to a a a as follows
τ a : = inf { n ∈ N 0 : S n = a } . \tau_a := \inf\{n \in\mathbb{N}_0 : S_n = a\}. τ a := inf { n ∈ N 0 : S n = a } . Now, suppose a < x < b a < x < b a < x < b . We wish to find P ( τ a < τ b ) \mathbb{P}(\tau_a < \tau_b) P ( τ a < τ b ) . To this end, we define
τ : = τ a ∧ τ b . \tau:= \tau_a \wedge\tau_b. τ := τ a ∧ τ b . Observe that τ \tau τ is an F \mathbb{F} F -stopping time and S S S is an F \mathbb{F} F martingale. The stopped process S τ : = ( S n ∧ τ ) n ∈ N S^{\tau} := (S_{n \wedge\tau})_{n \in\mathbb{N}} S τ := ( S n ∧ τ ) n ∈ N is a martingale by Theorem 2.4.3 . Moreover, because a < S n τ < b a < S_n^{\tau} < b a < S n τ < b we see that S τ S^{\tau} S τ satisfies (15) . Thus, we have by Theorem 2.4.5 that
S 0 τ = x = E S τ τ = a P ( τ a < τ b ) + b P ( τ b < τ a ) , S_0^{\tau} = x = \mathbb{E}S_{\tau}^{\tau} = a\mathbb{P}(\tau_a < \tau_b) + b\mathbb{P}(\tau_b < \tau_a), S 0 τ = x = E S τ τ = a P ( τ a < τ b ) + b P ( τ b < τ a ) ,
where we have used the fact that S τ τ = a S_{\tau}^{\tau} = a S τ τ = a when τ a < τ b \tau_a < \tau_b τ a < τ b and S τ τ = b S_{\tau}^{\tau} = b S τ τ = b when τ b < τ a \tau_b < \tau_a τ b < τ a . We also have
1 = P ( τ a < τ b ) + P ( τ b < τ a ) . 1 = \mathbb{P}(\tau_a < \tau_b) + \mathbb{P}(\tau_b < \tau_a). 1 = P ( τ a < τ b ) + P ( τ b < τ a ) .
Thus, using the above two equations, we find
P ( τ a < τ b ) = b − x b − a , P ( τ b < τ a ) = x − a b − a . \mathbb{P}(\tau_a < \tau_b) = \frac{b-x}{b-a}, \qquad\mathbb{P}(\tau_b < \tau_a) = \frac{x-a}{b-a}. P ( τ a < τ b ) = b − a b − x , P ( τ b < τ a ) = b − a x − a .
The condition given in (15) is essential in order for (16) to hold, as we shall see in the next example.
Example 2.4.7. Let B = ( B i ) i ∈ N \mathbf{B} = (B_i)_{i\in\mathbb{N}} B = ( B i ) i ∈ N be an iid sequence of Bernoulli random variables: B ∼ Ber ( p ) \mathbf{B} \sim\operatorname{Ber}(p) B ∼ Ber ( p ) with p = 1 / 2 p = 1/2 p = 1/2 . Construct a sequence X = ( X i ) \mathbf{X} = (X_i) X = ( X i ) of random variables as follows
X 0 = 1 , X n + 1 = 2 B n + 1 X n . X_0 = 1, \qquad X_{n+1} = 2B_{n+1}X_n. X 0 = 1 , X n + 1 = 2 B n + 1 X n . Define a filtration F = ( F n ) n ∈ N 0 \mathbb{F} = (\mathcal{F}_n)_{n\in\mathbb{N}_0} F = ( F n ) n ∈ N 0 where F n = σ ( B i , 0 ≤ i ≤ n ) \mathcal{F}_n = \sigma(B_i, 0 \le i \le n) F n = σ ( B i , 0 ≤ i ≤ n ) . Observe that both B \mathbf{B} B and X \mathbf{X} X are F \mathbb{F} F -adapted. It is easy to see that X \mathbf{X} X is a martingale with respect to F \mathbb{F} F because
E [ X n + 1 ∣ F n ] = 2 X n E [ B n + 1 ] = X n . \mathbb{E}[X_{n+1}\mid\mathcal{F}_n] = 2X_n\mathbb{E}[B_{n+1}] = X_n. E [ X n + 1 ∣ F n ] = 2 X n E [ B n + 1 ] = X n . Note, however, that condition (15) does not hold. Now, let us define the first hitting time of X \mathbf{X} X to zero: τ : = inf { n ≥ 0 : X n = 0 } \tau:= \inf\{n \ge0 : X_n = 0\} τ := inf { n ≥ 0 : X n = 0 } . We note that P ( τ < ∞ ) = 1 \mathbb{P}(\tau< \infty) = 1 P ( τ < ∞ ) = 1 . We have
E X τ = E 0 = 0 ≠ X 0 . \mathbb{E}X_\tau= \mathbb{E}0 = 0 \ne X_0. E X τ = E 0 = 0 = X 0 . Thus, we see that, without condition (15) , we cannot expect (16) to hold.
Exercises
Exercise 2.1. Let Ω = { a , b , c , d } \Omega= \{a,b,c,d\} Ω = { a , b , c , d } and let F = 2 Ω \mathcal{F} = 2^\Omega F = 2 Ω (the set of all subsets of Ω \Omega Ω ). We define a probability measure P \mathbb{P} P as follows
P ( a ) = 1 / 6 , P ( b ) = 1 / 3 , P ( c ) = 1 / 4 , P ( d ) = 1 / 4 , \mathbb{P}(a) = 1/6, \qquad\mathbb{P}(b) = 1/3, \qquad\mathbb{P}(c) = 1/4, \qquad\mathbb{P}(d) = 1/4, P ( a ) = 1/6 , P ( b ) = 1/3 , P ( c ) = 1/4 , P ( d ) = 1/4 ,
Next, define three random variables
X ( a ) = 1 , X ( b ) = 1 , X ( c ) = − 1 , X ( d ) = − 1 , X(a) = 1, \qquad X(b) = 1, \qquad X(c) = -1, \qquad X(d) = -1, X ( a ) = 1 , X ( b ) = 1 , X ( c ) = − 1 , X ( d ) = − 1 ,
Y ( a ) = 1 , Y ( b ) = − 1 , Y ( c ) = 1 , Y ( d ) = − 1 , Y(a) = 1, \qquad Y(b) = -1, \qquad Y(c) = 1, \qquad Y(d) = -1, Y ( a ) = 1 , Y ( b ) = − 1 , Y ( c ) = 1 , Y ( d ) = − 1 ,
and Z = X + Y Z = X + Y Z = X + Y . (a) List the sets in σ ( X ) \sigma(X) σ ( X ) . (b) What are the values of E [ Y ∣ X ] \mathbb{E}[Y\mid X] E [ Y ∣ X ] for { a , b , c , d } \{a,b,c,d\} { a , b , c , d } ? Verify the partial averaging property: E [ 1 A E [ Y ∣ X ] ] = E [ 1 A Y ] \mathbb{E}[\mathbf{1}_A\mathbb{E}[Y\mid X]] = \mathbb{E}[\mathbf{1}_A Y] E [ 1 A E [ Y ∣ X ]] = E [ 1 A Y ] for all A ∈ σ ( X ) A \in\sigma(X) A ∈ σ ( X ) . (c) What are the values of E [ Z ∣ X ] \mathbb{E}[Z\mid X] E [ Z ∣ X ] for { a , b , c , d } \{a,b,c,d\} { a , b , c , d } ? Verify the partial averaging property.
Exercise 2.2. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Let Y Y Y be a square integrable random variable: E Y 2 < ∞ \mathbb{E}Y^{2}<\infty E Y 2 < ∞ and let G \mathcal{G} G be a sub-σ \sigma σ -algebra of F \mathcal{F} F . Show that
V ( Y − E [ Y ∣ G ] ) ≤ V ( Y − X ) , ∀ X ∈ G . \operatorname{V}(Y-\mathbb{E}[Y\mid\mathcal{G}])\leq\operatorname{V}(Y-X),\qquad\forall X\in\mathcal{G}. V ( Y − E [ Y ∣ G ]) ≤ V ( Y − X ) , ∀ X ∈ G .
Exercise 2.3. Give an example of a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) , a random variable X X X and a function f f f such that σ ( f ( X ) ) \sigma(f(X)) σ ( f ( X )) is strictly smaller than σ ( X ) \sigma(X) σ ( X ) but σ ( f ( X ) ) ≠ { ∅ , Ω } \sigma(f(X))\ne\{\varnothing,\Omega\} σ ( f ( X )) = { ∅ , Ω } . Give a function g g g such that σ ( g ( X ) ) = { ∅ , Ω } \sigma(g(X))=\{\varnothing,\Omega\} σ ( g ( X )) = { ∅ , Ω } .
Exercise 2.4. On a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) define random variables X X X and Y 0 , Y 1 , Y 2 , … Y_{0},Y_{1},Y_{2},\ldots Y 0 , Y 1 , Y 2 , … and suppose E ∣ X ∣ < ∞ \mathbb{E}|X|<\infty E ∣ X ∣ < ∞ . Define F n : = σ ( Y 0 , Y 1 , … , Y n ) \mathcal{F}_{n}:=\sigma(Y_{0},Y_{1},\ldots,Y_{n}) F n := σ ( Y 0 , Y 1 , … , Y n ) and X n = E [ X ∣ F n ] X_{n}=\mathbb{E}[X\mid\mathcal{F}_{n}] X n = E [ X ∣ F n ] . Show that the sequence X 0 , X 1 , X 2 , … X_{0},X_{1},X_{2},\ldots X 0 , X 1 , X 2 , … is a martingale under P \mathbb{P} P with respect to the filtration ( F n ) n ≥ 0 (\mathcal{F}_{n})_{n\geq0} ( F n ) n ≥ 0 .
Exercise 2.5. In this exercise, we will construct a discrete-time model for a stock Z = ( Z n ) n ≥ 0 Z=(Z_{n})_{n\geq0} Z = ( Z n ) n ≥ 0 . Let X 0 , X 1 , … X_{0},X_{1},\ldots X 0 , X 1 , … be a sequence of i.i.d Bernoulli random variables with parameter p ∈ ( 0 , 1 ) p\in(0,1) p ∈ ( 0 , 1 ) (i.e., P ( X i = 1 ) = p \mathbb{P}(X_{i}=1)=p P ( X i = 1 ) = p ). Define S n = ∑ i = 1 n X i S_{n}=\sum_{i=1}^{n}X_{i} S n = ∑ i = 1 n X i where S 0 = 0 S_{0}=0 S 0 = 0 . Let Z Z Z be given by
Z n : = r 2 S n − n , r > 0 , n = 0 , 1 , 2 , … Z_{n}:=r^{2S_{n}-n},\qquad r>0,\qquad n=0,1,2,\ldots Z n := r 2 S n − n , r > 0 , n = 0 , 1 , 2 , …
(a) Let F = ( F n ) n ≥ 0 \mathcal{F}=(\mathcal{F}_{n})_{n\geq0} F = ( F n ) n ≥ 0 where F n : = σ ( X 0 , X 1 , … , X n ) \mathcal{F}_{n}:=\sigma(X_{0},X_{1},\ldots,X_{n}) F n := σ ( X 0 , X 1 , … , X n ) . For what value(s) of r r r is Z Z Z a ( P , F ) (\mathbb{P},\mathcal{F}) ( P , F ) martingale, submartingale and supermartingale? In particular, show that Z Z Z is a ( P , F ) (\mathbb{P},\mathcal{F}) ( P , F ) martingale when r = ( 1 − p ) / p r=(1-p)/p r = ( 1 − p ) / p .
(b) Let G = ( G n ) n ≥ 0 \mathcal{G}=(\mathcal{G}_{n})_{n\geq0} G = ( G n ) n ≥ 0 where G n : = σ ( S 0 , S 1 , … , S n ) \mathcal{G}_{n}:=\sigma(S_{0},S_{1},\ldots,S_{n}) G n := σ ( S 0 , S 1 , … , S n ) . Suppose r r r is such that Z Z Z is a ( P , F ) (\mathbb{P},\mathcal{F}) ( P , F ) martingale. Is Z Z Z also a ( P , G ) (\mathbb{P},\mathcal{G}) ( P , G ) martingale?
(c) Suppose that P ~ ( X i = 1 ) = p ~ ≠ p \widetilde{\mathbb{P}}(X_{i}=1)=\widetilde{p}\ne p P ( X i = 1 ) = p = p . Suppose r r r is such that Z Z Z is a ( P , F ) (\mathbb{P},\mathcal{F}) ( P , F ) martingale. Is Z Z Z also a ( P ~ , F ) (\widetilde{\mathbb{P}},\mathcal{F}) ( P , F ) martingale?
Exercise 2.6. This exercise is similar to Exercise 2.2. Suppose X X X and Y Y Y are random variables defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Define the following functional
J ( f ) : = E [ ( f ( X ) − h ( X , Y ) ) 2 ] , J(f):=\mathbb{E}[(f(X)-h(X,Y))^{2}], J ( f ) := E [( f ( X ) − h ( X , Y ) ) 2 ] ,
where E h 2 ( X , Y ) < ∞ \mathbb{E}h^{2}(X,Y)<\infty E h 2 ( X , Y ) < ∞ . We wish to find the function f ∗ f^{*} f ∗ that minimizes J J J . That is
J ( f ∗ ) = min f J ( f ) . J(f^{*})=\min_{f}J(f). J ( f ∗ ) = f min J ( f ) .
Note that, for any g g g , the following must hold
0 = d d ε J ( f ∗ + ε g ) ∣ ε = 0 . 0=\left.\frac{\mathrm{d}}{\mathrm{d}\varepsilon}J(f^{*}+\varepsilon g)\right|_{\varepsilon=0}. 0 = d ε d J ( f ∗ + ε g ) ε = 0 .
If you do not understand why, think about what happens to J ( f ∗ + ε g ) J(f^{*}+\varepsilon g) J ( f ∗ + ε g ) for values of ε \varepsilon ε near zero. Use this information to show that f ∗ ( x ) = E [ h ( X , Y ) ∣ X = x ] f^{*}(x)=\mathbb{E}[h(X,Y)\mid X=x] f ∗ ( x ) = E [ h ( X , Y ) ∣ X = x ] . If you find it helpful, you can assume ( X , Y ) (X,Y) ( X , Y ) are jointly continuous random variables, and therefore have a joint density p ( x , y ) p(x,y) p ( x , y ) . Though, this is not necessary to prove the result. Essentially, what this result shows, is that the σ ( X ) \sigma(X) σ ( X ) -measurable random variable Z Z Z that minimizes E [ ( Z − h ( X , Y ) ) 2 ] \mathbb{E}[(Z-h(X,Y))^{2}] E [( Z − h ( X , Y ) ) 2 ] is Z = E [ h ( X , Y ) ∣ X ] Z=\mathbb{E}[h(X,Y)\mid X] Z = E [ h ( X , Y ) ∣ X ] .
Generating and Characteristic Functions
The notes from this chapter are taken primarily from (Grimmett and Stirzaker, 2001, Chapter 5).
Generating functions
The generating function is a powerful tool for analyzing sums of random variables.
Definition 3.1.1. Suppose X X X is a discrete random variable taking values in { 0 } ∪ N \{0\} \cup\mathbb{N} { 0 } ∪ N . We define the probability generating function of X X X , written G X ( s ) G_X(s) G X ( s ) , by
G X ( s ) : = E s X = ∑ k s k f X ( k ) . G_X(s) := \mathbb{E}s^X = \sum_k s^k f_X(k). G X ( s ) := E s X = k ∑ s k f X ( k ) . where f X ( k ) f_X(k) f X ( k ) is the probability mass function of $X.
The generating function exists at least when ∣ s ∣ ≤ 1 \lvert s\rvert\le1 ∣ s ∣ ≤ 1 (since ∑ k f X ( k ) = 1 \sum_k f_X(k) = 1 ∑ k f X ( k ) = 1 ) and may exist in a larger interval. As we will see, we usually only care about the value of G X G_X G X and its derivatives at the point s = 1 s = 1 s = 1 . You can always integrate and differentiate a convergent power series term-by-term within its radius of convergence.
Why is G X G_X G X call the “probability generating function”? The reason is that the coefficient f X ( k ) f_X(k) f X ( k ) of the s k s^k s k term in the series expansion of G X ( s ) G_X(s) G X ( s ) is precisely P ( X = k ) \mathbb{P}(X = k) P ( X = k ) . So, one can expand G X G_X G X as a power series and obtain the probability mass function f X f_X f X .
Example 3.1.2. Suppose X ∼ Ber ( p ) X \sim\operatorname{Ber}(p) X ∼ Ber ( p ) as in Example 1.4.8 . Then
G X ( s ) = q s 0 + p s 1 = q + p s . G_X(s) = qs^0 + ps^1 = q + ps. G X ( s ) = q s 0 + p s 1 = q + p s .
Example 3.1.3. Suppose X ∼ Geo ( p ) X \sim\operatorname{Geo}(p) X ∼ Geo ( p ) as in Example 1.4.10 . Then
G X ( s ) = ∑ k = 1 ∞ s k p q k − 1 = s p ∑ k = 1 ∞ ( s q ) k − 1 = s p 1 − s q , ∣ s ∣ < 1 q . G_X(s)=\sum_{k=1}^{\infty}s^k p q^{k-1}=sp\sum_{k=1}^{\infty}(sq)^{k-1}=\frac{sp}{1-sq},\qquad\lvert s\rvert<\frac{1}{q}. G X ( s ) = k = 1 ∑ ∞ s k p q k − 1 = s p k = 1 ∑ ∞ ( s q ) k − 1 = 1 − s q s p , ∣ s ∣ < q 1 .
Theorem 3.1.4. Let G X ( n ) G_X^{(n)} G X ( n ) denote the n n n th derivative of G X G_X G X and let G X ( n ) ( 1 ) : = lim s ↗ 1 G X ( n ) ( s ) G_X^{(n)}(1):=\lim_{s\nearrow1}G_X^{(n)}(s) G X ( n ) ( 1 ) := lim s ↗ 1 G X ( n ) ( s ) , which may be + ∞ +\infty + ∞ . Then
G X ( n ) ( 1 ) = E X ! ( X − n ) ! , (17) G_X^{(n)}(1)=\mathbb{E}\frac{X!}{(X-n)!}, \tag*{(17)} G X ( n ) ( 1 ) = E ( X − n )! X ! , (17) where X ! / ( X − n ) ! = 0 X!/(X-n)!=0 X ! / ( X − n )! = 0 when X < n X<n X < n .
Proof. We have
G X ( n ) ( s ) = d n d s n G X ( s ) = E d n d s n s X = E X ! ( X − n ) ! s X − n , (18) G_X^{(n)}(s)=\frac{d^n}{ds^n}G_X(s)=\mathbb{E}\frac{d^n}{ds^n}s^X=\mathbb{E}\frac{X!}{(X-n)!}s^{X-n}, \tag*{(18)} G X ( n ) ( s ) = d s n d n G X ( s ) = E d s n d n s X = E ( X − n )! X ! s X − n , (18)
where passing the derivative through the expectation is allowed by dominated convergence. Taking s = 1 s=1 s = 1 in (18) yields (17) .
Example 3.1.5. How would you find V X V_X V X using G X G_X G X ? Note that
G X ′ ( 1 ) = E X , G X ′ ′ ( 1 ) = E X ( X − 1 ) = E X 2 − E X . (19) G_X'(1)=\mathbb{E}X,\qquad G_X''(1)=\mathbb{E}X(X-1)=\mathbb{E}X^2-\mathbb{E}X. \tag*{(19)} G X ′ ( 1 ) = E X , G X ′′ ( 1 ) = E X ( X − 1 ) = E X 2 − E X . (19) Using the above, we compute
V X = E X 2 − ( E X ) 2 = G X ′ ′ ( 1 ) − ( G X ′ ( 1 ) ) 2 + G X ′ ( 1 ) . V_X=\mathbb{E}X^2-(\mathbb{E}X)^2=G_X''(1)-(G_X'(1))^2+G_X'(1). V X = E X 2 − ( E X ) 2 = G X ′′ ( 1 ) − ( G X ′ ( 1 ) ) 2 + G X ′ ( 1 ) .
Given two independent random variables X X X and Y Y Y taking values in { 0 } ∪ N \{0\}\cup\mathbb{N} { 0 } ∪ N , one can compute P ( X + Y = k ) \mathbb{P}(X+Y=k) P ( X + Y = k ) using
P ( X + Y = n ) = ∑ k = 0 n P ( X = k ) P ( Y = n − k ) . \mathbb{P}(X+Y=n)=\sum_{k=0}^{n}\mathbb{P}(X=k)\mathbb{P}(Y=n-k). P ( X + Y = n ) = k = 0 ∑ n P ( X = k ) P ( Y = n − k ) .
Alternatively, one can compute the probability generating function of X + Y X+Y X + Y and then expand the generating function to obtain the probability P ( X + Y = n ) \mathbb{P}(X+Y=n) P ( X + Y = n ) .
Theorem 3.1.6. Suppose X ⊥ ⊥ Y X\perp\!\!\!\perp Y X ⊥ ⊥ Y . Then G X + Y ( s ) = G X ( s ) G Y ( s ) G_{X+Y}(s)=G_X(s)G_Y(s) G X + Y ( s ) = G X ( s ) G Y ( s ) .
Proof. We have
G X + Y ( s ) = E s X + Y = E s X ⋅ E s Y = G X ( s ) G Y ( s ) . G_{X+Y}(s)=\mathbb{E}s^{X+Y}=\mathbb{E}s^X\cdot\mathbb{E}s^Y=G_X(s)G_Y(s). G X + Y ( s ) = E s X + Y = E s X ⋅ E s Y = G X ( s ) G Y ( s ) .
The real use of generating functions arises when one wants to compute P ( ∑ i = 1 n X i = k ) \mathbb{P}(\sum_{i=1}^{n} X_i = k) P ( ∑ i = 1 n X i = k ) where ( X i ) i ≥ 1 (X_i)_{i\ge1} ( X i ) i ≥ 1 is an iid sequence of random variables. In this case, computing the generating function G S n ( s ) G_{S_n}(s) G S n ( s ) with S n = ∑ i = 1 n X i S_n = \sum_{i=1}^{n} X_i S n = ∑ i = 1 n X i is relatively easy. Upon computing the generating function G S n G_{S_n} G S n , one can compute probabilities of the form P ( S n = k ) \mathbb{P}(S_n = k) P ( S n = k ) by expanding G S n G_{S_n} G S n . By contrast, computing P ( S n = k ) \mathbb{P}(S_n = k) P ( S n = k ) directly from the probability mass function of X i X_i X i is difficult.
Theorem 3.1.7. Suppose ( X i ) i ≥ 1 (X_i)_{i\ge1} ( X i ) i ≥ 1 are iid with common distribution X X X . Define S n : = ∑ i = 1 n X i S_n := \sum_{i=1}^{n} X_i S n := ∑ i = 1 n X i . Then
G S n ( s ) = [ G X ( s ) ] n . G_{S_n}(s) = [G_X(s)]^n. G S n ( s ) = [ G X ( s ) ] n .
Proof. We have
G S n ( s ) = E s X 1 + X 2 + ⋯ + X n = ( E s X ) n = [ G X ( s ) ] n . G_{S_n}(s) = \mathbb{E}s^{X_1+X_2+\cdots+X_n} = (\mathbb{E}s^X)^n = [G_X(s)]^n. G S n ( s ) = E s X 1 + X 2 + ⋯ + X n = ( E s X ) n = [ G X ( s ) ] n .
Example 3.1.8. Let ( X i ) i ≥ 1 (X_i)_{i\ge1} ( X i ) i ≥ 1 be iid with each X i ∼ Ber ( p ) X_i \sim\operatorname{Ber}(p) X i ∼ Ber ( p ) . Define S n : = ∑ i = 1 n X i S_n := \sum_{i=1}^{n} X_i S n := ∑ i = 1 n X i . Recall from Example 1.4.9 that S n ∼ Bin ( n , p ) S_n \sim\operatorname{Bin}(n,p) S n ∼ Bin ( n , p ) . We have
G S n ( s ) = [ G X ( s ) ] n = ( q + p s ) n . G_{S_n}(s) = [G_X(s)]^n = (q + ps)^n. G S n ( s ) = [ G X ( s ) ] n = ( q + p s ) n .
Thus, we have obtained the probability generating function of the binomial random variable S n S_n S n . Although we could have computed G S n G_{S_n} G S n using the probability mass function f S n f_{S_n} f S n directly, this approach would have been much more work.
Theorem 3.1.9. Suppose ( X i ) i ≥ 1 (X_i)_{i\ge1} ( X i ) i ≥ 1 are iid with common distribution X X X . Define S n : = ∑ i = 1 n X i S_n := \sum_{i=1}^{n} X_i S n := ∑ i = 1 n X i . Let N N N be independent of ( X i ) i ≥ 1 (X_i)_{i\ge1} ( X i ) i ≥ 1 . Then
G S N ( s ) = G N ( G X ( s ) ) . G_{S_N}(s) = G_N(G_X(s)). G S N ( s ) = G N ( G X ( s )) .
Proof. We have
G S N ( s ) = E s X 1 + X 2 + ⋯ + X N = E E [ s X 1 + X 2 + ⋯ + X N ∣ N ] = E ( E s X ) N = E ( G X ( s ) ) N = G N ( G X ( s ) ) , \begin{aligned}
G_{S_N}(s) &= \mathbb{E}s^{X_1+X_2+\cdots+X_N} = \mathbb{E}\mathbb{E}[s^{X_1+X_2+\cdots+X_N}\mid N] \\
&= \mathbb{E}(\mathbb{E}s^X)^N = \mathbb{E}(G_X(s))^N = G_N(G_X(s)),
\end{aligned} G S N ( s ) = E s X 1 + X 2 + ⋯ + X N = EE [ s X 1 + X 2 + ⋯ + X N ∣ N ] = E ( E s X ) N = E ( G X ( s ) ) N = G N ( G X ( s )) ,
where, in the third equality, we have used the fact that the X i X_i X i are independent of each other and independent of N N N .
One can extend the notion of a probability generating function to multiple random variables in the obvious way.
Definition 3.1.10. Suppose X X X and Y Y Y are discrete random variables taking values in { 0 } ∪ N \{0\} \cup\mathbb{N} { 0 } ∪ N . We define the joint probability generating function of ( X , Y ) (X,Y) ( X , Y ) , written G X , Y ( s , t ) G_{X,Y}(s,t) G X , Y ( s , t ) , as
G X , Y ( s , t ) = E s X t Y = ∑ k ∑ m s k t m f X , Y ( k , m ) . G_{X,Y}(s,t) = \mathbb{E}s^X t^Y = \sum_k \sum_m s^k t^m f_{X,Y}(k,m). G X , Y ( s , t ) = E s X t Y = k ∑ m ∑ s k t m f X , Y ( k , m ) . where f X , Y ( k , m ) f_{X,Y}(k,m) f X , Y ( k , m ) is the joint probability mass function of ( X , Y ) (X,Y) ( X , Y ) .
The coefficient f X , Y ( k , m ) f_{X,Y}(k,m) f X , Y ( k , m ) of the O ( s k t m ) O(s^k t^m) O ( s k t m ) term in the power series expansion of G X , Y ( s , t ) G_{X,Y}(s,t) G X , Y ( s , t ) about ( 0 , 0 ) (0,0) ( 0 , 0 ) gives P ( X = k , Y = m ) \mathbb{P}(X = k, Y = m) P ( X = k , Y = m ) .
Theorem 3.1.11. Let G X , Y ( n , m ) G_{X,Y}^{(n,m)} G X , Y ( n , m ) denote the ( n , m ) (n,m) ( n , m ) th partial derivative of G X , Y G_{X,Y} G X , Y with respect to the first and second arguments. Then
G X , Y ( n , m ) ( 1 , 1 ) = E X ! ( X − n ) ! Y ! ( Y − m ) ! . G_{X,Y}^{(n,m)}(1,1) = \mathbb{E}\frac{X!}{(X-n)!}\frac{Y!}{(Y-m)!}. G X , Y ( n , m ) ( 1 , 1 ) = E ( X − n )! X ! ( Y − m )! Y ! .
Proof. The proof follows the proof the Theorem 3.1.4.
Branching processes
Suppose a population evolves in generations. Let Z n Z_n Z n be the size of the n n n th generation and assume Z 0 = 1 Z_0 = 1 Z 0 = 1 . We assume that the i i i th member of the n n n th generation gives birth to a random number X n , i X_{n,i} X n , i of individuals. Then, clearly, the number of individuals in the ( n + 1 ) (n+1) ( n + 1 ) th generation is given by
Z n + 1 = X n , 1 + X n , 2 + … + X n , Z n . (20) Z_{n+1} = X_{n,1} + X_{n,2} + \ldots+ X_{n,Z_n}. \tag*{(20)} Z n + 1 = X n , 1 + X n , 2 + … + X n , Z n . (20)
We would like to know what the probability mass function, mean and variance of Z n Z_n Z n are. Our route for obtaining these will be through the generating function G Z n G_{Z_n} G Z n .
Theorem 3.2.1. Assume Z n + 1 Z_{n+1} Z n + 1 is given by (20) and that the collection of random variables ( X n , i ) (X_{n,i}) ( X n , i ) are iid. Define G n ( s ) : = E s Z n G_n(s) := \mathbb{E}s^{Z_n} G n ( s ) := E s Z n and G ( s ) : = E s X G(s) := \mathbb{E}s^X G ( s ) := E s X . Then
G n + m ( s ) = G n ( G m ( s ) ) = G m ( G n ( s ) ) , and thus G n ( s ) = G ( G ( … G ( s ) … ) ) . G_{n+m}(s) = G_n(G_m(s)) = G_m(G_n(s)), \qquad\text{and thus} \qquad G_n(s) = G(G(\ldots G(s)\ldots)). G n + m ( s ) = G n ( G m ( s )) = G m ( G n ( s )) , and thus G n ( s ) = G ( G ( … G ( s ) … )) . where the right-hand side has n n n -fold iteration.
Proof. Let Y m , i Y_{m,i} Y m , i denote the number of offspring in the ( m + n ) (m+n) ( m + n ) th generation that descend from the i i i th member of the m m m th generation. Clearly, the ( Y m , i ) (Y_{m,i}) ( Y m , i ) are iid and Y m , i ∼ Z n Y_{m,i} \sim Z_n Y m , i ∼ Z n . The number of members of the ( m + n ) (m+n) ( m + n ) th generation is given by
Z m + n = Y m , 1 + Y m , 2 + … + Y m , Z m . Z_{m+n} = Y_{m,1} + Y_{m,2} + \ldots+ Y_{m,Z_m}. Z m + n = Y m , 1 + Y m , 2 + … + Y m , Z m .
Branching Processes
We have a random sum of iid random variables. By Theorem 3.1.9 it follows that G m + n ( s ) = G m ( G n ( s ) ) G_{m+n}(s)=G_m(G_n(s)) G m + n ( s ) = G m ( G n ( s )) . Obviously, we can interchange m ↔ n m \leftrightarrow n m ↔ n and obtain G m + n ( s ) = G n ( G m ( s ) ) G_{m+n}(s)=G_n(G_m(s)) G m + n ( s ) = G n ( G m ( s )) . Finally, we have
G n ( s ) = G ( G n − 1 ( s ) ) = G ( G ( G n − 2 ( s ) ) ) = ⋯ = G ( G ( … G ( s ) … ) ) , G_n(s)=G(G_{n-1}(s))=G(G(G_{n-2}(s)))=\cdots=G(G(\ldots G(s)\ldots)), G n ( s ) = G ( G n − 1 ( s )) = G ( G ( G n − 2 ( s ))) = ⋯ = G ( G ( … G ( s ) … )) ,
as claimed.
In principle, one can obtain the probability mass function f n f_n f n of the n n n th generation Z n Z_n Z n by expanding the generating function G n ( s ) G_n(s) G n ( s ) as a power series about s = 0 s=0 s = 0 . In practice, this may be difficult to carry out. However, moments of Z n Z_n Z n can usually be computed with relative ease.
Theorem 3.2.2. Suppose E Z 1 = μ \mathbb{E}Z_1=\mu E Z 1 = μ and V Z 1 = σ 2 \mathbb{V}Z_1=\sigma^2 V Z 1 = σ 2 . Then
E Z n = μ n , V Z n = { n σ 2 μ = 1 , σ 2 ( μ n − 1 ) μ n − 1 μ − 1 μ ≠ 1. (21) \mathbb{E}Z_n=\mu^n,\qquad
\mathbb{V}Z_n=
\begin{cases}
n\sigma^2 & \mu=1,\\
\frac{\sigma^2(\mu^n-1)\mu^{n-1}}{\mu-1} & \mu\ne1.
\end{cases} \tag*{(21)} E Z n = μ n , V Z n = { n σ 2 μ − 1 σ 2 ( μ n − 1 ) μ n − 1 μ = 1 , μ = 1. (21)
Proof. We have
E Z n = d d s G n ( s ) ∣ s = 1 = G n − 1 ′ ( G ( 1 ) ) ⋅ G ′ ( 1 ) = G n − 1 ′ ( 1 ) ⋅ μ = E Z n − 1 ⋅ μ , \mathbb{E}Z_n=\left.\frac{\mathrm{d}}{\mathrm{d}s}G_n(s)\right|_{s=1}=G'_{n-1}(G(1))\cdot G'(1)=G'_{n-1}(1)\cdot\mu=\mathbb{E}Z_{n-1}\cdot\mu, E Z n = d s d G n ( s ) s = 1 = G n − 1 ′ ( G ( 1 )) ⋅ G ′ ( 1 ) = G n − 1 ′ ( 1 ) ⋅ μ = E Z n − 1 ⋅ μ ,
where we have used G ( 1 ) = E Z 1 = 1 G(1)=\mathbb{E}Z_1=1 G ( 1 ) = E Z 1 = 1 . Iterating, we obtain E Z n = μ n \mathbb{E}Z_n=\mu^n E Z n = μ n . Next, from (19) , we compute
E Z n 2 − E Z n = d 2 d s 2 G n ( s ) ∣ s = 1 = d d s G n − 1 ′ ( G ( s ) ) ⋅ G ′ ( s ) ∣ s = 1 = G n − 1 ′ ′ ( G ( s ) ) ⋅ ( G ′ ( s ) ) 2 + G n − 1 ′ ( G ( s ) ) ⋅ G ′ ′ ( s ) ∣ s = 1 = G n − 1 ′ ′ ( 1 ) ⋅ ( G ′ ( 1 ) ) 2 + G n − 1 ′ ( 1 ) ⋅ G ′ ′ ( 1 ) = ( E Z n − 1 2 − E Z n − 1 ) ⋅ ( E Z 1 ) 2 + E Z n − 1 ⋅ ( E Z 1 2 − E Z 1 ) . (22) \begin{aligned}
\mathbb{E}Z_n^2-\mathbb{E}Z_n
&=\left.\frac{\mathrm{d}^2}{\mathrm{d}s^2}G_n(s)\right|_{s=1}
=\left.\frac{\mathrm{d}}{\mathrm{d}s}G'_{n-1}(G(s))\cdot G'(s)\right|_{s=1}\\
&=G''_{n-1}(G(s))\cdot(G'(s))^2+G'_{n-1}(G(s))\cdot G''(s)\big|_{s=1}\\
&=G''_{n-1}(1)\cdot(G'(1))^2+G'_{n-1}(1)\cdot G''(1)\\
&=(\mathbb{E}Z_{n-1}^2-\mathbb{E}Z_{n-1})\cdot(\mathbb{E}Z_1)^2+\mathbb{E}Z_{n-1}\cdot(\mathbb{E}Z_1^2-\mathbb{E}Z_1). \tag*{(22)}
\end{aligned} E Z n 2 − E Z n = d s 2 d 2 G n ( s ) s = 1 = d s d G n − 1 ′ ( G ( s )) ⋅ G ′ ( s ) s = 1 = G n − 1 ′′ ( G ( s )) ⋅ ( G ′ ( s ) ) 2 + G n − 1 ′ ( G ( s )) ⋅ G ′′ ( s ) s = 1 = G n − 1 ′′ ( 1 ) ⋅ ( G ′ ( 1 ) ) 2 + G n − 1 ′ ( 1 ) ⋅ G ′′ ( 1 ) = ( E Z n − 1 2 − E Z n − 1 ) ⋅ ( E Z 1 ) 2 + E Z n − 1 ⋅ ( E Z 1 2 − E Z 1 ) . (22)
Thus, using (22) , as well as E Z k = μ k \mathbb{E}Z_k=\mu^k E Z k = μ k and E Z k 2 = V Z k + ( μ k ) 2 \mathbb{E}Z_k^2=\mathbb{V}Z_k+(\mu^k)^2 E Z k 2 = V Z k + ( μ k ) 2 , we obtain
V Z n = E Z n 2 − ( E Z n ) 2 = ( E Z n − 1 2 − E Z n − 1 ) ⋅ ( E Z 1 ) 2 + E Z n − 1 ⋅ ( E Z 1 2 − E Z 1 ) − ( E Z n ) 2 + E Z n = ( V Z n − 1 + ( E Z n − 1 ) 2 − E Z n − 1 ) ⋅ ( E Z 1 ) 2 + E Z n − 1 ⋅ ( V Z 1 + ( E Z 1 ) 2 − E Z 1 ) − ( E Z n ) 2 + E Z n = ( V Z n − 1 + μ 2 ( n − 1 ) − μ n − 1 ) ⋅ μ 2 + μ n − 1 ⋅ ( σ 2 + μ 2 − μ ) − μ 2 n + μ n . \begin{aligned}
\mathbb{V}Z_n&=\mathbb{E}Z_n^2-(\mathbb{E}Z_n)^2\\
&=(\mathbb{E}Z_{n-1}^2-\mathbb{E}Z_{n-1})\cdot(\mathbb{E}Z_1)^2+\mathbb{E}Z_{n-1}\cdot(\mathbb{E}Z_1^2-\mathbb{E}Z_1)-(\mathbb{E}Z_n)^2+\mathbb{E}Z_n\\
&=(\mathbb{V}Z_{n-1}+(\mathbb{E}Z_{n-1})^2-\mathbb{E}Z_{n-1})\cdot(\mathbb{E}Z_1)^2+\mathbb{E}Z_{n-1}\cdot(\mathbb{V}Z_1+(\mathbb{E}Z_1)^2-\mathbb{E}Z_1)-(\mathbb{E}Z_n)^2+\mathbb{E}Z_n\\
&=(\mathbb{V}Z_{n-1}+\mu^{2(n-1)}-\mu^{n-1})\cdot\mu^2+\mu^{n-1}\cdot(\sigma^2+\mu^2-\mu)-\mu^{2n}+\mu^n.
\end{aligned} V Z n = E Z n 2 − ( E Z n ) 2 = ( E Z n − 1 2 − E Z n − 1 ) ⋅ ( E Z 1 ) 2 + E Z n − 1 ⋅ ( E Z 1 2 − E Z 1 ) − ( E Z n ) 2 + E Z n = ( V Z n − 1 + ( E Z n − 1 ) 2 − E Z n − 1 ) ⋅ ( E Z 1 ) 2 + E Z n − 1 ⋅ ( V Z 1 + ( E Z 1 ) 2 − E Z 1 ) − ( E Z n ) 2 + E Z n = ( V Z n − 1 + μ 2 ( n − 1 ) − μ n − 1 ) ⋅ μ 2 + μ n − 1 ⋅ ( σ 2 + μ 2 − μ ) − μ 2 n + μ n .
Thus, we have obtained an expression for V Z n \mathbb{V}Z_n V Z n in terms of V Z n − 1 \mathbb{V}Z_{n-1} V Z n − 1 , μ \mu μ and σ 2 \sigma^2 σ 2 . Solving this explicitly yields (21) .
Characteristic functions
Suppose a random variable X X X has a state space that is not contained in the non-negative integers. When we cannot use the probability generating function G X ( s ) G_X(s) G X ( s ) to analyze the properties of X X X . A natural candidate to replace the probability generating function is the moment generating function M X ( t ) : = E e t X M_X(t) := \mathbb{E}e^{tX} M X ( t ) := E e tX . Moment generating functions, however, are limited in their use because, for many random variables, E e t X = ∞ \mathbb{E}e^{tX} = \infty E e tX = ∞ for all t ≠ 0 t \ne0 t = 0 . However, there is a simple fix to this. We consider instead the characteristic function.
Definition 3.3.1. The characteristic function of a random variable X X X is the function φ X : R → C \varphi_X : \mathbb{R} \to\mathbb{C} φ X : R → C defined by
φ X ( t ) : = E e i t X , i = − 1 . \varphi_X(t) := \mathbb{E}e^{itX}, \qquad i = \sqrt{-1}. φ X ( t ) := E e i tX , i = − 1 .
The characteristic function (unlike the moment generating function) always exists since E ∣ e i t X ∣ = 1 \mathbb{E}|e^{itX}| = 1 E ∣ e i tX ∣ = 1 .
Clearly, if G X G_X G X and φ X \varphi_X φ X both exist, then we have
φ X ( t ) = G X ( e i t ) \varphi_X(t) = G_X(e^{it}) φ X ( t ) = G X ( e i t )
since
E e i t X = E s X , s = e i t . \mathbb{E}e^{itX} = \mathbb{E}s^X, \qquad s = e^{it}. E e i tX = E s X , s = e i t .
We have the following obvious properties.
Theorem 3.3.2. Characteristic functions have the following properties.
Let a , b a, b a , b be constants. Then φ a X + b ( t ) = e i b t φ ( a t ) \varphi_{aX+b}(t) = e^{ibt}\varphi(at) φ a X + b ( t ) = e ib t φ ( a t ) .
If X X X and Y Y Y are independent then φ X + Y ( t ) = φ X ( t ) φ Y ( t ) \varphi_{X+Y}(t) = \varphi_X(t)\varphi_Y(t) φ X + Y ( t ) = φ X ( t ) φ Y ( t ) .
If ( X i ) 1 ≤ i ≤ n (X_i)_{1 \le i \le n} ( X i ) 1 ≤ i ≤ n are iid and S n = ∑ i = 1 n X i S_n = \sum_{i=1}^{n} X_i S n = ∑ i = 1 n X i , then φ S n ( t ) = ( φ X ( t ) ) n \varphi_{S_n}(t) = (\varphi_X(t))^n φ S n ( t ) = ( φ X ( t ) ) n .
Example 3.3.3. Let X ∼ E ( λ ) X \sim\mathcal{E}(\lambda) X ∼ E ( λ ) as in Example Example 1.4.14 . Then
φ X ( t ) = ∫ 0 ∞ d x e i t x λ e − λ x = λ λ − i t . \varphi_X(t) = \int_0^\infty\mathrm{d}x\,e^{itx}\lambda e^{-\lambda x} = \frac{\lambda}{\lambda-it}. φ X ( t ) = ∫ 0 ∞ d x e i t x λ e − λ x = λ − i t λ .
Example 3.3.4. Let X ∼ N ( μ , σ 2 ) X \sim\mathcal{N}(\mu,\sigma^2) X ∼ N ( μ , σ 2 ) as in Example Example 1.4.15 . Then
φ X ( t ) = ∫ − ∞ ∞ d x e i t x 1 2 π σ 2 exp ( − ( x − μ ) 2 2 σ 2 ) = exp ( i μ t − 1 2 σ 2 t 2 ) . \varphi_X(t) = \int_{-\infty}^{\infty} \mathrm{d}x\,e^{itx}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) = \exp\left(i\mu t-\frac{1}{2}\sigma^2t^2\right). φ X ( t ) = ∫ − ∞ ∞ d x e i t x 2 π σ 2 1 exp ( − 2 σ 2 ( x − μ ) 2 ) = exp ( i μ t − 2 1 σ 2 t 2 ) .
Characteristic functions have several uses. First, they can be used to capture the moments of a random variable when they exist.
Theorem 3.3.5. Let φ X ( t ) \varphi_X(t) φ X ( t ) be the characteristic function of a random variable X X X . If E ∣ X ∣ n < ∞ \mathbb{E}|X|^n < \infty E ∣ X ∣ n < ∞ , then
φ ( n ) ( 0 ) = i n E X n . \varphi^{(n)}(0) = i^n\mathbb{E}X^n. φ ( n ) ( 0 ) = i n E X n .
Proof. We have
d n d t n φ X ( t ) = E d n d t n e i t X = i n E X n e i t X . \frac{\mathrm{d}^{n}}{\mathrm{d}t^{n}}\varphi_X(t)=\mathbb{E}\frac{\mathrm{d}^{n}}{\mathrm{d}t^{n}}e^{itX}=i^{n}\mathbb{E}X^{n}e^{itX}. d t n d n φ X ( t ) = E d t n d n e i tX = i n E X n e i tX .
Now set t = 0 t=0 t = 0 to complete the proof.
□ \square □
The characteristic function uniquely determines the distribution of a random variable. In other words, there is a one-to-one correspondence between F X F_X F X and φ X \varphi_X φ X . We show this for a continuous random variable.
Theorem 3.3.6 (Inversion). Suppose the characteristic function φ X \varphi_X φ X of a random variable X X X is absolutely integrable: ∫ R d t ∣ φ X ( t ) ∣ < ∞ \int_{\mathbb{R}}\mathrm{d}t\,|\varphi_X(t)|<\infty ∫ R d t ∣ φ X ( t ) ∣ < ∞ . Then X X X is a continuous random variable with a continuous density f X f_X f X , which is given by
f X ( x ) = 1 2 π ∫ R d t e − i t x φ X ( t ) , x ∈ R . f_X(x)=\frac{1}{2\pi}\int_{\mathbb{R}}\mathrm{d}t\,e^{-itx}\varphi_X(t),
\qquad x\in\mathbb{R}. f X ( x ) = 2 π 1 ∫ R d t e − i t x φ X ( t ) , x ∈ R . To obtain F X F_X F X from f X f_X f X simply use F X ( x ) = ∫ − ∞ x d y f X ( y ) F_X(x)=\int_{-\infty}^{x}\mathrm{d}y\,f_X(y) F X ( x ) = ∫ − ∞ x d y f X ( y ) .
Sketch of proof. The proof of Theorem 3.3.6 follows from standard Fourier results. Assuming X X X has a density f X f_X f X , Fourier inversion gives
f X ( x ) = 1 2 π ∫ R d t e − i t x ∫ R d y e i t y f X ( y ) = 1 2 π ∫ R d t e − i t x E e i t X = 1 2 π ∫ R d t e − i t x φ X ( t ) . f_X(x)=\frac{1}{2\pi}\int_{\mathbb{R}}\mathrm{d}t\,e^{-itx}\int_{\mathbb{R}}\mathrm{d}y\,e^{ity}f_X(y)=\frac{1}{2\pi}\int_{\mathbb{R}}\mathrm{d}t\,e^{-itx}\mathbb{E}e^{itX}=\frac{1}{2\pi}\int_{\mathbb{R}}\mathrm{d}t\,e^{-itx}\varphi_X(t). f X ( x ) = 2 π 1 ∫ R d t e − i t x ∫ R d y e i t y f X ( y ) = 2 π 1 ∫ R d t e − i t x E e i tX = 2 π 1 ∫ R d t e − i t x φ X ( t ) .
□ \square □
An inversion theorem for random variables that are not continuous also exists. Though, it is not particularly useful for the purposes of computation.
Perhaps the most important property of characteristic functions is they can be used to prove the convergence of a sequence of random variables to a limiting distribution.
Definition 3.3.7. We say that a sequence of distribution functions ( F n ) n ≥ 1 (F_n)_{n\geq1} ( F n ) n ≥ 1 converges to a distribution F F F , written F n → F F_n\to F F n → F , if lim n → ∞ F n ( x ) = F ( x ) \lim_{n\to\infty}F_n(x)=F(x) lim n → ∞ F n ( x ) = F ( x ) at all points x x x where F F F is continuous.
Theorem 3.3.8 (Continuity Theorem). Let ( F n ) n ≥ 1 (F_n)_{n\geq1} ( F n ) n ≥ 1 be a sequence of distributions functions with corresponding characteristic functions ( φ n ) n ≥ 1 (\varphi_n)_{n\geq1} ( φ n ) n ≥ 1 .
If F n → F F_n\to F F n → F where F F F is a distribution function with corresponding characteristic function φ \varphi φ , then φ n → φ \varphi_n\to\varphi φ n → φ (pointwise).
Conversely, if φ ( t ) : = lim n → ∞ φ n ( t ) \varphi(t):=\lim_{n\to\infty}\varphi_n(t) φ ( t ) := lim n → ∞ φ n ( t ) exists and is continuous at t = 0 t=0 t = 0 , then φ \varphi φ is the characteristic function of a distribution F F F and F n → F F_n\to F F n → F .
Item 2 in Theorem 3.3.8 is particularly powerful. If F n F_n F n and φ n \varphi_n φ n are, respectively, the distribution and characteristic function of a sum of n n n independent random variables, it is often easier to compute φ n \varphi_n φ n than F n F_n F n . If we can compute φ n \varphi_n φ n and find its limit, then we can obtain F F F .
Example 3.3.9. In Example 3.1.3 we found that
X ∼ Geo ( p ) ⇒ G X ( s ) = s p 1 − s q . X \sim\operatorname{Geo}(p) \quad\Rightarrow\quad G_X(s)=\frac{sp}{1-sq}. X ∼ Geo ( p ) ⇒ G X ( s ) = 1 − s q s p .
From this, one can easily show that E X = 1 / p \mathbb{E}X=1/p E X = 1/ p . Now consider a sequence of random variables ( Y n ) n ≥ 0 (Y_n)_{n\geq0} ( Y n ) n ≥ 0 defined by
Y n = 1 n X n , X n ∼ Geo ( λ / n ) , Y_n=\frac{1}{n}X_n,\qquad X_n\sim\operatorname{Geo}(\lambda/n), Y n = n 1 X n , X n ∼ Geo ( λ / n ) ,
where λ > 0 \lambda>0 λ > 0 is a fixed constant. For n n n large enough, λ / n < 1 \lambda/n<1 λ / n < 1 . Note that E Y n = 1 n E X n = 1 n n λ = 1 λ \mathbb{E}Y_n=\frac{1}{n}\mathbb{E}X_n=\frac{1}{n}\frac{n}{\lambda}=\frac{1}{\lambda} E Y n = n 1 E X n = n 1 λ n = λ 1 for all n n n . We would like to know what the limiting distribution of the sequence ( Y n ) n ≥ 0 (Y_n)_{n\geq0} ( Y n ) n ≥ 0 is; we will use the Continuity Theorem 3.3.8 to do this. We have
φ Y n ( t ) = φ X n ( t / n ) = e i t / n ( λ / n ) 1 − e i t / n ( 1 − λ / n ) = e i t / n λ n − e i t / n ( n − λ ) = e i t / n λ n ( 1 − e i t / n ) + e i t / n λ = ( 1 + i t / n ) λ n ( 1 − 1 − i t / n − ⋯ ) + ( 1 + i t / n + ⋯ ) λ ⟶ λ λ − i t = φ Z ( t ) , \begin{aligned}
\varphi_{Y_n}(t)&=\varphi_{X_n}(t/n)=\frac{e^{it/n}(\lambda/n)}{1-e^{it/n}(1-\lambda/n)}
=\frac{e^{it/n}\lambda}{n-e^{it/n}(n-\lambda)}\\
&=\frac{e^{it/n}\lambda}{n(1-e^{it/n})+e^{it/n}\lambda}\\
&=\frac{(1+it/n)\lambda}{n(1-1-it/n-\cdots)+(1+it/n+\cdots)\lambda}
\longrightarrow\frac{\lambda}{\lambda-it}=\varphi_Z(t),
\end{aligned} φ Y n ( t ) = φ X n ( t / n ) = 1 − e i t / n ( 1 − λ / n ) e i t / n ( λ / n ) = n − e i t / n ( n − λ ) e i t / n λ = n ( 1 − e i t / n ) + e i t / n λ e i t / n λ = n ( 1 − 1 − i t / n − ⋯ ) + ( 1 + i t / n + ⋯ ) λ ( 1 + i t / n ) λ ⟶ λ − i t λ = φ Z ( t ) ,
where Z ∼ E ( λ ) Z\sim\mathcal{E}(\lambda) Z ∼ E ( λ ) (see Example 3.3.3). By the Continuity Theorem 3.3.8 , since φ Y n → φ Z \varphi_{Y_n}\to\varphi_Z φ Y n → φ Z it follows that F Y n → F Z F_{Y_n}\to F_Z F Y n → F Z .
LLN and the CLT
In this section we will use Theorem 3.3.8 to prove the Law of Large Numbers (LLN) and the Central Limit Theorem (CLT). First, we must define the notion of convergence in distribution.
Definition 3.4.1. We say that a sequence of random variables ( X n ) n ≥ 1 (X_n)_{n\geq1} ( X n ) n ≥ 1 converges in distribution to a random variable X X X , written X n → D X X_n \xrightarrow{\mathcal{D}} X X n D X , if F X n → F X F_{X_n}\to F_X F X n → F X .
Theorem 3.4.2 (Law of Large Numbers). Let ( X n ) n ≥ 1 (X_n)_{n\geq1} ( X n ) n ≥ 1 be a sequence of iid random variables with E X n = μ \mathbb{E}X_n=\mu E X n = μ . Define a sequence of random variables ( S n ) n ≥ 1 (S_n)_{n\geq1} ( S n ) n ≥ 1 by S n : = 1 n ∑ i = 1 n X i S_n:=\frac{1}{n}\sum_{i=1}^{n}X_i S n := n 1 ∑ i = 1 n X i . Then S n → D μ S_n\xrightarrow{\mathcal{D}}\mu S n D μ .
Proof. From Theorem 3.3.2 we have
φ S n ( t ) = ( φ X ( t / n ) ) n = ( 1 + i μ t n + O ( ( t / n ) 2 ) ) n → e i μ t , as n → ∞ . \varphi_{S_n}(t)=\left(\varphi_X(t/n)\right)^n=\left(1+\frac{i\mu t}{n}+O((t/n)^2)\right)^n\to e^{i\mu t},\qquad\text{as }n\to\infty. φ S n ( t ) = ( φ X ( t / n ) ) n = ( 1 + n i μ t + O (( t / n ) 2 ) ) n → e i μ t , as n → ∞. Note that e i μ t = φ μ ( t ) e^{i\mu t}=\varphi_\mu(t) e i μ t = φ μ ( t ) , where φ μ \varphi_\mu φ μ is the characteristic function of a constant random variable μ \mu μ . Thus, we have shown that φ S n → φ μ \varphi_{S_n}\to\varphi_\mu φ S n → φ μ . By Theorem 3.3.8 we have F S n → F μ F_{S_n}\to F_\mu F S n → F μ and thus, from Definition 3.4.1 , we have S n → D μ S_n\xrightarrow{\mathcal{D}}\mu S n D μ .
In the proof of Theorem 3.4.2 we actually assumed that E X n 2 < ∞ \mathbb{E}X_n^2 < \infty E X n 2 < ∞ when we wrote the O ( ( t / n ) 2 ) O((t/n)^2) O (( t / n ) 2 ) . In fact, the Law of Large Numbers (LLN) holds even when E X n 2 = ∞ \mathbb{E}X_n^2 = \infty E X n 2 = ∞ .
Theorem 3.4.3 (Central Limit Theorem). Let ( X n ) n ≥ 1 (X_n)_{n\geq1} ( X n ) n ≥ 1 be a sequence of iid random variables with E X n = μ \mathbb{E}X_n = \mu E X n = μ and V X n = σ 2 ∈ ( 0 , ∞ ) \mathop{\mathrm{V}}X_n = \sigma^2 \in(0,\infty) V X n = σ 2 ∈ ( 0 , ∞ ) . Define two sequences of random variables ( S n ) n ≥ 1 (S_n)_{n\geq1} ( S n ) n ≥ 1 and ( U n ) n ≥ 1 (U_n)_{n\geq1} ( U n ) n ≥ 1 by
S n = ∑ i = 1 n X i , U n = S n − n μ n σ 2 . \begin{aligned}
S_n &= \sum_{i=1}^{n} X_i, \\
U_n &= \frac{S_n-n\mu}{\sqrt{n\sigma^2}}.
\end{aligned} S n U n = i = 1 ∑ n X i , = n σ 2 S n − n μ . Then U n → D Z U_n \xrightarrow{\mathcal{D}} Z U n D Z where Z ∼ N ( 0 , 1 ) Z \sim\mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) .
Proof. First, we re-write U n U_n U n as follows
U n = 1 n ∑ i = 1 n Y i , Y i : = X i − μ σ . \begin{aligned}
U_n &= \frac{1}{\sqrt{n}}\sum_{i=1}^{n}Y_i, &
Y_i &:= \frac{X_i-\mu}{\sigma}.
\end{aligned} U n = n 1 i = 1 ∑ n Y i , Y i := σ X i − μ .
Note that E Y n = 0 \mathbb{E}Y_n = 0 E Y n = 0 and V Y n = 1 \mathop{\mathrm{V}}Y_n = 1 V Y n = 1 . Next, we compute the characteristic function of U n U_n U n . From Theorem 3.3.2 we have
φ U n ( t ) = ( φ Y ( t / n ) ) n = ( 1 − t 2 2 n + O ( ( t / n ) 3 ) ) n → exp ( − 1 2 t 2 ) , n → ∞ . \varphi_{U_n}(t)=\left(\varphi_Y(t/\sqrt{n})\right)^n=\left(1-\frac{t^2}{2n}+O((t/n)^3)\right)^n\to\exp\left(-\frac{1}{2}t^2\right),\qquad n\to\infty. φ U n ( t ) = ( φ Y ( t / n ) ) n = ( 1 − 2 n t 2 + O (( t / n ) 3 ) ) n → exp ( − 2 1 t 2 ) , n → ∞.
From Example 3.3.4, we see that exp ( − 1 2 t 2 ) = φ Z ( t ) \exp\left(-\frac{1}{2}t^2\right)=\varphi_Z(t) exp ( − 2 1 t 2 ) = φ Z ( t ) where Z ∼ N ( 0 , 1 ) Z\sim\mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) . Thus, we have shown that φ U n → φ Z \varphi_{U_n}\to\varphi_Z φ U n → φ Z . By Theorem 3.3.8 we have F U n → F Z F_{U_n}\to F_Z F U n → F Z and thus, from Definition 3.4.1 , we have U n → D Z U_n\xrightarrow{\mathcal{D}}Z U n D Z , as claimed. □
In the proof of Theorem 3.4.3 we assume E ∣ X i ∣ 3 < ∞ \mathbb{E}|X_i|^3 < \infty E ∣ X i ∣ 3 < ∞ when we wrote the O ( ( t / n ) 3 ) O((t/n)^3) O (( t / n ) 3 ) term. The Central Limit Theorem (CLT) holds even when E ∣ X i ∣ 3 = ∞ \mathbb{E}|X_i|^3 = \infty E ∣ X i ∣ 3 = ∞ .
Large Deviations Principle
Let S n S_n S n be the sum of n n n iid random variables ( X i ) (X_i) ( X i ) with common distribution X X X whose mean and variance are μ \mu μ and σ \sigma σ , respectively
S n = ∑ i = 1 n X i , E X = μ , V X = σ 2 . \begin{aligned}
S_n &= \sum_{i=1}^{n}X_i, &
\mathbb{E}X &= \mu, &
\mathop{\mathrm{V}}X &= \sigma^2.
\end{aligned} S n = i = 1 ∑ n X i , E X = μ , V X = σ 2 .
The SLLN asserts that, as n → ∞ n\to\infty n → ∞ , we roughly have S n ∼ n μ S_n\sim n\mu S n ∼ n μ . Similarly, the CLT asserts that, as n → ∞ n\to\infty n → ∞ , we roughly have ∣ S n − n μ ∣ ∼ n |S_n-n\mu|\sim\sqrt{n} ∣ S n − n μ ∣ ∼ n . But, of course, S n S_n S n is a random variable, not a constant. And it is quite possible that ∣ S n − n μ ∣ |S_n-n\mu| ∣ S n − n μ ∣ is O ( n α ) O(n^\alpha) O ( n α ) where α > 1 / 2 \alpha>1/2 α > 1/2 . The theory of large deviations studies the asymptotic behavior of P ( ∣ S n − n μ ∣ > n α ) \mathbb{P}(\lvert S_n-n\mu\rvert> n^\alpha) P (∣ S n − n μ ∣ > n α ) as n → ∞ n \to\infty n → ∞ for α > 1 / 2 \alpha> 1/2 α > 1/2 .
Let us introduce Λ X \Lambda_X Λ X , the cumulant generating function of X X X , which is given by
Λ X ( t ) = log M X ( t ) , M X ( t ) = E e t X , (23) \Lambda_X(t)=\log M_X(t), \qquad M_X(t)=\mathbb{E}e^{tX}, \tag*{(23)} Λ X ( t ) = log M X ( t ) , M X ( t ) = E e tX , (23)
assuming M X ( t ) M_X(t) M X ( t ) exists in a neighborhood of zero. We note that Λ X ′ ( 0 ) = μ \Lambda_X'(0)=\mu Λ X ′ ( 0 ) = μ because
Λ X ′ ( 0 ) = M X ′ ( 0 ) M X ( 0 ) = E X e t X E e t X ∣ t = 0 = E X = μ . \Lambda_X'(0)=\frac{M_X'(0)}{M_X(0)}=\left.\frac{\mathbb{E}Xe^{tX}}{\mathbb{E}e^{tX}}\right|_{t=0}=\mathbb{E}X=\mu. Λ X ′ ( 0 ) = M X ( 0 ) M X ′ ( 0 ) = E e tX E X e tX t = 0 = E X = μ .
We also note that Λ X \Lambda_X Λ X is convex because
Λ X ′ ′ ( t ) = M X ( t ) M X ′ ′ ( t ) − ( M X ′ ( t ) ) 2 M X 2 ( t ) = E X e t X E X 2 e t X − ( E X e t X ) 2 M X 2 ( t ) ≥ 0 , (24) \Lambda_X''(t)=\frac{M_X(t)M_X''(t)-(M_X'(t))^2}{M_X^2(t)}
=\frac{\mathbb{E}Xe^{tX}\mathbb{E}X^2e^{tX}-(\mathbb{E}Xe^{tX})^2}{M_X^2(t)}\geq0, \tag*{(24)} Λ X ′′ ( t ) = M X 2 ( t ) M X ( t ) M X ′′ ( t ) − ( M X ′ ( t ) ) 2 = M X 2 ( t ) E X e tX E X 2 e tX − ( E X e tX ) 2 ≥ 0 , (24)
where we have used the Cauchy-Schwarz inequality
( E Y Z ) 2 ≤ E Y 2 E Z 2 , with Y = X e t X / 2 , Z = e t X / 2 . (\mathbb{E}YZ)^2\leq\mathbb{E}Y^2\mathbb{E}Z^2,
\qquad\text{with}\qquad
Y=Xe^{tX/2},\qquad Z=e^{tX/2}. ( E Y Z ) 2 ≤ E Y 2 E Z 2 , with Y = X e tX /2 , Z = e tX /2 .
Next, we define Λ X ∗ \Lambda_X^* Λ X ∗ , the Fenchel-Legendre transform for Λ X \Lambda_X Λ X as follows
Λ X ∗ ( a ) = sup t ∈ R { a t − Λ X ( t ) } , a ∈ R . (25) \Lambda_X^*(a)=\sup_{t\in\mathbb{R}}\{at-\Lambda_X(t)\},\qquad a\in\mathbb{R}. \tag*{(25)} Λ X ∗ ( a ) = t ∈ R sup { a t − Λ X ( t )} , a ∈ R . (25)
We are now in a position to state the main theorem of this section
Theorem 3.5.1 (Large Deviations Principle). Let ( X i ) (X_i) ( X i ) be a sequence of iid random variables with common distribution X X X . Define μ : = E X \mu:=\mathbb{E}X μ := E X and suppose the moment generating function M X ( t ) : = E e t X M_X(t):=\mathbb{E}e^{tX} M X ( t ) := E e tX is finite in some neighborhood of t = 0 t=0 t = 0 . Let Λ X \Lambda_X Λ X and Λ X ∗ \Lambda_X^* Λ X ∗ be given by (23) and (25) , respectively. Suppose a > μ a>\mu a > μ and P ( X > a ) > 0 \mathbb{P}(X>a)>0 P ( X > a ) > 0 . Then Λ X ∗ ( a ) > 0 \Lambda_X^*(a)>0 Λ X ∗ ( a ) > 0 and
lim n → ∞ 1 n log P ( S n > n a ) = − Λ X ∗ ( a ) , S n = ∑ i = 1 n X i . (26) \lim_{n\to\infty}\frac{1}{n}\log\mathbb{P}(S_n>na)=-\Lambda_X^*(a),\qquad S_n=\sum_{i=1}^{n}X_i. \tag*{(26)} n → ∞ lim n 1 log P ( S n > na ) = − Λ X ∗ ( a ) , S n = i = 1 ∑ n X i . (26)
Theorem 3.5.1 asserts that, under appropriate conditions, we have P ( S n > n a ) ∼ exp ( − n Λ X ∗ ( a ) ) \mathbb{P}(S_n>na)\sim\exp(-n\Lambda_X^*(a)) P ( S n > na ) ∼ exp ( − n Λ X ∗ ( a )) . Although the theorem appears to deal only with deviations of S n S_n S n in excess of its mean, the corresponding result for deviations of S n S_n S n below the mean can be obtained by considering the sequence of iid random variables ( − X i ) (-X_i) ( − X i ) .
Proof of Theorem 3.5.1. Without loss of generality, we may assume that μ = 0 \mu=0 μ = 0 (if μ ≠ 0 \mu\neq0 μ = 0 , we can define Y i = X i − μ Y_i=X_i-\mu Y i = X i − μ and translate the result accordingly). We begin by proving that Λ X ∗ ( a ) > 0 \Lambda_X^*(a)>0 Λ X ∗ ( a ) > 0 . We have
a t − Λ X ( t ) = log ( e a t M X ( t ) ) = log ( 1 + a t + O ( t 2 ) 1 + σ 2 t 2 / 2 + O ( t 3 ) ) , where σ 2 : = V X . at-\Lambda_X(t)=\log\left(\frac{e^{at}}{M_X(t)}\right)=\log\left(\frac{1+at+O(t^2)}{1+\sigma^2t^2/2+O(t^3)}\right),\qquad\text{where }\sigma^2:=\mathbb{V}X. a t − Λ X ( t ) = log ( M X ( t ) e a t ) = log ( 1 + σ 2 t 2 /2 + O ( t 3 ) 1 + a t + O ( t 2 ) ) , where σ 2 := V X .
Large Deviations Principle
For t > 0 t > 0 t > 0 sufficiently small we have
1 + a t + O ( t 2 ) 1 + σ 2 t 2 / 2 + O ( t 3 ) > 1. \frac{1 + at + O(t^2)}{1 + \sigma^2t^2/2 + O(t^3)} > 1. 1 + σ 2 t 2 /2 + O ( t 3 ) 1 + a t + O ( t 2 ) > 1.
Hence, we have
Λ X ∗ ( a ) = sup t ∈ R { a t − Λ X ( t ) } ≥ sup t ∈ R + log ( 1 + a t + O ( t 2 ) 1 + σ 2 t 2 / 2 + O ( t 3 ) ) > 0. \Lambda_X^*(a) = \sup_{t\in\mathbb{R}}\{at - \Lambda_X(t)\} \ge\sup_{t\in\mathbb{R}_+}\log\left(\frac{1 + at + O(t^2)}{1 + \sigma^2t^2/2 + O(t^3)}\right) > 0. Λ X ∗ ( a ) = t ∈ R sup { a t − Λ X ( t )} ≥ t ∈ R + sup log ( 1 + σ 2 t 2 /2 + O ( t 3 ) 1 + a t + O ( t 2 ) ) > 0.
We make two notes for future use. First, by assumption we have a > μ = 0 a > \mu= 0 a > μ = 0 . As Λ X \Lambda_X Λ X is convex with Λ X ′ ( 0 ) = μ = 0 \Lambda_X'(0) = \mu= 0 Λ X ′ ( 0 ) = μ = 0 and Λ X ( 0 ) = 0 \Lambda_X(0) = 0 Λ X ( 0 ) = 0 it follows that
Λ X ∗ ( a ) = sup t > 0 { a t − Λ X ( t ) } , a > 0. (27) \Lambda_X^*(a) = \sup_{t>0}\{at - \Lambda_X(t)\}, \qquad a > 0. \tag*{(27)} Λ X ∗ ( a ) = t > 0 sup { a t − Λ X ( t )} , a > 0. (27)
To see (27) , it may help to draw a picture. Second,
Λ X is strictly convex at points where Λ X ′ ′ exists. (28) \Lambda_X \text{ is strictly convex at points where } \Lambda_X'' \text{ exists.} \tag*{(28)} Λ X is strictly convex at points where Λ X ′′ exists. (28)
To see this, note that Var X > 0 \operatorname{Var}X > 0 Var X > 0 under the hypotheses of the theorem, implying by (24) and the Cauchy-Schwarz inequality that Λ X ′ ′ ( t ) > 0 \Lambda_X''(t) > 0 Λ X ′′ ( t ) > 0 .
We now proceed to derive an upper bound for P ( S n > n a ) \mathbb{P}(S_n > na) P ( S n > na ) . Using the fact that e t S n > e n a t 1 { S n > n a } e^{tS_n} > e^{nat}\mathbf{1}_{\{S_n>na\}} e t S n > e na t 1 { S n > na } for all t > 0 t > 0 t > 0 we obtain
P ( S n > n a ) = E 1 { S n > n a } ≤ e − n a t E e t S n = ( e − a t M X ( t ) ) n = e − n ( a t − Λ X ( t ) ) , ∀ t > 0. (29) \mathbb{P}(S_n > na) = \mathbb{E}\mathbf{1}_{\{S_n>na\}} \le e^{-nat}\mathbb{E}e^{tS_n} = \left(e^{-at}M_X(t)\right)^n = e^{-n(at-\Lambda_X(t))}, \qquad\forall\ t > 0. \tag*{(29)} P ( S n > na ) = E 1 { S n > na } ≤ e − na t E e t S n = ( e − a t M X ( t ) ) n = e − n ( a t − Λ X ( t )) , ∀ t > 0. (29)
Note that (29) is valid for all t > 0 t > 0 t > 0 and thus
1 n log P ( S n > n a ) ≤ − sup t > 0 { a t − Λ X ( t ) } = − Λ X ∗ ( a ) , (30) \frac{1}{n}\log\mathbb{P}(S_n > na) \le-\sup_{t>0}\{at-\Lambda_X(t)\} = -\Lambda_X^*(a), \tag*{(30)} n 1 log P ( S n > na ) ≤ − t > 0 sup { a t − Λ X ( t )} = − Λ X ∗ ( a ) , (30)
where the equality follows from (27) .
Before obtaining lower bound for P ( S n > n a ) \mathbb{P}(S_n > na) P ( S n > na ) let us define
T : = sup { t : M X ( t ) < ∞ } . T := \sup\{t : M_X(t) < \infty\}. T := sup { t : M X ( t ) < ∞ } .
Throughout this proof we will assume that
t ∗ : = argmax t { a t − Λ X ( t ) } ∈ ( 0 , T ) . (31) t^* := \underset{t}{\operatorname{argmax}}\{at-\Lambda_X(t)\} \in(0,T). \tag*{(31)} t ∗ := t argmax { a t − Λ X ( t )} ∈ ( 0 , T ) . (31)
This assumption is not necessary, but simplifies the proof considerably. As a t − Λ X ( t ) at-\Lambda_X(t) a t − Λ X ( t ) has a maximum at t ∗ t^* t ∗ , and as Λ X \Lambda_X Λ X is C ∞ C^\infty C ∞ on ( 0 , T ) (0,T) ( 0 , T ) , the derivative of a t − Λ X ( t ) at-\Lambda_X(t) a t − Λ X ( t ) equals zero at t = t ∗ t=t^* t = t ∗ and hence
Λ X ′ ( t ∗ ) = a . \Lambda_X'(t^*) = a. Λ X ′ ( t ∗ ) = a .
Let us denote by F X F_X F X the distribution of X X X . We can define an exponentially tilted distribution F ~ X \widetilde{F}_X F X as follows
F ~ X ( x ) = 1 M X ( t ∗ ) ∫ − ∞ x e t ∗ y d F X ( y ) . (32) \widetilde{F}_X(x)=\frac{1}{M_X(t^*)}\int_{-\infty}^{x}e^{t^*y}\,dF_X(y). \tag*{(32)} F X ( x ) = M X ( t ∗ ) 1 ∫ − ∞ x e t ∗ y d F X ( y ) . (32)
Let ( X ~ i ) (\widetilde{X}_i) ( X i ) be a sequence of iid random variables with common distribution F ~ X \widetilde{F}_X F X and define S ~ n = ∑ i = 1 n X ~ i \widetilde{S}_n=\sum_{i=1}^{n}\widetilde{X}_i S n = ∑ i = 1 n X i . Observe that
M X ~ ( t ) : = ∫ R e t x d F ~ X ( x ) = ∫ R e ( t + t ∗ ) x M X ( t ∗ ) d F X ( x ) = M X ( t + t ∗ ) M X ( t ∗ ) . M_{\widetilde{X}}(t):=\int_{\mathbb{R}}e^{tx}\,d\widetilde{F}_X(x)=\int_{\mathbb{R}}\frac{e^{(t+t^*)x}}{M_X(t^*)}\,dF_X(x)=\frac{M_X(t+t^*)}{M_X(t^*)}. M X ( t ) := ∫ R e t x d F X ( x ) = ∫ R M X ( t ∗ ) e ( t + t ∗ ) x d F X ( x ) = M X ( t ∗ ) M X ( t + t ∗ ) .
The first two moments of X ~ \widetilde{X} X can be obtained by direct computation
E X ~ = M X ~ ′ ( 0 ) = M X ′ ( t ∗ ) M X ( t ∗ ) = Λ X ′ ( t ∗ ) = a , \mathbb{E}\widetilde{X}=M_{\widetilde{X}}'(0)=\frac{M_X'(t^*)}{M_X(t^*)}=\Lambda_X'(t^*)=a, E X = M X ′ ( 0 ) = M X ( t ∗ ) M X ′ ( t ∗ ) = Λ X ′ ( t ∗ ) = a ,
V a r X ~ = E X ~ 2 − ( E X ~ ) 2 = M X ~ ′ ′ ( 0 ) − ( M X ~ ′ ( 0 ) ) 2 = Λ X ′ ′ ( t ∗ ) ∈ ( 0 , ∞ ) . \mathop{\mathrm{Var}}\widetilde{X}=\mathbb{E}\widetilde{X}^{2}-(\mathbb{E}\widetilde{X})^{2}=M_{\widetilde{X}}''(0)-\left(M_{\widetilde{X}}'(0)\right)^{2}=\Lambda_X''(t^*)\in(0,\infty). Var X = E X 2 − ( E X ) 2 = M X ′′ ( 0 ) − ( M X ′ ( 0 ) ) 2 = Λ X ′′ ( t ∗ ) ∈ ( 0 , ∞ ) .
where, the fact that 0 < Λ X ′ ′ ( t ∗ ) < ∞ 0<\Lambda_X''(t^*)<\infty 0 < Λ X ′′ ( t ∗ ) < ∞ follow from (28) and the assumption (31) . Noting that S n S_n S n and S ~ n \widetilde{S}_n S n are sums of iid random variables, we have
M S n ( t ) = M X n ( t ) , M S ~ n ( t ) = M X ~ n ( t ) . M_{S_n}(t)=M_X^n(t),\qquad M_{\widetilde{S}_n}(t)=M_{\widetilde{X}}^n(t). M S n ( t ) = M X n ( t ) , M S n ( t ) = M X n ( t ) .
Using the above and (32) , it is easy to show that F S n F_{S_n} F S n and F ~ S ~ n \widetilde{F}_{\widetilde{S}_n} F S n are related as follows
F ~ S ~ n ( x ) = 1 M X n ( t ∗ ) ∫ − ∞ x e t ∗ y d F S n ( y ) . \widetilde{F}_{\widetilde{S}_n}(x)=\frac{1}{M_X^n(t^*)}\int_{-\infty}^{x}e^{t^*y}\,dF_{S_n}(y). F S n ( x ) = M X n ( t ∗ ) 1 ∫ − ∞ x e t ∗ y d F S n ( y ) .
Now, let b > a b>a b > a . We have
P ( S n > n a ) = ∫ n a ∞ d F S n ( x ) = ∫ n a ∞ M X n ( t ∗ ) e − t ∗ x d F ~ S ~ n ( x ) ≥ M X n ( t ∗ ) e − n t ∗ b ∫ n a n b d F ~ S ~ n ( x ) ≥ e − n ( t ∗ b − Λ X ( t ∗ ) ) P ( n a < S ~ n < n b ) . \begin{aligned}
\mathbb{P}(S_n>na)&=\int_{na}^{\infty}dF_{S_n}(x)\\
&=\int_{na}^{\infty}M_X^n(t^*)e^{-t^*x}\,d\widetilde{F}_{\widetilde{S}_n}(x)\\
&\ge M_X^n(t^*)e^{-nt^*b}\int_{na}^{nb}d\widetilde{F}_{\widetilde{S}_n}(x)\\
&\ge e^{-n(t^*b-\Lambda_X(t^*))}\mathbb{P}(na<\widetilde{S}_n<nb).
\end{aligned} P ( S n > na ) = ∫ na ∞ d F S n ( x ) = ∫ na ∞ M X n ( t ∗ ) e − t ∗ x d F S n ( x ) ≥ M X n ( t ∗ ) e − n t ∗ b ∫ na nb d F S n ( x ) ≥ e − n ( t ∗ b − Λ X ( t ∗ )) P ( na < S n < nb ) .
As E X ~ = a \mathbb{E}\widetilde{X}=a E X = a and V a r X ~ > 0 \mathop{\mathrm{Var}}\widetilde{X}>0 Var X > 0 we have by the CLT that P ( S ~ n > n a ) → 1 / 2 \mathbb{P}(\widetilde{S}_n>na)\to1/2 P ( S n > na ) → 1/2 as n → ∞ n\to\infty n → ∞ , and by the SLLN we have that P ( S ~ n < n b ) → 1 \mathbb{P}(\widetilde{S}_n<nb)\to1 P ( S n < nb ) → 1 . Therefore, we have
1 n log P ( S n > n a ) ≥ − ( t ∗ b − Λ X ( t ∗ ) ) + 1 n log P ( n a < S ~ n < n b ) → − ( t ∗ b − Λ X ( t ∗ ) ) as n → ∞ → − ( t ∗ a − Λ X ( t ∗ ) ) = − Λ X ∗ ( a ) as b → a . (33) \begin{aligned}
\frac{1}{n}\log\mathbb{P}(S_n>na)&\ge-(t^*b-\Lambda_X(t^*))+\frac{1}{n}\log\mathbb{P}(na<\widetilde{S}_n<nb)\\
&\to-(t^*b-\Lambda_X(t^*))&&\text{as }n\to\infty\\
&\to-(t^*a-\Lambda_X(t^*))=-\Lambda_X^*(a)&&\text{as }b\to a. \tag*{(33)}
\end{aligned} n 1 log P ( S n > na ) ≥ − ( t ∗ b − Λ X ( t ∗ )) + n 1 log P ( na < S n < nb ) → − ( t ∗ b − Λ X ( t ∗ )) → − ( t ∗ a − Λ X ( t ∗ )) = − Λ X ∗ ( a ) as n → ∞ as b → a . (33)
Equation (26) follows from (30) and (33) .
Example 3.5.2. Let ( X i ) (X_i) ( X i ) be a sequence of iid random variables with P ( X i = 1 ) = P ( X i = − 1 ) = 1 / 2 \mathbb{P}(X_i=1)=\mathbb{P}(X_i=-1)=1/2 P ( X i = 1 ) = P ( X i = − 1 ) = 1/2 . We claim that
P ( S n > a n ) 1 / n → 1 ( 1 + a ) 1 + a ( 1 − a ) 1 − a , 0 < a < 1 , \mathbb{P}(S_n>an)^{1/n}\to\frac{1}{\sqrt{(1+a)^{1+a}(1-a)^{1-a}}}, \qquad0<a<1, P ( S n > an ) 1/ n → ( 1 + a ) 1 + a ( 1 − a ) 1 − a 1 , 0 < a < 1 , as n → ∞ n\to\infty n → ∞ . To see this, we have from (26) that
P ( S n > n a ) 1 / n → e − Λ X ∗ ( a ) \mathbb{P}(S_n>na)^{1/n}\to e^{-\Lambda_X^*(a)} P ( S n > na ) 1/ n → e − Λ X ∗ ( a ) We must compute Λ X ∗ ( a ) \Lambda_X^*(a) Λ X ∗ ( a ) . First, we have
Λ X ( t ) = log E e t X = log ( 1 2 e − t + 1 2 e t ) = log cosh t . \Lambda_X(t)=\log\mathbb{E}e^{tX}=\log\left(\tfrac{1}{2}e^{-t}+\tfrac{1}{2}e^t\right)=\log\cosh t. Λ X ( t ) = log E e tX = log ( 2 1 e − t + 2 1 e t ) = log cosh t . Next, to find t ∗ : = argmax { a t − Λ X ( t ) } t^*:=\operatorname{argmax}\{at-\Lambda_X(t)\} t ∗ := argmax { a t − Λ X ( t )} we compute
0 = ∂ t [ a t − Λ X ( t ) ] ∣ t = t ∗ = a − tanh t ∗ , ⇒ t ∗ = tanh − 1 a . 0=\left.\partial_t\left[at-\Lambda_X(t)\right]\right|_{t=t^*}=a-\tanh t^*,\qquad\Rightarrow\qquad t^*=\tanh^{-1}a. 0 = ∂ t [ a t − Λ X ( t ) ] ∣ t = t ∗ = a − tanh t ∗ , ⇒ t ∗ = tanh − 1 a . Therefore, we have
Λ X ∗ ( a ) = sup t { a t − Λ X ( t ) } = a t ∗ − Λ X ( t ∗ ) = a tanh − 1 a − log cosh tanh − 1 a . \Lambda_X^*(a)=\sup_t\{at-\Lambda_X(t)\}=at^*-\Lambda_X(t^*)=a\tanh^{-1}a-\log\cosh\tanh^{-1}a. Λ X ∗ ( a ) = t sup { a t − Λ X ( t )} = a t ∗ − Λ X ( t ∗ ) = a tanh − 1 a − log cosh tanh − 1 a . Lastly, using 2 tanh − 1 a = log ( 1 + a ) − log ( 1 − a ) 2\tanh^{-1}a=\log(1+a)-\log(1-a) 2 tanh − 1 a = log ( 1 + a ) − log ( 1 − a ) , we have
e − Λ X ∗ ( a ) = e − a tanh − 1 a + log cosh tanh − 1 a = e − a tanh − 1 a 1 − a 2 e^{-\Lambda_X^*(a)}=e^{-a\tanh^{-1}a+\log\cosh\tanh^{-1}a}=\frac{e^{-a\tanh^{-1}a}}{\sqrt{1-a^2}} e − Λ X ∗ ( a ) = e − a t a n h − 1 a + l o g c o s h t a n h − 1 a = 1 − a 2 e − a t a n h − 1 a = ( 1 + a ) − a ( 1 − a ) a ( 1 + a ) ( 1 − a ) = 1 ( 1 + a ) 1 + a ( 1 − a ) 1 − a , =\frac{\sqrt{(1+a)^{-a}(1-a)^a}}{\sqrt{(1+a)(1-a)}}=\frac{1}{\sqrt{(1+a)^{1+a}(1-a)^{1-a}}}, = ( 1 + a ) ( 1 − a ) ( 1 + a ) − a ( 1 − a ) a = ( 1 + a ) 1 + a ( 1 − a ) 1 − a 1 , as claimed. Note that, if a ∉ ( 0 , 1 ) a\notin(0,1) a ∈ / ( 0 , 1 ) , then we have P ( S n > n a ) = 0 \mathbb{P}(S_n>na)=0 P ( S n > na ) = 0 for all n n n .
Exercises
Exercise 3.1. Let X ∼ Bin ( n , U ) X\sim\operatorname{Bin}(n,U) X ∼ Bin ( n , U ) where U ∼ U ( ( 0 , 1 ) ) U\sim\mathcal{U}((0,1)) U ∼ U (( 0 , 1 )) . What is the probability Generating function G X ( s ) G_X(s) G X ( s ) of $X? What is P ( X = k ) \mathbb{P}(X=k) P ( X = k ) where k ∈ { 0 , 1 , 2 , … , n } k\in\{0,1,2,\ldots,n\} k ∈ { 0 , 1 , 2 , … , n } ?
Exercise 3.2. Let Z n Z_n Z n be the size of the n n n th generation in an ordinary branching process with Z 0 = 1 Z_0=1 Z 0 = 1 , E Z 1 = μ \mathbb{E}Z_1=\mu E Z 1 = μ and V Z 1 > 0 \mathop{\mathrm{V}}Z_1>0 V Z 1 > 0 . Show that E Z n Z m = μ n − m E Z m 2 \mathbb{E}Z_nZ_m=\mu^{n-m}\mathbb{E}Z_m^2 E Z n Z m = μ n − m E Z m 2 for m ≤ n m\le n m ≤ n . Use this to find the correlation coefficient ρ ( Z m , Z n ) \rho(Z_m,Z_n) ρ ( Z m , Z n ) in terms of μ \mu μ , n n n , and m m m . Consider the case μ = 1 \mu=1 μ = 1 and the case μ ≠ 1 \mu\ne1 μ = 1 .
Exercise 3.3. Consider a branching process with generation sizes Z n Z_n Z n satisfying Z 0 = 1 Z_0=1 Z 0 = 1 and P ( Z 1 = 0 ) = 0 \mathbb{P}(Z_1=0)=0 P ( Z 1 = 0 ) = 0 . Pick two individuals at random with replacement from the n n n th generation and let L L L be the index of the generation which contains their most recent common ancestor. Show that P ( L = r ) = E Z r − 1 − E Z r + 1 − 1 \mathbb{P}(L=r)=\mathbb{E}Z_r^{-1}-\mathbb{E}Z_{r+1}^{-1} P ( L = r ) = E Z r − 1 − E Z r + 1 − 1 for 0 ≤ r < n 0\le r<n 0 ≤ r < n .
Exercise 3.4. Consider a branching process with immigration
Z 0 = 1 , Z n + 1 = ∑ i = 1 Z n X n , i + Y n . Z_0=1,\qquad Z_{n+1}=\sum_{i=1}^{Z_n}X_{n,i}+Y_n. Z 0 = 1 , Z n + 1 = i = 1 ∑ Z n X n , i + Y n .
where the ( X n , i ) (X_{n,i}) ( X n , i ) are iid with common distribution X X X , the ( Y n ) (Y_n) ( Y n ) are iid with common distribution Y Y Y and the ( X n , i ) (X_{n,i}) ( X n , i ) and ( Y n ) (Y_n) ( Y n ) are independent. What is G Z n + 1 ( s ) G_{Z_{n+1}}(s) G Z n + 1 ( s ) in terms of G Z n ( s ) G_{Z_n}(s) G Z n ( s ) , G X ( s ) G_X(s) G X ( s ) and G Y ( s ) G_Y(s) G Y ( s ) ? Write G Z 2 ( s ) G_{Z_2}(s) G Z 2 ( s ) explicitly in terms of G X ( s ) G_X(s) G X ( s ) and G Y ( s ) G_Y(s) G Y ( s ) .
Exercise 3.5. Find ϕ X 2 ( t ) : = E e i t X 2 \phi_{X^2}(t):=\mathbb{E}e^{itX^2} ϕ X 2 ( t ) := E e i t X 2 where X ∼ N ( μ , σ 2 ) X\sim\mathcal{N}(\mu,\sigma^2) X ∼ N ( μ , σ 2 ) .
Exercise 3.6. Let X n X_n X n have cumulative distribution function
F X n ( x ) = ( x − sin ( 2 n π x ) 2 n π ) 1 { 0 ≤ x ≤ 1 } + 1 { x > 1 } . F_{X_n}(x)=\left(x-\frac{\sin(2n\pi x)}{2n\pi}\right)\mathbf{1}_{\{0\le x\le1\}}+\mathbf{1}_{\{x>1\}}. F X n ( x ) = ( x − 2 nπ sin ( 2 nπ x ) ) 1 { 0 ≤ x ≤ 1 } + 1 { x > 1 } .
(a) Show that F X n F_{X_n} F X n is a distribution function and find the corresponding density function f X n f_{X_n} f X n . (b) Show that F X n F_{X_n} F X n converges to the uniform distribution function F U F_U F U as n → ∞ n\to\infty n → ∞ , but that the density functions f X n f_{X_n} f X n do not converge to f U f_U f U . Here, U ∼ U ( ( 0 , 1 ) ) U\sim U((0,1)) U ∼ U (( 0 , 1 )) .
Exercise 3.7. A coin is tossed repeatedly, with heads turning up with probability p p p on each toss. Let N N N be the minimum number of tosses required to obtain k k k heads. Show that, as p → 0 p\to0 p → 0 , the distribution function of 2 N p 2Np 2 N p converges to that of a gamma distribution. Note that, if X ∼ Γ ( λ , r ) X\sim\Gamma(\lambda,r) X ∼ Γ ( λ , r ) then
f X ( x ) = 1 Γ ( r ) λ r x r − 1 e − λ x 1 { x ≥ 0 } . f_X(x)=\frac{1}{\Gamma(r)}\lambda^r x^{r-1}e^{-\lambda x}\mathbf{1}_{\{x\ge0\}}. f X ( x ) = Γ ( r ) 1 λ r x r − 1 e − λ x 1 { x ≥ 0 } .
Exercise 3.8. Show that, as n → ∞ n\to\infty n → ∞ , we have
2 − n ∑ k ∈ A n ( n k ) ⟶ ∫ − x x 1 2 π e − y 2 / 2 d y . 2^{-n}\sum_{k\in A_n}\binom{n}{k}\longrightarrow\int_{-x}^{x}\frac{1}{\sqrt{2\pi}}e^{-y^2/2}\,dy. 2 − n k ∈ A n ∑ ( k n ) ⟶ ∫ − x x 2 π 1 e − y 2 /2 d y .
where A n : = { k : ∣ k − n / 2 ∣ ≤ x n / 2 } A_n:=\{k:|k-n/2|\le x\sqrt{n}/2\} A n := { k : ∣ k − n /2∣ ≤ x n /2 } .
Exercise 3.9. Let ( X j ) j ∈ N (X_j)_{j\in\mathbb{N}} ( X j ) j ∈ N be a sequence of i.i.d standard normal N ( 0 , 1 ) N(0,1) N ( 0 , 1 ) random variables. Define
Y n : = 1 n ∑ j = 1 n j X j , n ∈ N . Y_n:=\frac{1}{n}\sum_{j=1}^{n}\sqrt{j}X_j,\qquad n\in\mathbb{N}. Y n := n 1 j = 1 ∑ n j X j , n ∈ N .
Show that Y n → D Y Y_n\xrightarrow{\mathcal{D}}Y Y n D Y and identify the distribution of Y Y Y . Hint: recall that if
Z ∼ N ( μ , σ 2 ) , then E e i u Z = e i u μ − 1 2 u 2 σ 2 . Z\sim\mathcal{N}(\mu,\sigma^2),\qquad\text{then}\qquad\mathbb{E}e^{iuZ}=e^{iu\mu-\frac{1}{2}u^2\sigma^2}. Z ∼ N ( μ , σ 2 ) , then E e i u Z = e i uμ − 2 1 u 2 σ 2 .
Exercises
Exercise 3.10. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Recall that, if Z Z Z is a Poisson random variable with parameter λ > 0 \lambda> 0 λ > 0 and X X X is a Bernoulli random variable with parameter p ∈ ( 0 , 1 ) p \in(0,1) p ∈ ( 0 , 1 ) then their probability mass functions are given by
f Z ( k ) = λ k e − λ k ! , k ∈ { 0 , 1 , 2 , … } , f X ( k ) = { p , k = 1 , 1 − p , k = 0. f_Z(k)=\frac{\lambda^k e^{-\lambda}}{k!}, \qquad k \in\{0,1,2,\ldots\}, \qquad f_X(k)=\begin{cases}p, & k=1,\\ 1-p, & k=0.\end{cases} f Z ( k ) = k ! λ k e − λ , k ∈ { 0 , 1 , 2 , … } , f X ( k ) = { p , 1 − p , k = 1 , k = 0. (a) Compute the characteristic functions φ Z ( t ) : = E e i t Z \varphi_Z(t):=\mathbb{E}e^{itZ} φ Z ( t ) := E e i tZ and φ X ( t ) : = E e i t X \varphi_X(t):=\mathbb{E}e^{itX} φ X ( t ) := E e i tX .
(b) For every n ∈ N n \in\mathbb{N} n ∈ N , let ( X j ( n ) ) 1 ≤ j ≤ n (X_j^{(n)})_{1\leq j\leq n} ( X j ( n ) ) 1 ≤ j ≤ n be a sequence of i.i.d. Bernoulli random variables with parameter λ / n \lambda/n λ / n . Define
S n : = ∑ j = 1 n X j ( n ) . S_n:=\sum_{j=1}^{n}X_j^{(n)}. S n := j = 1 ∑ n X j ( n ) . Show that S n → D Z S_n \xrightarrow{\mathcal{D}} Z S n D Z . The following may be useful
e x = ∑ n = 0 ∞ x n n ! , e^x=\sum_{n=0}^{\infty}\frac{x^n}{n!}, e x = n = 0 ∑ ∞ n ! x n , lim n → ∞ ( 1 + x n ) n = e x . \lim_{n\to\infty}\left(1+\frac{x}{n}\right)^n=e^x. n → ∞ lim ( 1 + n x ) n = e x .
Discrete time Markov chains
The notes from this chapter are primarily taken from (Grimmett and Stirzaker, 2001, Chapter 6).
Overview of discrete time Markov chains
We have already defined what we mean by Markov process in Definition 2.3.6. In this chapter, we will derive some results that are specific for Markov discrete time processes that have a countable state space.
Definition 4.1.1. A discrete time Markov chain X = ( X n ) n ∈ N 0 X=(X_n)_{n\in\mathbb{N}_0} X = ( X n ) n ∈ N 0 is a discrete-time Markov process with a countable state space S S S .
Recall from (13) that a Markov process defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) , which is equipped with a filtration F = ( F n ) n ∈ N 0 \mathbb{F}=(\mathcal{F}_n)_{n\in\mathbb{N}_0} F = ( F n ) n ∈ N 0 , is a process X = ( X n ) n ∈ N 0 X=(X_n)_{n\in\mathbb{N}_0} X = ( X n ) n ∈ N 0 that satisfies P ( X n + k ∈ A ∣ F n ) = P ( X n + k ∈ A ∣ X n ) \mathbb{P}(X_{n+k}\in A\mid\mathcal{F}_n)=\mathbb{P}(X_{n+k}\in A\mid X_n) P ( X n + k ∈ A ∣ F n ) = P ( X n + k ∈ A ∣ X n ) . What the Markov property means in practice is that the evolution of a Markov chain is described by its one-step transition probabilities P ( X n + 1 = j ∣ X n = i ) \mathbb{P}(X_{n+1}=j\mid X_n=i) P ( X n + 1 = j ∣ X n = i ) . In general, such probabilities may depend on the time n n n . Throughout this Chapter, we will restrict our attention to Markov chains whose transition probabilities do not depend on n n n .
Definition 4.1.2. A discrete time Markov chain X X X is called homogeneous if
P ( X n + 1 = j ∣ X n = i ) = P ( X 1 = j ∣ X 0 = i ) = : p ( i , j ) ∀ n ∈ N 0 , ∀ i , j ∈ S . \mathbb{P}(X_{n+1}=j\mid X_n=i)=\mathbb{P}(X_1=j\mid X_0=i)=:p(i,j)\qquad\forall n\in\mathbb{N}_0,\ \forall i,j\in S. P ( X n + 1 = j ∣ X n = i ) = P ( X 1 = j ∣ X 0 = i ) =: p ( i , j ) ∀ n ∈ N 0 , ∀ i , j ∈ S . We call the ∣ S ∣ × ∣ S ∣ |S|\times|S| ∣ S ∣ × ∣ S ∣ matrix P = ( p ( i , j ) ) \mathbf{P}=(p(i,j)) P = ( p ( i , j )) the one-step transition matrix .
The one-step transition matrix P \mathbf{P} P satisfies some obvious properties:
∑ j p ( i , j ) = 1 \sum_j p(i,j)=1 ∑ j p ( i , j ) = 1 .
p ( i , j ) ≥ 0 p(i,j) \ge0 p ( i , j ) ≥ 0 for all i , j ∈ S i,j \in S i , j ∈ S .
Since P ( X n + 1 = j ∣ X n = i ) = P ( X 1 = j ∣ X 0 = i ) \mathbb{P}(X_{n+1}=j\mid X_n=i)=\mathbb{P}(X_1=j\mid X_0=i) P ( X n + 1 = j ∣ X n = i ) = P ( X 1 = j ∣ X 0 = i ) , it follows that P ( X n + m = j ∣ X m = i ) = P ( X n = j ∣ X 0 = i ) \mathbb{P}(X_{n+m}=j\mid X_m=i)=\mathbb{P}(X_n=j\mid X_0=i) P ( X n + m = j ∣ X m = i ) = P ( X n = j ∣ X 0 = i ) .
Definition 4.1.3. Let X X X be a discrete time Markov chain and let
p n ( i , j ) : = P ( X n = j ∣ X 0 = i ) ∀ n ∈ N 0 , ∀ i , j ∈ S . p_n(i,j):=\mathbb{P}(X_n=j\mid X_0=i) \qquad\forall n\in\mathbb{N}_0,\ \forall i,j\in S. p n ( i , j ) := P ( X n = j ∣ X 0 = i ) ∀ n ∈ N 0 , ∀ i , j ∈ S . We call the ∣ S ∣ × ∣ S ∣ |S|\times|S| ∣ S ∣ × ∣ S ∣ matrix P n = ( p n ( i , j ) ) \mathbf{P}_n=(p_n(i,j)) P n = ( p n ( i , j )) the n n n -step transition matrix.
From the one-step transition matrix P \mathbf{P} P , one can easily derive the n n n -step transition matrix P n \mathbf{P}_n P n .
Theorem 4.1.4 (Chapman-Kolmogorov Equation). Let P \mathbf{P} P and P n \mathbf{P}_n P n be the one-step and n n n -step transition matrices of a homogeneous discrete-time Markov chain. Then
P m + n = P m P n , P n = P n \mathbf{P}_{m+n}=\mathbf{P}_m\mathbf{P}_n,\qquad\mathbf{P}_n=\mathbf{P}^n P m + n = P m P n , P n = P n where P n \mathbf{P}^n P n is the n n n th power of P \mathbf{P} P .
Proof. For any i , j ∈ S i,j\in S i , j ∈ S we have
p m + n ( i , j ) = P ( X m + n = j ∣ X 0 = i ) = ∑ k P ( X n + m = j ∣ X m = k ) P ( X m = k ∣ X 0 = i ) = ∑ k P ( X n = j ∣ X 0 = k ) P ( X m = k ∣ X 0 = i ) = ∑ k p m ( i , k ) p n ( k , j ) . \begin{aligned}
p_{m+n}(i,j)&=\mathbb{P}(X_{m+n}=j\mid X_0=i)\\
&=\sum_k\mathbb{P}(X_{n+m}=j\mid X_m=k)\mathbb{P}(X_m=k\mid X_0=i)\\
&=\sum_k\mathbb{P}(X_n=j\mid X_0=k)\mathbb{P}(X_m=k\mid X_0=i)
=\sum_k p_m(i,k)p_n(k,j).
\end{aligned} p m + n ( i , j ) = P ( X m + n = j ∣ X 0 = i ) = k ∑ P ( X n + m = j ∣ X m = k ) P ( X m = k ∣ X 0 = i ) = k ∑ P ( X n = j ∣ X 0 = k ) P ( X m = k ∣ X 0 = i ) = k ∑ p m ( i , k ) p n ( k , j ) .
Thus, we have established that P m + n = P m P n \mathbf{P}_{m+n}=\mathbf{P}_m\mathbf{P}_n P m + n = P m P n . Next, note that
P n = P n − 1 P = P n − 2 P 2 = … = P n , \mathbf{P}_n=\mathbf{P}_{n-1}\mathbf{P}=\mathbf{P}_{n-2}\mathbf{P}^2=\ldots=\mathbf{P}^n, P n = P n − 1 P = P n − 2 P 2 = … = P n ,
as claimed.
Lemma 4.1.5. Let X X X be a homogeneous discrete time Markov chain. Denote the probability mass function of X n X_n X n by
μ n ( i ) : = P ( X n = i ) , μ n = ( μ n ( 1 ) , μ n ( 2 ) , … , μ n ( ∣ S ∣ ) ) . \mu_n(i):=\mathbb{P}(X_n=i),\qquad\boldsymbol{\mu}_n=(\mu_n(1),\mu_n(2),\ldots,\mu_n(|S|)). μ n ( i ) := P ( X n = i ) , μ n = ( μ n ( 1 ) , μ n ( 2 ) , … , μ n ( ∣ S ∣ )) . Then μ n + m = μ n P m \boldsymbol{\mu}_{n+m}=\boldsymbol{\mu}_n\mathbf{P}_m μ n + m = μ n P m and thus μ m = μ 0 P m \boldsymbol{\mu}_m=\boldsymbol{\mu}_0\mathbf{P}^m μ m = μ 0 P m .
Proof. For any j ∈ S j\in S j ∈ S we have
μ n + m ( j ) = P ( X n + m = j ) = ∑ i P ( X n + m = j ∣ X n = i ) P ( X n = i ) = ∑ i μ n ( i ) p m ( i , j ) . \mu_{n+m}(j)=\mathbb{P}(X_{n+m}=j)=\sum_i\mathbb{P}(X_{n+m}=j\mid X_n=i)\mathbb{P}(X_n=i)=\sum_i\mu_n(i)p_m(i,j). μ n + m ( j ) = P ( X n + m = j ) = i ∑ P ( X n + m = j ∣ X n = i ) P ( X n = i ) = i ∑ μ n ( i ) p m ( i , j ) .
Thus, we have established that μ n + m = μ n P m \boldsymbol{\mu}_{n+m}=\boldsymbol{\mu}_n\mathbf{P}_m μ n + m = μ n P m . To finish the proof, take n = 0 n=0 n = 0 and use P m = P m \mathbf{P}_m=\mathbf{P}^m P m = P m .
Example 4.1.6. Consider a random walk on a circle with n n n nodes. At each step a particle moves clockwise with probability p p p and counter-clockwise with probability q q q . Let us write the one-step transition matrix. We have
P = ( 0 p 0 0 ⋯ 0 0 q q 0 p 0 ⋯ 0 0 0 0 q 0 p ⋯ 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ 0 0 0 0 ⋯ q 0 p p 0 0 0 ⋯ 0 q 0 ) . \mathbf{P} =
\begin{pmatrix}
0 & p & 0 & 0 & \cdots& 0 & 0 & q \\
q & 0 & p & 0 & \cdots& 0 & 0 & 0 \\
0 & q & 0 & p & \cdots& 0 & 0 & 0 \\
\vdots& \vdots& \vdots& \vdots& \ddots& \vdots& \vdots& \vdots\\
0 & 0 & 0 & 0 & \cdots& q & 0 & p \\
p & 0 & 0 & 0 & \cdots& 0 & q & 0
\end{pmatrix}. P = 0 q 0 ⋮ 0 p p 0 q ⋮ 0 0 0 p 0 ⋮ 0 0 0 0 p ⋮ 0 0 ⋯ ⋯ ⋯ ⋱ ⋯ ⋯ 0 0 0 ⋮ q 0 0 0 0 ⋮ 0 q q 0 0 ⋮ p 0 .
Example 4.1.7. Consider a random walk on N 0 \mathbb{N}_0 N 0 . At each step a particle moves to the up one unit with probability p p p and returns to the origin with probability q q q . Let us write the one-step transition matrix. We have
P = ( q p 0 0 0 ⋯ q 0 p 0 0 ⋯ q 0 0 p 0 ⋯ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱ ) . \mathbf{P} =
\begin{pmatrix}
q & p & 0 & 0 & 0 & \cdots\\
q & 0 & p & 0 & 0 & \cdots\\
q & 0 & 0 & p & 0 & \cdots\\
\vdots& \vdots& \vdots& \vdots& \vdots& \ddots
\end{pmatrix}. P = q q q ⋮ p 0 0 ⋮ 0 p 0 ⋮ 0 0 p ⋮ 0 0 0 ⋮ ⋯ ⋯ ⋯ ⋱ .
Classification of states
Definition 4.2.1. Let X be a Markov chain. We say that state i i i is persistent or recurrent if
P ( X n = i for some n ∣ X 0 = i ) = 1. \mathbb{P}(X_n = i \text{ for some } n \mid X_0 = i) = 1. P ( X n = i for some n ∣ X 0 = i ) = 1. Otherwise, we say that state i i i is transient .
If a Markov chain visits a persistent state once, then it is guaranteed to return to that state. In fact, it is guaranteed to return to that state infinitely often .
We would like to find conditions on the transition matrix P \mathbf{P} P that enable us to classify a state as either persistent or transient.
Definition 4.2.2. Let X be a Markov chain. We define the first passage to state j j j by
τ j : = inf { n ≥ 1 : X n = j } \tau_j := \inf\{n \ge1 : X_n = j\} τ j := inf { n ≥ 1 : X n = j } and we denote by f i j f_{ij} f ij the probability mass function of τ j \tau_j τ j given X 0 = i X_0 = i X 0 = i . That is
f i j ( n ) = P ( τ j = n ∣ X 0 = i ) . f_{ij}(n) = \mathbb{P}(\tau_j = n \mid X_0 = i). f ij ( n ) = P ( τ j = n ∣ X 0 = i ) .
Theorem 4.2.3. Define the following generating functions
P i j ( s ) : = ∑ n = 0 ∞ s n p n ( i , j ) , p 0 ( i , j ) : = δ i , j , F i j ( s ) : = ∑ n = 0 ∞ s n f i j ( n ) , f i j ( 0 ) : = 0. P_{ij}(s) := \sum_{n=0}^{\infty} s^n p_n(i,j), \qquad p_0(i,j) := \delta_{i,j}, \qquad F_{ij}(s) := \sum_{n=0}^{\infty} s^n f_{ij}(n), \qquad f_{ij}(0) := 0. P ij ( s ) := n = 0 ∑ ∞ s n p n ( i , j ) , p 0 ( i , j ) := δ i , j , F ij ( s ) := n = 0 ∑ ∞ s n f ij ( n ) , f ij ( 0 ) := 0. Then, for any ∣ s ∣ < 1 |s| < 1 ∣ s ∣ < 1 , we have
P i i ( s ) = 1 + F i i ( s ) P i i ( s ) , P i j ( s ) = F i j ( s ) P j j ( s ) , i ≠ j . (34) P_{ii}(s) = 1 + F_{ii}(s)P_{ii}(s), \qquad P_{ij}(s) = F_{ij}(s)P_{jj}(s), \qquad i \ne j. \tag*{(34)} P ii ( s ) = 1 + F ii ( s ) P ii ( s ) , P ij ( s ) = F ij ( s ) P j j ( s ) , i = j . (34)
Note that we must restrict ∣ s ∣ < 1 |s| < 1 ∣ s ∣ < 1 because P is not a probability generating function. When we need s = 1 s = 1 s = 1 we can take a limit as s ↗ 1 s \nearrow1 s ↗ 1 and use Abel's Theorem . which states that
lim s ↗ 1 P i j ( s ) = ∑ n = 0 ∞ p n ( i , j ) . (35) \lim_{s \nearrow1} P_{ij}(s) = \sum_{n=0}^{\infty} p_n(i,j). \tag*{(35)} s ↗ 1 lim P ij ( s ) = n = 0 ∑ ∞ p n ( i , j ) . (35)
Proof. Fix i , j ∈ S i,j \in S i , j ∈ S and define
A m = { X m = j } , B m = { τ j = m } . A_m = \{X_m = j\}, \qquad B_m = \{\tau_j = m\}. A m = { X m = j } , B m = { τ j = m } .
Since the sets ( B m ) m ≥ 1 (B_m)_{m \ge1} ( B m ) m ≥ 1 are disjoint we have
p m ( i , j ) = P ( A m ∣ X 0 = i ) = ∑ k = 1 m P ( A m ∩ B k ∣ X 0 = i ) . (36) p_m(i,j) = \mathbb{P}(A_m \mid X_0 = i) = \sum_{k=1}^{m} \mathbb{P}(A_m \cap B_k \mid X_0 = i). \tag*{(36)} p m ( i , j ) = P ( A m ∣ X 0 = i ) = k = 1 ∑ m P ( A m ∩ B k ∣ X 0 = i ) . (36)
Note that
P ( A m ∩ B k ∣ X 0 = i ) = P ( A m ∣ B k , X 0 = i ) ⋅ P ( B k ∣ X 0 = i ) = P ( A m ∣ X k = j ) ⋅ P ( B k ∣ X 0 = i ) = p m − k ( j , j ) ⋅ f i j ( k ) . (37) \begin{aligned}
\mathbb{P}(A_m \cap B_k \mid X_0 = i) &= \mathbb{P}(A_m \mid B_k, X_0 = i) \cdot\mathbb{P}(B_k \mid X_0 = i) \\
&= \mathbb{P}(A_m \mid X_k = j) \cdot\mathbb{P}(B_k \mid X_0 = i) = p_{m-k}(j,j) \cdot f_{ij}(k). \tag*{(37)}
\end{aligned} P ( A m ∩ B k ∣ X 0 = i ) = P ( A m ∣ B k , X 0 = i ) ⋅ P ( B k ∣ X 0 = i ) = P ( A m ∣ X k = j ) ⋅ P ( B k ∣ X 0 = i ) = p m − k ( j , j ) ⋅ f ij ( k ) . (37)
Inserting (37) into (36) we obtain
p m ( i , j ) = ∑ k = 1 m f i j ( k ) p m − k ( j , j ) . p_m(i,j) = \sum_{k=1}^{m} f_{ij}(k)p_{m-k}(j,j). p m ( i , j ) = k = 1 ∑ m f ij ( k ) p m − k ( j , j ) .
Thus
P i j ( s ) = ∑ m = 0 ∞ s m p m ( i , j ) = δ i j + ∑ m = 1 ∞ s m ∑ k = 1 m f i j ( k ) p m − k ( j , j ) = δ i j + ∑ m = 1 ∞ ∑ k = 1 m s k f i j ( k ) s m − k p m − k ( j , j ) = δ i j + ∑ k = 1 ∞ s k f i j ( k ) ⋅ ∑ m = k ∞ s m − k p m − k ( j , j ) = δ i j + F i j ( s ) P j j ( s ) , \begin{aligned}
P_{ij}(s) &= \sum_{m=0}^{\infty} s^m p_m(i,j) \\
&= \delta_{ij} + \sum_{m=1}^{\infty} s^m \sum_{k=1}^{m} f_{ij}(k)p_{m-k}(j,j) \\
&= \delta_{ij} + \sum_{m=1}^{\infty} \sum_{k=1}^{m} s^k f_{ij}(k)s^{m-k}p_{m-k}(j,j) \\
&= \delta_{ij} + \sum_{k=1}^{\infty} s^k f_{ij}(k) \cdot\sum_{m=k}^{\infty} s^{m-k}p_{m-k}(j,j) \\
&= \delta_{ij} + F_{ij}(s)P_{jj}(s),
\end{aligned} P ij ( s ) = m = 0 ∑ ∞ s m p m ( i , j ) = δ ij + m = 1 ∑ ∞ s m k = 1 ∑ m f ij ( k ) p m − k ( j , j ) = δ ij + m = 1 ∑ ∞ k = 1 ∑ m s k f ij ( k ) s m − k p m − k ( j , j ) = δ ij + k = 1 ∑ ∞ s k f ij ( k ) ⋅ m = k ∑ ∞ s m − k p m − k ( j , j ) = δ ij + F ij ( s ) P j j ( s ) ,
which is precisely the claimed result. ∎
Corollary 4.2.4. The following holds:
State j j j is persistent if ∑ n p n ( j , j ) = ∞ \sum_n p_n(j,j)=\infty ∑ n p n ( j , j ) = ∞ , and if this holds then ∑ n p n ( i , j ) = ∞ \sum_n p_n(i,j)=\infty ∑ n p n ( i , j ) = ∞ for all i i i such that F i j ( 1 ) > 0 F_{ij}(1)>0 F ij ( 1 ) > 0 .
State j j j is transient if ∑ n p n ( j , j ) < ∞ \sum_n p_n(j,j)<\infty ∑ n p n ( j , j ) < ∞ , and if this holds then ∑ n p n ( i , j ) < ∞ \sum_n p_n(i,j)<\infty ∑ n p n ( i , j ) < ∞ for all i i i .
Proof. We will show that j j j is persistent if and only if ∑ n p n ( j , j ) = ∞ \sum_n p_n(j,j)=\infty ∑ n p n ( j , j ) = ∞ . First, by Definition 4.2.1 , we have
j is persistent ⟺ F j j ( 1 ) = ∑ n = 1 ∞ f j j ( n ) = 1. j\text{ is persistent}\quad\Longleftrightarrow\quad F_{jj}(1)=\sum_{n=1}^{\infty}f_{jj}(n)=1. j is persistent ⟺ F j j ( 1 ) = n = 1 ∑ ∞ f j j ( n ) = 1.
Next, from (34) and Abel’s Theorem (4.2), we have
∑ n p n ( j , j ) = lim s ↗ 1 P j j ( s ) = lim s ↗ 1 1 1 − F j j ( s ) = 1 1 − F j j ( 1 ) . \sum_n p_n(j,j)=\lim_{s\nearrow1}P_{jj}(s)=\lim_{s\nearrow1}\frac{1}{1-F_{jj}(s)}=\frac{1}{1-F_{jj}(1)}. n ∑ p n ( j , j ) = s ↗ 1 lim P j j ( s ) = s ↗ 1 lim 1 − F j j ( s ) 1 = 1 − F j j ( 1 ) 1 .
Thus, j j j is persistent if and only if ∑ n p n ( j , j ) = ∞ \sum_n p_n(j,j)=\infty ∑ n p n ( j , j ) = ∞ . We also have from (34) and Abel’s Theorem (4.2) that
∑ n p n ( j , j ) = lim s ↗ 1 P j j ( s ) = lim s ↗ 1 P i j ( s ) F i j ( 1 ) = ∑ n p n ( i , j ) F i j ( 1 ) . \sum_n p_n(j,j)=\lim_{s\nearrow1}P_{jj}(s)=\frac{\lim_{s\nearrow1}P_{ij}(s)}{F_{ij}(1)}=\frac{\sum_n p_n(i,j)}{F_{ij}(1)}. n ∑ p n ( j , j ) = s ↗ 1 lim P j j ( s ) = F ij ( 1 ) lim s ↗ 1 P ij ( s ) = F ij ( 1 ) ∑ n p n ( i , j ) .
Thus, if j j j is persistent then ∑ n p n ( i , j ) = ∞ \sum_n p_n(i,j)=\infty ∑ n p n ( i , j ) = ∞ whenever F i j ( 1 ) > 0 F_{ij}(1)>0 F ij ( 1 ) > 0 .
Corollary 4.2.5. If j j j is transient then p n ( i , j ) → 0 p_n(i,j)\to0 p n ( i , j ) → 0 as n → ∞ n\to\infty n → ∞ for all i i i .
Proof. From Corollary 4.2.4, if j j j is transient, we have ∑ n p n ( i , j ) < ∞ \sum_n p_n(i,j)<\infty ∑ n p n ( i , j ) < ∞ for all i i i . It follows that p n ( i , j ) → 0 p_n(i,j)\to0 p n ( i , j ) → 0 as n → ∞ n\to\infty n → ∞ for all i i i .
There are a few other state classifications that are important
Definition 4.2.6. The mean recurrence time of a state j j j is defined as
τ ˉ j = E [ τ j ∣ X 0 = j ] , \bar{\tau}_j=\mathbb{E}[\tau_j\mid X_0=j], τ ˉ j = E [ τ j ∣ X 0 = j ] , where τ j \tau_j τ j is defined in Definition 4.2.2 .
Clearly, τ ˉ j = ∞ \bar{\tau}_j=\infty τ ˉ j = ∞ if a state is transient. However, τ ˉ j \bar{\tau}_j τ ˉ j may be infinite even if j j j is recurrent.
Definition 4.2.7. A recurrent state j j j is said to be null if τ ˉ j = ∞ \bar{\tau}_j=\infty τ ˉ j = ∞ and non-null or positive if τ ˉ j < ∞ \bar{\tau}_j<\infty τ ˉ j < ∞ .
This is a simple condition which differentiates null recurrent from positive recurrent states.
Theorem 4.2.8. A persistent state j j j is null if and only if p n ( j , j ) → 0 p_n(j,j)\to0 p n ( j , j ) → 0 as n → ∞ n\to\infty n → ∞ ; if this holds then p n ( i , j ) → 0 p_n(i,j)\to0 p n ( i , j ) → 0 as n → ∞ n\to\infty n → ∞ for all $i.
We will not prove of Theorem 4.2.8 .
Definition 4.2.9. The period of a state i i i , denoted d ( i ) d(i) d ( i ) , is defined by d ( i ) = gcd { n : p n ( i , i ) > 0 } d(i) = \gcd\{n : p_n(i,i) > 0\} d ( i ) = g cd{ n : p n ( i , i ) > 0 } . If d ( i ) = 1 d(i) = 1 d ( i ) = 1 we say that state i i i is aperiodic . Here gcd means “greatest common divisor.”
Example 4.2.10. Let X be a Markov Chain with one-step transition matrix
P = ( 0 1 0 0 0 1 1 0 0 ) . \mathbf{P} =
\begin{pmatrix}
0 & 1 & 0 \\
0 & 0 & 1 \\
1 & 0 & 0
\end{pmatrix}. P = 0 0 1 1 0 0 0 1 0 . One can easily see that
p n ( i , i ) = { 1 n = 3 , 6 , 9 , … 0 n ≠ 3 , 6 , 9 , … p_n(i,i) =
\begin{cases}
1 & n = 3,6,9,\ldots\\
0 & n \ne3,6,9,\ldots
\end{cases} p n ( i , i ) = { 1 0 n = 3 , 6 , 9 , … n = 3 , 6 , 9 , … Since gcd { 3 , 6 , 9 , 12 , … } = 3 \gcd\{3,6,9,12,\ldots\} = 3 g cd{ 3 , 6 , 9 , 12 , … } = 3 , we have d ( i ) = 3 d(i) = 3 d ( i ) = 3 for all i i i .
Definition 4.2.11. A state is called ergodic if it is persistent, non-null, and aperiodic.
Classification of chains
Definition 4.3.1. We say state i i i communicates with state j j j , written i → j i \to j i → j , if p n ( i , j ) > 0 p_n(i,j) > 0 p n ( i , j ) > 0 for some n ≥ 1 n \ge1 n ≥ 1 (in other words, it is possible to reach state j j j from state i i i ). We say that states i i i and j j j intercommunicate , written i ↔ j i \leftrightarrow j i ↔ j , if i → j i \to j i → j and j → i j \to i j → i .
Theorem 4.3.2. Suppose i ↔ j i \leftrightarrow j i ↔ j . Then
i i i is transient if an only if j j j is transient;
i i i is null persistent if and only if j j j is null persistent.
i i i and j j j have the same period: d ( i ) = d ( j ) d(i) = d(j) d ( i ) = d ( j ) .
Proof. We will prove Item 1. If i ↔ j i \leftrightarrow j i ↔ j then there exists k , n k,n k , n such that α : = p k ( i , j ) p n ( j , i ) > 0 \alpha:= p_k(i,j)p_n(j,i) > 0 α := p k ( i , j ) p n ( j , i ) > 0 . Therefore, for any m m m , we have
p k + m + n ( i , i ) ≥ p k ( i , j ) p m ( j , j ) p n ( j , i ) = α p m ( j , j ) . p_{k+m+n}(i,i) \ge p_k(i,j)p_m(j,j)p_n(j,i) = \alpha p_m(j,j). p k + m + n ( i , i ) ≥ p k ( i , j ) p m ( j , j ) p n ( j , i ) = α p m ( j , j ) .
Therefore, if i i i is transient. by Corollary 4.2.4, we have
∞ > ∑ m p m ( i , i ) ≥ ∑ m p k + m + n ( i , i ) ≥ α ∑ m p m ( j , j ) , \infty> \sum_m p_m(i,i) \ge\sum_m p_{k+m+n}(i,i) \ge\alpha\sum_m p_m(j,j), ∞ > m ∑ p m ( i , i ) ≥ m ∑ p k + m + n ( i , i ) ≥ α m ∑ p m ( j , j ) ,
and thus j j j is transient is well. To show the converse simply switch i i i and j j j in the above argument. ∎
Definition 4.3.3. Let C ⊂ S C \subset S C ⊂ S . We say that C C C is closed if p ( i , j ) = 0 p(i,j)=0 p ( i , j ) = 0 for all i ∈ C i \in C i ∈ C and j ∉ C j \notin C j ∈ / C . We say that C C C is irreducible if i ↔ j i \leftrightarrow j i ↔ j for all i , j ∈ C i,j \in C i , j ∈ C .
If a closed set C C C consists of a single state (e.g., C = { j } C=\{j\} C = { j } ), then we call this state absorbing .
Theorem 4.3.4 (Markov chain decomposition). The state space S S S of a Markov chain can be uniquely partitioned as follows
S = T ∪ C 1 ∪ C 2 ∪ ⋯ , S = T \cup C_1 \cup C_2 \cup\cdots, S = T ∪ C 1 ∪ C 2 ∪ ⋯ , where T T T is the set of transient states and C 1 , C 2 , … C_1,C_2,\ldots C 1 , C 2 , … are closed, irreducible sets of persistent states.
The importance of the decomposition Theorem is as follows. Suppose X 0 ∈ C k X_0 \in C_k X 0 ∈ C k , where C k C_k C k is a set of closed persistent states. Then X X X remains in C k C_k C k forever; as such, we may as well analyze only C k C_k C k rather than the entire states space S S S . On the other hand, if X 0 ∈ T X_0 \in T X 0 ∈ T , then the chain X X X either remains in T T T forever or it eventually visits one of the closed persistent states C k C_k C k , where it remains forever. If ∣ S ∣ < ∞ |S|<\infty ∣ S ∣ < ∞ , it turns out that it is impossible for X X X to remain in T T T forever.
Lemma 4.3.5. If S S S is finite, then at least one state is persistent and all persistent states are non-null.
Proof. First, suppose all state are transient. Then, by Corollary 4.2.5, we have
0 = ∑ j lim n → ∞ p n ( i , j ) = lim n → ∞ ∑ j p n ( i , j ) = 1. 0 = \sum_j \lim_{n\to\infty} p_n(i,j) = \lim_{n\to\infty} \sum_j p_n(i,j) = 1. 0 = j ∑ n → ∞ lim p n ( i , j ) = n → ∞ lim j ∑ p n ( i , j ) = 1. Thus, we have a contradiction and we conclude that at least one state is persistent. Now, suppose j ∈ C k j \in C_k j ∈ C k , where C k C_k C k is a closed set of persistent states. Suppose j j j is null. Then by Theorem 4.3.2 , all states i ∈ C k i \in C_k i ∈ C k are null. Then, by Theorem 4.2.8 we have
0 = ∑ j lim n → ∞ p n ( i , j ) = lim n → ∞ ∑ j p n ( i , j ) = 1. 0 = \sum_j \lim_{n\to\infty} p_n(i,j) = \lim_{n\to\infty} \sum_j p_n(i,j) = 1. 0 = j ∑ n → ∞ lim p n ( i , j ) = n → ∞ lim j ∑ p n ( i , j ) = 1. Thus, j j j cannot be null.
Example 4.3.6. Let S = { 1 , 2 , 3 , 4 , 5 , 6 } S=\{1,2,3,4,5,6\} S = { 1 , 2 , 3 , 4 , 5 , 6 } and
P = ( p q 0 0 0 0 q p 0 0 0 0 r s t u 0 0 0 0 u t s r 0 0 0 0 p q 0 0 0 0 q p ) , q + p = 1 , r + s + t + u = 1. \mathbf{P} =
\begin{pmatrix}
p & q & 0 & 0 & 0 & 0 \\
q & p & 0 & 0 & 0 & 0 \\
r & s & t & u & 0 & 0 \\
0 & 0 & u & t & s & r \\
0 & 0 & 0 & 0 & p & q \\
0 & 0 & 0 & 0 & q & p
\end{pmatrix},
\qquad q+p=1,\qquad r+s+t+u=1. P = p q r 0 0 0 q p s 0 0 0 0 0 t u 0 0 0 0 u t 0 0 0 0 0 s p q 0 0 0 r q p , q + p = 1 , r + s + t + u = 1.
The states subsets C 1 = { 1 , 2 } C_1 = \{1,2\} C 1 = { 1 , 2 } and C 2 = { 5 , 6 } C_2 = \{5,6\} C 2 = { 5 , 6 } are closed, persistent, non-null sets, since, once the chain visits these sets, it cannot escape. The subset T = { 3 , 4 } T = \{3,4\} T = { 3 , 4 } is transient. If X 0 ∈ T X_0 \in T X 0 ∈ T , it will eventually move to either C 1 C_1 C 1 or C 2 C_2 C 2 , where it will remain forever.
Stationary distributions and the limit theorem
In this section, we will be interested in studying the long-run behavior of a Markov chain X X X .
Definition 4.4.1. Let X X X be a Markov chain with one-step transition matrix P \mathbf{P} P . We say that a row vector
π = ( π ( 1 ) , π ( 2 ) , … , π ( ∣ S ∣ ) ) , \boldsymbol{\pi} = \left(\pi(1), \pi(2), \ldots, \pi(|S|)\right), π = ( π ( 1 ) , π ( 2 ) , … , π ( ∣ S ∣ ) ) , is a stationary or invariant distribution of X X X if it satisfies
π = π P , \boldsymbol{\pi} = \boldsymbol{\pi}\mathbf{P}, π = π P , ∑ i π ( i ) = 1 , \sum_i \pi(i) = 1, i ∑ π ( i ) = 1 , and π ( i ) ≥ 0 \pi(i) \geq0 π ( i ) ≥ 0 for all i i i .
The vector π \boldsymbol{\pi} π is called a stationary distribution for the following reason: suppose the Markov chain X X X has an initial distribution μ 0 = π \boldsymbol{\mu}_0 = \boldsymbol{\pi} μ 0 = π . Then we have
μ n = π P n = π P P n − 1 = π P n − 1 = π P P n − 2 = ⋯ = π P = π . \begin{aligned}
\boldsymbol{\mu}_n &= \boldsymbol{\pi}\mathbf{P}^{n} = \boldsymbol{\pi}\mathbf{P}\mathbf{P}^{n-1}\\
&= \boldsymbol{\pi}\mathbf{P}^{n-1} = \boldsymbol{\pi}\mathbf{P}\mathbf{P}^{n-2}\\
&= \cdots\\
&= \boldsymbol{\pi}\mathbf{P} = \boldsymbol{\pi}.
\end{aligned} μ n = π P n = π P P n − 1 = π P n − 1 = π P P n − 2 = ⋯ = π P = π . Thus, if μ 0 = π \boldsymbol{\mu}_0 = \boldsymbol{\pi} μ 0 = π then μ n = π \boldsymbol{\mu}_n = \boldsymbol{\pi} μ n = π for all n n n .
Example 4.4.2. Consider a two-state Markov chain with state space S = { 1 , 2 } S = \{1,2\} S = { 1 , 2 } and with one-step transition matrix
P = ( 1 − p p q 1 − q ) , \mathbf{P} =
\begin{pmatrix}
1-p & p\\
q & 1-q
\end{pmatrix}, P = ( 1 − p q p 1 − q ) , q , p ∈ ( 0 , 1 ) q,p \in(0,1) q , p ∈ ( 0 , 1 ) .
Let us find π \boldsymbol{\pi} π . From π = π P \boldsymbol{\pi} = \boldsymbol{\pi}\mathbf{P} π = π P we derive
π ( 1 ) = π ( 1 ) ( 1 − p ) + π ( 2 ) q , \pi(1) = \pi(1)(1-p) + \pi(2)q, π ( 1 ) = π ( 1 ) ( 1 − p ) + π ( 2 ) q , π ( 2 ) = π ( 1 ) p + π ( 2 ) ( 1 − q ) . \pi(2) = \pi(1)p + \pi(2)(1-q). π ( 2 ) = π ( 1 ) p + π ( 2 ) ( 1 − q ) . We also have π ( 1 ) + π ( 2 ) = 1 \pi(1) + \pi(2) = 1 π ( 1 ) + π ( 2 ) = 1 . Solving for π ( 1 ) \pi(1) π ( 1 ) and π ( 2 ) \pi(2) π ( 2 ) , we obtain
π ( 1 ) = q p + q , \pi(1) = \frac{q}{p+q}, π ( 1 ) = p + q q , π ( 2 ) = p p + q . \pi(2) = \frac{p}{p+q}. π ( 2 ) = p + q p .
For a given Markov chain X X X , a stationary distribution may not exist. And, if it does exist, it may not be unique.
Theorem 4.4.3. An irreducible chain X X X has a stationary distribution π \boldsymbol{\pi} π if and only if all the states are non-null persistent; in this case π \boldsymbol{\pi} π is unique and is given by
π ( i ) = 1 / τ ˉ i , ∀ i ∈ S , \pi(i) = 1/\bar{\tau}_i,\qquad\forall\ i \in S, π ( i ) = 1/ τ ˉ i , ∀ i ∈ S , where τ ˉ i \bar{\tau}_i τ ˉ i is given in Definition 4.2.6.
It is typically much easier to compute π \boldsymbol{\pi} π rather than the recurrence times τ ˉ i \bar{\tau}_i τ ˉ i . Theorem 4.4.3 gives us a method of computing τ ˉ i \bar{\tau}_i τ ˉ i from π ( i ) \pi(i) π ( i ) .
Example 4.4.4. Consider a Markov chain with state space S = { 0 , 1 , 2 , … } S = \{0,1,2,\ldots\} S = { 0 , 1 , 2 , … } and with one-step transition matrix
P = ( q p 0 0 0 ⋯ q 0 p 0 0 ⋯ 0 q 0 p 0 ⋯ 0 0 q 0 p ⋯ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱ ) , p + q = 1. \mathbf{P} =
\begin{pmatrix}
q & p & 0 & 0 & 0 & \cdots\\
q & 0 & p & 0 & 0 & \cdots\\
0 & q & 0 & p & 0 & \cdots\\
0 & 0 & q & 0 & p & \cdots\\
\vdots& \vdots& \vdots& \vdots& \vdots& \ddots
\end{pmatrix},
\qquad p+q=1. P = q q 0 0 ⋮ p 0 q 0 ⋮ 0 p 0 q ⋮ 0 0 p 0 ⋮ 0 0 0 p ⋮ ⋯ ⋯ ⋯ ⋯ ⋱ , p + q = 1. Note that the chain is irreducible. Thus, if an invariant distribution π \boldsymbol{\pi} π exists, it is unique. Let us find π \boldsymbol{\pi} π (if it exists). From π = π P \boldsymbol{\pi} = \boldsymbol{\pi}\mathbf{P} π = π P we have
π ( 0 ) = q π ( 0 ) + q π ( 1 ) , ⇒ π ( 1 ) = 1 − q q π ( 0 ) = p q π ( 0 ) , \pi(0)=q\pi(0)+q\pi(1), \qquad\Rightarrow\qquad\pi(1)=\frac{1-q}{q}\pi(0)=\frac{p}{q}\pi(0), π ( 0 ) = q π ( 0 ) + q π ( 1 ) , ⇒ π ( 1 ) = q 1 − q π ( 0 ) = q p π ( 0 ) , π ( 1 ) = p π ( 0 ) + q π ( 2 ) , ⇒ π ( 2 ) = 1 q ( π ( 1 ) − p π ( 0 ) ) = ( p q ) 2 π ( 0 ) , \pi(1)=p\pi(0)+q\pi(2), \qquad\Rightarrow\qquad\pi(2)=\frac{1}{q}\left(\pi(1)-p\pi(0)\right)=\left(\frac{p}{q}\right)^2\pi(0), π ( 1 ) = p π ( 0 ) + q π ( 2 ) , ⇒ π ( 2 ) = q 1 ( π ( 1 ) − p π ( 0 ) ) = ( q p ) 2 π ( 0 ) , π ( 2 ) = p π ( 1 ) + q π ( 3 ) , ⇒ π ( 3 ) = 1 q ( π ( 2 ) − p π ( 1 ) ) = ( p q ) 3 π ( 0 ) . \pi(2)=p\pi(1)+q\pi(3), \qquad\Rightarrow\qquad\pi(3)=\frac{1}{q}\left(\pi(2)-p\pi(1)\right)=\left(\frac{p}{q}\right)^3\pi(0). π ( 2 ) = p π ( 1 ) + q π ( 3 ) , ⇒ π ( 3 ) = q 1 ( π ( 2 ) − p π ( 1 ) ) = ( q p ) 3 π ( 0 ) . This suggests that π ( n ) = ( p / q ) n π ( 0 ) \pi(n)=(p/q)^n\pi(0) π ( n ) = ( p / q ) n π ( 0 ) . Assume this is true for every i ≤ n i\le n i ≤ n . We show it holds for n + 1 n+1 n + 1 (and thus, for every n n n ). We have
π ( n ) = p π ( n − 1 ) + q π ( n + 1 ) , ⇒ π ( n + 1 ) = 1 q ( π ( n ) − p π ( n − 1 ) ) = 1 q ( ( p q ) n π ( 0 ) − p ( p q ) n − 1 π ( 0 ) ) = ( p q ) n + 1 π ( 0 ) . \begin{aligned}
\pi(n)&=p\pi(n-1)+q\pi(n+1), \qquad\Rightarrow\qquad\pi(n+1)=\frac{1}{q}\left(\pi(n)-p\pi(n-1)\right) \\
&=\frac{1}{q}\left(\left(\frac{p}{q}\right)^n\pi(0)-p\left(\frac{p}{q}\right)^{n-1}\pi(0)\right) \\
&=\left(\frac{p}{q}\right)^{n+1}\pi(0).
\end{aligned} π ( n ) = p π ( n − 1 ) + q π ( n + 1 ) , ⇒ π ( n + 1 ) = q 1 ( π ( n ) − p π ( n − 1 ) ) = q 1 ( ( q p ) n π ( 0 ) − p ( q p ) n − 1 π ( 0 ) ) = ( q p ) n + 1 π ( 0 ) .
Next, we use ∑ n π ( n ) = 1 \sum_{n} \pi(n) = 1 ∑ n π ( n ) = 1 to find π ( 0 ) \pi(0) π ( 0 ) . We have
1 = ∑ n = 0 ∞ π ( n ) = π ( 0 ) ∑ n = 0 ∞ ( p / q ) n = π ( 0 ) 1 1 − ( p / q ) , 1 = \sum_{n=0}^{\infty} \pi(n) = \pi(0) \sum_{n=0}^{\infty} (p/q)^n = \pi(0) \frac{1}{1-(p/q)}, 1 = n = 0 ∑ ∞ π ( n ) = π ( 0 ) n = 0 ∑ ∞ ( p / q ) n = π ( 0 ) 1 − ( p / q ) 1 ,
if ( p / q ) < 1 (p/q) < 1 ( p / q ) < 1 .
Thus, if ( p / q ) < 1 (p/q) < 1 ( p / q ) < 1 then π ( 0 ) = 1 − ( p / q ) \pi(0) = 1-(p/q) π ( 0 ) = 1 − ( p / q ) . Finally, let us find the mean recurrence time of the n n n th state. We have
τ ˉ n = 1 π ( n ) = 1 π ( 0 ) ( q p ) n = 1 1 − ( p / q ) ( q p ) n . \bar{\tau}_n = \frac{1}{\pi(n)} = \frac{1}{\pi(0)}\left(\frac{q}{p}\right)^n = \frac{1}{1-(p/q)}\left(\frac{q}{p}\right)^n. τ ˉ n = π ( n ) 1 = π ( 0 ) 1 ( p q ) n = 1 − ( p / q ) 1 ( p q ) n .
Note that if ( p / q ) > 1 (p/q) > 1 ( p / q ) > 1 , then there is no invariant distribution π \pi π . In this case, the chain is transient and the mean recurrence time for every state is infinite. What happens if p = q p=q p = q ? It turns out that in this case, all states are null recurrent (need to check).
Reversibility
Definition 4.5.1. Let X = ( X n ) 0 ≤ n ≤ N X=(X_n)_{0\leq n\leq N} X = ( X n ) 0 ≤ n ≤ N be a Markov chain. The time reversal of X X X is the process Y = ( Y n ) 0 ≤ n ≤ N Y=(Y_n)_{0\leq n\leq N} Y = ( Y n ) 0 ≤ n ≤ N , where
Y n = X N − n . Y_n=X_{N-n}. Y n = X N − n .
Theorem 4.5.2. Let X = ( X n ) 0 ≤ n ≤ N X=(X_n)_{0\leq n\leq N} X = ( X n ) 0 ≤ n ≤ N be an irreducible Markov chain with a one-step transition matrix P \mathbf{P} P and invariant distribution π \boldsymbol{\pi} π . Suppose X 0 ∼ π X_0\sim\boldsymbol{\pi} X 0 ∼ π (so that μ n = π \boldsymbol{\mu}_n=\boldsymbol{\pi} μ n = π for every n n n ). Then Y Y Y , the time reversal of X X X , is a Markov chain and its one-step transition matrix, denoted Q = ( q ( i , j ) ) i j \mathbf{Q}=(q(i,j))_{ij} Q = ( q ( i , j ) ) ij , is given by
q ( i , j ) = π ( j ) π ( i ) p ( j , i ) . (38) q(i,j)=\frac{\pi(j)}{\pi(i)}p(j,i). \tag*{(38)} q ( i , j ) = π ( i ) π ( j ) p ( j , i ) . (38)
Proof. Let F n = σ ( Y k , 0 ≤ k ≤ n ) \mathcal{F}_n=\sigma(Y_k,0\leq k\leq n) F n = σ ( Y k , 0 ≤ k ≤ n ) . We cannot assume, a priori that Y Y Y is Markov. Thus, in computing P ( Y n + 1 = i n + 1 ∣ F n ) \mathbb{P}(Y_{n+1}=i_{n+1}\mid\mathcal{F}_n) P ( Y n + 1 = i n + 1 ∣ F n ) we must condition on the entire history F n \mathcal{F}_n F n of Y Y Y rather than conditioning on Y n Y_n Y n only. We have
P ( Y n + 1 = i n + 1 ∣ Y k = i k , 0 ≤ k ≤ n ) = P ( Y k = i k , 0 ≤ k ≤ n + 1 ) P ( Y k = i k , 0 ≤ k ≤ n ) = P ( X N − k = i k , 0 ≤ k ≤ n + 1 ) P ( X N − k = i k , 0 ≤ k ≤ n ) = P ( X N − k = i k , 0 ≤ k ≤ n − 1 ∣ X N − n = i n ) P ( X N − n = i n ∣ X N − ( n + 1 ) = i n + 1 ) P ( X N − ( n + 1 ) = i n + 1 ) P ( X N − k = i k , 0 ≤ k ≤ n − 1 ∣ X N − n = i n ) P ( X N − n = i n ) \begin{aligned}
\mathbb{P}(Y_{n+1}=i_{n+1}\mid Y_k=i_k,0\leq k\leq n)
&=\frac{\mathbb{P}(Y_k=i_k,0\leq k\leq n+1)}{\mathbb{P}(Y_k=i_k,0\leq k\leq n)}\\
&=\frac{\mathbb{P}(X_{N-k}=i_k,0\leq k\leq n+1)}{\mathbb{P}(X_{N-k}=i_k,0\leq k\leq n)}\\
&=\frac{\mathbb{P}(X_{N-k}=i_k,0\leq k\leq n-1\mid X_{N-n}=i_n)\mathbb{P}(X_{N-n}=i_n\mid X_{N-(n+1)}=i_{n+1})\mathbb{P}(X_{N-(n+1)}=i_{n+1})}{\mathbb{P}(X_{N-k}=i_k,0\leq k\leq n-1\mid X_{N-n}=i_n)\mathbb{P}(X_{N-n}=i_n)}
\end{aligned} P ( Y n + 1 = i n + 1 ∣ Y k = i k , 0 ≤ k ≤ n ) = P ( Y k = i k , 0 ≤ k ≤ n ) P ( Y k = i k , 0 ≤ k ≤ n + 1 ) = P ( X N − k = i k , 0 ≤ k ≤ n ) P ( X N − k = i k , 0 ≤ k ≤ n + 1 ) = P ( X N − k = i k , 0 ≤ k ≤ n − 1 ∣ X N − n = i n ) P ( X N − n = i n ) P ( X N − k = i k , 0 ≤ k ≤ n − 1 ∣ X N − n = i n ) P ( X N − n = i n ∣ X N − ( n + 1 ) = i n + 1 ) P ( X N − ( n + 1 ) = i n + 1 )
= P ( X N − n = i n ∣ X N − ( n + 1 ) = i n + 1 ) P ( X N − ( n + 1 ) = i n + 1 ) P ( X N − n = i n ) = p ( i n + 1 , i n ) π ( i n + 1 ) π ( i n ) . \begin{aligned}
&= \frac{\mathbb{P}(X_{N-n}=i_n\mid X_{N-(n+1)}=i_{n+1})\mathbb{P}(X_{N-(n+1)}=i_{n+1})}{\mathbb{P}(X_{N-n}=i_n)} \\
&= \frac{p(i_{n+1},i_n)\pi(i_{n+1})}{\pi(i_n)}.
\end{aligned} = P ( X N − n = i n ) P ( X N − n = i n ∣ X N − ( n + 1 ) = i n + 1 ) P ( X N − ( n + 1 ) = i n + 1 ) = π ( i n ) p ( i n + 1 , i n ) π ( i n + 1 ) .
Thus, we have show that Y Y Y is Markov and its one-step transition matrix is defined by (38) .
Definition 4.5.3. Let X = ( X n ) n ≥ 0 X=(X_n)_{n\ge0} X = ( X n ) n ≥ 0 be an irreducible Markov chain with a one-step transition matrix P \mathbf{P} P and invariant distribution π \boldsymbol{\pi} π . Suppose X 0 ∼ π X_0\sim\boldsymbol{\pi} X 0 ∼ π (so that μ n = π \boldsymbol{\mu}_n=\boldsymbol{\pi} μ n = π for every n n n ). Let Y Y Y be the time reversal of X X X and denote by Q \mathbf{Q} Q the one-step transition matrix of Y Y Y . We say that X X X is reversible if P = Q \mathbf{P}=\mathbf{Q} P = Q , or equivalently (by Theorem 4.5.2 ), if
π ( i ) p ( i , j ) = π ( j ) p ( j , i ) , ∀ i , j ∈ S . \pi(i)p(i,j)=\pi(j)p(j,i), \qquad\forall i,j\in S. π ( i ) p ( i , j ) = π ( j ) p ( j , i ) , ∀ i , j ∈ S .
Definition 4.5.4. Let P \mathbf{P} P be an ∣ S ∣ × ∣ S ∣ |S|\times|S| ∣ S ∣ × ∣ S ∣ one step transition matrix. We say that a distribution λ = ( λ ( 1 ) , λ ( 2 ) , … , λ ( ∣ S ∣ ) ) \boldsymbol{\lambda}=(\lambda(1),\lambda(2),\ldots,\lambda(|S|)) λ = ( λ ( 1 ) , λ ( 2 ) , … , λ ( ∣ S ∣ )) is in detailed balance with P \mathbf{P} P if
λ ( i ) p ( i , j ) = λ ( j ) p ( j , i ) , ∀ i , j ∈ S . (39) \lambda(i)p(i,j)=\lambda(j)p(j,i), \qquad\forall i,j\in S. \tag*{(39)} λ ( i ) p ( i , j ) = λ ( j ) p ( j , i ) , ∀ i , j ∈ S . (39)
Theorem 4.5.5. Let P \mathbf{P} P be the one step transition matrix of an irreducible Markov chain X X X . Suppose there exists a distribution λ \boldsymbol{\lambda} λ such that (39) holds. Then λ \boldsymbol{\lambda} λ is a stationary distribution for X X X . That is, λ = λ P \boldsymbol{\lambda}=\boldsymbol{\lambda}\mathbf{P} λ = λ P .
Proof. Assuming (39) holds, we have
∑ i λ ( i ) p ( i , j ) = ∑ i λ ( j ) p ( j , i ) = λ ( j ) ∑ i p ( j , i ) = λ ( j ) . \sum_i\lambda(i)p(i,j)=\sum_i\lambda(j)p(j,i)=\lambda(j)\sum_i p(j,i)=\lambda(j). i ∑ λ ( i ) p ( i , j ) = i ∑ λ ( j ) p ( j , i ) = λ ( j ) i ∑ p ( j , i ) = λ ( j ) .
Thus, we have proved λ = λ P \boldsymbol{\lambda}=\boldsymbol{\lambda}\mathbf{P} λ = λ P , and thus λ \boldsymbol{\lambda} λ is a stationary distribution for X X X .
Chains with finitely many states
In this section, we will give some important results that apply to Markov chains with finitely many states. By Theorem 4.3.2 and Lemma 4.3.5 , if the state space S S S is finite and irreducible then all states are necessarily non-null persistent and have the same period.
Theorem 4.6.1 (Perron-Frobenius). If P \mathbf{P} P is a one-step transition matrix of a finite irreducible Markov chain X X X with period d d d , then
λ 1 = 1 \lambda_1=1 λ 1 = 1 is an eigenvalue of P \mathbf{P} P .
The d d d complex roots of unity
λ n = ω n − 1 , ω = e 2 π i / d , n = 1 , 2 , … , d , \lambda_n=\omega^{n-1},\qquad\omega=e^{2\pi i/d},\qquad n=1,2,\ldots,d, λ n = ω n − 1 , ω = e 2 π i / d , n = 1 , 2 , … , d ,
are eigenvalues of P \mathbf{P} P .
The remaining eigenvalues λ d + 1 , λ d + 2 , … , λ ∣ S ∣ \lambda_{d+1},\lambda_{d+2},\ldots,\lambda_{|S|} λ d + 1 , λ d + 2 , … , λ ∣ S ∣ , satisfy ∣ λ j ∣ < 1 |\lambda_j|<1 ∣ λ j ∣ < 1 .
Suppose the eigenvalues of P \mathbf{P} P are distinct. Then it is well-known that P \mathbf{P} P has the decomposition
P = U − 1 Λ U , U = ( e 1 e 2 ⋮ e ∣ S ∣ ) , Λ = ( λ 1 0 ⋯ 0 0 λ 2 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ λ ∣ S ∣ ) , \mathbf{P}=\mathbf{U}^{-1}\boldsymbol{\Lambda}\mathbf{U},\qquad
\mathbf{U}=
\begin{pmatrix}
\mathbf{e}_1\\
\mathbf{e}_2\\
\vdots\\
\mathbf{e}_{|S|}
\end{pmatrix},
\qquad
\boldsymbol{\Lambda}=
\begin{pmatrix}
\lambda_1&0&\cdots&0\\
0&\lambda_2&\cdots&0\\
\vdots&\vdots&\ddots&\vdots\\
0&0&\cdots&\lambda_{|S|}
\end{pmatrix}, P = U − 1 Λ U , U = e 1 e 2 ⋮ e ∣ S ∣ , Λ = λ 1 0 ⋮ 0 0 λ 2 ⋮ 0 ⋯ ⋯ ⋱ ⋯ 0 0 ⋮ λ ∣ S ∣ ,
where ( e i ) 1 ≤ i ≤ ∣ S ∣ (\mathbf{e}_i)_{1\leq i\leq|S|} ( e i ) 1 ≤ i ≤ ∣ S ∣ are the left eigenvectors of P \mathbf{P} P .
P n = U − 1 Λ U ⋅ U − 1 Λ U ⋅ … ⋅ U − 1 Λ U ⏟ n t i m e s = U − 1 Λ n U , (40) \mathbf{P}^n=\underbrace{\mathbf{U}^{-1}\boldsymbol{\Lambda}\mathbf{U}\cdot\mathbf{U}^{-1}\boldsymbol{\Lambda}\mathbf{U}\cdot\ldots\cdot\mathbf{U}^{-1}\boldsymbol{\Lambda}\mathbf{U}}_{n\ \mathrm{times}}=\mathbf{U}^{-1}\boldsymbol{\Lambda}^n\mathbf{U}, \tag*{(40)} P n = n times U − 1 Λ U ⋅ U − 1 Λ U ⋅ … ⋅ U − 1 Λ U = U − 1 Λ n U , (40)
where we have used the fact that U − 1 U = I \mathbf{U}^{-1}\mathbf{U}=\mathbf{I} U − 1 U = I .
Expression (40) allows us to study the long-run behavior n → ∞ n\to\infty n → ∞ of a Markov chain X \mathbf{X} X . In what follows, assume the period of X \mathbf{X} X is one so that the eigenvalues of P \mathbf{P} P satisfy 1 = λ 1 > ∣ λ 2 ∣ > ∣ λ 3 ∣ > … > ∣ λ ∣ S ∣ ∣ 1=\lambda_1>|\lambda_2|>|\lambda_3|>\ldots>|\lambda_{|S|}| 1 = λ 1 > ∣ λ 2 ∣ > ∣ λ 3 ∣ > … > ∣ λ ∣ S ∣ ∣ . Let X 0 = μ 0 \mathbf{X}_0=\boldsymbol{\mu}_0 X 0 = μ 0 . As the eigenvectors ( e i ) 1 ≤ i ≤ ∣ S ∣ (\mathbf{e}_i)_{1\leq i\leq|S|} ( e i ) 1 ≤ i ≤ ∣ S ∣ form a basis for R ∣ S ∣ \mathbb{R}^{|S|} R ∣ S ∣ we can express μ 0 \boldsymbol{\mu}_0 μ 0 as
μ 0 = ∑ i = 1 ∣ S ∣ c i e i , (41) \boldsymbol{\mu}_0=\sum_{i=1}^{|S|}c_i\mathbf{e}_i, \tag*{(41)} μ 0 = i = 1 ∑ ∣ S ∣ c i e i , (41)
for some constants ( c i ) (c_i) ( c i ) . Next, using (40) and (41) , we obtain
μ n = μ 0 P n = ∑ i = 1 ∣ S ∣ c i e i U − 1 Λ n U = ∑ i = 1 ∣ S ∣ c i λ i n e i = c 1 α π + ∑ i = 2 ∣ S ∣ c i λ i n e i , \boldsymbol{\mu}_n=\boldsymbol{\mu}_0\mathbf{P}^n=\sum_{i=1}^{|S|}c_i\mathbf{e}_i\mathbf{U}^{-1}\boldsymbol{\Lambda}^n\mathbf{U}=\sum_{i=1}^{|S|}c_i\lambda_i^n\mathbf{e}_i=c_1\alpha\boldsymbol{\pi}+\sum_{i=2}^{|S|}c_i\lambda_i^n\mathbf{e}_i, μ n = μ 0 P n = i = 1 ∑ ∣ S ∣ c i e i U − 1 Λ n U = i = 1 ∑ ∣ S ∣ c i λ i n e i = c 1 α π + i = 2 ∑ ∣ S ∣ c i λ i n e i ,
where, in the last step, we used λ 1 = 1 \lambda_1=1 λ 1 = 1 and e 1 = α π \mathbf{e}_1=\alpha\boldsymbol{\pi} e 1 = α π for some constant α \alpha α . As n → ∞ n\to\infty n → ∞ , the terms in the sum go to zero at least as fast as ∣ λ 2 ∣ n = exp ( n log ∣ λ 2 ∣ ) |\lambda_2|^n=\exp(n\log|\lambda_2|) ∣ λ 2 ∣ n = exp ( n log ∣ λ 2 ∣ ) . Thus, μ n → π \boldsymbol{\mu}_n\to\boldsymbol{\pi} μ n → π exponentially fast with a rate determined by the second eigenvalue.
Exercises
Exercise 4.1. A six-sided die is rolled repeatedly. Which of the following a Markov chains? For those that are, find the one-step transition matrix. (a) X n X_n X n is the largest number rolled up to the n n n th roll. (b) X n X_n X n is the number of sixes rolled in the first n n n rolls. (c) At time n n n , X n X_n X n is the time since the last six was rolled. (d) At time n n n , X n X_n X n is the time until the next six is rolled.
Exercises
Exercise 4.2. Let Y n = X 2 n Y_n = X_{2n} Y n = X 2 n . Compute the transition matrix for Y Y Y when (a) X X X is a simple random walk (i.e., X X X increases by one with probability p p p and decreases by 1 with probability q q q and (b) X X X is a branching process where G G G is the generating function of the number of offspring from each individual.
Exercise 4.3. Let X X X be a Markov chain with state space S S S and absorbing state k k k (i.e., p ( k , j ) = 0 p(k,j) = 0 p ( k , j ) = 0 for all j ∈ S j \in S j ∈ S ). Suppose j → k j \to k j → k for all j ∈ S j \in S j ∈ S . Show that all states other than k k k are transient.
Exercise 4.4. Suppose two distinct states i , j i,j i , j satisfy
P ( τ j < τ i ∣ X 0 = i ) = P ( τ i < τ j ∣ X 0 = j ) , \mathbb{P}(\tau_j < \tau_i \mid X_0 = i) = \mathbb{P}(\tau_i < \tau_j \mid X_0 = j), P ( τ j < τ i ∣ X 0 = i ) = P ( τ i < τ j ∣ X 0 = j ) ,
where τ k : = inf { n ≥ 1 : X n = j } \tau_k := \inf\{n \ge1 : X_n = j\} τ k := inf { n ≥ 1 : X n = j } . Show that, if X 0 = i X_0 = i X 0 = i , the expected number of visits to j j j prior to re-visiting i i i is one.
Exercise 4.5. Let X X X be a Markov chain with transition matrix
P = ( 1 − 2 p 2 p 0 p 1 − 2 p p 0 2 p 1 − 2 p ) , p ∈ ( 0 , 1 ) . \mathbf{P} =
\begin{pmatrix}
1 - 2p & 2p & 0 \\
p & 1 - 2p & p \\
0 & 2p & 1 - 2p
\end{pmatrix},
\qquad p \in(0,1). P = 1 − 2 p p 0 2 p 1 − 2 p 2 p 0 p 1 − 2 p , p ∈ ( 0 , 1 ) .
Find P n \mathbf{P}^n P n , the invariant distribution π \boldsymbol{\pi} π and the mean-recurrence times τ ˉ j \bar{\tau}_j τ ˉ j for j = 1 , 2 , 3 j = 1,2,3 j = 1 , 2 , 3 .
Exercise 4.6. Let X n X_n X n be the number of mistakes in the n n n th addition of a book. Between the n n n th and the ( n + 1 ) (n+1) ( n + 1 ) th addition an editor corrects each mistake independently with probability p p p and introduces Y n Y_n Y n new mistakes where the ( Y n ) (Y_n) ( Y n ) are iid and Poisson distributed with parameter λ \lambda λ . Find the invariant distribution π \boldsymbol{\pi} π of the number of mistakes in the book.
Exercise 4.7. Give an example of a transition matrix P \mathbf{P} P that admits multiple stationary distributions π \boldsymbol{\pi} π .
Exercise 4.8. A Markov chain on S = { 0 , 1 , 2 , … , n } S = \{0,1,2,\ldots,n\} S = { 0 , 1 , 2 , … , n } has transition probabilities p ( 0 , 0 ) = 1 − λ 0 p(0,0) = 1 - \lambda_0 p ( 0 , 0 ) = 1 − λ 0 , p ( i , i + 1 ) = λ i p(i,i+1) = \lambda_i p ( i , i + 1 ) = λ i and p ( i + 1 , i ) = μ i + 1 p(i+1,i) = \mu_{i+1} p ( i + 1 , i ) = μ i + 1 for i = 0 , 1 , … , n − 1 i = 0,1,\ldots,n-1 i = 0 , 1 , … , n − 1 , and p ( n , n ) = 1 − μ n p(n,n) = 1 - \mu_n p ( n , n ) = 1 − μ n . Show that the process is reversible in equilibrium.
Continuous time Markov chains
The notes from this chapter are primarily taken from (Grimmett and Stirzaker, 2001, Chapter 6).
The Poisson process
Many stochastic process change values at random times rather than at fixed intervals. For example, the value of a stock does not change once per second every second. Rather, a stock price changes when there is a buy or sell order. And these orders arrive at random times. Thus, it makes no sense to model a stock price as a discrete time Markov chain; rather, we must move to continuous time. The simplest continuous time process is a counting process .
Definition 5.1.1. A counting process is a stochastic process N = ( N t ) t ≥ 0 N = (N_t)_{t \ge0} N = ( N t ) t ≥ 0 taking values in S = { 0 , 1 , 2 , … } S = \{0, 1, 2, \ldots\} S = { 0 , 1 , 2 , … } such that the following hold:
N 0 = 0 N_0 = 0 N 0 = 0 .
If s < t s < t s < t then N s ≤ N t N_s \le N_t N s ≤ N t .
As its name suggests, a counting process simply counts the number of times an event occurs prior to a given time. For example, we could define N t : = { number of buy orders prior to time t } N_t := \{\text{number of buy orders prior to time }t\} N t := { number of buy orders prior to time t } . Of course, there are many ways to model a counting process. And, depending on how we model N N N , the counting process may or may not be Markov. However, because Markov processes allow for many analytically tractable computations, we will focus on Markov counting processes; the most basic of these is the Poisson process.
Definition 5.1.2. A Poisson process with intensity λ \lambda λ is a stochastic process N = ( N t ) t ≥ 0 N = (N_t)_{t \ge0} N = ( N t ) t ≥ 0 taking values in S = { 0 , 1 , 2 , … } S = \{0, 1, 2, \ldots\} S = { 0 , 1 , 2 , … } such that the following hold.
N 0 = 0 N_0 = 0 N 0 = 0 .
If s < t s < t s < t then N s ≤ N t N_s \le N_t N s ≤ N t .
If s < t s < t s < t then ( N t − N s ) ⊥ ⊥ F s (N_t-N_s) \perp\!\!\!\perp\mathcal{F}_s ( N t − N s ) ⊥ ⊥ F s , where F s = σ ( N r , 0 ≤ r ≤ s ) \mathcal{F}_s=\sigma(N_r,0\le r\le s) F s = σ ( N r , 0 ≤ r ≤ s ) .
Lastly,
P ( N t + s = n + m ∣ N t = n ) = { λ s + O ( s 2 ) m = 1 , O ( s 2 ) m ≥ 2 , 1 − λ s + O ( s 2 ) m = 0 , \mathbb{P}(N_{t+s}=n+m\mid N_t=n)=
\begin{cases}
\lambda s+O(s^2) & m=1,\\
O(s^2) & m\ge2,\\
1-\lambda s+O(s^2) & m=0,
\end{cases} P ( N t + s = n + m ∣ N t = n ) = ⎩ ⎨ ⎧ λ s + O ( s 2 ) O ( s 2 ) 1 − λ s + O ( s 2 ) m = 1 , m ≥ 2 , m = 0 ,
as s → 0 + s\to0^+ s → 0 + .
Note that Item 3 implies that N N N is Markov. To see this, define F t : = σ ( N s , 0 ≤ s ≤ t ) \mathcal{F}_t:=\sigma(N_s,0\le s\le t) F t := σ ( N s , 0 ≤ s ≤ t ) and observe that
E [ g ( N t ) ∣ F s ] = E [ g ( N t − N s + N s ) ∣ N s ] = ∑ n = 0 ∞ g ( n + N s ) f N t − N s ( n ) . \mathbb{E}[g(N_t)\mid\mathcal{F}_s]=\mathbb{E}[g(N_t-N_s+N_s)\mid N_s]=\sum_{n=0}^{\infty}g(n+N_s)f_{N_t-N_s}(n). E [ g ( N t ) ∣ F s ] = E [ g ( N t − N s + N s ) ∣ N s ] = n = 0 ∑ ∞ g ( n + N s ) f N t − N s ( n ) .
Let us find the probability mass function of N t N_t N t .
Theorem 5.1.3. Let N = ( N t ) t ≥ 0 N=(N_t)_{t\ge0} N = ( N t ) t ≥ 0 be a Poisson process with parameter λ \lambda λ . Then, for all t ≥ 0 t\ge0 t ≥ 0 we have N t ∼ Poi ( λ t ) N_t\sim\operatorname{Poi}(\lambda t) N t ∼ Poi ( λ t ) . That is
p t ( j ) : = P ( N t = j ) = ( λ t ) j j ! e − λ t . (42) p_t(j):=\mathbb{P}(N_t=j)=\frac{(\lambda t)^j}{j!}e^{-\lambda t}. \tag*{(42)} p t ( j ) := P ( N t = j ) = j ! ( λ t ) j e − λ t . (42)
Proof. We have
P ( N t + s = j ) = ∑ i = 0 j P ( N t + s = j ∣ N t = i ) P ( N t = i ) = ∑ i = 0 j P ( N t + s − N t = j − i ) P ( N t = i ) = P ( N t + s − N t = 0 ) P ( N t = j ) + P ( N t + s − N t = 1 ) P ( N t = j − 1 ) + ∑ i = 0 j − 2 P ( N t + s − N t = j − i ) P ( N t = i ) = ( 1 − λ s ) P ( N t = j ) + λ s P ( N t = j − 1 ) + O ( s 2 ) . \begin{aligned}
\mathbb{P}(N_{t+s}=j)=\sum_{i=0}^{j}\mathbb{P}(N_{t+s}=j\mid N_t=i)\mathbb{P}(N_t=i) \\
&=\sum_{i=0}^{j}\mathbb{P}(N_{t+s}-N_t=j-i)\mathbb{P}(N_t=i) \\
&=\mathbb{P}(N_{t+s}-N_t=0)\mathbb{P}(N_t=j)+\mathbb{P}(N_{t+s}-N_t=1)\mathbb{P}(N_t=j-1)
+\sum_{i=0}^{j-2}\mathbb{P}(N_{t+s}-N_t=j-i)\mathbb{P}(N_t=i) \\
&=(1-\lambda s)\mathbb{P}(N_t=j)+\lambda s\mathbb{P}(N_t=j-1)+O(s^2).
\end{aligned} P ( N t + s = j ) = i = 0 ∑ j P ( N t + s = j ∣ N t = i ) P ( N t = i ) = i = 0 ∑ j P ( N t + s − N t = j − i ) P ( N t = i ) = P ( N t + s − N t = 0 ) P ( N t = j ) + P ( N t + s − N t = 1 ) P ( N t = j − 1 ) + i = 0 ∑ j − 2 P ( N t + s − N t = j − i ) P ( N t = i ) = ( 1 − λ s ) P ( N t = j ) + λ s P ( N t = j − 1 ) + O ( s 2 ) .
Subtracting P ( N t = j ) = : p t ( j ) \mathbb{P}(N_t=j)=:p_t(j) P ( N t = j ) =: p t ( j ) from both sides, dividing by s s s and taking a limit as s → 0 s\to0 s → 0 we obtain
∂ t p t ( 0 ) = − λ p t ( 0 ) , p 0 ( 0 ) = 1 , (43) \partial_t p_t(0)=-\lambda p_t(0),\qquad p_0(0)=1, \tag*{(43)} ∂ t p t ( 0 ) = − λ p t ( 0 ) , p 0 ( 0 ) = 1 , (43)
∂ t p t ( j ) = − λ p t ( j ) + λ p t ( j − 1 ) , p 0 ( j ) = 0 , j ≥ 1 , (44) \partial_t p_t(j)=-\lambda p_t(j)+\lambda p_t(j-1),\qquad p_0(j)=0,\qquad j\ge1, \tag*{(44)} ∂ t p t ( j ) = − λ p t ( j ) + λ p t ( j − 1 ) , p 0 ( j ) = 0 , j ≥ 1 , (44)
where we have included the obvious boundary conditions. Thus, we have obtained a sequence of nested ODEs for ( p t ( j ) ) j ≥ 0 (p_t(j))_{j\ge0} ( p t ( j ) ) j ≥ 0 . One can easily verify that (42) satisfies (43) -(44) .
The Poisson Process
For any counting process (whether or not it is Poisson), we may be interested to know the arrival time of the n n n th event and the time between the n n n th and ( n + 1 ) (n+1) ( n + 1 ) th events.
Definition 5.1.4. Let N = ( N t ) t ≥ 0 \mathbb{N}=(N_t)_{t\geq0} N = ( N t ) t ≥ 0 be a counting process. We define S n S_n S n , the the n n n th arrival time , by
S 0 = 0 , S n = inf { t ≥ 0 : N t = n } , n ≥ 1. (45) S_0=0,\qquad S_n=\inf\{t\geq0:N_t=n\},\qquad n\geq1. \tag*{(45)} S 0 = 0 , S n = inf { t ≥ 0 : N t = n } , n ≥ 1. (45) with inf { ∅ } = ∞ \inf\{\varnothing\}=\infty inf { ∅ } = ∞ . We define τ n \tau_n τ n , the n n n th inter-arrival time , by
τ n : = S n − S n − 1 , n ≥ 1. (46) \tau_n:=S_n-S_{n-1},\qquad n\geq1. \tag*{(46)} τ n := S n − S n − 1 , n ≥ 1. (46)
Given the full history of N \mathbb{N} N , we can construct the sequence ( τ n ) n ≥ 1 (\tau_n)_{n\geq1} ( τ n ) n ≥ 1 using (45) –(46) . Alternatively, given the sequence ( τ n ) n ≥ 1 (\tau_n)_{n\geq1} ( τ n ) n ≥ 1 , we can construct the path of N \mathbb{N} N using
S n = ∑ i = 1 n τ i , n ≥ 1 , N t = ∑ n = 1 ∞ n 1 { S n ≤ t < S n + 1 } . (47) S_n=\sum_{i=1}^{n}\tau_i,\qquad n\geq1,\qquad N_t=\sum_{n=1}^{\infty}n\mathbf{1}_{\{S_n\leq t<S_{n+1}\}}. \tag*{(47)} S n = i = 1 ∑ n τ i , n ≥ 1 , N t = n = 1 ∑ ∞ n 1 { S n ≤ t < S n + 1 } . (47)
We now undertake this alternative construction.
Theorem 5.1.5. Suppose the inter-arrival times τ i \tau_i τ i (see Definition 5.1.4 ) of a counting process N = ( N t ) t ≥ 0 \mathbb{N}=(N_t)_{t\geq0} N = ( N t ) t ≥ 0 are iid and exponentially distributed with parameter λ \lambda λ . Then N \mathbb{N} N is a Poisson process with parameter λ \lambda λ .
Proof. We will show that if the inter-arrival times ( τ i ) i ≥ 1 (\tau_i)_{i\geq1} ( τ i ) i ≥ 1 are iid with τ i ∼ E ( λ ) \tau_i\sim\mathcal{E}(\lambda) τ i ∼ E ( λ ) then N t N_t N t , given by (47) has a probability mass function given by (42) . First, we show that S n S_n S n , as defined in (47) , has a Gamma density
P ( S n ∈ d x ) = f S n ( x ) d x , f S n ( x ) = 1 { x ≥ 0 } ( λ x ) n − 1 ( n − 1 ) ! λ e − λ x . (48) \mathbb{P}(S_n\in dx)=f_{S_n}(x)\,dx,\qquad f_{S_n}(x)=\mathbf{1}_{\{x\geq0\}}\frac{(\lambda x)^{n-1}}{(n-1)!}\lambda e^{-\lambda x}. \tag*{(48)} P ( S n ∈ d x ) = f S n ( x ) d x , f S n ( x ) = 1 { x ≥ 0 } ( n − 1 )! ( λ x ) n − 1 λ e − λ x . (48)
Note that S 1 = τ 1 S_1=\tau_1 S 1 = τ 1 and f τ ( x ) = f S 1 ( x ) = 1 { x ≥ 0 } λ e − λ x f_{\tau}(x)=f_{S_1}(x)=\mathbf{1}_{\{x\geq0\}}\lambda e^{-\lambda x} f τ ( x ) = f S 1 ( x ) = 1 { x ≥ 0 } λ e − λ x . Thus, (48) is correct for n = 1 n=1 n = 1 . We now assume (48) holds for n n n and show that it holds for n + 1 n+1 n + 1 . As S n + 1 = S n + τ n + 1 S_{n+1}=S_n+\tau_{n+1} S n + 1 = S n + τ n + 1 we have
f S n + 1 ( x ) = ∫ R 2 d y d z f S n ( y ) ⋅ f τ ( z ) ⋅ δ ( y + z − x ) = ∫ R d y f S n ( y ) ⋅ f τ ( x − y ) = ∫ R d y 1 { y ≥ 0 } ( λ y ) n − 1 ( n − 1 ) ! λ e − λ y ⋅ 1 { x − y ≥ 0 } λ e − λ ( x − y ) = 1 { x ≥ 0 } λ n + 1 e − λ x ( n − 1 ) ! ∫ 0 x d y y n − 1 = 1 { x ≥ 0 } ( λ x ) n n ! λ e − λ x . \begin{aligned}
f_{S_{n+1}}(x)
&=\int_{\mathbb{R}^2}dy\,dz\,f_{S_n}(y)\cdot f_{\tau}(z)\cdot\delta(y+z-x)\\
&=\int_{\mathbb{R}}dy\,f_{S_n}(y)\cdot f_{\tau}(x-y)\\
&=\int_{\mathbb{R}}dy\,\mathbf{1}_{\{y\geq0\}}\frac{(\lambda y)^{n-1}}{(n-1)!}\lambda e^{-\lambda y}\cdot\mathbf{1}_{\{x-y\geq0\}}\lambda e^{-\lambda(x-y)}\\
&=\mathbf{1}_{\{x\geq0\}}\frac{\lambda^{n+1}e^{-\lambda x}}{(n-1)!}\int_0^xdy\,y^{n-1}
=\mathbf{1}_{\{x\geq0\}}\frac{(\lambda x)^n}{n!}\lambda e^{-\lambda x}.
\end{aligned} f S n + 1 ( x ) = ∫ R 2 d y d z f S n ( y ) ⋅ f τ ( z ) ⋅ δ ( y + z − x ) = ∫ R d y f S n ( y ) ⋅ f τ ( x − y ) = ∫ R d y 1 { y ≥ 0 } ( n − 1 )! ( λ y ) n − 1 λ e − λ y ⋅ 1 { x − y ≥ 0 } λ e − λ ( x − y ) = 1 { x ≥ 0 } ( n − 1 )! λ n + 1 e − λ x ∫ 0 x d y y n − 1 = 1 { x ≥ 0 } n ! ( λ x ) n λ e − λ x .
which agrees with (48) . Thus, by induction, (48) holds for every n n n . Using the above result, we now show that N t N_t N t , as defined in (47) , has probability mass function (42) . For n ≥ 1 n \ge1 n ≥ 1 we clearly have
P ( N t ≥ n ) = P ( S n ≤ t ) = ∫ 0 t d x f S n ( x ) = ∫ 0 t d x ( λ x ) n − 1 ( n − 1 ) ! λ e − λ x \mathbb{P}(N_t \ge n) = \mathbb{P}(S_n \le t) = \int_0^t \mathrm{d}x\, f_{S_n}(x) = \int_0^t \mathrm{d}x\, \frac{(\lambda x)^{n-1}}{(n-1)!}\lambda e^{-\lambda x} P ( N t ≥ n ) = P ( S n ≤ t ) = ∫ 0 t d x f S n ( x ) = ∫ 0 t d x ( n − 1 )! ( λ x ) n − 1 λ e − λ x
Using this result, we compute
P ( N t ≥ n + 1 ) = ∫ 0 t d x ( λ x ) n n ! λ e − λ x = − ( λ t ) n n ! e − λ t + P ( N t ≥ n ) , \mathbb{P}(N_t \ge n+1) = \int_0^t \mathrm{d}x\, \frac{(\lambda x)^n}{n!}\lambda e^{-\lambda x} = -\frac{(\lambda t)^n}{n!}e^{-\lambda t} + \mathbb{P}(N_t \ge n), P ( N t ≥ n + 1 ) = ∫ 0 t d x n ! ( λ x ) n λ e − λ x = − n ! ( λ t ) n e − λ t + P ( N t ≥ n ) ,
where we have used integration by parts to obtain the second equality. Thus, we have
P ( N t = n ) = P ( N t ≥ n ) − P ( N t ≥ n + 1 ) = ( λ t ) n n ! e − λ t , \mathbb{P}(N_t=n) = \mathbb{P}(N_t \ge n) - \mathbb{P}(N_t \ge n+1) = \frac{(\lambda t)^n}{n!}e^{-\lambda t}, P ( N t = n ) = P ( N t ≥ n ) − P ( N t ≥ n + 1 ) = n ! ( λ t ) n e − λ t ,
which agrees with (42) . To complete the proof, we observe that
P ( N t = 0 ) = P ( S 1 > t ) = P ( τ 1 > t ) = e − λ t , \mathbb{P}(N_t=0) = \mathbb{P}(S_1>t) = \mathbb{P}(\tau_1>t) = e^{-\lambda t}, P ( N t = 0 ) = P ( S 1 > t ) = P ( τ 1 > t ) = e − λ t ,
which shows that (42) holds for n = 0 n=0 n = 0 .
The inter-arrival times ( τ i ) i ≥ 1 (\tau_i)_{i\ge1} ( τ i ) i ≥ 1 of a Poisson process N N N are memoryless in the sense that
P ( τ i > s + t ∣ τ i > s ) = P ( τ i > s + t , τ i > s ) P ( τ i > s ) = P ( τ i > s + t ) P ( τ i > s ) = e − λ ( t + s ) e − λ s = e − λ t = P ( τ i > t ) . \mathbb{P}(\tau_i>s+t\mid\tau_i>s) = \frac{\mathbb{P}(\tau_i>s+t,\tau_i>s)}{\mathbb{P}(\tau_i>s)} = \frac{\mathbb{P}(\tau_i>s+t)}{\mathbb{P}(\tau_i>s)} = \frac{e^{-\lambda(t+s)}}{e^{-\lambda s}} = e^{-\lambda t} = \mathbb{P}(\tau_i>t). P ( τ i > s + t ∣ τ i > s ) = P ( τ i > s ) P ( τ i > s + t , τ i > s ) = P ( τ i > s ) P ( τ i > s + t ) = e − λ s e − λ ( t + s ) = e − λ t = P ( τ i > t ) .
The memoryless property of the inter-arrival times is what gives the Poisson process its Markov property. If we were to construct a counting process whose inter-arrival times were not exponentially distributed, we would find that this process is not Markov. The reason is that if inter-arrival times are not exponentially distributed, then the amount of time we have to wait for the next event arrival will depend on how much time has passed since the last event arrival.
It should be clear from the construction of N N N that the Poisson process has stationary increments
N t − N s ∼ N t − s , N_t-N_s \sim N_{t-s}, N t − N s ∼ N t − s ,
since N t − N s ⊥ ⊥ N s N_t-N_s \perp\!\!\!\perp N_s N t − N s ⊥ ⊥ N s .
Overview of continuous time Markov chains
We have already defined what we mean by Markov process in Definition 2.3.6 . In this section, we will derive some results that are specific for Markov continuous time processes that have a countable state space.
Overview of Continuous Time Markov Chains
Definition 5.2.1. A continuous time Markov chain X = ( X t ) t ≥ 0 \mathbf{X}=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 is a Markov process with a countable state space S \mathcal{S} S .
Recall from (13) that a Markov process satisfies P ( X t + s ∈ A ∣ F s ) = P ( X t + s ∈ A ∣ X s ) \mathbb{P}(X_{t+s}\in A\mid\mathcal{F}_s)=\mathbb{P}(X_{t+s}\in A\mid X_s) P ( X t + s ∈ A ∣ F s ) = P ( X t + s ∈ A ∣ X s ) . Here, we take the filtration F s \mathcal{F}_s F s to be the natural filtration for X \mathbf{X} X . That is, F s = σ ( X u , 0 ≤ u ≤ s ) \mathcal{F}_s=\sigma(X_u,0\leq u\leq s) F s = σ ( X u , 0 ≤ u ≤ s ) . What the Markov property means in practice is that the evolution of a Markov chain is described by its one-step transition probabilities P ( X t + s = j ∣ X s = i ) \mathbb{P}(X_{t+s}=j\mid X_s=i) P ( X t + s = j ∣ X s = i ) . In general, such probabilities may depend on the times t , s t,s t , s . Throughout this Chapter, we will restrict our attention to Markov chains whose transition probabilities do not depend on s s s .
Definition 5.2.2. A continuous time Markov chain X \mathbf{X} X is called homogeneous if
P ( X t + s = j ∣ X s = i ) = P ( X t = j ∣ X 0 = i ) = : p t ( i , j ) ∀ t , s ∈ R + , ∀ i , j ∈ S . \mathbb{P}(X_{t+s}=j\mid X_s=i)=\mathbb{P}(X_t=j\mid X_0=i)=:p_t(i,j)\qquad\forall\ t,s\in\mathbb{R}_+,\ \forall\ i,j\in\mathcal{S}. P ( X t + s = j ∣ X s = i ) = P ( X t = j ∣ X 0 = i ) =: p t ( i , j ) ∀ t , s ∈ R + , ∀ i , j ∈ S . Let P t \mathbf{P}_t P t be the ∣ S ∣ × ∣ S ∣ |\mathcal{S}|\times|\mathcal{S}| ∣ S ∣ × ∣ S ∣ matrix P t = ( p t ( i , j ) ) \mathbf{P}_t=(p_t(i,j)) P t = ( p t ( i , j )) . We call the collection of matrices ( P t ) t ≥ 0 (\mathbf{P}_t)_{t\geq0} ( P t ) t ≥ 0 the transition semigroup .
Theorem 5.2.3. The transition semigroup ( P t ) t ≥ 0 (\mathbf{P}_t)_{t\geq0} ( P t ) t ≥ 0 satisfies the following properties.
P 0 = I \mathbf{P}_0=\mathbf{I} P 0 = I .
∑ j p t ( i , j ) = 1 \sum_j p_t(i,j)=1 ∑ j p t ( i , j ) = 1 .
P t P s = P t + s \mathbf{P}_t\mathbf{P}_s=\mathbf{P}_{t+s} P t P s = P t + s .
Proof. Item 1 follows from P ( X 0 = j ∣ X 0 = i ) = δ i , j \mathbb{P}(X_0=j\mid X_0=i)=\delta_{i,j} P ( X 0 = j ∣ X 0 = i ) = δ i , j . Item 2 follows from ∑ j P ( X t = j ∣ X 0 = i ) = 1 \sum_j\mathbb{P}(X_t=j\mid X_0=i)=1 ∑ j P ( X t = j ∣ X 0 = i ) = 1 . Lastly, to show Item 3 note that
p t + s ( i , j ) = P ( X t + s = j ∣ X 0 = i ) = ∑ k P ( X t + s = j ∣ X t = k ) ⋅ P ( X t = k ∣ X 0 = i ) = ∑ k P ( X s = j ∣ X 0 = k ) ⋅ P ( X t = k ∣ X 0 = i ) = ∑ k p t ( i , k ) p s ( k , j ) . \begin{aligned}
p_{t+s}(i,j)=\mathbb{P}(X_{t+s}=j\mid X_0=i)=\sum_k\mathbb{P}(X_{t+s}=j\mid X_t=k)\cdot\mathbb{P}(X_t=k\mid X_0=i) \\
&=\sum_k\mathbb{P}(X_s=j\mid X_0=k)\cdot\mathbb{P}(X_t=k\mid X_0=i)=\sum_k p_t(i,k)p_s(k,j).
\end{aligned} p t + s ( i , j ) = P ( X t + s = j ∣ X 0 = i ) = k ∑ P ( X t + s = j ∣ X t = k ) ⋅ P ( X t = k ∣ X 0 = i ) = k ∑ P ( X s = j ∣ X 0 = k ) ⋅ P ( X t = k ∣ X 0 = i ) = k ∑ p t ( i , k ) p s ( k , j ) .
We now construct a continuous-time Markov chain in very much the same manner we constructed the Poisson process in Definition 5.1.2. The following derivation is purely formal (i.e., not rigorous ). Some of what we write is not true in general, but holds in many practical applications.
Definition 5.2.4. A continuous time Markov chain with generator G = ( g ( i , j ) ) i , j \mathbf{G}=(g(i,j))_{i,j} G = ( g ( i , j ) ) i , j is a stochastic process satisfying
Markov property: if s < t s<t s < t then P ( X t = j ∣ F s X ) = P ( X t = j ∣ X s ) \mathbb{P}(X_t=j\mid\mathcal{F}^{\mathbf{X}}_s)=\mathbb{P}(X_t=j\mid X_s) P ( X t = j ∣ F s X ) = P ( X t = j ∣ X s ) for all j ∈ S j\in\mathcal{S} j ∈ S , where F s X : = σ ( X u , 0 ≤ u ≤ s ) \mathcal{F}^{\mathbf{X}}_s:=\sigma(X_u,0\leq u\leq s) F s X := σ ( X u , 0 ≤ u ≤ s ) .
As s → 0 s \to0 s → 0 we have P ( X t + s = j ∣ X t = i ) = δ i j + g ( i , j ) s + O ( s 2 ) \mathbb{P}(X_{t+s}=j\mid X_t=i)=\delta_{ij}+g(i,j)s+O(s^2) P ( X t + s = j ∣ X t = i ) = δ ij + g ( i , j ) s + O ( s 2 ) .
The essence of Definition 5.2.4 is as follows. In a small time interval [ t , t + s ) [t,t+s) [ t , t + s ) the process X either remains at its current state X t = i X_t=i X t = i or it jumps to a new state j ≠ i j\ne i j = i . The probability of remaining at i i i is 1 + g ( i , i ) s + O ( s 2 ) 1+g(i,i)s+O(s^2) 1 + g ( i , i ) s + O ( s 2 ) . The probability of jumping to state j j j is g ( i , j ) s + O ( s 2 ) g(i,j)s+O(s^2) g ( i , j ) s + O ( s 2 ) . Since probabilities should always fall in [ 0 , 1 ] [0,1] [ 0 , 1 ] we must have
g ( i , i ) ≤ 0 , g ( i , j ) ≥ 0 , i ≠ j . g(i,i)\le0,\qquad g(i,j)\ge0,\qquad i\ne j. g ( i , i ) ≤ 0 , g ( i , j ) ≥ 0 , i = j .
Moreover, since X should be found somewhere with probability one we must also have
1 = ∑ j P ( X t + s = j ∣ X t = i ) = ∑ j ( δ i j + g ( i , j ) s ) + O ( s 2 ) = 1 + s ∑ j g ( i , j ) + O ( s 2 ) ⇒ 0 = ∑ j g ( i , j ) . \begin{aligned}
1=\sum_j\mathbb{P}(X_{t+s}=j\mid X_t=i) \\
&=\sum_j(\delta_{ij}+g(i,j)s)+O(s^2)=1+s\sum_j g(i,j)+O(s^2)
\qquad\Rightarrow\qquad
0=\sum_j g(i,j).
\end{aligned} 1 = j ∑ P ( X t + s = j ∣ X t = i ) = j ∑ ( δ ij + g ( i , j ) s ) + O ( s 2 ) = 1 + s j ∑ g ( i , j ) + O ( s 2 ) ⇒ 0 = j ∑ g ( i , j ) .
Note that item 2 in Definition 5.2.4 can be written more compactly as
G = lim s → 0 + 1 s ( P s − I ) . \mathbf{G}=\lim_{s\to0^+}\frac{1}{s}(\mathbf{P}_s-\mathbf{I}). G = s → 0 + lim s 1 ( P s − I ) .
Thus, it is clear that we can obtain G \mathbf{G} G from knowledge of P t \mathbf{P}_t P t . We can also obtain P t \mathbf{P}_t P t from G \mathbf{G} G .
Theorem 5.2.5. Let X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 be a continuous time Markov chain with generator G \mathbf{G} G and a finite state space S S S . The transition semigroup of X satisfies the following ODEs
d d t P t = P t G , (49) \frac{\mathrm{d}}{\mathrm{d}t}\mathbf{P}_t=\mathbf{P}_t\mathbf{G}, \tag*{(49)} d t d P t = P t G , (49) d d t P t = G P t . (50) \frac{\mathrm{d}}{\mathrm{d}t}\mathbf{P}_t=\mathbf{G}\mathbf{P}_t. \tag*{(50)} d t d P t = G P t . (50)
Proof. To derive the forward equation (5.8) we computed P ( X t + s = j ∣ X 0 = i ) \mathbb{P}(X_{t+s}=j\mid X_0=i) P ( X t + s = j ∣ X 0 = i ) by conditioning on the value of X t X_t X t . We have
p t + s ( i , j ) = ∑ k P ( X t + s = j ∣ X t = k ) P ( X t = k ∣ X 0 = i ) = ∑ k p t ( i , k ) p s ( k , j ) = p t ( i , j ) ( 1 + g ( j , j ) s ) + ∑ k : k ≠ j p t ( i , k ) g ( k , j ) s + O ( s 2 ) . \begin{aligned}
p_{t+s}(i,j)=\sum_k\mathbb{P}(X_{t+s}=j\mid X_t=k)\mathbb{P}(X_t=k\mid X_0=i)=\sum_k p_t(i,k)p_s(k,j) \\
&=p_t(i,j)(1+g(j,j)s)+\sum_{k:k\ne j}p_t(i,k)g(k,j)s+O(s^2).
\end{aligned} p t + s ( i , j ) = k ∑ P ( X t + s = j ∣ X t = k ) P ( X t = k ∣ X 0 = i ) = k ∑ p t ( i , k ) p s ( k , j ) = p t ( i , j ) ( 1 + g ( j , j ) s ) + k : k = j ∑ p t ( i , k ) g ( k , j ) s + O ( s 2 ) .
Subtracting p t ( i , j ) p_t(i,j) p t ( i , j ) from both sides, dividing by s s s and taking a limit as s s s goes to zero, we obtain
lim s ↓ 0 1 s ( p t + s ( i , j ) − p t ( i , j ) ) = d d t p t ( i , j ) = ∑ k p t ( i , k ) g ( k , j ) , \lim_{s\downarrow0}\frac{1}{s}\left(p_{t+s}(i,j)-p_t(i,j)\right)=\frac{\mathrm{d}}{\mathrm{d}t}p_t(i,j)=\sum_k p_t(i,k)g(k,j), s ↓ 0 lim s 1 ( p t + s ( i , j ) − p t ( i , j ) ) = d t d p t ( i , j ) = k ∑ p t ( i , k ) g ( k , j ) ,
which is the component-wise representation of (49) . To derive the backward equation (50) we computed P ( X t + s = j ∣ X 0 = i ) \mathbb{P}(X_{t+s}=j\mid X_0=i) P ( X t + s = j ∣ X 0 = i ) by conditioning on the value of X s X_s X s . We have
p t + s ( i , j ) = ∑ k P ( X t + s = j ∣ X s = k ) P ( X s = k ∣ X 0 = i ) = ∑ k p s ( i , k ) p t ( k , j ) = ( 1 + g ( i , i ) s ) p t ( i , j ) + ∑ k : k ≠ i g ( i , k ) s p t ( k , j ) + O ( s 2 ) . \begin{aligned}
p_{t+s}(i,j)=\sum_k\mathbb{P}(X_{t+s}=j\mid X_s=k)\mathbb{P}(X_s=k\mid X_0=i)=\sum_k p_s(i,k)p_t(k,j) \\
&=(1+g(i,i)s)p_t(i,j)+\sum_{k:k\ne i}g(i,k)s\,p_t(k,j)+O(s^2).
\end{aligned} p t + s ( i , j ) = k ∑ P ( X t + s = j ∣ X s = k ) P ( X s = k ∣ X 0 = i ) = k ∑ p s ( i , k ) p t ( k , j ) = ( 1 + g ( i , i ) s ) p t ( i , j ) + k : k = i ∑ g ( i , k ) s p t ( k , j ) + O ( s 2 ) .
Subtracting p t ( i , j ) p_t(i,j) p t ( i , j ) from both sides, dividing by s s s and taking a limit as s s s goes to zero, we obtain
lim s ↓ 0 1 s ( p t + s ( i , j ) − p t ( i , j ) ) = d d t p t ( i , j ) = ∑ k g ( i , k ) p t ( k , j ) , \lim_{s\downarrow0}\frac{1}{s}\left(p_{t+s}(i,j)-p_t(i,j)\right)=\frac{\mathrm{d}}{\mathrm{d}t}p_t(i,j)=\sum_k g(i,k)p_t(k,j), s ↓ 0 lim s 1 ( p t + s ( i , j ) − p t ( i , j ) ) = d t d p t ( i , j ) = k ∑ g ( i , k ) p t ( k , j ) ,
which is the component-wise representation of (50) .
If P t \mathbf{P}_t P t solves the forward equation (49) then it also solves the backward equation (50) and vice versa. The solution to (49) and (50) , subject to the initial condition P 0 = I \mathbf{P}_0=\mathbf{I} P 0 = I is
P t = e t G : = ∑ n = 0 ∞ 1 n ! t n G n . \mathbf{P}_t=e^{t\mathbf{G}}:=\sum_{n=0}^{\infty}\frac{1}{n!}t^n\mathbf{G}^n. P t = e t G := n = 0 ∑ ∞ n ! 1 t n G n .
If G \mathbf{G} G has an eigenvector decomposition G = U − 1 Λ U \mathbf{G}=\mathbf{U}^{-1}\mathbf{\Lambda}\mathbf{U} G = U − 1 ΛU as in § 11.6 then
P t = U − 1 ( ∑ n = 0 ∞ 1 n ! ( t Λ ) n ) U = U − 1 e t Λ U . (51) \mathbf{P}_t=\mathbf{U}^{-1}\left(\sum_{n=0}^{\infty}\frac{1}{n!}(t\mathbf{\Lambda})^n\right)\mathbf{U}=\mathbf{U}^{-1}e^{t\mathbf{\Lambda}}\mathbf{U}. \tag*{(51)} P t = U − 1 ( n = 0 ∑ ∞ n ! 1 ( t Λ ) n ) U = U − 1 e t Λ U . (51)
Example 5.2.6. Let the generator G \mathbf{G} G of a continuous time Markov chain X X X be given by
G = ( − α α β − β ) , α , β > 0. (52) \mathbf{G}=\begin{pmatrix}-\alpha& \alpha\\ \beta& -\beta\end{pmatrix},\qquad\alpha,\beta>0. \tag*{(52)} G = ( − α β α − β ) , α , β > 0. (52) We wish to find the semigroup P t \mathbf{P}_t P t generated by G \mathbf{G} G . We note that G \mathbf{G} G has left eigenvectors and eigenvalues given by
λ 0 = 0 , e 0 = 1 α 2 + β 2 ( β , α ) , λ 1 = − α − β , e 1 = 1 2 ( 1 , − 1 ) . \lambda_0=0,\qquad\mathbf{e}_0=\frac{1}{\sqrt{\alpha^2+\beta^2}}(\beta,\alpha),\qquad\lambda_1=-\alpha-\beta,\qquad\mathbf{e}_1=\frac{1}{\sqrt{2}}(1,-1). λ 0 = 0 , e 0 = α 2 + β 2 1 ( β , α ) , λ 1 = − α − β , e 1 = 2 1 ( 1 , − 1 ) . Thus, from (51) we have
P t = U − 1 e t Λ U = ( e 0 e 1 ) − 1 ( e t λ 0 0 0 e t λ 1 ) ( e 0 e 1 ) \mathbf{P}_t=\mathbf{U}^{-1}e^{t\mathbf{\Lambda}}\mathbf{U}=
\begin{pmatrix}\mathbf{e}_0\\\mathbf{e}_1\end{pmatrix}^{-1}
\begin{pmatrix}e^{t\lambda_0}&0\\0&e^{t\lambda_1}\end{pmatrix}
\begin{pmatrix}\mathbf{e}_0\\\mathbf{e}_1\end{pmatrix} P t = U − 1 e t Λ U = ( e 0 e 1 ) − 1 ( e t λ 0 0 0 e t λ 1 ) ( e 0 e 1 ) = 1 α + β ( β + α e − ( α + β ) t α − α e − ( α + β ) t β − β e − ( α + β ) t α + β e − ( α + β ) t ) . (53) =\frac{1}{\alpha+\beta}
\begin{pmatrix}
\beta+\alpha e^{-(\alpha+\beta)t} & \alpha-\alpha e^{-(\alpha+\beta)t}\\
\beta-\beta e^{-(\alpha+\beta)t} & \alpha+\beta e^{-(\alpha+\beta)t}
\end{pmatrix}. \tag*{(53)} = α + β 1 ( β + α e − ( α + β ) t β − β e − ( α + β ) t α − α e − ( α + β ) t α + β e − ( α + β ) t ) . (53)
For a Poisson process with intensity λ \lambda λ , we found that we could construct N N N from a sequence of iid inter-arrival times ( τ i ) i ≥ 1 (\tau_i)_{i\ge1} ( τ i ) i ≥ 1 that were exponentially distributed: τ i ∼ E ( λ ) \tau_i \sim\mathcal{E}(\lambda) τ i ∼ E ( λ ) . A continuous time Markov chain can be constructed in a similar manner.
Let G = ( g ( i , j ) ) i , j ∈ S \mathbf{G}=(g(i,j))_{i,j\in S} G = ( g ( i , j ) ) i , j ∈ S be the generator of continuous time Markov chain. Consider a discrete time Markov chain Y = ( Y n ) n ∈ N 0 \mathbf{Y}=(Y_n)_{n\in\mathbb{N}_0} Y = ( Y n ) n ∈ N 0 with one-step transition matrix P = ( p ( i , j ) ) i , j ∈ S \mathbf{P}=(p(i,j))_{i,j\in S} P = ( p ( i , j ) ) i , j ∈ S . Suppose the entries of P \mathbf{P} P are given by
p ( i , j ) = − g ( i , j ) / g ( i , i ) , i ≠ j , p ( i , i ) = 0. p(i,j)=-g(i,j)/g(i,i),\qquad i\ne j,\qquad p(i,i)=0. p ( i , j ) = − g ( i , j ) / g ( i , i ) , i = j , p ( i , i ) = 0.
Observe that p ( i , j ) ≥ 0 ∀ i , j ∈ S p(i,j)\ge0\ \forall i,j\in S p ( i , j ) ≥ 0 ∀ i , j ∈ S and ∑ j p ( i , j ) = 1 ∀ i \sum_j p(i,j)=1\ \forall i ∑ j p ( i , j ) = 1 ∀ i as required. Now, let ( τ i ) i ∈ N (\tau_i)_{i\in\mathbb{N}} ( τ i ) i ∈ N be a sequence of independent random variables satisfying
τ n ∣ Y n ∼ E ( − g ( Y n , Y n ) ) . \tau_n\mid Y_n\sim\mathcal{E}(-g(Y_n,Y_n)). τ n ∣ Y n ∼ E ( − g ( Y n , Y n )) .
Let ( S n ) n ∈ N 0 (S_n)_{n\in\mathbb{N}_0} ( S n ) n ∈ N 0 be given by
S n : = ∑ i = 1 n τ i , S 0 : = 0. S_n:=\sum_{i=1}^{n}\tau_i,\qquad S_0:=0. S n := i = 1 ∑ n τ i , S 0 := 0.
We can construct a continuous time process X = ( X t ) t ≥ 0 \mathbf{X}=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 as follows
X t = ∑ n = 0 ∞ Y n 1 { S n ≤ t < S n + 1 } . X_t=\sum_{n=0}^{\infty}Y_n\mathbf{1}_{\{S_n\le t<S_{n+1}\}}. X t = n = 0 ∑ ∞ Y n 1 { S n ≤ t < S n + 1 } .
We claim that the process X \mathbf{X} X is a continuous time Markov chain with generator G \mathbf{G} G . We call ( S n ) n ∈ N 0 (S_n)_{n\in\mathbb{N}_0} ( S n ) n ∈ N 0 the jump times and ( τ n ) n ∈ N (\tau_n)_{n\in\mathbb{N}} ( τ n ) n ∈ N the inter-jump times . Compare this construction of a Markov chain to the second construction we gave for a Poisson process (see Theorem 5.1.5 and the text preceding it).
The above construction makes clear what the dynamics of a CTMC X \mathbf{X} X with generator G \mathbf{G} G are. Specifically, once X \mathbf{X} X jumps into state i i i , the amount of time it remains in that state is exponentially distributed with parameter − g ( i , i ) -g(i,i) − g ( i , i ) . When a jump occurs, the probability that X \mathbf{X} X jumps from i i i to j j j is − g ( i , j ) / g ( i , i ) -g(i,j)/g(i,i) − g ( i , j ) / g ( i , i ) .
To see how this construction is equivalent to Definition 5.2.4, note that, for small s s s , the probability of two jumps in the time interval [ 0 , s ) [0,s) [ 0 , s ) is O ( s 2 ) O(s^2) O ( s 2 ) because
P ( S 2 ≤ s ) ≤ P ( τ 1 ≤ s ) P ( τ 2 ≤ s ) ≤ ( 1 − e g s ) ( 1 − e g s ) = ( 1 − ( 1 + g s ) + O ( s 2 ) ) 2 = O ( s 2 ) . \mathbb{P}(S_2\le s)\le\mathbb{P}(\tau_1\le s)\mathbb{P}(\tau_2\le s)\le(1-e^{gs})(1-e^{gs})=(1-(1+gs)+O(s^2))^2=O(s^2). P ( S 2 ≤ s ) ≤ P ( τ 1 ≤ s ) P ( τ 2 ≤ s ) ≤ ( 1 − e g s ) ( 1 − e g s ) = ( 1 − ( 1 + g s ) + O ( s 2 ) ) 2 = O ( s 2 ) .
where we have defined g : = inf i ∈ S { − g ( i , i ) } g:=\inf_{i\in S}\{-g(i,i)\} g := inf i ∈ S { − g ( i , i )} . As the probability of two jumps is O ( s 2 ) O(s^2) O ( s 2 ) , for small s s s we have
p s ( i , i ) = P ( X s = i ∣ X 0 = i ) = P ( τ 1 > s ∣ X 0 = i ) + O ( s 2 ) = exp ( g ( i , i ) s ) + O ( s 2 ) = 1 + g ( i , i ) s + O ( s 2 ) , \begin{aligned}
p_s(i,i)=\mathbb{P}(X_s=i\mid X_0=i)=\mathbb{P}(\tau_1>s\mid X_0=i)+O(s^2) \\
&= \exp(g(i,i)s) + O(s^2) = 1 + g(i,i)s + O(s^2),
\end{aligned} p s ( i , i ) = P ( X s = i ∣ X 0 = i ) = P ( τ 1 > s ∣ X 0 = i ) + O ( s 2 ) = exp ( g ( i , i ) s ) + O ( s 2 ) = 1 + g ( i , i ) s + O ( s 2 ) ,
p s ( i , j ) = P ( X s = j ∣ X 0 = i ) = P ( τ 1 ≤ s ∣ X 0 = i ) ⋅ P ( X τ 1 = j ∣ X 0 = i ) + O ( s 2 ) = ( 1 − exp ( g ( i , i ) s ) ) ⋅ g ( i , j ) − g ( i , i ) + O ( s 2 ) = − g ( i , i ) s ⋅ g ( i , j ) − g ( i , i ) + O ( s 2 ) = g ( i , j ) s + O ( s 2 ) , \begin{aligned}
p_s(i,j) = \mathbb{P}(X_s=j\mid X_0=i) = \mathbb{P}(\tau_1\le s\mid X_0=i)\cdot\mathbb{P}(X_{\tau_1}=j\mid X_0=i) + O(s^2) \\
&= (1-\exp(g(i,i)s))\cdot\frac{g(i,j)}{-g(i,i)}+O(s^2) \\
&= -g(i,i)s\cdot\frac{g(i,j)}{-g(i,i)}+O(s^2)=g(i,j)s+O(s^2),
\end{aligned} p s ( i , j ) = P ( X s = j ∣ X 0 = i ) = P ( τ 1 ≤ s ∣ X 0 = i ) ⋅ P ( X τ 1 = j ∣ X 0 = i ) + O ( s 2 ) = ( 1 − exp ( g ( i , i ) s )) ⋅ − g ( i , i ) g ( i , j ) + O ( s 2 ) = − g ( i , i ) s ⋅ − g ( i , i ) g ( i , j ) + O ( s 2 ) = g ( i , j ) s + O ( s 2 ) ,
which agrees with Item 2 in Definition 5.2.4. The independent increments property X t − X s ⊥ ⊥ X s X_t-X_s\perp\!\!\!\perp X_s X t − X s ⊥ ⊥ X s follows from the memoryless property of the exponentially distributed waiting times.
For discrete time Markov chains, we found that μ n + m = μ n P m \boldsymbol{\mu}_{n+m}=\boldsymbol{\mu}_nP_m μ n + m = μ n P m . We defined an invariant distribution π \boldsymbol{\pi} π as any probability mass function that satisfied π = π P \boldsymbol{\pi}=\boldsymbol{\pi}P π = π P , which implied that π = π P n \boldsymbol{\pi}=\boldsymbol{\pi}P_n π = π P n for all n n n . In continuous time, we have the following analogs
μ t + s = μ t P s , ∀ t , s ≥ 0 , \boldsymbol{\mu}_{t+s}=\boldsymbol{\mu}_tP_s,\qquad\forall\ t,s\ge0, μ t + s = μ t P s , ∀ t , s ≥ 0 ,
π = π P t , ∀ t ≥ 0. \boldsymbol{\pi}=\boldsymbol{\pi}P_t,\qquad\forall\ t\ge0. π = π P t , ∀ t ≥ 0.
It is not always easy to find an invariant distribution π \boldsymbol{\pi} π from P t P_t P t . However, π \boldsymbol{\pi} π can also be obtained from the generator G \mathbf{G} G .
Theorem 5.2.7. Let X X X be a continuous time Markov chain with generator G \mathbf{G} G and a finite state space S S S , and let P t P_t P t be the semigroup generated by G \mathbf{G} G . Then
π = π P t ∀ t ≥ 0 , ⟺ π G = 0. \boldsymbol{\pi}=\boldsymbol{\pi}P_t\quad\forall\ t\ge0,\qquad\Longleftrightarrow\qquad\boldsymbol{\pi}\mathbf{G}=0. π = π P t ∀ t ≥ 0 , ⟺ π G = 0.
Proof. Recalling that G 0 ≡ I \mathbf{G}^0\equiv\mathbf{I} G 0 ≡ I we have
π G = 0 ⟺ π G n = 0 , ∀ n ≥ 1 , \boldsymbol{\pi}\mathbf{G}=0\qquad\Longleftrightarrow\qquad\boldsymbol{\pi}\mathbf{G}^n=0,\quad\forall\ n\ge1, π G = 0 ⟺ π G n = 0 , ∀ n ≥ 1 ,
⟺ ∑ n = 1 ∞ 1 n ! t n π G n = 0 \Longleftrightarrow\qquad\sum_{n=1}^{\infty}\frac{1}{n!}t^n\boldsymbol{\pi}\mathbf{G}^n=0 ⟺ n = 1 ∑ ∞ n ! 1 t n π G n = 0
⟺ π ∑ n = 0 ∞ 1 n ! t n G n = π \Longleftrightarrow\qquad\boldsymbol{\pi}\sum_{n=0}^{\infty}\frac{1}{n!}t^n\mathbf{G}^n=\boldsymbol{\pi} ⟺ π n = 0 ∑ ∞ n ! 1 t n G n = π
⟺ π P t = π . \Longleftrightarrow\qquad\boldsymbol{\pi}P_t=\boldsymbol{\pi}. ⟺ π P t = π .
Example 5.2.8. Let us return to Example 5.2.6 . Let us find the stationary distribution for the Markov process X X X with generator G \mathbf{G} G given by (52) . We have
0 = ( π ( 0 ) , π ( 1 ) ) ( − α α β − β ) , 1 = π ( 0 ) + π ( 1 ) , π ( i ) ≥ 0 , ⟹ π = ( β α + β , α α + β ) . 0=(\pi(0),\pi(1))
\begin{pmatrix}
-\alpha& \alpha\\
\beta& -\beta
\end{pmatrix},
\qquad
1=\pi(0)+\pi(1),\qquad
\pi(i)\ge0,
\qquad\Longrightarrow\qquad
\boldsymbol{\pi}=\left(\frac{\beta}{\alpha+\beta},\frac{\alpha}{\alpha+\beta}\right). 0 = ( π ( 0 ) , π ( 1 )) ( − α β α − β ) , 1 = π ( 0 ) + π ( 1 ) , π ( i ) ≥ 0 , ⟹ π = ( α + β β , α + β α ) .
Note that from (53) we have for any μ 0 \boldsymbol{\mu}_0 μ 0 that
lim t → ∞ μ t = μ 0 lim t → ∞ P t = ( μ 0 ( 0 ) , μ 0 ( 1 ) ) 1 α + β ( β α β α ) = π , \lim_{t\to\infty}\boldsymbol{\mu}_t=\boldsymbol{\mu}_0\lim_{t\to\infty}\boldsymbol{P}_t=(\mu_0(0),\mu_0(1))\frac{1}{\alpha+\beta}\begin{pmatrix}\beta& \alpha\\ \beta& \alpha\end{pmatrix}=\boldsymbol{\pi}, t → ∞ lim μ t = μ 0 t → ∞ lim P t = ( μ 0 ( 0 ) , μ 0 ( 1 )) α + β 1 ( β β α α ) = π ,
where we have used μ 0 ( 0 ) + μ 1 ( 1 ) = 1 \mu_0(0)+\mu_1(1)=1 μ 0 ( 0 ) + μ 1 ( 1 ) = 1 .
As in the discrete time case, for a given Markov chain X X X a stationary distribution π \boldsymbol{\pi} π may not exist. If a stationary distribution does exist, then it may not be unique. Given the existence of a stationary distribution, the condition for uniqueness is that
for all i , j ∈ S , there exists t > 0 such that p t ( i , j ) > 0. (54) \text{for all }i,j\in S,\text{ there exists }t>0\text{ such that }p_t(i,j)>0. \tag*{(54)} for all i , j ∈ S , there exists t > 0 such that p t ( i , j ) > 0. (54)
If (54) is satisfied, then we say the chain X X X is irreducible .
Example 5.2.9. Consider a continuous time Markov chain X X X with generator
G = ( − λ 0 λ 0 0 0 0 ⋯ μ 1 − ( μ 1 + λ 1 ) λ 1 0 0 ⋯ 0 μ 2 − ( μ 2 + λ 2 ) λ 2 0 ⋯ 0 0 μ 3 − ( μ 3 + λ 3 ) λ 3 ⋯ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱ ) . G=\begin{pmatrix}
-\lambda_0 & \lambda_0 & 0 & 0 & 0 & \cdots\\
\mu_1 & -(\mu_1+\lambda_1) & \lambda_1 & 0 & 0 & \cdots\\
0 & \mu_2 & -(\mu_2+\lambda_2) & \lambda_2 & 0 & \cdots\\
0 & 0 & \mu_3 & -(\mu_3+\lambda_3) & \lambda_3 & \cdots\\
\vdots& \vdots& \vdots& \vdots& \vdots& \ddots
\end{pmatrix}. G = − λ 0 μ 1 0 0 ⋮ λ 0 − ( μ 1 + λ 1 ) μ 2 0 ⋮ 0 λ 1 − ( μ 2 + λ 2 ) μ 3 ⋮ 0 0 λ 2 − ( μ 3 + λ 3 ) ⋮ 0 0 0 λ 3 ⋮ ⋯ ⋯ ⋯ ⋯ ⋱ . The chains X X X is called a birth-death process since, when X t = i X_t=i X t = i it either jumps to i + 1 i+1 i + 1 (i.e., a birth) or jumps down to i − 1 i-1 i − 1 (i.e., a death). The expected waiting time τ \tau τ in state i i i is exponentially distributed τ ∼ E ( λ i + μ i ) \tau\sim\mathcal{E}(\lambda_i+\mu_i) τ ∼ E ( λ i + μ i ) . When the process does jump out of state i i i , the probability of an up-jump is λ i / ( λ i + μ i ) \lambda_i/(\lambda_i+\mu_i) λ i / ( λ i + μ i ) and the probability of a down jump is μ i / ( λ i + μ i ) \mu_i/(\lambda_i+\mu_i) μ i / ( λ i + μ i ) . If λ 0 > 0 \lambda_0>0 λ 0 > 0 then the chain is irreducible. Let us see if we can find the stationary distribution π \boldsymbol{\pi} π . We see π \boldsymbol{\pi} π such that π G = 0 \boldsymbol{\pi}G=0 π G = 0 , which leads to
0 = − λ 0 π ( 0 ) + μ 1 π ( 1 ) , 0=-\lambda_0\pi(0)+\mu_1\pi(1), 0 = − λ 0 π ( 0 ) + μ 1 π ( 1 ) , 0 = λ n − 1 π ( n − 1 ) − ( μ n + λ n ) π ( n ) + μ n + 1 π ( n + 1 ) , n ≥ 1. 0=\lambda_{n-1}\pi(n-1)-(\mu_n+\lambda_n)\pi(n)+\mu_{n+1}\pi(n+1),\qquad n\geq1. 0 = λ n − 1 π ( n − 1 ) − ( μ n + λ n ) π ( n ) + μ n + 1 π ( n + 1 ) , n ≥ 1. One can check by direct substitution that
π ( n ) = λ 0 λ 1 ⋯ λ n − 1 μ 1 μ 2 ⋯ μ n π ( 0 ) . \pi(n)=\frac{\lambda_0\lambda_1\cdots\lambda_{n-1}}{\mu_1\mu_2\cdots\mu_n}\pi(0). π ( n ) = μ 1 μ 2 ⋯ μ n λ 0 λ 1 ⋯ λ n − 1 π ( 0 ) . In order for π \boldsymbol{\pi} π to be a distribution, the following normalization condition must be satisfied
1 = ∑ n = 0 ∞ π ( n ) = π ( 0 ) + π ( 0 ) ∑ n = 1 ∞ λ 0 λ 1 ⋯ λ n − 1 μ 1 μ 2 ⋯ μ n , ⇒ π ( 0 ) = ( 1 + ∑ n = 1 ∞ λ 0 λ 1 ⋯ λ n − 1 μ 1 μ 2 ⋯ μ n ) − 1 . 1=\sum_{n=0}^{\infty}\pi(n)=\pi(0)+\pi(0)\sum_{n=1}^{\infty}\frac{\lambda_0\lambda_1\cdots\lambda_{n-1}}{\mu_1\mu_2\cdots\mu_n},
\qquad\Rightarrow\qquad
\pi(0)=\left(1+\sum_{n=1}^{\infty}\frac{\lambda_0\lambda_1\cdots\lambda_{n-1}}{\mu_1\mu_2\cdots\mu_n}\right)^{-1}. 1 = n = 0 ∑ ∞ π ( n ) = π ( 0 ) + π ( 0 ) n = 1 ∑ ∞ μ 1 μ 2 ⋯ μ n λ 0 λ 1 ⋯ λ n − 1 , ⇒ π ( 0 ) = ( 1 + n = 1 ∑ ∞ μ 1 μ 2 ⋯ μ n λ 0 λ 1 ⋯ λ n − 1 ) − 1 . Clearly, we can only normalize if
∑ n = 1 ∞ λ 0 λ 1 ⋯ λ n − 1 μ 1 μ 2 ⋯ μ n < ∞ . \sum_{n=1}^{\infty}\frac{\lambda_0\lambda_1\cdots\lambda_{n-1}}{\mu_1\mu_2\cdots\mu_n}<\infty. n = 1 ∑ ∞ μ 1 μ 2 ⋯ μ n λ 0 λ 1 ⋯ λ n − 1 < ∞.
While a stationary distribution π \boldsymbol{\pi} π can often be computed with relative ease, it is almost always difficult (or impossible) to compute μ t \boldsymbol{\mu}_t μ t explicitly. Nevertheless, we can still obtain moments E X t n \mathbb{E}X_t^n E X t n if we can compute the generating function G X t ( s ) : = E s X t G_{X_t}(s):=\mathbb{E}s^{X_t} G X t ( s ) := E s X t .
Example 5.2.10. Consider the birth-death process given in Example 5.2.9 and assume
λ n = n λ , μ n = n μ , λ , μ > 0. \lambda_n=n\lambda,\qquad\mu_n=n\mu,\qquad\lambda,\mu>0. λ n = nλ , μ n = n μ , λ , μ > 0. Note that the state X t = 0 X_t=0 X t = 0 is an absorbing state since, λ 0 = 0 \lambda_0=0 λ 0 = 0 . Assume X 0 = i ≥ 1 X_0=i\geq1 X 0 = i ≥ 1 . The Kolmogorov forward equations (49) become
d d t p t ( i , 0 ) = μ p t ( i , 1 ) , \frac{\mathrm{d}}{\mathrm{d}t}p_t(i,0)=\mu p_t(i,1), d t d p t ( i , 0 ) = μ p t ( i , 1 ) , d d t p t ( i , j ) = λ ( j − 1 ) p t ( i , j − 1 ) − ( λ + μ ) j p t ( i , j ) + μ ( j + 1 ) p t ( i , j + 1 ) , j ≥ 1. \frac{\mathrm{d}}{\mathrm{d}t}p_t(i,j)=\lambda(j-1)p_t(i,j-1)-(\lambda+\mu)jp_t(i,j)+\mu(j+1)p_t(i,j+1),\qquad j\geq1. d t d p t ( i , j ) = λ ( j − 1 ) p t ( i , j − 1 ) − ( λ + μ ) j p t ( i , j ) + μ ( j + 1 ) p t ( i , j + 1 ) , j ≥ 1. Multiplying the j j j th equation by s j s^j s j and summing all of the equations, we obtain
∂ ∂ t G X t = λ s 2 ∂ G X t ∂ s − ( λ + μ ) s ∂ G X t ∂ s + μ ∂ G X t ∂ s , (55) \frac{\partial}{\partial t}G_{X_t}=\lambda s^2\frac{\partial G_{X_t}}{\partial s}-(\lambda+\mu)s\frac{\partial G_{X_t}}{\partial s}+\mu\frac{\partial G_{X_t}}{\partial s}, \tag*{(55)} ∂ t ∂ G X t = λ s 2 ∂ s ∂ G X t − ( λ + μ ) s ∂ s ∂ G X t + μ ∂ s ∂ G X t , (55) where we have used
G X t ( s ) : = E [ s X t ∣ X 0 = i ] = ∑ j = 0 ∞ s j p t ( i , j ) , G_{X_t}(s):=\mathbb{E}[s^{X_t}\mid X_0=i]=\sum_{j=0}^{\infty}s^jp_t(i,j), G X t ( s ) := E [ s X t ∣ X 0 = i ] = j = 0 ∑ ∞ s j p t ( i , j ) , from which it follows that
∂ G X t ( s ) ∂ t = ∑ j = 0 ∞ s j ∂ p t ( i , j ) ∂ t , ∂ G X t ( s ) ∂ s = ∑ j = 1 ∞ j s j − 1 p t ( i , j ) . \frac{\partial G_{X_t}(s)}{\partial t}=\sum_{j=0}^{\infty}s^j\frac{\partial p_t(i,j)}{\partial t},\qquad
\frac{\partial G_{X_t}(s)}{\partial s}=\sum_{j=1}^{\infty}js^{j-1}p_t(i,j). ∂ t ∂ G X t ( s ) = j = 0 ∑ ∞ s j ∂ t ∂ p t ( i , j ) , ∂ s ∂ G X t ( s ) = j = 1 ∑ ∞ j s j − 1 p t ( i , j ) . One can check by direct substitution that the solution to PDE (55) with initial condition G X 0 ( s ) = s i G_{X_0}(s)=s^i G X 0 ( s ) = s i is given by
G X t ( s ) = ( μ ( 1 − s ) − ( μ − λ s ) e − ( λ − μ ) t λ ( 1 − s ) − ( μ − λ s ) e − ( λ − μ ) t ) i , μ ≠ λ . G_{X_t}(s)=\left(\frac{\mu(1-s)-(\mu-\lambda s)e^{-(\lambda-\mu)t}}{\lambda(1-s)-(\mu-\lambda s)e^{-(\lambda-\mu)t}}\right)^i,\qquad\mu\ne\lambda. G X t ( s ) = ( λ ( 1 − s ) − ( μ − λ s ) e − ( λ − μ ) t μ ( 1 − s ) − ( μ − λ s ) e − ( λ − μ ) t ) i , μ = λ . From the generating function we can compute the first to moments of X t X_t X t easily. We have
E X t = i e ( λ − μ ) t , V X t = i λ + μ λ − μ e ( λ − μ ) t ( e ( λ − μ ) t − 1 ) , μ ≠ λ . \mathbb{E}X_t=ie^{(\lambda-\mu)t},\qquad
\mathop{\mathrm{V}}X_t=i\frac{\lambda+\mu}{\lambda-\mu}e^{(\lambda-\mu)t}\left(e^{(\lambda-\mu)t}-1\right),\qquad\mu\ne\lambda. E X t = i e ( λ − μ ) t , V X t = i λ − μ λ + μ e ( λ − μ ) t ( e ( λ − μ ) t − 1 ) , μ = λ .
Reversibility
In Section 4.5, we introduced a notion of reversibility for discrete time Markov chains. In this section, we will discuss reversibility of continuous time Markov chains. As we shall see, most of the definitions and theorems discussed below are direct analogs of the definitions and theorems given in Section 4.5.
Definition 5.3.1. Let X = ( X t ) 0 ≤ t ≤ T \mathbf{X}=(X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T be a continuous time Markov chain. The time reversal of X \mathbf{X} X is the process Y = ( Y t ) 0 ≤ t ≤ T \mathbf{Y}=(Y_t)_{0\leq t\leq T} Y = ( Y t ) 0 ≤ t ≤ T defined as follows
Y t : = lim δ ↓ 0 X T − t + δ = X ( T − t ) + . Y_t:=\lim_{\delta\downarrow0}X_{T-t+\delta}=X_{(T-t)+}. Y t := δ ↓ 0 lim X T − t + δ = X ( T − t ) + .
Compare Definition 5.3.1 for the time reversal of a continuous time Markov chain with Definition 4.5.1 for the time reversal of a discrete time Markov Chain. Notice the similarities.
Theorem 5.3.2. Let X = ( X t ) 0 ≤ t ≤ T \mathbf{X}=(X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T be an irreducible continuous time Markov chain with generator G \mathbf{G} G , transition semigroup P t = e t G \mathbf{P}_t=e^{t\mathbf{G}} P t = e t G and invariant distribution π \boldsymbol{\pi} π and suppose X 0 ∼ π X_0\sim\boldsymbol{\pi} X 0 ∼ π . Let Y = ( Y t ) 0 ≤ t ≤ T \mathbf{Y}=(Y_t)_{0\leq t\leq T} Y = ( Y t ) 0 ≤ t ≤ T be the time reversal of X \mathbf{X} X . Then Y \mathbf{Y} Y is a continuous time Markov chain with generator H \mathbf{H} H and transition semigroup Q t = e t H \mathbf{Q}_t=e^{t\mathbf{H}} Q t = e t H , which satisfy
h ( i , j ) = π ( j ) π ( i ) g ( j , i ) , q t ( i , j ) = π ( j ) π ( i ) p t ( j , i ) . (56) h(i,j)=\frac{\pi(j)}{\pi(i)}g(j,i),\qquad q_t(i,j)=\frac{\pi(j)}{\pi(i)}p_t(j,i). \tag*{(56)} h ( i , j ) = π ( i ) π ( j ) g ( j , i ) , q t ( i , j ) = π ( i ) π ( j ) p t ( j , i ) . (56)
Rather than prove Theorem 5.3.2 , we will explain intuitively why it is true. Fix a large N ∈ N N\in\mathbb{N} N ∈ N , define δ : = T / N ≪ 1 \delta:=T/N\ll1 δ := T / N ≪ 1 and consider the discrete-time processes X ^ = ( X ^ n ) 0 ≤ n ≤ N \widehat{\mathbf{X}}=(\widehat{X}_n)_{0\leq n\leq N} X = ( X n ) 0 ≤ n ≤ N where X ^ n : = X n δ \widehat{X}_n:=X_{n\delta} X n := X n δ . The process X ^ \widehat{\mathbf{X}} X is a discrete time Markov chain with transition probabilities p ^ ( i , j ) : = P ( X ^ n + 1 = j ∣ X ^ n = i ) \widehat{p}(i,j):=\mathbb{P}(\widehat{X}_{n+1}=j\mid\widehat{X}_n=i) p ( i , j ) := P ( X n + 1 = j ∣ X n = i ) given by
p ^ ( i , j ) = δ i , j + g ( i , j ) δ + O ( δ 2 ) . (57) \widehat{p}(i,j)=\delta_{i,j}+g(i,j)\delta+O(\delta^2). \tag*{(57)} p ( i , j ) = δ i , j + g ( i , j ) δ + O ( δ 2 ) . (57)
Now, let Y ^ = ( Y ^ n ) 0 ≤ n ≤ N \widehat{\mathbf{Y}}=(\widehat{Y}_n)_{0\leq n\leq N} Y = ( Y n ) 0 ≤ n ≤ N be the time-reversal of X ^ \widehat{\mathbf{X}} X . That is Y ^ n : = X ^ N − n = X T − n δ = Y n δ \widehat{Y}_n:=\widehat{X}_{N-n}=X_{T-n\delta}=Y_{n\delta} Y n := X N − n = X T − n δ = Y n δ . We know from Theorem 4.5.2 that Y ^ \widehat{\mathbf{Y}} Y is a Markov process with transition probabilities q ^ ( i , j ) : = P ( Y ^ n + 1 = j ∣ Y ^ n = i ) \widehat{q}(i,j):=\mathbb{P}(\widehat{Y}_{n+1}=j\mid\widehat{Y}_n=i) q ( i , j ) := P ( Y n + 1 = j ∣ Y n = i ) given by
q ^ ( i , j ) = π ( j ) π ( i ) p ^ ( j , i ) . (58) \widehat{q}(i,j)=\frac{\pi(j)}{\pi(i)}\widehat{p}(j,i). \tag*{(58)} q ( i , j ) = π ( i ) π ( j ) p ( j , i ) . (58)
If we accept that Y \mathbf{Y} Y , like its discrete-time counterpart Y ^ \widehat{\mathbf{Y}} Y , is a Markov process, then we must have
q ^ ( i , j ) = δ i , j + h ( i , j ) δ + O ( δ 2 ) . (59) \widehat{q}(i,j)=\delta_{i,j}+h(i,j)\delta+O(\delta^2). \tag*{(59)} q ( i , j ) = δ i , j + h ( i , j ) δ + O ( δ 2 ) . (59)
for some generating matrix H = ( h ( i , j ) ) i , j ∈ S \mathbf{H}=(h(i,j))_{i,j\in S} H = ( h ( i , j ) ) i , j ∈ S . Inserting (57) and (59) in (58) we obtain
δ i , j + h ( i , j ) δ = π ( j ) π ( i ) ( δ j , i + g ( j , i ) δ ) + O ( δ 2 ) \delta_{i,j}+h(i,j)\delta=\frac{\pi(j)}{\pi(i)}\left(\delta_{j,i}+g(j,i)\delta\right)+O(\delta^2) δ i , j + h ( i , j ) δ = π ( i ) π ( j ) ( δ j , i + g ( j , i ) δ ) + O ( δ 2 )
The terms of order O ( 1 ) O(1) O ( 1 ) cancel (you can check they are equal for both for i = j i=j i = j and for $i\ne j). In order for the O ( δ ) O(\delta) O ( δ ) terms to match, we must have
h ( i , j ) = π ( j ) π ( i ) g ( j , i ) , h(i,j)=\frac{\pi(j)}{\pi(i)}g(j,i), h ( i , j ) = π ( i ) π ( j ) g ( j , i ) ,
which is the left-hand side of equation (56) . Now, note that
π ( i ) h ( i , j ) = π ( j ) g ( j , i ) ⇒ π ( i ) ( H n ) i j = π ( j ) ( G n ) j i , ∀ n ∈ N 0 , \pi(i)h(i,j)=\pi(j)g(j,i)\quad\Rightarrow\quad\pi(i)(\mathbf{H}^{n})_{ij}=\pi(j)(\mathbf{G}^{n})_{ji},\qquad\forall n\in\mathbb{N}_{0}, π ( i ) h ( i , j ) = π ( j ) g ( j , i ) ⇒ π ( i ) ( H n ) ij = π ( j ) ( G n ) j i , ∀ n ∈ N 0 ,
which implies
π ( i ) ( e t H ) i j = π ( j ) ( e t G ) j i , ∀ t ∈ [ 0 , T ] . \pi(i)(e^{t\mathbf{H}})_{ij}=\pi(j)(e^{t\mathbf{G}})_{ji},\qquad\forall t\in[0,T]. π ( i ) ( e t H ) ij = π ( j ) ( e t G ) j i , ∀ t ∈ [ 0 , T ] .
The right-hand side of (56) follows from ( e t G ) i j = ( P t ) i , j = p t ( i , j ) (e^{t\mathbf{G}})_{ij}=(\mathbf{P}_{t})_{i,j}=p_{t}(i,j) ( e t G ) ij = ( P t ) i , j = p t ( i , j ) and ( e t H ) i j = ( Q t ) i j = q t ( i , j ) (e^{t\mathbf{H}})_{ij}=(\mathbf{Q}_{t})_{ij}=q_{t}(i,j) ( e t H ) ij = ( Q t ) ij = q t ( i , j ) .
Definition 5.3.3. Let X = ( X t ) 0 ≤ t ≤ T \mathbf{X}=(X_{t})_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T be an irreducible continuous-time Markov Chain with transition semigroup P t = e t G \mathbf{P}_{t}=e^{t\mathbf{G}} P t = e t G and invariant distribution π \boldsymbol{\pi} π . Suppose X 0 ∼ π X_{0}\sim\boldsymbol{\pi} X 0 ∼ π . Let Y Y Y be the time reversal of X X X and denote by Q t = e t H \mathbf{Q}_{t}=e^{t\mathbf{H}} Q t = e t H its time-reversal. We say that X X X is reversible if P t = Q t \mathbf{P}_{t}=\mathbf{Q}_{t} P t = Q t for all t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] , or equivalently (by Theorem 5.3.2 ), if either of the following holds
π ( i ) g ( i , j ) = π ( j ) g ( j , i ) , or π ( i ) p t ( i , j ) = π ( j ) p t ( j , i ) , ∀ t ∈ [ 0 , T ] . \pi(i)g(i,j)=\pi(j)g(j,i),\qquad\text{or}\qquad\pi(i)p_{t}(i,j)=\pi(j)p_{t}(j,i),\qquad\forall t\in[0,T]. π ( i ) g ( i , j ) = π ( j ) g ( j , i ) , or π ( i ) p t ( i , j ) = π ( j ) p t ( j , i ) , ∀ t ∈ [ 0 , T ] .
Once again, it is instructive to compare Definition 5.3.3 with its discrete time analog Definition 4.5.3.
Definition 5.3.4. Let P t = e t G \mathbf{P}_{t}=e^{t\mathbf{G}} P t = e t G the transition semigroup of a continuous time Markov process X X X with state space S S S . We say that a distribution λ = ( λ ( 1 ) , λ ( 2 ) , … , λ ( ∣ S ∣ ) ) \boldsymbol{\lambda}=(\lambda(1),\lambda(2),\ldots,\lambda(|S|)) λ = ( λ ( 1 ) , λ ( 2 ) , … , λ ( ∣ S ∣ )) is in detailed balance with P t \mathbf{P}_{t} P t or G \mathbf{G} G if
λ ( i ) g ( i , j ) = λ ( j ) g ( j , i ) , or λ ( i ) p t ( i , j ) = λ ( j ) p t ( j , i ) , ∀ t ∈ [ 0 , T ] . (60) \lambda(i)g(i,j)=\lambda(j)g(j,i),\qquad\text{or}\qquad\lambda(i)p_{t}(i,j)=\lambda(j)p_{t}(j,i),\qquad\forall t\in[0,T]. \tag*{(60)} λ ( i ) g ( i , j ) = λ ( j ) g ( j , i ) , or λ ( i ) p t ( i , j ) = λ ( j ) p t ( j , i ) , ∀ t ∈ [ 0 , T ] . (60)
Theorem 5.3.5. Let P t = e t G \mathbf{P}_{t}=e^{t\mathbf{G}} P t = e t G be transition semigroup of an irreducible Markov chain X X X . Suppose there exists a distribution λ \boldsymbol{\lambda} λ such that (60) holds. Then λ \boldsymbol{\lambda} λ is a stationary distribution for X X X . That is, λ G = 0 \boldsymbol{\lambda}\mathbf{G}=0 λ G = 0 and λ = λ P t \boldsymbol{\lambda}=\boldsymbol{\lambda}\mathbf{P}_{t} λ = λ P t for all t t t .
Proof. Assuming (60) holds, we have
∑ i λ ( i ) p t ( i , j ) = ∑ i λ ( j ) p t ( j , i ) = λ ( j ) ∑ i p t ( j , i ) = λ ( j ) . \sum_{i}\lambda(i)p_{t}(i,j)=\sum_{i}\lambda(j)p_{t}(j,i)=\lambda(j)\sum_{i}p_{t}(j,i)=\lambda(j). i ∑ λ ( i ) p t ( i , j ) = i ∑ λ ( j ) p t ( j , i ) = λ ( j ) i ∑ p t ( j , i ) = λ ( j ) .
Thus, we have proved λ = λ P t \boldsymbol{\lambda}=\boldsymbol{\lambda}\mathbf{P}_{t} λ = λ P t , and thus λ \boldsymbol{\lambda} λ is a stationary distribution for X X X .
Potential and entropy production
Definition 5.3.6. The generator G \mathbf{G} G of a continuous time Markov chain X X X is said to satisfy the Kolmogorov cycle condition if, for any finite and feasible cycle of states ( s 0 , s 1 , … , s n , s 0 ) (s_{0},s_{1},\ldots,s_{n},s_{0}) ( s 0 , s 1 , … , s n , s 0 ) we have
g ( s 0 , s 1 ) g ( s 1 , s 2 ) ⋯ g ( s n − 1 , s n ) g ( s n , s 0 ) g ( s 1 , s 0 ) g ( s 2 , s 1 ) ⋯ g ( s n , s n − 1 ) g ( s 0 , s n ) = 1. (61) \frac{g(s_{0},s_{1})g(s_{1},s_{2})\cdots g(s_{n-1},s_{n})g(s_{n},s_{0})}{g(s_{1},s_{0})g(s_{2},s_{1})\cdots g(s_{n},s_{n-1})g(s_{0},s_{n})}=1. \tag*{(61)} g ( s 1 , s 0 ) g ( s 2 , s 1 ) ⋯ g ( s n , s n − 1 ) g ( s 0 , s n ) g ( s 0 , s 1 ) g ( s 1 , s 2 ) ⋯ g ( s n − 1 , s n ) g ( s n , s 0 ) = 1. (61) Here, “feasible” means that X X X can jump directly from s k s_{k} s k to s k + 1 s_{k+1} s k + 1 for all k k k and also from s n s_{n} s n to s 0 s_{0} s 0 .
Theorem 5.3.7. Let X = ( X t ) 0 ≤ t ≤ T \mathbf{X}=(X_t)_{0\le t\le T} X = ( X t ) 0 ≤ t ≤ T be an irreducible continuous-time Markov Chain with state space S \mathrm{S} S , transition semigroup P t = e t G \mathbf{P}_t=e^{t\mathbf{G}} P t = e t G and stationary distribution π \boldsymbol{\pi} π . Then
π is in detailed balance with G ⟺ G satisfies the Kolmogorov cycle condition. \boldsymbol{\pi}\text{ is in detailed balance with }\mathbf{G}
\quad\Longleftrightarrow\quad
\mathbf{G}\text{ satisfies the Kolmogorov cycle condition.} π is in detailed balance with G ⟺ G satisfies the Kolmogorov cycle condition.
Proof. ( ⇒ ) (\Rightarrow) ( ⇒ ) Suppose π \boldsymbol{\pi} π is in detailed balance with G \mathbf{G} G , then π ( i ) g ( i , j ) = π ( j ) g ( j , i ) \pi(i)g(i,j)=\pi(j)g(j,i) π ( i ) g ( i , j ) = π ( j ) g ( j , i ) . Thus
1 = π ( s 0 ) g ( s 0 , s 1 ) π ( s 1 ) g ( s 1 , s 2 ) ⋯ π ( s n − 1 ) g ( s n − 1 , s n ) π ( s n ) g ( s n , s 0 ) π ( s 1 ) g ( s 1 , s 0 ) π ( s 2 ) g ( s 2 , s 1 ) ⋯ π ( s n ) g ( s n , s n − 1 ) π ( s 0 ) g ( s 0 , s n ) = g ( s 0 , s 1 ) g ( s 1 , s 2 ) ⋯ g ( s n − 1 , s n ) g ( s n , s 0 ) g ( s 1 , s 0 ) g ( s 2 , s 1 ) ⋯ g ( s n , s n − 1 ) g ( s 0 , s n ) . 1=\frac{\pi(s_0)g(s_0,s_1)\pi(s_1)g(s_1,s_2)\cdots\pi(s_{n-1})g(s_{n-1},s_n)\pi(s_n)g(s_n,s_0)}
{\pi(s_1)g(s_1,s_0)\pi(s_2)g(s_2,s_1)\cdots\pi(s_n)g(s_n,s_{n-1})\pi(s_0)g(s_0,s_n)}
=\frac{g(s_0,s_1)g(s_1,s_2)\cdots g(s_{n-1},s_n)g(s_n,s_0)}
{g(s_1,s_0)g(s_2,s_1)\cdots g(s_n,s_{n-1})g(s_0,s_n)}. 1 = π ( s 1 ) g ( s 1 , s 0 ) π ( s 2 ) g ( s 2 , s 1 ) ⋯ π ( s n ) g ( s n , s n − 1 ) π ( s 0 ) g ( s 0 , s n ) π ( s 0 ) g ( s 0 , s 1 ) π ( s 1 ) g ( s 1 , s 2 ) ⋯ π ( s n − 1 ) g ( s n − 1 , s n ) π ( s n ) g ( s n , s 0 ) = g ( s 1 , s 0 ) g ( s 2 , s 1 ) ⋯ g ( s n , s n − 1 ) g ( s 0 , s n ) g ( s 0 , s 1 ) g ( s 1 , s 2 ) ⋯ g ( s n − 1 , s n ) g ( s n , s 0 ) .
( ⇐ ) (\Leftarrow) ( ⇐ ) The Kolmogorov cycle condition (61) is equivalent to a conservative vector field:
0 = ∑ cycle { s k } log ( g ( s k , s k + 1 ) g ( s k + 1 , s k ) ) . 0=\sum_{\operatorname{cycle}\{s_k\}}\log\left(\frac{g(s_k,s_{k+1})}{g(s_{k+1},s_k)}\right). 0 = cycle { s k } ∑ log ( g ( s k + 1 , s k ) g ( s k , s k + 1 ) ) .
Therefore, we can define a potential function U : S → R \mathrm{U}:\mathrm{S}\to\mathbb{R} U : S → R relative to a reference state s 0 ∈ S s_0\in\mathrm{S} s 0 ∈ S via
U ( s j ) : = ∑ k = 0 j − 1 log ( g ( s k , s k + 1 ) g ( s k + 1 , s k ) ) . \mathrm{U}(s_j):=\sum_{k=0}^{j-1}\log\left(\frac{g(s_k,s_{k+1})}{g(s_{k+1},s_k)}\right). U ( s j ) := k = 0 ∑ j − 1 log ( g ( s k + 1 , s k ) g ( s k , s k + 1 ) ) .
From the potential function U ( s ) \mathrm{U}(s) U ( s ) , we can construct a distribution λ = ( λ ( 1 ) , λ ( 2 ) , … , λ ( ∣ S ∣ ) ) \boldsymbol{\lambda}=(\lambda(1),\lambda(2),\ldots,\lambda(|\mathrm{S}|)) λ = ( λ ( 1 ) , λ ( 2 ) , … , λ ( ∣ S ∣ )) for any state s ∈ S s\in\mathrm{S} s ∈ S as follows
λ ( s ) : = A e U ( s ) , A = ( ∑ s k ∈ S e U ( s k ) ) − 1 . \lambda(s):=A e^{\mathrm{U}(s)},\qquad A=\left(\sum_{s_k\in\mathrm{S}}e^{\mathrm{U}(s_k)}\right)^{-1}. λ ( s ) := A e U ( s ) , A = ( s k ∈ S ∑ e U ( s k ) ) − 1 .
Note that A A A is just a normalization factor. It is easy to check that λ ( i ) \lambda(i) λ ( i ) and λ ( j ) \lambda(j) λ ( j ) satisfy
λ ( i ) g ( i , j ) = λ ( j ) g ( j , i ) , \lambda(i)g(i,j)=\lambda(j)g(j,i), λ ( i ) g ( i , j ) = λ ( j ) g ( j , i ) ,
for any states i i i and j j j . Therefore, by Definition (5.3.4), λ \boldsymbol{\lambda} λ is in detailed balance with G \mathbf{G} G . Additionally, by Theorem (5.3.5), λ \boldsymbol{\lambda} λ is a stationary distribution for X \mathbf{X} X . As X \mathbf{X} X is irreducible, the stationary distribution is unique, which implies λ = π \boldsymbol{\lambda}=\boldsymbol{\pi} λ = π . Therefore, π \boldsymbol{\pi} π is in detailed balance with G \mathbf{G} G .
For any distributions μ \boldsymbol{\mu} μ and ν \boldsymbol{\nu} ν on some countable state space S \mathrm{S} S we can define
E n t r o p y : H [ μ ] : = − ∑ i ∈ S μ ( i ) log μ ( i ) , \mathit{Entropy}:\qquad H[\boldsymbol{\mu}]:=-\sum_{i\in\mathrm{S}}\mu(i)\log\mu(i), Entropy : H [ μ ] := − i ∈ S ∑ μ ( i ) log μ ( i ) ,
R e l a t i v e E n t r o p y : K [ μ ∥ ν ] : = ∑ i ∈ S μ ( i ) log μ ( i ) ν ( i ) . \mathit{Relative\ Entropy}:\qquad K[\boldsymbol{\mu}\Vert\boldsymbol{\nu}]:=\sum_{i\in\mathrm{S}}\mu(i)\log\frac{\mu(i)}{\nu(i)}. Relative Entropy : K [ μ ∥ ν ] := i ∈ S ∑ μ ( i ) log ν ( i ) μ ( i ) .
Relative Entropy is sometimes called Kullback–Leibler divergence .
Consider a physical system with N ≫ 1 N \gg1 N ≫ 1 particles, each of which must be in some state s ∈ S s \in S s ∈ S . Let us define
μ t ( i ) : = # { particles in state i at time t } N . \mu_t(i) := \frac{\#\{\text{particles in state }i\text{ at time }t\}}{N}. μ t ( i ) := N # { particles in state i at time t } .
Then μ t \mu_t μ t is a distribution on S S S . Suppose this system has an invariant distribution π \boldsymbol{\pi} π . We can compute the relative entropy of μ t \boldsymbol{\mu}_t μ t with respect to π \boldsymbol{\pi} π . We have
K [ μ t ∥ π ] = − ( − ∑ i ∈ S μ t ( i ) log μ t ( i ) + ( − ∑ i ∈ S μ t ( i ) log π ( i ) ) ) . K[\boldsymbol{\mu}_t\mathbin\Vert\boldsymbol{\pi}]
= -\left(-\sum_{i\in S}\mu_t(i)\log\mu_t(i)+\left(-\sum_{i\in S}\mu_t(i)\log\pi(i)\right)\right). K [ μ t ∥ π ] = − ( − i ∈ S ∑ μ t ( i ) log μ t ( i ) + ( − i ∈ S ∑ μ t ( i ) log π ( i ) ) ) .
where we have indicated the names that the above quantities are given in physics.
Theorem 5.3.8. Let X X X be an irreducible Markov chain with semigroup P t = e t G \mathbf{P}_t=e^{t\mathbf{G}} P t = e t G and invariant distribution π \boldsymbol{\pi} π . Suppose π \boldsymbol{\pi} π is in detailed balance with G \mathbf{G} G . Then we have
d d t K [ μ t ∥ π ] ≤ 0. \frac{\mathrm{d}}{\mathrm{d}t}K[\boldsymbol{\mu}_t\mathbin\Vert\boldsymbol{\pi}] \le0. d t d K [ μ t ∥ π ] ≤ 0. That is, the relative entropy K [ μ t ∥ π ] K[\boldsymbol{\mu}_t\mathbin\Vert\boldsymbol{\pi}] K [ μ t ∥ π ] is non-increasing.
Proof. Suppose X 0 ∼ δ i X_0\sim\delta_i X 0 ∼ δ i (i.e., all mass at i i i ). Then μ t ( j ) = p t ( i , j ) \mu_t(j)=p_t(i,j) μ t ( j ) = p t ( i , j ) and we have
d d t K [ μ t ∥ π ] = d d t ∑ j ∈ S μ t ( j ) log μ t ( j ) π ( j ) = ∑ j ∈ S ( 1 + log μ t ( j ) π ( j ) ) d d t μ t ( j ) = ∑ j ∈ S ( 1 + log p t ( i , j ) π ( j ) ) d d t p t ( i , j ) = d d t ∑ j ∈ S p t ( i , j ) + ∑ j ∈ S ( log p t ( i , j ) π ( j ) ) d d t p t ( i , j ) = ∑ j ∈ S ( log p t ( i , j ) π ( j ) ) d d t p t ( i , j ) ( as ∑ j ∈ S p t ( i , j ) = 1 ) = ∑ j ∈ S ( log p t ( i , j ) π ( j ) ) ∑ k ∈ S ( p t ( i , k ) g ( k , j ) ) (by the KFE) = ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) π ( j ) ) ( p t ( i , k ) g ( k , j ) ) = ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) π ( j ) ) ( p t ( i , k ) g ( k , j ) ) \begin{aligned}
\frac{\mathrm{d}}{\mathrm{d}t}K[\boldsymbol{\mu}_t\mathbin\Vert\boldsymbol{\pi}]
&=\frac{\mathrm{d}}{\mathrm{d}t}\sum_{j\in S}\mu_t(j)\log\frac{\mu_t(j)}{\pi(j)}\\
&=\sum_{j\in S}\left(1+\log\frac{\mu_t(j)}{\pi(j)}\right)\frac{\mathrm{d}}{\mathrm{d}t}\mu_t(j)\\
&=\sum_{j\in S}\left(1+\log\frac{p_t(i,j)}{\pi(j)}\right)\frac{\mathrm{d}}{\mathrm{d}t}p_t(i,j)\\
&=\frac{\mathrm{d}}{\mathrm{d}t}\sum_{j\in S}p_t(i,j)+\sum_{j\in S}\left(\log\frac{p_t(i,j)}{\pi(j)}\right)\frac{\mathrm{d}}{\mathrm{d}t}p_t(i,j)\\
&=\sum_{j\in S}\left(\log\frac{p_t(i,j)}{\pi(j)}\right)\frac{\mathrm{d}}{\mathrm{d}t}p_t(i,j)
\qquad\left(\text{as }\sum_{j\in S}p_t(i,j)=1\right)\\
&=\sum_{j\in S}\left(\log\frac{p_t(i,j)}{\pi(j)}\right)\sum_{k\in S}\left(p_t(i,k)g(k,j)\right)
\qquad\text{(by the KFE)}\\
&=\sum_{j\in S}\sum_{k\in S}\left(\log\frac{p_t(i,j)}{\pi(j)}\right)\left(p_t(i,k)g(k,j)\right)\\
&=\sum_{j\in S}\sum_{k\in S}\left(\log\frac{p_t(i,j)}{\pi(j)}\right)\left(p_t(i,k)g(k,j)\right)
\end{aligned} d t d K [ μ t ∥ π ] = d t d j ∈ S ∑ μ t ( j ) log π ( j ) μ t ( j ) = j ∈ S ∑ ( 1 + log π ( j ) μ t ( j ) ) d t d μ t ( j ) = j ∈ S ∑ ( 1 + log π ( j ) p t ( i , j ) ) d t d p t ( i , j ) = d t d j ∈ S ∑ p t ( i , j ) + j ∈ S ∑ ( log π ( j ) p t ( i , j ) ) d t d p t ( i , j ) = j ∈ S ∑ ( log π ( j ) p t ( i , j ) ) d t d p t ( i , j ) as j ∈ S ∑ p t ( i , j ) = 1 = j ∈ S ∑ ( log π ( j ) p t ( i , j ) ) k ∈ S ∑ ( p t ( i , k ) g ( k , j ) ) (by the KFE) = j ∈ S ∑ k ∈ S ∑ ( log π ( j ) p t ( i , j ) ) ( p t ( i , k ) g ( k , j ) ) = j ∈ S ∑ k ∈ S ∑ ( log π ( j ) p t ( i , j ) ) ( p t ( i , k ) g ( k , j ) )
+ ∑ k ∈ S ( log π ( k ) p t ( i , k ) ) p t ( i , k ) ∑ j ∈ S g ( k , j ) ∑ k ∈ S ( log π ( k ) p t ( i , k ) ) p t ( i , k ) ∑ j ∈ S g ( k , j ) ⏟ this term is zero since ∑ j ∈ S g ( k , j ) = 0 = ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) π ( k ) π ( j ) p t ( i , k ) ) ( p t ( i , k ) g ( k , j ) ) (algebra) = ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) g ( j , k ) p t ( i , k ) g ( k , j ) ) ( p t ( i , k ) g ( k , j ) ) ( π is in detailed balance with G ) = 1 2 ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) g ( j , k ) p t ( i , k ) g ( k , j ) ) × ( p t ( i , k ) g ( k , j ) + p t ( i , k ) g ( k , j ) ) = 1 2 ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) g ( j , k ) p t ( i , k ) g ( k , j ) ) × ( − p t ( i , j ) g ( j , k ) + p t ( i , k ) g ( k , j ) ) (algebra) = − 1 2 ∑ j ∈ S ∑ k ∈ S ( log p t ( i , j ) g ( j , k ) p t ( i , k ) g ( k , j ) ) ⏟ A × ( p t ( i , j ) g ( j , k ) − p t ( i , k ) g ( k , j ) ) ⏟ B = : f ( i ) ≤ 0 , (as A and B have the same sign) \begin{aligned}
&+ \sum_{k\in S}\left(\log\frac{\pi(k)}{p_t(i,k)}\right)p_t(i,k)\sum_{j\in S}g(k,j)\\
&\qquad\underbrace{\phantom{\sum_{k\in S}\left(\log\frac{\pi(k)}{p_t(i,k)}\right)p_t(i,k)\sum_{j\in S}g(k,j)}}_{\text{this term is zero since }\sum_{j\in S}g(k,j)=0}\\
&= \sum_{j\in S}\sum_{k\in S}\left(\log\frac{p_t(i,j)\pi(k)}{\pi(j)p_t(i,k)}\right)\left(p_t(i,k)g(k,j)\right) \qquad\text{(algebra)}\\
&= \sum_{j\in S}\sum_{k\in S}\left(\log\frac{p_t(i,j)g(j,k)}{p_t(i,k)g(k,j)}\right)\left(p_t(i,k)g(k,j)\right) \qquad\text{(}\pi\text{ is in detailed balance with }G\text{)}\\
&= \frac{1}{2}\sum_{j\in S}\sum_{k\in S}\left(\log\frac{p_t(i,j)g(j,k)}{p_t(i,k)g(k,j)}\right)\\
&\qquad\times\left(p_t(i,k)g(k,j)+p_t(i,k)g(k,j)\right)\\
&= \frac{1}{2}\sum_{j\in S}\sum_{k\in S}\left(\log\frac{p_t(i,j)g(j,k)}{p_t(i,k)g(k,j)}\right)\\
&\qquad\times\left(-p_t(i,j)g(j,k)+p_t(i,k)g(k,j)\right) \qquad\text{(algebra)}\\
&= -\frac{1}{2}\sum_{j\in S}\sum_{k\in S}\underbrace{\left(\log\frac{p_t(i,j)g(j,k)}{p_t(i,k)g(k,j)}\right)}_{A}\\
&\qquad\times\underbrace{\left(p_t(i,j)g(j,k)-p_t(i,k)g(k,j)\right)}_{B}\\
&=: f(i)\le0, \qquad\text{(as A and B have the same sign)}
\end{aligned} + k ∈ S ∑ ( log p t ( i , k ) π ( k ) ) p t ( i , k ) j ∈ S ∑ g ( k , j ) this term is zero since ∑ j ∈ S g ( k , j ) = 0 k ∈ S ∑ ( l o g p t ( i , k ) π ( k ) ) p t ( i , k ) j ∈ S ∑ g ( k , j ) = j ∈ S ∑ k ∈ S ∑ ( log π ( j ) p t ( i , k ) p t ( i , j ) π ( k ) ) ( p t ( i , k ) g ( k , j ) ) (algebra) = j ∈ S ∑ k ∈ S ∑ ( log p t ( i , k ) g ( k , j ) p t ( i , j ) g ( j , k ) ) ( p t ( i , k ) g ( k , j ) ) ( π is in detailed balance with G ) = 2 1 j ∈ S ∑ k ∈ S ∑ ( log p t ( i , k ) g ( k , j ) p t ( i , j ) g ( j , k ) ) × ( p t ( i , k ) g ( k , j ) + p t ( i , k ) g ( k , j ) ) = 2 1 j ∈ S ∑ k ∈ S ∑ ( log p t ( i , k ) g ( k , j ) p t ( i , j ) g ( j , k ) ) × ( − p t ( i , j ) g ( j , k ) + p t ( i , k ) g ( k , j ) ) (algebra) = − 2 1 j ∈ S ∑ k ∈ S ∑ A ( log p t ( i , k ) g ( k , j ) p t ( i , j ) g ( j , k ) ) × B ( p t ( i , j ) g ( j , k ) − p t ( i , k ) g ( k , j ) ) =: f ( i ) ≤ 0 , (as A and B have the same sign)
Thus, for a general starting condition X 0 ∼ μ 0 X_0\sim\mu_0 X 0 ∼ μ 0 we have
d d t K [ μ t ∥ π ] = ∑ i ∈ S μ 0 ( i ) f ( i ) ≤ 0. \frac{\mathrm{d}}{\mathrm{d}t}K[\mu_t\|\pi]=\sum_{i\in S}\mu_0(i)f(i)\le0. d t d K [ μ t ∥ π ] = i ∈ S ∑ μ 0 ( i ) f ( i ) ≤ 0.
In statistical physics, the term A is called the thermodynamic force , the term B is called thermodynamic flux , the product A⋅ \cdot ⋅ B is called entropy production or free energy dissipation and the result d d t K [ μ t ∥ π ] ≤ 0 \frac{\mathrm{d}}{\mathrm{d}t}K[\mu_t\|\pi]\le0 d t d K [ μ t ∥ π ] ≤ 0 is the Second Law of Thermodynamics.
Exercises
Exercise 5.1. Patients arrive at an emergency room as a Poisson process with intensity λ \lambda λ . The time to treat each patient is an independent exponential random variable with parameter μ \mu μ . Let X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 be the number of patients in the system (either being treated or waiting). Write down the generator of X X X .
Exercises
Show that X X X has an invariant distribution π \boldsymbol{\pi} π if and only if λ < μ \lambda< \mu λ < μ . Find π \boldsymbol{\pi} π . What is the total expected time (waiting + treatment) a patient waits when the system is in its invariant distribution?
Exercise 5.2. Let X = ( X t ) t ≥ 0 X = (X_t)_{t\geq0} X = ( X t ) t ≥ 0 be a Markov chain with stationary distribution π \boldsymbol{\pi} π . Let N N N be an independent Poisson process with intensity λ \lambda λ and denote by τ n \tau_n τ n the time of the n n n th arrival of N N N . Define Y n : = X τ n + Y_n := X_{\tau_n+} Y n := X τ n + (i.e., Y n Y_n Y n is the value of X X X immediately after the n n n th jump). Show that Y Y Y is a discrete time Markov chain with the same stationary distribution as X X X .
Exercise 5.3. Let X = ( X t ) t ≥ t X = (X_t)_{t\geq t} X = ( X t ) t ≥ t be a Markov chain with state space S = { 0 , 1 , 2 , … } S = \{0,1,2,\ldots\} S = { 0 , 1 , 2 , … } and with a generator G \mathbf{G} G whose i i i th row has entries
g i , i − 1 = i μ , g i , i = − i μ − λ , g i , i + 1 = λ g_{i,i-1} = i\mu,\qquad g_{i,i} = -i\mu-\lambda,\qquad g_{i,i+1} = \lambda g i , i − 1 = i μ , g i , i = − i μ − λ , g i , i + 1 = λ
with all other entries being zero (the zeroth row has only two entries: g 0 , 0 g_{0,0} g 0 , 0 and g 0 , 1 g_{0,1} g 0 , 1 ). Assume X 0 = j X_0 = j X 0 = j . Find G X t ( s ) : = E s X t G_{X_t}(s) := \mathbb{E}s^{X_t} G X t ( s ) := E s X t . What is the distribution of X t X_t X t as t → ∞ t \to\infty t → ∞ ?
Exercise 5.4. Let N N N be a time-inhomogeneous Poisson process with intensity function λ ( t ) \lambda(t) λ ( t ) . That is, the probability of a jumps of size one in the time interval ( t , t + d t ) (t,t+dt) ( t , t + d t ) is λ ( t ) d t \lambda(t)dt λ ( t ) d t and the probability of two jumps in that interval of time is O ( d t 2 ) O(dt^2) O ( d t 2 ) . Write down the Kolmogorov forward and backward equations of N N N and solve them. Let N 0 = 0 N_0 = 0 N 0 = 0 and let τ 1 \tau_1 τ 1 be the time of the first jump of N N N . If λ ( t ) = c / ( 1 + t ) \lambda(t) = c/(1+t) λ ( t ) = c / ( 1 + t ) show that E τ 1 < ∞ \mathbb{E}\tau_1 < \infty E τ 1 < ∞ if and only if c > 1 c > 1 c > 1 .
Exercise 5.5. Let N N N be a Poisson process with a random intensity Λ \Lambda Λ with is equal to λ 1 \lambda_1 λ 1 with probability p p p and λ 2 \lambda_2 λ 2 with probability 1 − p 1-p 1 − p . Find G N t ( s ) : = E s N t G_{N_t}(s) := \mathbb{E}s^{N_t} G N t ( s ) := E s N t . What is the mean and variance of N t N_t N t ?
Exercise 5.6. Let X = ( X t ) 0 ≤ t ≤ T X = (X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T be an irreducible continuous-time Markov Chain with state space S S S , transition semigroup P t = e t G \mathbf{P}_t = e^{t\mathbf{G}} P t = e t G and stationary distribution π \boldsymbol{\pi} π . Prove the following six statements are equivalent
(i) Its stationary distribution satisfies detailed balance: π ( i ) g ( i , j ) = π ( j ) g ( j , i ) \pi(i)g(i,j) = \pi(j)g(j,i) π ( i ) g ( i , j ) = π ( j ) g ( j , i ) , ∀ i , j ∈ S \forall i,j \in S ∀ i , j ∈ S
(ii) Any path connecting states i i i and j j j : i ≡ s 0 , s 1 , s 2 , … , s n ≡ j i \equiv s_0, s_1, s_2,\ldots,s_n \equiv j i ≡ s 0 , s 1 , s 2 , … , s n ≡ j , has a path independent
log ( g ( s 0 , s 1 ) g ( s 1 , s 0 ) ) + log ( g ( s 1 , s 2 ) g ( s 2 , s 1 ) ) + ⋯ + log ( g ( s n − 1 , s n ) g ( s n , s n − 1 ) ) = log π ( s n ) − log π ( s 0 ) \log\left(\frac{g(s_0,s_1)}{g(s_1,s_0)}\right) + \log\left(\frac{g(s_1,s_2)}{g(s_2,s_1)}\right) + \cdots+ \log\left(\frac{g(s_{n-1},s_n)}{g(s_n,s_{n-1})}\right) = \log\pi(s_n) - \log\pi(s_0) log ( g ( s 1 , s 0 ) g ( s 0 , s 1 ) ) + log ( g ( s 2 , s 1 ) g ( s 1 , s 2 ) ) + ⋯ + log ( g ( s n , s n − 1 ) g ( s n − 1 , s n ) ) = log π ( s n ) − log π ( s 0 )
(iii) It defines a time reversible stationary Markov process.
(iv) Its G \mathbf{G} G matrix satisfies Kolmogorov cycle condition for every sequence of states.
(v) There exists a positive diagonal matrix Π \boldsymbol{\Pi} Π such that matrix G Π \mathbf{G}\boldsymbol{\Pi} G Π is symmetric.
(vi) Its stationary process has zero entropy production rate. Entropy production rate, e p ( μ t ) e_p(\mu_t) e p ( μ t ) , is defined by
e p ( μ t ) = 1 2 ∑ i , j ( μ t ( i ) g ( i , j ) − μ t ( j ) g ( j , i ) ) log ( μ t ( i ) g ( i , j ) μ t ( j ) g ( j , i ) ) e_p(\mu_t) = \frac{1}{2}\sum_{i,j}\left(\mu_t(i)g(i,j)-\mu_t(j)g(j,i)\right)\log\left(\frac{\mu_t(i)g(i,j)}{\mu_t(j)g(j,i)}\right) e p ( μ t ) = 2 1 i , j ∑ ( μ t ( i ) g ( i , j ) − μ t ( j ) g ( j , i ) ) log ( μ t ( j ) g ( j , i ) μ t ( i ) g ( i , j ) )
Convergence of random variables
The notes from this chapter are primarily taken from (Grimmett and Stirzaker, 2001, Chapter 7). The goals of this chapter are (i) to introduce various modes of convergence of random variables and (ii) to establish when convergence of one mode implies convergence of another mode.
A motivating example
We begin this chapter with a motivating example.
Example 6.1.1. Consider the infinite sequence of coin tosses described in § 4.3.2 . Suppose you have an initial wealth X 0 X_0 X 0 . Just before the n th n^{\text{th}} n th coin toss, you bet your entire wealth on the outcome of the n th n^{\text{th}} n th toss being a heads. It is easy to see that, at time n n n your wealth is given by
X n ( ω ) = { 2 n X 0 if ω i = H for i = 1 , 2 , … , n , 0 else . X_n(\omega) =
\begin{cases}
2^n X_0 & \text{if } \omega_i = \mathrm{H} \text{ for } i = 1,2,\ldots,n, \\
0 & \text{else}.
\end{cases} X n ( ω ) = { 2 n X 0 0 if ω i = H for i = 1 , 2 , … , n , else . Suppose the probability of a heads is p ∈ ( 0 , 1 ) p \in(0,1) p ∈ ( 0 , 1 ) . As each of the coin tosses are independent, we deduce that
P ( X n = 2 n X 0 ) = p n , P ( X n = 0 ) = 1 − p n . \mathbb{P}(X_n = 2^n X_0) = p^n, \qquad\mathbb{P}(X_n = 0) = 1-p^n. P ( X n = 2 n X 0 ) = p n , P ( X n = 0 ) = 1 − p n . As p n → 0 p^n \to0 p n → 0 as n → ∞ n \to\infty n → ∞ we see that
lim n → ∞ P ( X n = 0 ) = lim n → ∞ 1 − p n = 1 \lim_{n\to\infty} \mathbb{P}(X_n = 0) = \lim_{n\to\infty} 1-p^n = 1 n → ∞ lim P ( X n = 0 ) = n → ∞ lim 1 − p n = 1 Thus, it seems reasonable to say X n → X ∞ X_n \to X_\infty X n → X ∞ where X ∞ ( ω ) : = 0 X_\infty(\omega) := 0 X ∞ ( ω ) := 0 . Note that
lim n → ∞ P ( ∣ X n − X ∞ ∣ > ε ) = 0 , ∀ ε > 0. (62) \lim_{n\to\infty} \mathbb{P}(|X_n-X_\infty|>\varepsilon)=0, \qquad\forall\ \varepsilon>0. \tag*{(62)} n → ∞ lim P ( ∣ X n − X ∞ ∣ > ε ) = 0 , ∀ ε > 0. (62)
From this computation one might expect that ∣ X n − X ∞ ∣ → 0 \lvert X_n-X_\infty\rvert\to0 ∣ X n − X ∞ ∣ → 0 as n → ∞ n\to\infty n → ∞ . However, this is not correct, as
E ∣ X n − X ∞ ∣ = E X n = 2 n X 0 ⋅ p n → { ∞ if p ∈ ( 1 / 2 , 1 ) , X 0 if p = 1 / 2 , 0 if p ∈ ( 0 , 1 / 2 ) . \mathbb{E}\lvert X_n-X_\infty\rvert=\mathbb{E}X_n=2^nX_0\cdot p^n\to
\begin{cases}
\infty& \text{if }p\in(1/2,1),\\
X_0 & \text{if }p=1/2,\\
0 & \text{if }p\in(0,1/2).
\end{cases} E ∣ X n − X ∞ ∣ = E X n = 2 n X 0 ⋅ p n → ⎩ ⎨ ⎧ ∞ X 0 0 if p ∈ ( 1/2 , 1 ) , if p = 1/2 , if p ∈ ( 0 , 1/2 ) .
Thus, we see that (62) does not imply E ∣ X n − X ∞ ∣ → 0 \mathbb{E}\lvert X_n-X_\infty\rvert\to0 E ∣ X n − X ∞ ∣ → 0 . This simple example forces us to consider more carefully what we mean by “convergence” of a random variable.
Modes of convergence defined
There are many (many!) ways to define convergence of a sequence of random variables. The most important of these modes are the four we define here
Definition 6.2.1. Consider a sequence of random variables ( X n ) n ≥ 0 (X_n)_{n\geq0} ( X n ) n ≥ 0 defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . We say that X n X_n X n converges to X ∞ X_\infty X ∞ almost surely , written X n → a . s . X ∞ X_n \xrightarrow{\mathrm{a.s.}} X_\infty X n a.s. X ∞ , if
P ( lim n → ∞ ∣ X n − X ∞ ∣ = 0 ) = 1. \mathbb{P}\left(\lim_{n\to\infty}\lvert X_n-X_\infty\rvert=0\right)=1. P ( n → ∞ lim ∣ X n − X ∞ ∣ = 0 ) = 1. We say that X n X_n X n converges to X ∞ X_\infty X ∞ in L p L^p L p , written X n → L p X ∞ X_n \xrightarrow{L^p} X_\infty X n L p X ∞ , if E ∣ X n ∣ p < ∞ \mathbb{E}\lvert X_n\rvert^p<\infty E ∣ X n ∣ p < ∞ for all n n n and
lim n → ∞ E ∣ X n − X ∞ ∣ p = 0. (63) \lim_{n\to\infty}\mathbb{E}\lvert X_n-X_\infty\rvert^p=0. \tag*{(63)} n → ∞ lim E ∣ X n − X ∞ ∣ p = 0. (63) We say that X n X_n X n converges to X ∞ X_\infty X ∞ in probability , written X n → P X ∞ X_n \xrightarrow{\mathbb{P}} X_\infty X n P X ∞ , if
lim n → ∞ P ( ∣ X n − X ∞ ∣ > ε ) = 0 , ∀ ε > 0. \lim_{n\to\infty}\mathbb{P}(\lvert X_n-X_\infty\rvert>\varepsilon)=0,\qquad\forall\ \varepsilon>0. n → ∞ lim P (∣ X n − X ∞ ∣ > ε ) = 0 , ∀ ε > 0. And we say that X n X_n X n converges to X ∞ X_\infty X ∞ in distribution , written X n → D X ∞ X_n\xrightarrow{\mathcal{D}}X_\infty X n D X ∞ , if
lim n → ∞ ∣ F X n ( x ) − F X ∞ ( x ) ∣ = 0 , \lim_{n\to\infty}\lvert F_{X_n}(x)-F_{X_\infty}(x)\rvert=0, n → ∞ lim ∣ F X n ( x ) − F X ∞ ( x )∣ = 0 , for all points x ∈ R x\in\mathbb{R} x ∈ R at which F X ∞ F_{X_\infty} F X ∞ is continuous.
It will be helpful to comment a bit more on the above definitions. Observe that almost sure convergence does not require that X n ( ω ) → X ∞ ( ω ) X_n(\omega)\to X_\infty(\omega) X n ( ω ) → X ∞ ( ω ) for every ω ∈ Ω \omega\in\Omega ω ∈ Ω . It may be helpful to define
A : = { ω : lim n → ∞ ∣ X n ( ω ) − X ∞ ( ω ) ∣ = 0 } . A:=\{\omega:\lim_{n\to\infty}\lvert X_n(\omega)-X_\infty(\omega)\rvert=0\}. A := { ω : n → ∞ lim ∣ X n ( ω ) − X ∞ ( ω )∣ = 0 } .
Almost sure convergence simply asks that P ( A ) = 1 \mathbb{P}(A)=1 P ( A ) = 1 ; it does not ask that A = Ω A=\Omega A = Ω . There may be infinitely many ω ∈ A c \omega\in A^c ω ∈ A c (uncountably infinite, in fact). But, if X n → a . s . X ∞ X_n \xrightarrow{\mathrm{a.s.}} X_\infty X n a.s. X ∞ then P ( A c ) = 0 \mathbb{P}(A^c)=0 P ( A c ) = 0 . At times, it is helpful to write that X n → X ∞ X_n\to X_\infty X n → X ∞ P \mathbb{P} P -a.s. to indicate that X n X_n X n converges to X ∞ X_\infty X ∞ almost surely under a specific probability measure P \mathbb{P} P . Note that, that X n → X ∞ X_n\to X_\infty X n → X ∞ P \mathbb{P} P -a.s. does not imply that X n → X ∞ X_n\to X_\infty X n → X ∞ P ~ \widetilde{\mathbb{P}} P -a.s.. However, if two measure are equivalent P ∼ P ~ \mathbb{P}\sim\widetilde{\mathbb{P}} P ∼ P then almost sure convergence under P \mathbb{P} P implies almost sure convergence under P ~ \widetilde{\mathbb{P}} P and vice-versa because P ( A ) = 1 ⇔ P ~ ( A ) = 1 \mathbb{P}(A)=1\Leftrightarrow\widetilde{\mathbb{P}}(A)=1 P ( A ) = 1 ⇔ P ( A ) = 1 . Finally, we mention that the phrases convergence with probability one and convergence almost everywhere (a.e.) are sometimes used to indicate almost sure convergence.
For any p ≥ 1 p\geq1 p ≥ 1 we can define L p ( Ω , F , P ) L^p(\Omega,\mathcal{F},\mathbb{P}) L p ( Ω , F , P ) , the space of random variables X : Ω → R X:\Omega\to\mathbb{R} X : Ω → R with a finite p p p th moment.
E ∣ X ∣ p < ∞ . \mathbb{E}|X|^p<\infty. E ∣ X ∣ p < ∞.
We can define the p p p -norm ∥ ⋅ ∥ p \|\cdot\|_p ∥ ⋅ ∥ p of a random variable X ∈ L p ( Ω , F , P ) X\in L^p(\Omega,\mathcal{F},\mathbb{P}) X ∈ L p ( Ω , F , P ) as
∥ X ∥ p : = ( E ∣ X ∣ p ) 1 / p . \|X\|_p:=\left(\mathbb{E}|X|^p\right)^{1/p}. ∥ X ∥ p := ( E ∣ X ∣ p ) 1/ p .
Then, from (63) we see that convergence in L p L^p L p is equivalent to convergence in the p p p -norm
lim n → ∞ ∥ X n − X ∞ ∥ p = 0. \lim_{n\to\infty}\|X_n-X_\infty\|_p=0. n → ∞ lim ∥ X n − X ∞ ∥ p = 0.
As with almost sure convergence, it is sometimes necessary to specify a particular probability measure under which L p L^p L p convergence occurs by writing X n → X ∞ X_n\to X_\infty X n → X ∞ in L p ( P ) L^p(\mathbb{P}) L p ( P ) or, even more specifically, L p ( Ω , F , P ) L^p(\Omega,\mathcal{F},\mathbb{P}) L p ( Ω , F , P ) . It is common to say X n → X ∞ X_n\to X_\infty X n → X ∞ in mean and in mean square when X n → X ∞ X_n\to X_\infty X n → X ∞ in L 1 L^1 L 1 and L 2 L^2 L 2 , respectively.
Convergence in probability is based on the following intuition: two random variables X X X and Y Y Y are “close to each other” if there is a low probability that their difference is very large: P ( ∣ X − Y ∣ > ε ) ≈ 0 \mathbb{P}(|X-Y|>\varepsilon)\approx0 P ( ∣ X − Y ∣ > ε ) ≈ 0 for ε ≪ 1 \varepsilon\ll1 ε ≪ 1 . In some sense, the “distance” between X X X and Y Y Y is measured by d ε ( X , Y ) : = P ( ∣ X − Y ∣ > ε ) d_\varepsilon(X,Y):=\mathbb{P}(|X-Y|>\varepsilon) d ε ( X , Y ) := P ( ∣ X − Y ∣ > ε ) . Convergence in probability simply asks that this measure of distance goes to zero for any ε > 0 \varepsilon>0 ε > 0 as n → ∞ n\to\infty n → ∞ . That is,
lim n → ∞ d ε ( X n , X ∞ ) = 0. \lim_{n\to\infty}d_\varepsilon(X_n,X_\infty)=0. n → ∞ lim d ε ( X n , X ∞ ) = 0.
Note that we have already defined convergence in distribution (see Definition 3.4.1 ). We have simply repeated the definition here for convenience. In a sense, convergence in distribution is the weakest of the four modes of convergence, as the definition makes no reference to the underlying probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Indeed, suppose Z ∼ N ( 0 , 1 ) Z\sim\mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) and X n = − Z X_n=-Z X n = − Z for all n n n . Then it is clear that F X n = F Z F_{X_n}=F_Z F X n = F Z for all n n n . But, it is easy to see that X n X_n X n does not converge to Z Z Z in probability, in L p L^p L p or almost surely. Convergence in distribution is sometimes alternatively called weak convergence , written X n → w X ∞ X_n\xrightarrow{\mathrm{w}}X_\infty X n w X ∞ , or convergence in law , written X n → L X ∞ X_n\xrightarrow{\mathcal{L}}X_\infty X n L X ∞ .
Sometimes, we may wish to see if a sequence of random variables ( X n ) (X_n) ( X n ) converges (in some sense) to a random variable X ∞ X_\infty X ∞ , but we do not know what the random variable X ∞ X_\infty X ∞ is. In such cases, it can be helpful to check for Cauchy convergence of some sort.
Definition 6.2.2. Consider a sequence of random variables ( X n ) n ≥ 0 (X_n)_{n\geq0} ( X n ) n ≥ 0 defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . We say that ( X n ) (X_n) ( X n ) is Cauchy convergent almost surely if
P ( lim n , m → ∞ ∣ X n − X m ∣ = 0 ) = 1. \mathbb{P}\left(\lim_{n,m\to\infty}|X_n-X_m|=0\right)=1. P ( n , m → ∞ lim ∣ X n − X m ∣ = 0 ) = 1. We say that ( X n ) (X_n) ( X n ) is Cauchy convergent in L p L^p L p if E ∣ X n ∣ p < ∞ \mathbb{E}|X_n|^p<\infty E ∣ X n ∣ p < ∞ for all n n n and
lim n , m → ∞ E ∣ X n − X m ∣ p = 0. \lim_{n,m\to\infty}\mathbb{E}|X_n-X_m|^p=0. n , m → ∞ lim E ∣ X n − X m ∣ p = 0. And we say that X n X_n X n is Cauchy convergent in probability if
lim n , m → ∞ P ( ∣ X n − X m ∣ > ε ) = 0 , ∀ ε > 0. \lim_{n,m\to\infty}\mathbb{P}(|X_n-X_m|>\varepsilon)=0,\qquad\forall\varepsilon>0. n , m → ∞ lim P ( ∣ X n − X m ∣ > ε ) = 0 , ∀ ε > 0.
We will not prove the following Theorem.
Theorem 6.2.3. Consider a sequence of random variables ( X n ) (X_n) ( X n ) defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let X ∞ X_\infty X ∞ be a random variable defined on the same probability space.
∃ X ∞ s.t. X n → a . s . X ∞ ⟺ ( X n ) is Cauchy convergent almost surely. \exists X_\infty\ \text{s.t.}\ X_n \xrightarrow{\mathrm{a.s.}} X_\infty
\quad\Longleftrightarrow\quad
(X_n)\ \text{is Cauchy convergent almost surely.} ∃ X ∞ s.t. X n a.s. X ∞ ⟺ ( X n ) is Cauchy convergent almost surely. ∃ X ∞ s.t. X n → L p X ∞ ⟺ ( X n ) is Cauchy convergent in L p . \exists X_\infty\ \text{s.t.}\ X_n \xrightarrow{L^p} X_\infty
\quad\Longleftrightarrow\quad
(X_n)\ \text{is Cauchy convergent in }L^p. ∃ X ∞ s.t. X n L p X ∞ ⟺ ( X n ) is Cauchy convergent in L p . ∃ X ∞ s.t. X n → P X ∞ ⟺ ( X n ) is Cauchy convergent in probability. \exists X_\infty\ \text{s.t.}\ X_n \xrightarrow{\mathbb{P}} X_\infty
\quad\Longleftrightarrow\quad
(X_n)\ \text{is Cauchy convergent in probability.} ∃ X ∞ s.t. X n P X ∞ ⟺ ( X n ) is Cauchy convergent in probability.
The usefulness of Theorem 6.2.3 is that it allows us to establish the existence of a limit X ∞ X_\infty X ∞ of a sequence of random variables ( X n ) (X_n) ( X n ) (in some sense) without having knowledge of what X ∞ X_\infty X ∞ is.
In Example 6.1.1 we saw that the wealth process ( X n ) (X_n) ( X n ) converged in probability to zero: X n → P 0 X_n \xrightarrow{\mathbb{P}} 0 X n P 0 . Suppose P \mathbb{P} P was such that P ( ω i = H ) = p ∈ ( 0 , 1 / 2 ) \mathbb{P}(\omega_i=H)=p\in(0,1/2) P ( ω i = H ) = p ∈ ( 0 , 1/2 ) . Then the wealth process ( X n ) (X_n) ( X n ) also converged in mean to zero: X n → 0 X_n\to0 X n → 0 in L 1 ( P ) L^1(\mathbb{P}) L 1 ( P ) . However, if P ~ ( ω i = H ) = p ~ ∈ [ 1 / 2 , 1 ) \widetilde{\mathbb{P}}(\omega_i=H)=\widetilde{p}\in[1/2,1) P ( ω i = H ) = p ∈ [ 1/2 , 1 ) , then ( X n ) (X_n) ( X n ) did not converge in L 1 ( P ~ ) L^1(\widetilde{\mathbb{P}}) L 1 ( P ) . It would be very helpful to know when one mode of convergence implies convergence in another mode. This is the subject of the next section.
Convergence in one mode implies convergence in another?
Let us get right to it and state the main result.
Theorem 6.3.1. Let ( X n ) (X_n) ( X n ) be a sequence of random variables defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let X ∞ X_\infty X ∞ be a random variable on the same probability space. Suppose 1 ≤ q ≤ p 1 \le q \le p 1 ≤ q ≤ p . Then the following implications hold
X n → a . s . X ∞ X n → L p X ∞ ⇒ X n → L q X ∞ ⇒ X n → P X ∞ ⇒ X n → D X ∞ . \begin{aligned}
X_n &\xrightarrow{\mathrm{a.s.}} X_\infty\\
X_n &\xrightarrow{L^p} X_\infty\Rightarrow X_n \xrightarrow{L^q} X_\infty
\end{aligned}
\quad\Rightarrow\quad
X_n \xrightarrow{\mathbb{P}} X_\infty\Rightarrow X_n \xrightarrow{\mathcal{D}} X_\infty. X n X n a.s. X ∞ L p X ∞ ⇒ X n L q X ∞ ⇒ X n P X ∞ ⇒ X n D X ∞ . No other implications hold in general.
Note: the right bracket “} \} } ” indicates that convergence in probability holds if X n X_n X n converges to X ∞ X_\infty X ∞ either in L q L^q L q or P \mathbb{P} P -a.s.. One does not need for X n X_n X n to converge to X ∞ X_\infty X ∞ in both L q L^q L q and P \mathbb{P} P -a.s. in order for convergence in probability to hold. We will prove Theorem 6.3.1 in a series of steps. Along the way, we will introduce some related (and useful!) inequalities.
Proof that X n → P X ∞ ⇒ X n → D X ∞ X_n \xrightarrow{\mathbb{P}} X_\infty\Rightarrow X_n \xrightarrow{\mathcal{D}} X_\infty X n P X ∞ ⇒ X n D X ∞ . First we note that, if ε > 0 \varepsilon> 0 ε > 0 , then
F X n ( x ) = P ( X n ≤ x ) = P ( X n ≤ x , X ∞ ≤ x + ε ) + P ( X n ≤ x , X ∞ > x + ε ) ≤ P ( X ∞ ≤ x + ε ) + P ( ∣ X n − X ∞ ∣ > ε ) = F X ∞ ( x + ε ) + P ( ∣ X n − X ∞ ∣ > ε ) . (64) \begin{aligned}
F_{X_n}(x) &= \mathbb{P}(X_n \le x) \\
&= \mathbb{P}(X_n \le x, X_\infty\le x+\varepsilon) + \mathbb{P}(X_n \le x, X_\infty> x+\varepsilon) \\
&\le\mathbb{P}(X_\infty\le x+\varepsilon) + \mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon) \\
&= F_{X_\infty}(x+\varepsilon) + \mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon). \tag*{(64)}
\end{aligned} F X n ( x ) = P ( X n ≤ x ) = P ( X n ≤ x , X ∞ ≤ x + ε ) + P ( X n ≤ x , X ∞ > x + ε ) ≤ P ( X ∞ ≤ x + ε ) + P (∣ X n − X ∞ ∣ > ε ) = F X ∞ ( x + ε ) + P (∣ X n − X ∞ ∣ > ε ) . (64)
If you find the above inequality difficult to derive, it may help to draw a pictures of regions in the X n X_n X n -X ∞ X_\infty X ∞ plane to which the above probabilities correspond. Likewise, we have
F X ∞ ( x − ε ) = P ( X ∞ ≤ x − ε ) = P ( X ∞ ≤ x − ε , X n ≤ x ) + P ( X ∞ ≤ x − ε , X n > x ) ≤ P ( X n ≤ x ) + P ( ∣ X n − X ∞ ∣ > ε ) = F X n ( x ) + P ( ∣ X n − X ∞ ∣ > ε ) . (65) \begin{aligned}
F_{X_\infty}(x-\varepsilon) &= \mathbb{P}(X_\infty\le x-\varepsilon) \\
&= \mathbb{P}(X_\infty\le x-\varepsilon, X_n \le x) + \mathbb{P}(X_\infty\le x-\varepsilon, X_n > x) \\
&\le\mathbb{P}(X_n \le x) + \mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon) \\
&= F_{X_n}(x) + \mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon). \tag*{(65)}
\end{aligned} F X ∞ ( x − ε ) = P ( X ∞ ≤ x − ε ) = P ( X ∞ ≤ x − ε , X n ≤ x ) + P ( X ∞ ≤ x − ε , X n > x ) ≤ P ( X n ≤ x ) + P (∣ X n − X ∞ ∣ > ε ) = F X n ( x ) + P (∣ X n − X ∞ ∣ > ε ) . (65)
Combining (64) with (65) we obtain
F X ∞ ( x − ε ) − P ( ∣ X n − X ∞ ∣ > ε ) ≤ F X n ( x ) ≤ F X ∞ ( x + ε ) + P ( ∣ X n − X ∞ ∣ > ε ) . F_{X_\infty}(x-\varepsilon)-\mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon)
\le F_{X_n}(x)
\le F_{X_\infty}(x+\varepsilon)+\mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon). F X ∞ ( x − ε ) − P (∣ X n − X ∞ ∣ > ε ) ≤ F X n ( x ) ≤ F X ∞ ( x + ε ) + P (∣ X n − X ∞ ∣ > ε ) .
As X n → P X ∞ X_n \xrightarrow{\mathbb{P}} X_\infty X n P X ∞ by assumption, we have that lim n → ∞ P ( ∣ X n − X ∞ ∣ > ε ) = 0 \lim_{n\to\infty}\mathbb{P}(\lvert X_n-X_\infty\rvert> \varepsilon)=0 lim n → ∞ P (∣ X n − X ∞ ∣ > ε ) = 0 . Thus
F X ∞ ( x − ε ) ≤ lim inf n → ∞ F X n ( x ) ≤ lim sup n → ∞ F X n ( x ) ≤ F X ∞ ( x + ε ) . F_{X_\infty}(x-\varepsilon)
\le\liminf_{n\to\infty} F_{X_n}(x)
\le\limsup_{n\to\infty} F_{X_n}(x)
\le F_{X_\infty}(x+\varepsilon). F X ∞ ( x − ε ) ≤ n → ∞ lim inf F X n ( x ) ≤ n → ∞ lim sup F X n ( x ) ≤ F X ∞ ( x + ε ) .
Finally, if F X ∞ ( x ) F_{X_\infty}(x) F X ∞ ( x ) is continuous at x x x , then lim ε → 0 F X ∞ ( x ± ε ) = F X ∞ ( x ) \lim_{\varepsilon\to0}F_{X_\infty}(x\pm\varepsilon)=F_{X_\infty}(x) lim ε → 0 F X ∞ ( x ± ε ) = F X ∞ ( x ) . Therefore, letting ε → 0 \varepsilon\to0 ε → 0 we obtain
F X ∞ ( x ) ≤ lim inf n → ∞ F X n ( x ) ≤ lim sup n → ∞ F X n ( x ) ≤ F X ∞ ( x ) , F_{X_\infty}(x)
\le\liminf_{n\to\infty} F_{X_n}(x)
\le\limsup_{n\to\infty} F_{X_n}(x)
\le F_{X_\infty}(x), F X ∞ ( x ) ≤ n → ∞ lim inf F X n ( x ) ≤ n → ∞ lim sup F X n ( x ) ≤ F X ∞ ( x ) ,
which implies that lim n → ∞ F X n ( x ) = F X ∞ ( x ) \lim_{n\to\infty}F_{X_n}(x)=F_{X_\infty}(x) lim n → ∞ F X n ( x ) = F X ∞ ( x ) at any point x x x where F X ∞ F_{X_\infty} F X ∞ is continuous. And this establishes that X n → D X ∞ X_n \xrightarrow{\mathcal{D}} X_\infty X n D X ∞ .
We shall need the following result, which we state without proof.
Theorem 6.3.2 (Hölder’s inequality). Let X X X and Y Y Y be random variables defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let p , q ≥ 1 p,q\ge1 p , q ≥ 1 satisfy 1 / p + 1 / q = 1 1/p+1/q=1 1/ p + 1/ q = 1 . Then we have
E ∣ X Y ∣ ≤ ( E ∣ X ∣ p ) 1 / p ( E ∣ Y ∣ q ) 1 / q . (66) \mathbb{E}|XY|\leq\left(\mathbb{E}|X|^p\right)^{1/p}\left(\mathbb{E}|Y|^q\right)^{1/q}. \tag*{(66)} E ∣ X Y ∣ ≤ ( E ∣ X ∣ p ) 1/ p ( E ∣ Y ∣ q ) 1/ q . (66)
Corollary 6.3.3 (Lyapunov’s inequality). Let Z Z Z be a random variable defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and suppose 1 ≤ q ≤ p 1\leq q\leq p 1 ≤ q ≤ p . Then we have
( E ∣ Z ∣ q ) 1 / q ≤ ( E ∣ Z ∣ p ) 1 / p . (67) \left(\mathbb{E}|Z|^q\right)^{1/q}\leq\left(\mathbb{E}|Z|^p\right)^{1/p}. \tag*{(67)} ( E ∣ Z ∣ q ) 1/ q ≤ ( E ∣ Z ∣ p ) 1/ p . (67)
Proof. Let r , p ≥ 1 r,p\geq1 r , p ≥ 1 . Applying Hölder’s inequality (66) with Y = 1 Y=1 Y = 1 and X = ∣ Z ∣ r X=|Z|^r X = ∣ Z ∣ r we obtain
E ∣ Z ∣ r ≤ ( E ∣ Z ∣ r p ) 1 / p . \mathbb{E}|Z|^r\leq\left(\mathbb{E}|Z|^{rp}\right)^{1/p}. E ∣ Z ∣ r ≤ ( E ∣ Z ∣ r p ) 1/ p .
Defining q : = r p q:=rp q := r p , we have
E ∣ Z ∣ r ≤ ( E ∣ Z ∣ q ) r / q ⇒ ( E ∣ Z ∣ r ) 1 / r ≤ ( E ∣ Z ∣ q ) 1 / q . \mathbb{E}|Z|^r\leq\left(\mathbb{E}|Z|^q\right)^{r/q}
\quad\Rightarrow\quad
\left(\mathbb{E}|Z|^r\right)^{1/r}\leq\left(\mathbb{E}|Z|^q\right)^{1/q}. E ∣ Z ∣ r ≤ ( E ∣ Z ∣ q ) r / q ⇒ ( E ∣ Z ∣ r ) 1/ r ≤ ( E ∣ Z ∣ q ) 1/ q .
Noting the q ≥ r q\geq r q ≥ r , we have proved (67) .
Proof that X n → L p X ∞ ⇒ X n → L q X ∞ X_n \xrightarrow{L^p} X_\infty\Rightarrow X_n \xrightarrow{L^q} X_\infty X n L p X ∞ ⇒ X n L q X ∞ with p ≥ q ≥ 1 p\geq q\geq1 p ≥ q ≥ 1 . Apply (67) with Z = X n − X ∞ Z=X_n-X_\infty Z = X n − X ∞ , take a limit as n → ∞ n\to\infty n → ∞ and use the fact that E ∣ X n − X ∞ ∣ p → 0 \mathbb{E}|X_n-X_\infty|^p\to0 E ∣ X n − X ∞ ∣ p → 0 by assumption.
Lemma 6.3.4 (Markov’s inequality). Let Z Z Z be a random variable defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . For any a > 0 a>0 a > 0 we have
P ( ∣ Z ∣ > a ) ≤ 1 a E ∣ Z ∣ . (68) \mathbb{P}(|Z|>a)\leq\frac{1}{a}\mathbb{E}|Z|. \tag*{(68)} P ( ∣ Z ∣ > a ) ≤ a 1 E ∣ Z ∣. (68)
Proof. By direct computation, we have
P ( ∣ Z ∣ > a ) = E 1 { ∣ Z ∣ > a } = 1 a E a 1 { ∣ Z ∣ > a } ≤ 1 a E ∣ Z ∣ , \mathbb{P}(|Z|>a)=\mathbb{E}\mathbf{1}_{\{|Z|>a\}}=\frac{1}{a}\mathbb{E}a\mathbf{1}_{\{|Z|>a\}}\leq\frac{1}{a}\mathbb{E}|Z|, P ( ∣ Z ∣ > a ) = E 1 { ∣ Z ∣ > a } = a 1 E a 1 { ∣ Z ∣ > a } ≤ a 1 E ∣ Z ∣ ,
where we have used ∣ Z ∣ ≥ a 1 { ∣ Z ∣ > a } |Z|\geq a\mathbf{1}_{\{|Z|>a\}} ∣ Z ∣ ≥ a 1 { ∣ Z ∣ > a } .
Proof that X n → L p X ∞ ⇒ X n → P X ∞ X_n \xrightarrow{L^p} X_\infty\Rightarrow X_n \xrightarrow{\mathbb{P}} X_\infty X n L p X ∞ ⇒ X n P X ∞ for p ≥ 1 p\geq1 p ≥ 1 . Fix ε > 0 \varepsilon>0 ε > 0 . Applying Markov’s inequality (68) with Z = X n − X ∞ Z=X_n-X_\infty Z = X n − X ∞ and a = ε a=\varepsilon a = ε and taking a limit as n → ∞ n\to\infty n → ∞ we obtain
lim n → ∞ P ( ∣ X n − X ∞ ∣ > ε ) ≤ 1 ε lim n → ∞ E ∣ X n − X ∞ ∣ = 0 , \lim_{n\to\infty}\mathbb{P}(|X_n-X_\infty|>\varepsilon)\leq\frac{1}{\varepsilon}\lim_{n\to\infty}\mathbb{E}|X_n-X_\infty|=0, n → ∞ lim P ( ∣ X n − X ∞ ∣ > ε ) ≤ ε 1 n → ∞ lim E ∣ X n − X ∞ ∣ = 0 ,
where, to deduce the last equality, we have used the fact that convergence in X n → L p X ∞ ⇒ X n → L 1 X ∞ X_n \xrightarrow{L^p} X_\infty\Rightarrow X_n \xrightarrow{L^1} X_\infty X n L p X ∞ ⇒ X n L 1 X ∞ .
We will not prove the following result.
Theorem 6.3.5 (Lebesgue’s dominated convergence theorem). Let ( X n ) (X_n) ( X n ) be a sequence of random variables defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and suppose X n → a . s . X ∞ X_n \mathrel{\mathop{\to}^{\mathrm{a.s.}}} X_\infty X n → a.s. X ∞ . If there exists non-negative random variable Y Y Y such that (i) ∣ X n ∣ ≤ Y |X_n| \leq Y ∣ X n ∣ ≤ Y for every n n n and (ii) E Y < ∞ \mathbb{E}Y < \infty E Y < ∞ , then
lim n → ∞ E X n = E lim n → ∞ X n = E X ∞ . (69) \lim_{n\to\infty} \mathbb{E}X_n = \mathbb{E}\lim_{n\to\infty} X_n = \mathbb{E}X_\infty. \tag*{(69)} n → ∞ lim E X n = E n → ∞ lim X n = E X ∞ . (69)
We can now prove the remaining part of Theorem 6.3.1 .
Proof that X n → a . s . X ∞ ⇒ X n → P X ∞ X_n \mathrel{\mathop{\to}^{\mathrm{a.s.}}} X_\infty\Rightarrow X_n \mathrel{\mathop{\to}^{\mathbb{P}}} X_\infty X n → a.s. X ∞ ⇒ X n → P X ∞ . Fix ε > 0 \varepsilon> 0 ε > 0 and define Y n : = 1 { ∣ X n − X ∞ ∣ > ε } Y_n := \mathbf{1}_{\{|X_n-X_\infty|>\varepsilon\}} Y n := 1 { ∣ X n − X ∞ ∣ > ε } . As X n → a . s . X ∞ X_n \mathrel{\mathop{\to}^{\mathrm{a.s.}}} X_\infty X n → a.s. X ∞ it follows that Y n → a . s . 0 Y_n \mathrel{\mathop{\to}^{\mathrm{a.s.}}} 0 Y n → a.s. 0 . Moreover, Y n ≤ 1 Y_n \leq1 Y n ≤ 1 for all n n n and E 1 < ∞ \mathbb{E}1 < \infty E 1 < ∞ (clearly!). Thus, using Lebesgue’s dominated convergence Theorem (69) we obtain
lim n → ∞ P ( ∣ X n − X ∞ ∣ > ε ) = lim n → ∞ E Y n = E lim n → ∞ Y n = 0 , \lim_{n\to\infty}\mathbb{P}(|X_n-X_\infty|>\varepsilon) = \lim_{n\to\infty}\mathbb{E}Y_n = \mathbb{E}\lim_{n\to\infty}Y_n = 0, n → ∞ lim P ( ∣ X n − X ∞ ∣ > ε ) = n → ∞ lim E Y n = E n → ∞ lim Y n = 0 ,
which implies X n → P X ∞ X_n \mathrel{\mathop{\to}^{\mathbb{P}}} X_\infty X n → P X ∞ .
Continuity of probability measures
Much like we can define the limit (if it exists) of a sequence of real numbers ( x n ) (x_n) ( x n ) , we can define the limit (if it exists) of a sequence of sets.
Definition 6.4.1. We say a sequence of sets ( A n ) (A_n) ( A n ) increases to A A A , written A n ↑ A A_n \uparrow A A n ↑ A , if
A n ⊆ A n + 1 ∀ n ∈ N , and A = ⋃ n = 1 ∞ A n . A_n \subseteq A_{n+1} \qquad\forall n \in\mathbb{N}, \qquad\text{and} \qquad A = \bigcup_{n=1}^{\infty} A_n. A n ⊆ A n + 1 ∀ n ∈ N , and A = n = 1 ⋃ ∞ A n . We say a sequence of sets ( A n ) (A_n) ( A n ) decreases to A A A , written A n ↓ A A_n \downarrow A A n ↓ A , if
A n ⊇ A n + 1 ∀ n ∈ N , and A = ⋂ n = 1 ∞ A n . A_n \supseteq A_{n+1} \qquad\forall n \in\mathbb{N}, \qquad\text{and} \qquad A = \bigcap_{n=1}^{\infty} A_n. A n ⊇ A n + 1 ∀ n ∈ N , and A = n = 1 ⋂ ∞ A n .
A very important property of probability measures is that they are continuous in the sense described by the following Theorem.
Theorem 6.4.2 (Continuity of Probability Measures). Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Suppose ( A n ) (A_n) ( A n ) is a sequence of subsets of F \mathcal{F} F . Then
A n ↑ A , ⇒ lim n → ∞ P ( A n ) = P ( lim n → ∞ ⋃ k = 1 n A k ) = P ( A ) , A_n \uparrow A, \qquad\Rightarrow\qquad\lim_{n\to\infty}\mathbb{P}(A_n) = \mathbb{P}\left(\lim_{n\to\infty}\bigcup_{k=1}^{n} A_k\right) = \mathbb{P}(A), A n ↑ A , ⇒ n → ∞ lim P ( A n ) = P ( n → ∞ lim k = 1 ⋃ n A k ) = P ( A ) , A n ↓ A , ⇒ lim n → ∞ P ( A n ) = P ( lim n → ∞ ⋂ k = 1 n A k ) = P ( A ) . A_n \downarrow A, \qquad\Rightarrow\qquad\lim_{n\to\infty}\mathbb{P}(A_n) = \mathbb{P}\left(\lim_{n\to\infty}\bigcap_{k=1}^{n} A_k\right) = \mathbb{P}(A). A n ↓ A , ⇒ n → ∞ lim P ( A n ) = P ( n → ∞ lim k = 1 ⋂ n A k ) = P ( A ) .
Proof. Suppose A n ↑ A A_n \uparrow A A n ↑ A . Define
B n : = A n ∖ A n − 1 , ∀ n ∈ N , A 0 = ∅ , B_n := A_n \setminus A_{n-1}, \qquad\forall n \in\mathbb{N}, \qquad A_0 = \varnothing, B n := A n ∖ A n − 1 , ∀ n ∈ N , A 0 = ∅ ,
and observe that the sets ( B n ) (B_n) ( B n ) are disjoint and
A n = ⋃ k = 1 n B k , A : = ⋃ n = 1 ∞ A n = ⋃ n = 1 ∞ B n . A_n = \bigcup_{k=1}^{n} B_k, \qquad A := \bigcup_{n=1}^{\infty} A_n = \bigcup_{n=1}^{\infty} B_n. A n = k = 1 ⋃ n B k , A := n = 1 ⋃ ∞ A n = n = 1 ⋃ ∞ B n .
Hence, using the countably additive property of P \mathbb{P} P we have
P ( A ) = ∑ k = 1 ∞ P ( B k ) = lim n → ∞ ∑ k = 1 n P ( B k ) = lim n → ∞ P ( ⋃ k = 1 n B k ) = lim n → ∞ P ( A n ) . \mathbb{P}(A) = \sum_{k=1}^{\infty} \mathbb{P}(B_k) = \lim_{n\to\infty} \sum_{k=1}^{n} \mathbb{P}(B_k) = \lim_{n\to\infty} \mathbb{P}\left(\bigcup_{k=1}^{n} B_k\right) = \lim_{n\to\infty} \mathbb{P}(A_n). P ( A ) = k = 1 ∑ ∞ P ( B k ) = n → ∞ lim k = 1 ∑ n P ( B k ) = n → ∞ lim P ( k = 1 ⋃ n B k ) = n → ∞ lim P ( A n ) .
Next, suppose A n ↓ A A_n \downarrow A A n ↓ A . Then A n c ↑ A c A_n^c \uparrow A^c A n c ↑ A c . Thus, we have
P ( A ) = 1 − P ( A c ) = 1 − lim n → ∞ P ( A n c ) = lim n → ∞ ( 1 − P ( A n c ) ) = lim n → ∞ P ( A n ) . \mathbb{P}(A) = 1 - \mathbb{P}(A^c) = 1 - \lim_{n\to\infty} \mathbb{P}(A_n^c) = \lim_{n\to\infty} \left(1-\mathbb{P}(A_n^c)\right) = \lim_{n\to\infty} \mathbb{P}(A_n). P ( A ) = 1 − P ( A c ) = 1 − n → ∞ lim P ( A n c ) = n → ∞ lim ( 1 − P ( A n c ) ) = n → ∞ lim P ( A n ) .
and of sets
Much like we can define the lim sup and lim inf of a sequence of real numbers ( x n ) (x_n) ( x n ) , we can define the lim sup and lim inf of a sequence of sets.
Definition 6.5.1. Let ( A n ) (A_n) ( A n ) be a sequence of sets. We define
lim sup n → ∞ A n : = ⋂ m = 1 ∞ ⋃ n = m ∞ A n , lim inf n → ∞ A n : = ⋃ m = 1 ∞ ⋂ n = m ∞ A n . \limsup_{n\to\infty} A_n := \bigcap_{m=1}^{\infty} \bigcup_{n=m}^{\infty} A_n, \qquad\liminf_{n\to\infty} A_n := \bigcup_{m=1}^{\infty} \bigcap_{n=m}^{\infty} A_n. n → ∞ lim sup A n := m = 1 ⋂ ∞ n = m ⋃ ∞ A n , n → ∞ lim inf A n := m = 1 ⋃ ∞ n = m ⋂ ∞ A n .
To get a more intuitive understanding of the above definitions, it may be helpful to note the following
lim sup n → ∞ A n = { ω ∈ Ω : ∀ m ∃ n ( ω ) ≥ m s.t. ω ∈ A n ( ω ) } = { ω ∈ Ω : ω ∈ A n for infinitely many n } , \begin{aligned}
\limsup_{n\to\infty} A_n = \{\omega\in\Omega: \forall m\ \exists n(\omega) \ge m\ \text{s.t.}\ \omega\in A_{n(\omega)}\} \\
&= \{\omega\in\Omega: \omega\in A_n\ \text{for infinitely many }n\},
\end{aligned} n → ∞ lim sup A n = { ω ∈ Ω : ∀ m ∃ n ( ω ) ≥ m s.t. ω ∈ A n ( ω ) } = { ω ∈ Ω : ω ∈ A n for infinitely many n } ,
lim inf n → ∞ A n = { ω ∈ Ω : ∃ m ( ω ) s.t. ω ∈ A n ∀ n ≥ m ( ω ) } = { ω ∈ Ω : ω ∈ A n for all but finitely many n } . \begin{aligned}
\liminf_{n\to\infty} A_n = \{\omega\in\Omega: \exists m(\omega)\ \text{s.t.}\ \omega\in A_n\ \forall n \ge m(\omega)\} \\
&= \{\omega\in\Omega: \omega\in A_n\ \text{for all but finitely many }n\}.
\end{aligned} n → ∞ lim inf A n = { ω ∈ Ω : ∃ m ( ω ) s.t. ω ∈ A n ∀ n ≥ m ( ω )} = { ω ∈ Ω : ω ∈ A n for all but finitely many n } .
For this reason, we sometimes write
lim sup n → ∞ A n = { A n , i.o. } , i.o. = “infinitely often,” \limsup_{n\to\infty} A_n = \{A_n,\ \text{i.o.}\}, \qquad\text{i.o.} = \text{“infinitely often,”} n → ∞ lim sup A n = { A n , i.o. } , i.o. = “infinitely often,”
lim inf n → ∞ A n = { A n , a.b.f.m. } , a.b.f.m. = “all but finitely many.” \liminf_{n\to\infty} A_n = \{A_n,\ \text{a.b.f.m.}\}, \qquad\text{a.b.f.m.} = \text{“all but finitely many.”} n → ∞ lim inf A n = { A n , a.b.f.m. } , a.b.f.m. = “all but finitely many.”
Clearly, we have
lim inf n → ∞ A n ⊆ lim sup n → ∞ A n . \liminf_{n\to\infty} A_n \subseteq\limsup_{n\to\infty} A_n. n → ∞ lim inf A n ⊆ n → ∞ lim sup A n .
It is also helpful to note that
( A n , i . o ) c = ( A n c , a . b . f . m ) , ( A n , a . b . f . m . ) c = ( A n c , i . o ) . (A_n,\mathrm{i.o})^c=(A_n^c,\mathrm{a.b.f.m}),
\qquad
(A_n,\mathrm{a.b.f.m.})^c=(A_n^c,\mathrm{i.o}). ( A n , i.o ) c = ( A n c , a.b.f.m ) , ( A n , a.b.f.m. ) c = ( A n c , i.o ) .
Note that the lim inf \liminf lim inf and lim sup \limsup lim sup of a sequence of sets always exists (just like the lim inf \liminf lim inf and lim sup \limsup lim sup of a sequence of real numbers always exists). To see this, observe that, if we define
B m : = ⋃ n = m ∞ A n , C m : = ⋂ n = m ∞ A n , B_m:=\bigcup_{n=m}^{\infty} A_n,
\qquad
C_m:=\bigcap_{n=m}^{\infty} A_n, B m := n = m ⋃ ∞ A n , C m := n = m ⋂ ∞ A n ,
then, for all n ∈ N n\in\mathbb{N} n ∈ N we have B n ⊇ B n + 1 B_n\supseteq B_{n+1} B n ⊇ B n + 1 and C n ⊆ C n + 1 C_n\subseteq C_{n+1} C n ⊆ C n + 1 . We further have
lim sup n → ∞ A n = ⋂ m = 1 ∞ B m , lim inf n → ∞ A n = ⋃ m = 1 ∞ C m . \limsup_{n\to\infty} A_n=\bigcap_{m=1}^{\infty} B_m,
\qquad
\liminf_{n\to\infty} A_n=\bigcup_{m=1}^{\infty} C_m. n → ∞ lim sup A n = m = 1 ⋂ ∞ B m , n → ∞ lim inf A n = m = 1 ⋃ ∞ C m .
Stated another way, we have
B m ↓ lim sup n → ∞ A n , C n ↑ lim inf n → ∞ A n . B_m\downarrow\limsup_{n\to\infty} A_n,
\qquad
C_n\uparrow\liminf_{n\to\infty} A_n. B m ↓ n → ∞ lim sup A n , C n ↑ n → ∞ lim inf A n .
It follows by the continuity of P \mathbb{P} P that
P ( lim sup n → ∞ A n ) = lim n → ∞ P ( B n ) , P ( lim inf n → ∞ A n ) = lim n → ∞ P ( C n ) . \mathbb{P}\left(\limsup_{n\to\infty} A_n\right)=\lim_{n\to\infty}\mathbb{P}(B_n),
\qquad
\mathbb{P}\left(\liminf_{n\to\infty} A_n\right)=\lim_{n\to\infty}\mathbb{P}(C_n). P ( n → ∞ lim sup A n ) = n → ∞ lim P ( B n ) , P ( n → ∞ lim inf A n ) = n → ∞ lim P ( C n ) .
Example 6.5.2. Consider the following sequence of sets
A 2 n = B , A 2 n + 1 = C , n = 0 , 1 , 2 , … , A_{2n}=B,
\qquad
A_{2n+1}=C,
\qquad
n=0,1,2,\ldots, A 2 n = B , A 2 n + 1 = C , n = 0 , 1 , 2 , … ,
where B B B and C C C are some arbitrary sets in a sample space Ω \Omega Ω . To compute the lim inf n A n \liminf_n A_n lim inf n A n consider the following question: for which ω ∈ Ω \omega\in\Omega ω ∈ Ω does the event ω ∈ A n \omega\in A_n ω ∈ A n occur all but finitely many times? The answer is all of the ω ∈ B ∩ C \omega\in B\cap C ω ∈ B ∩ C . To compute the lim sup n A n \limsup_n A_n lim sup n A n consider the following question: for which ω ∈ Ω \omega\in\Omega ω ∈ Ω does the event ω ∈ A n \omega\in A_n ω ∈ A n infinitely often? The answer is all of the ω ∈ B ∪ C \omega\in B\cup C ω ∈ B ∪ C . Thus, we have
lim inf n → ∞ A n = B ∩ C , lim sup n → ∞ A n = B ∪ C . \liminf_{n\to\infty} A_n=B\cap C,
\qquad
\limsup_{n\to\infty} A_n=B\cup C. n → ∞ lim inf A n = B ∩ C , n → ∞ lim sup A n = B ∪ C .
Borel-Cantelli Lemma
Lemma 6.6.1 (Borel-Cantelli). Let ( A n ) (A_n) ( A n ) be a sequence of sets. Then
∑ n = 1 ∞ P ( A n ) < ∞ ⇒ P ( A n , i . o ) = 0. (70) \sum_{n=1}^{\infty}\mathbb{P}(A_n)<\infty
\qquad\Rightarrow\qquad
\mathbb{P}(A_n,\mathrm{i.o})=0. \tag*{(70)} n = 1 ∑ ∞ P ( A n ) < ∞ ⇒ P ( A n , i.o ) = 0. (70)
If the events ( A n ) (A_n) ( A n ) are independent , then
∑ n = 1 ∞ P ( A n ) = ∞ ⇒ P ( A n , i . o . ) = 1. (71) \sum_{n=1}^{\infty} \mathbb{P}(A_n)=\infty
\quad\Rightarrow\quad
\mathbb{P}(A_n,\ i.o.)=1. \tag*{(71)} n = 1 ∑ ∞ P ( A n ) = ∞ ⇒ P ( A n , i . o . ) = 1. (71)
Proof. To prove (70) we note that
{ A n , i . o . } = ⋂ m = 1 ∞ ⋃ n = m ∞ A n ⊆ ⋃ n = m ∞ A n . \{A_n,\ i.o.\}=\bigcap_{m=1}^{\infty}\bigcup_{n=m}^{\infty}A_n\subseteq\bigcup_{n=m}^{\infty}A_n. { A n , i . o . } = m = 1 ⋂ ∞ n = m ⋃ ∞ A n ⊆ n = m ⋃ ∞ A n .
Therefore, we have
P ( A n , i . o . ) ≤ P ( ⋃ n = m ∞ A n ) ≤ ∑ n = m ∞ P ( A n ) . \mathbb{P}(A_n,\ i.o.)\leq\mathbb{P}\left(\bigcup_{n=m}^{\infty}A_n\right)\leq\sum_{n=m}^{\infty}\mathbb{P}(A_n). P ( A n , i . o . ) ≤ P ( n = m ⋃ ∞ A n ) ≤ n = m ∑ ∞ P ( A n ) .
The right-hand side above goes to zero as m → ∞ m\to\infty m → ∞ if ∑ n = 1 ∞ P ( A n ) < ∞ \sum_{n=1}^{\infty}\mathbb{P}(A_n)<\infty ∑ n = 1 ∞ P ( A n ) < ∞ , which proves (70) . To prove (71) we note that
P ( ⋂ n = m ∞ A n c ) = lim k → ∞ P ( ⋂ n = m k A n c ) = ∏ n = m ∞ ( 1 − P ( A n ) ) (by independence) ≤ ∏ n = m ∞ exp ( − P ( A n ) ) (as 1 − x ≤ e − x if x ≥ 0 ) = exp ( − ∑ n = m ∞ P ( A n ) ) . \begin{aligned}
\mathbb{P}\left(\bigcap_{n=m}^{\infty}A_n^c\right)
=\lim_{k\to\infty}\mathbb{P}\left(\bigcap_{n=m}^{k}A_n^c\right) \\
&=\prod_{n=m}^{\infty}(1-\mathbb{P}(A_n))
\qquad\text{(by independence)} \\
&\leq\prod_{n=m}^{\infty}\exp(-\mathbb{P}(A_n))
\qquad\text{(as }1-x\leq e^{-x}\text{ if }x\geq0\text{)} \\
&=\exp\left(-\sum_{n=m}^{\infty}\mathbb{P}(A_n)\right).
\end{aligned} P ( n = m ⋂ ∞ A n c ) = k → ∞ lim P ( n = m ⋂ k A n c ) = n = m ∏ ∞ ( 1 − P ( A n )) (by independence) ≤ n = m ∏ ∞ exp ( − P ( A n )) (as 1 − x ≤ e − x if x ≥ 0 ) = exp ( − n = m ∑ ∞ P ( A n ) ) .
The last term equals zero if ∑ n = 1 ∞ P ( A n ) = ∞ \sum_{n=1}^{\infty}\mathbb{P}(A_n)=\infty ∑ n = 1 ∞ P ( A n ) = ∞ , and in this case we have
P ( { A n , i . o . } c ) = P ( ⋃ m = 1 ∞ ⋂ n = m ∞ A n c ) = lim m → ∞ P ( ⋂ n = m ∞ A n c ) = 0 , \mathbb{P}(\{A_n,\ i.o.\}^{c})
=\mathbb{P}\left(\bigcup_{m=1}^{\infty}\bigcap_{n=m}^{\infty}A_n^c\right)
=\lim_{m\to\infty}\mathbb{P}\left(\bigcap_{n=m}^{\infty}A_n^c\right)=0, P ({ A n , i . o . } c ) = P ( m = 1 ⋃ ∞ n = m ⋂ ∞ A n c ) = m → ∞ lim P ( n = m ⋂ ∞ A n c ) = 0 ,
which establishes that P ( A n , i . o . ) = 1 \mathbb{P}(A_n,\ i.o.)=1 P ( A n , i . o . ) = 1 , as claimed.
We can use the Borel-Cantelli Lemma to establish that fast convergence in probability implies almost sure convergence.
Theorem 6.6.2. Suppose X n → P X ∞ X_n\xrightarrow{\mathbb{P}}X_\infty X n P X ∞ and
∑ n = 1 ∞ P ( ∣ X n − X ∞ ∣ > ε ) < ∞ , ∀ ε > 0. \sum_{n=1}^{\infty}\mathbb{P}(|X_n-X_\infty|>\varepsilon)<\infty,
\qquad\forall\ \varepsilon>0. n = 1 ∑ ∞ P ( ∣ X n − X ∞ ∣ > ε ) < ∞ , ∀ ε > 0. Then X n → a . s . X ∞ X_n\xrightarrow{\mathrm{a.s.}}X_\infty X n a.s. X ∞ .
Proof. Fix ε > 0 \varepsilon>0 ε > 0 and take A n = { ∣ X n − X ∞ ∣ > ε } A_n=\{|X_n-X_\infty|>\varepsilon\} A n = { ∣ X n − X ∞ ∣ > ε } . We have by assumption that ∑ n P ( A n ) < ∞ \sum_n\mathbb{P}(A_n)<\infty ∑ n P ( A n ) < ∞ . It follows from the Borel-Cantelli Lemma 6.6.1, that
P ( A n , i . o . ) = P ( ∣ X n − X ∞ ∣ > ε , i . o . ) = 0. \mathbb{P}(A_n,\ i.o.)=\mathbb{P}(|X_n-X_\infty|>\varepsilon,\ i.o.)=0. P ( A n , i . o . ) = P ( ∣ X n − X ∞ ∣ > ε , i . o . ) = 0.
Thus, X n → a . s . X ∞ X_n\xrightarrow{\mathrm{a.s.}}X_\infty X n a.s. X ∞ .
Martingale convergence theorems
It is often the case that a sequence of random variables ( X n ) (X_n) ( X n ) represents the value of a process at discrete periods of time. This was the case in Example 6.1.1 , in which X n X_n X n represented the wealth of a gambler after the n th n^{\text{th}} n th flip of a coin. A very special class of processes that we have previously encountered is the class of martingale processes (see Definition 2.3.5 ). For the reader’s convenience, we restate the definition of a martingale for a discrete-time process here.
Definition 6.7.1. Let X = ( X n ) n ∈ N 0 X=(X_n)_{n\in\mathbb{N}_0} X = ( X n ) n ∈ N 0 be a sequence of random variables defined on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . The sequence X X X is said to be a martingale with respect to a filtration ( F n ) n ∈ N 0 (\mathcal{F}_n)_{n\in\mathbb{N}_0} ( F n ) n ∈ N 0 if the following hold:
(i) E ∣ X n ∣ < ∞ \mathbb{E}|X_n|<\infty E ∣ X n ∣ < ∞ for all n ∈ N 0 n\in\mathbb{N}_0 n ∈ N 0 ,
(ii) E [ X n + m ∣ F n ] = X n \mathbb{E}[X_{n+m}\mid\mathcal{F}_n]=X_n E [ X n + m ∣ F n ] = X n for all n , m ∈ N 0 n,m\in\mathbb{N}_0 n , m ∈ N 0 . Alternatively, E [ X n + 1 ∣ F n ] = X n \mathbb{E}[X_{n+1}\mid\mathcal{F}_n]=X_n E [ X n + 1 ∣ F n ] = X n for all n ∈ N 0 n\in\mathbb{N}_0 n ∈ N 0 .
The special structure of martingales will allow us to prove some very powerful theorems about the behavior of a martingale X X X as n → ∞ n\to\infty n → ∞ . Before stating and proving these theorems, however, let us take a look at some example of discrete-time martingales.
Example 6.7.2 (Martingales from branching processes). Suppose Z n Z_n Z n is the size of the n th n^{\text{th}} n th generation of a branching process with Z 0 = 1 Z_0=1 Z 0 = 1 (see § 9.2 ). We have
Z n + 1 = ∑ i = 1 Z n X n , i . Z_{n+1}=\sum_{i=1}^{Z_n}X_{n,i}. Z n + 1 = i = 1 ∑ Z n X n , i . where the ( X n , i ) (X_{n,i}) ( X n , i ) are i.i.d with common distribution X X X . Let F n : = σ ( Z 0 , Z 1 , … , Z n ) \mathcal{F}_n:=\sigma(Z_0,Z_1,\ldots,Z_n) F n := σ ( Z 0 , Z 1 , … , Z n ) . We have
E [ Z n + 1 ∣ F n ] = E [ ∑ i = 1 Z n X n , i ∣ Z n ] = Z n E X = Z n μ , μ : = E X . \mathbb{E}[Z_{n+1}\mid\mathcal{F}_n]=\mathbb{E}\left[\sum_{i=1}^{Z_n}X_{n,i}\mid Z_n\right]=Z_n\mathbb{E}X=Z_n\mu,\qquad\mu:=\mathbb{E}X. E [ Z n + 1 ∣ F n ] = E [ i = 1 ∑ Z n X n , i ∣ Z n ] = Z n E X = Z n μ , μ := E X . From this, one obtains by induction that E Z n = μ n \mathbb{E}Z_n=\mu^n E Z n = μ n . Clearly the process Z Z Z is not a martingale when μ ≠ 1 \mu\ne1 μ = 1 . However, consider W n : = μ − n Z n W_n:=\mu^{-n}Z_n W n := μ − n Z n . We have
E [ W n + 1 ∣ F n ] = μ − ( n + 1 ) E [ Z n + 1 ∣ F n ] = μ − ( n + 1 ) Z n μ = μ − n Z n = W n . \mathbb{E}[W_{n+1}\mid\mathcal{F}_n]=\mu^{-(n+1)}\mathbb{E}[Z_{n+1}\mid\mathcal{F}_n]=\mu^{-(n+1)}Z_n\mu=\mu^{-n}Z_n=W_n. E [ W n + 1 ∣ F n ] = μ − ( n + 1 ) E [ Z n + 1 ∣ F n ] = μ − ( n + 1 ) Z n μ = μ − n Z n = W n . Thus, the process W = ( W n ) W=(W_n) W = ( W n ) is a martingale with respect to ( F n ) (\mathcal{F}_n) ( F n ) . Next, suppose η \eta η is the smallest solution of
G ( η ) = η , where G ( s ) : = E s X . G(\eta)=\eta,\qquad\text{where}\qquad G(s):=\mathbb{E}s^X. G ( η ) = η , where G ( s ) := E s X . Defining V n : = η Z n V_n:=\eta^{Z_n} V n := η Z n , we find
E [ V n + 1 ∣ F n ] = E [ η Z n + 1 ∣ F n ] = E [ η ∑ i = 1 Z n X n , i ∣ F n ] = ( E η X ) Z n \mathbb{E}[V_{n+1}\mid\mathcal{F}_n]=\mathbb{E}[\eta^{Z_{n+1}}\mid\mathcal{F}_n]=\mathbb{E}\left[\eta^{\sum_{i=1}^{Z_n}X_{n,i}}\mid\mathcal{F}_n\right]=(\mathbb{E}\eta^X)^{Z_n} E [ V n + 1 ∣ F n ] = E [ η Z n + 1 ∣ F n ] = E [ η ∑ i = 1 Z n X n , i ∣ F n ] = ( E η X ) Z n
= ( G ( η ) ) Z n = η Z n = V n . = (\mathrm{G}(\eta))^{\mathbb{Z}_n} = \eta^{\mathbb{Z}_n} = V_n. = ( G ( η ) ) Z n = η Z n = V n .
Thus, the process V = ( V n ) V=(V_n) V = ( V n ) is a martingale with respect to ( F n ) (\mathcal{F}_n) ( F n ) .
Example 6.7.3 (Martingales from Markov chains). Let X = ( X n ) X=(X_n) X = ( X n ) be a discrete time Markov chain with state space S S S and one-step transition matrix P = ( p ( i , j ) ) \mathbf{P}=(p(i,j)) P = ( p ( i , j )) . Suppose ψ \psi ψ is a right eigenvector of P \mathbf{P} P with corresponding eigenvalue λ = 1 \lambda=1 λ = 1 . That is
P ψ = ψ , or component-wise ∑ j ∈ S p ( i , j ) ψ ( j ) = ψ ( i ) , ∀ i ∈ S . \mathbf{P}\psi=\psi,
\qquad
\text{or component-wise}
\qquad
\sum_{j\in S}p(i,j)\psi(j)=\psi(i),
\qquad
\forall i\in S. P ψ = ψ , or component-wise j ∈ S ∑ p ( i , j ) ψ ( j ) = ψ ( i ) , ∀ i ∈ S . Define Y n : = ψ ( X n ) Y_n:=\psi(X_n) Y n := ψ ( X n ) . Letting F n : = σ ( X 0 , X 1 , … , X n ) \mathcal{F}_n:=\sigma(X_0,X_1,\ldots,X_n) F n := σ ( X 0 , X 1 , … , X n ) , we have
E [ Y n + 1 ∣ F n ] = E [ ψ ( X n + 1 ) ∣ X n ] = ∑ j ∈ S p ( X n , j ) ψ ( j ) = ψ ( X n ) = Y n . \mathbb{E}[Y_{n+1}\mid\mathcal{F}_n]
=\mathbb{E}[\psi(X_{n+1})\mid X_n]
=\sum_{j\in S}p(X_n,j)\psi(j)
=\psi(X_n)
=Y_n. E [ Y n + 1 ∣ F n ] = E [ ψ ( X n + 1 ) ∣ X n ] = j ∈ S ∑ p ( X n , j ) ψ ( j ) = ψ ( X n ) = Y n . This, the process Y = ( Y n ) Y=(Y_n) Y = ( Y n ) is a martingale with respect to ( F n ) (\mathcal{F}_n) ( F n ) .
To prove the main result of this section (Theorem 6.7.5 ) we require the following result:
Theorem 6.7.4 (Doob-Kolmogorov inequality). Let M = ( M n ) M=(M_n) M = ( M n ) be a martingale with respect to a filtration ( F n ) (\mathcal{F}_n) ( F n ) . Then
P ( sup 0 ≤ i ≤ n ∣ M i ∣ ≥ ε ) ≤ 1 ε 2 E M n 2 , ∀ ε > 0. (72) \mathbb{P}\left(\sup_{0\leq i\leq n}|M_i|\geq\varepsilon\right)
\leq\frac{1}{\varepsilon^2}\mathbb{E}M_n^2,
\qquad\forall\ \varepsilon>0. \tag*{(72)} P ( 0 ≤ i ≤ n sup ∣ M i ∣ ≥ ε ) ≤ ε 2 1 E M n 2 , ∀ ε > 0. (72)
Proof. We begin by defining the following sets
A 0 : = Ω , A k : = { sup 0 ≤ i ≤ k ∣ M i ∣ < ε } , B k : = A k − 1 ∩ { ∣ M k ∣ ≥ ε } , k ≥ 1. A_0:=\Omega,
\qquad
A_k:=\left\{\sup_{0\leq i\leq k}|M_i|<\varepsilon\right\},
\qquad
B_k:=A_{k-1}\cap\{|M_k|\geq\varepsilon\},
\qquad
k\geq1. A 0 := Ω , A k := { 0 ≤ i ≤ k sup ∣ M i ∣ < ε } , B k := A k − 1 ∩ { ∣ M k ∣ ≥ ε } , k ≥ 1.
Observe that A k A_k A k is the event that ∣ M ∣ |M| ∣ M ∣ does not reach ε \varepsilon ε prior to time k k k and B k B_k B k is the event that ∣ M ∣ |M| ∣ M ∣ first reaches or exceeds ε \varepsilon ε for the first time at time k k k . Thus, for any k k k , the events A k , B 1 , B 2 , … , B k A_k,B_1,B_2,\ldots,B_k A k , B 1 , B 2 , … , B k are a partition of Ω \Omega Ω . That is, for all k k k , we have
Ω = A k ∪ ( ⋃ i = 1 k B i ) , A k ∩ B i = ∅ , i ≤ k , B i ∩ B j = ∅ , i ≠ j . \Omega=A_k\cup\left(\bigcup_{i=1}^{k}B_i\right),
\qquad
A_k\cap B_i=\varnothing,
\qquad
i\leq k,
\qquad
B_i\cap B_j=\varnothing,
\qquad
i\ne j. Ω = A k ∪ ( i = 1 ⋃ k B i ) , A k ∩ B i = ∅ , i ≤ k , B i ∩ B j = ∅ , i = j .
It follows that
E M n 2 = E M n 2 1 A n + ∑ i = 1 n E M n 2 1 B i ≤ ∑ i = 1 n E M n 2 1 B i . \mathbb{E}M_n^2
=\mathbb{E}M_n^2\mathbf{1}_{A_n}
+\sum_{i=1}^{n}\mathbb{E}M_n^2\mathbf{1}_{B_i}
\leq\sum_{i=1}^{n}\mathbb{E}M_n^2\mathbf{1}_{B_i}. E M n 2 = E M n 2 1 A n + i = 1 ∑ n E M n 2 1 B i ≤ i = 1 ∑ n E M n 2 1 B i .
Moreover, we have
E M n 2 1 B i = E ( M n − M i + M i ) 2 1 B i = E ( M n − M i ) 2 1 B i + 2 E ( M n − M i ) M i 1 B i + E M i 2 1 B i . \begin{aligned}
\mathbb{E}M_n^2\mathbf{1}_{B_i}
=\mathbb{E}(M_n-M_i+M_i)^2\mathbf{1}_{B_i} \\
&= \mathbb{E}(M_n-M_i)^2\mathbf{1}_{B_i}+2\mathbb{E}(M_n-M_i)M_i\mathbf{1}_{B_i}+\mathbb{E}M_i^2\mathbf{1}_{B_i}.
\end{aligned} E M n 2 1 B i = E ( M n − M i + M i ) 2 1 B i = E ( M n − M i ) 2 1 B i + 2 E ( M n − M i ) M i 1 B i + E M i 2 1 B i .
Let us look at these three terms one-by-one. Clearly, the first term satisfies α ≥ 0 \alpha\ge0 α ≥ 0 . Next, using the fact that M M M is a martingale, the second term satisfies
β = E ( M n − M i ) M i 1 B i = E M i 1 B i E [ ( M n − M i ) ∣ F i ] = 0. \beta=\mathbb{E}(M_n-M_i)M_i\mathbf{1}_{B_i}=\mathbb{E}M_i\mathbf{1}_{B_i}\mathbb{E}[(M_n-M_i)\mid\mathcal{F}_i]=0. β = E ( M n − M i ) M i 1 B i = E M i 1 B i E [( M n − M i ) ∣ F i ] = 0.
And the last term satisfies
γ = E M i 2 1 B i ≥ E ε 2 1 B i = ε 2 P ( B i ) . \gamma=\mathbb{E}M_i^2\mathbf{1}_{B_i}\ge\mathbb{E}\varepsilon^2\mathbf{1}_{B_i}=\varepsilon^2\mathbb{P}(B_i). γ = E M i 2 1 B i ≥ E ε 2 1 B i = ε 2 P ( B i ) .
Using the above results, we have
E M n 2 1 B i = α + 2 β + γ ≥ ε 2 P ( B i ) \mathbb{E}M_n^2\mathbf{1}_{B_i}=\alpha+2\beta+\gamma\ge\varepsilon^2\mathbb{P}(B_i) E M n 2 1 B i = α + 2 β + γ ≥ ε 2 P ( B i )
and hence
E M n 2 ≤ ∑ i = 1 n E M n 2 1 B i ≤ ∑ i = 1 n ε 2 P ( B i ) = ε 2 P ( sup 0 ≤ i ≤ n ∣ M i ∣ ≥ ε ) , \mathbb{E}M_n^2\le\sum_{i=1}^{n}\mathbb{E}M_n^2\mathbf{1}_{B_i}\le\sum_{i=1}^{n}\varepsilon^2\mathbb{P}(B_i)=\varepsilon^2\mathbb{P}\left(\sup_{0\le i\le n}|M_i|\ge\varepsilon\right), E M n 2 ≤ i = 1 ∑ n E M n 2 1 B i ≤ i = 1 ∑ n ε 2 P ( B i ) = ε 2 P ( 0 ≤ i ≤ n sup ∣ M i ∣ ≥ ε ) ,
which proves (72) .
We are now ready to prove the main result of this section:
Theorem 6.7.5. Let M = ( M n ) M=(M_n) M = ( M n ) be a martingale with respect to a filtration ( F n ) (\mathcal{F}_n) ( F n ) . Suppose that sup n E M n 2 < ∞ \sup_n\mathbb{E}M_n^2<\infty sup n E M n 2 < ∞ . Then there exists a random variable M ∞ M_\infty M ∞ such that
M n → a . s . M ∞ , and M n → L 2 M ∞ . M_n\xrightarrow{\mathrm{a.s.}}M_\infty,\qquad\text{and}\qquad M_n\xrightarrow{L^2}M_\infty. M n a.s. M ∞ , and M n L 2 M ∞ .
Proof. Observe that, for any n , m ∈ N 0 n,m\in\mathbb{N}_0 n , m ∈ N 0 , we have
E M n + m 2 = E ( M n + m − M n + M n ) 2 = E ( M n + m − M n ) 2 + E M n 2 + 2 E M n ( M n + m − M n ) = E ( M n + m − M n ) 2 + E M n 2 + 2 E M n E [ ( M n + m − M n ) ∣ F n ] = E ( M n + m − M n ) 2 + E M n 2 ≥ E M n 2 . \begin{aligned}
\mathbb{E}M_{n+m}^2=\mathbb{E}(M_{n+m}-M_n+M_n)^2 \\
&=\mathbb{E}(M_{n+m}-M_n)^2+\mathbb{E}M_n^2+2\mathbb{E}M_n(M_{n+m}-M_n) \\
&=\mathbb{E}(M_{n+m}-M_n)^2+\mathbb{E}M_n^2+2\mathbb{E}M_n\mathbb{E}[(M_{n+m}-M_n)\mid\mathcal{F}_n] \\
&=\mathbb{E}(M_{n+m}-M_n)^2+\mathbb{E}M_n^2\ge\mathbb{E}M_n^2.
\end{aligned} E M n + m 2 = E ( M n + m − M n + M n ) 2 = E ( M n + m − M n ) 2 + E M n 2 + 2 E M n ( M n + m − M n ) = E ( M n + m − M n ) 2 + E M n 2 + 2 E M n E [( M n + m − M n ) ∣ F n ] = E ( M n + m − M n ) 2 + E M n 2 ≥ E M n 2 .
Thus, the sequence ( E M n 2 ) (\mathbb{E}M_n^2) ( E M n 2 ) is strictly increasing and (by assumption) bounded. The sequence therefore has a limit ℓ : = lim n → ∞ E M n 2 \ell:=\lim_{n\to\infty}\mathbb{E}M_n^2 ℓ := lim n → ∞ E M n 2 . We will now show that the sequence ( M n ) (M_n) ( M n ) is Cauchy convergent almost surely (see Definition 6.2.2 ). By Theorem 6.2.3 , this will establish that M n → a . s . M ∞ M_n\xrightarrow{\mathrm{a.s.}}M_\infty M n a.s. M ∞ . Define C C C as the set of ω ∈ Ω \omega\in\Omega ω ∈ Ω for which the sequence ( M n ) (M_n) ( M n ) is Cauchy convergent almost surely
C : = { ∀ ε > 0 ∃ m s.t. ∣ M m + i − M m + j ∣ < ε ∀ i , j ∈ N } = ⋂ ε > 0 ⋃ m = 1 ∞ { ∣ M m + i − M m ∣ < ε ∀ i ∈ N } \begin{aligned}
C:=\{\forall\varepsilon>0\ \exists m\ \text{s.t. }|M_{m+i}-M_{m+j}|<\varepsilon\ \forall i,j\in\mathbb{N}\} \\
&= \bigcap_{\varepsilon>0}\bigcup_{m=1}^{\infty}\{\lvert M_{m+i}-M_m\rvert<\varepsilon\ \forall i\in\mathbb{N}\}
\end{aligned} C := { ∀ ε > 0 ∃ m s.t. ∣ M m + i − M m + j ∣ < ε ∀ i , j ∈ N } = ε > 0 ⋂ m = 1 ⋃ ∞ {∣ M m + i − M m ∣ < ε ∀ i ∈ N }
The complement of C is given by
C c = ⋃ ε > 0 ⋃ m = 1 ∞ { ∃ i ∈ N s.t. ∣ M m + i − M m ∣ ≥ ε } = ⋃ ε > 0 ⋃ m = 1 ∞ A m ( ε ) , C^{c}=\bigcup_{\varepsilon>0}\bigcup_{m=1}^{\infty}\{\exists i\in\mathbb{N}\ \text{s.t.}\ \lvert M_{m+i}-M_m\rvert\ge\varepsilon\}=\bigcup_{\varepsilon>0}\bigcup_{m=1}^{\infty}A_m(\varepsilon), C c = ε > 0 ⋃ m = 1 ⋃ ∞ { ∃ i ∈ N s.t. ∣ M m + i − M m ∣ ≥ ε } = ε > 0 ⋃ m = 1 ⋃ ∞ A m ( ε ) ,
A m ( ε ) : = { ∃ i ∈ N s.t. ∣ M m + i − M m ∣ ≥ ε } . A_m(\varepsilon):=\{\exists i\in\mathbb{N}\ \text{s.t.}\ \lvert M_{m+i}-M_m\rvert\ge\varepsilon\}. A m ( ε ) := { ∃ i ∈ N s.t. ∣ M m + i − M m ∣ ≥ ε } .
Noting that A m ( ε ) ⊆ A m ( ε ′ ) A_m(\varepsilon)\subseteq A_m(\varepsilon') A m ( ε ) ⊆ A m ( ε ′ ) if ε ≥ ε ′ \varepsilon\ge\varepsilon' ε ≥ ε ′ , it follows that
P ( C c ) = lim ε ↓ 0 P ( ⋂ m = 1 ∞ A m ( ε ) ) ≤ lim ε ↓ 0 lim m → ∞ P ( A m ( ε ) ) . \mathbb{P}(C^{c})=\lim_{\varepsilon\downarrow0}\mathbb{P}\left(\bigcap_{m=1}^{\infty}A_m(\varepsilon)\right)\le\lim_{\varepsilon\downarrow0}\lim_{m\to\infty}\mathbb{P}(A_m(\varepsilon)). P ( C c ) = ε ↓ 0 lim P ( m = 1 ⋂ ∞ A m ( ε ) ) ≤ ε ↓ 0 lim m → ∞ lim P ( A m ( ε )) .
Thus, to prove that P ( C c ) = 0 \mathbb{P}(C^{c})=0 P ( C c ) = 0 , it is sufficient to show that P ( A n ( ε ) ) → 0 \mathbb{P}(A_n(\varepsilon))\to0 P ( A n ( ε )) → 0 as n → ∞ n\to\infty n → ∞ for all ε > 0 \varepsilon>0 ε > 0 . To this end, for a fixed m ∈ N m\in\mathbb{N} m ∈ N define the sequence Y = ( Y n ) Y=(Y_n) Y = ( Y n ) by Y n : = M n + m − M m Y_n:=M_{n+m}-M_m Y n := M n + m − M m . The process ( Y n ) (Y_n) ( Y n ) is a martingale with respect to ( G n ) (\mathcal{G}_n) ( G n ) where G n = σ ( Y 1 , Y 2 , … , Y n ) \mathcal{G}_n=\sigma(Y_1,Y_2,\ldots,Y_n) G n = σ ( Y 1 , Y 2 , … , Y n ) since
E [ Y n + 1 ∣ G n ] = E [ M n + 1 + m − M m ∣ G n ] = E [ E [ M n + 1 + m − M m ∣ F n + m ] ∣ G n ] = E [ M n + m − M m ∣ G n ] = E [ Y n ∣ G n ] = Y n . \begin{aligned}
\mathbb{E}[Y_{n+1}\mid\mathcal{G}_n]=\mathbb{E}[M_{n+1+m}-M_m\mid\mathcal{G}_n]=\mathbb{E}[\mathbb{E}[M_{n+1+m}-M_m\mid\mathcal{F}_{n+m}]\mid\mathcal{G}_n] \\
&=\mathbb{E}[M_{n+m}-M_m\mid\mathcal{G}_n]=\mathbb{E}[Y_n\mid\mathcal{G}_n]=Y_n.
\end{aligned} E [ Y n + 1 ∣ G n ] = E [ M n + 1 + m − M m ∣ G n ] = E [ E [ M n + 1 + m − M m ∣ F n + m ] ∣ G n ] = E [ M n + m − M m ∣ G n ] = E [ Y n ∣ G n ] = Y n .
As the process Y Y Y is a martingale, we have by the Doob–Kolmogorov inequality (Theorem 6.7.4 ) that
P ( sup 0 ≤ i ≤ n ∣ Y i ∣ ≥ ε ) ≤ 1 ε 2 E Y n 2 , ∀ ε > 0 , ⇒ P ( sup 0 ≤ i ≤ n ∣ M m + i − M m ∣ ≥ ε ) ≤ 1 ε 2 E ( M m + n − M m ) 2 = 1 ε 2 ( E M m + n 2 − E M m 2 ) , ∀ ε > 0. \begin{aligned}
\mathbb{P}\left(\sup_{0\le i\le n}\lvert Y_i\rvert\ge\varepsilon\right)\le\frac{1}{\varepsilon^2}\mathbb{E}Y_n^2,\qquad\forall\varepsilon>0, \\
&\Rightarrow\quad\mathbb{P}\left(\sup_{0\le i\le n}\lvert M_{m+i}-M_m\rvert\ge\varepsilon\right)\le\frac{1}{\varepsilon^2}\mathbb{E}(M_{m+n}-M_m)^2 \\
&=\frac{1}{\varepsilon^2}\left(\mathbb{E}M_{m+n}^2-\mathbb{E}M_m^2\right),\qquad\forall\varepsilon>0.
\end{aligned} P ( 0 ≤ i ≤ n sup ∣ Y i ∣ ≥ ε ) ≤ ε 2 1 E Y n 2 , ∀ ε > 0 , ⇒ P ( 0 ≤ i ≤ n sup ∣ M m + i − M m ∣ ≥ ε ) ≤ ε 2 1 E ( M m + n − M m ) 2 = ε 2 1 ( E M m + n 2 − E M m 2 ) , ∀ ε > 0.
Taking a limit as n → ∞ n\to\infty n → ∞ we obtain
P ( A m ( ε ) ) ≤ 1 ε 2 ( ℓ − E M m 2 ) , ⇒ lim m → ∞ P ( A m ( ε ) ) ≤ 1 ε 2 ( ℓ − ℓ ) = 0 , \begin{aligned}
\mathbb{P}(A_m(\varepsilon))\le\frac{1}{\varepsilon^2}\left(\ell-\mathbb{E}M_m^2\right), \\
&\Rightarrow\quad\lim_{m\to\infty}\mathbb{P}(A_m(\varepsilon))\le\frac{1}{\varepsilon^2}(\ell-\ell)=0,
\end{aligned} P ( A m ( ε )) ≤ ε 2 1 ( ℓ − E M m 2 ) , ⇒ m → ∞ lim P ( A m ( ε )) ≤ ε 2 1 ( ℓ − ℓ ) = 0 ,
which establishes that ( M n ) (M_n) ( M n ) is Cauchy convergent almost surely. Hence, there exists M ∞ M_\infty M ∞ such that
M n → a . s . M ∞ . M_n\xrightarrow{\mathrm{a.s.}}M_\infty. M n a.s. M ∞ .
Next using Fatou’s Lemma1 we have
E ( M n − M ∞ ) 2 = E lim inf m → ∞ ( M n − M m ) 2 ≤ lim inf m → ∞ E ( M n − M m ) 2 = lim inf m → ∞ E ( M n − M m ) 2 = lim m → ∞ E M m 2 − E M n 2 = ℓ − E M n 2 → 0 as n → ∞ , \begin{aligned}
\mathbb{E}(M_n-M_\infty)^2=\mathbb{E}\liminf_{m\to\infty}(M_n-M_m)^2\le\liminf_{m\to\infty}\mathbb{E}(M_n-M_m)^2 \\
&=\liminf_{m\to\infty}\mathbb{E}(M_n-M_m)^2=\lim_{m\to\infty}\mathbb{E}M_m^2-\mathbb{E}M_n^2 \\
&=\ell-\mathbb{E}M_n^2\to0\quad\text{as }n\to\infty,
\end{aligned} E ( M n − M ∞ ) 2 = E m → ∞ lim inf ( M n − M m ) 2 ≤ m → ∞ lim inf E ( M n − M m ) 2 = m → ∞ lim inf E ( M n − M m ) 2 = m → ∞ lim E M m 2 − E M n 2 = ℓ − E M n 2 → 0 as n → ∞ ,
which establishes that M n → L 2 M ∞ M_n\xrightarrow{L^2}M_\infty M n L 2 M ∞ .
Let us see an example of how Theorem 6.7.5 can be applied.
Example 6.7.6. Suppose ( X n ) (X_n) ( X n ) is a Markov chain with one-step transition probabilities given by
p ( i , j ) = P ( X n + 1 = j ∣ X n = i ) = ( N j ) ( i N ) j ( 1 − i N ) N − j , i , j ∈ { 0 , 1 , … , N } . p(i,j)=\mathbb{P}(X_{n+1}=j\mid X_n=i)=\binom{N}{j}\left(\frac{i}{N}\right)^j\left(1-\frac{i}{N}\right)^{N-j},\qquad i,j\in\{0,1,\ldots,N\}. p ( i , j ) = P ( X n + 1 = j ∣ X n = i ) = ( j N ) ( N i ) j ( 1 − N i ) N − j , i , j ∈ { 0 , 1 , … , N } .
Let F n = σ ( X 1 , X 2 , … , X n ) \mathcal{F}_n=\sigma(X_1,X_2,\ldots,X_n) F n = σ ( X 1 , X 2 , … , X n ) . The process ( X n ) (X_n) ( X n ) is a martingale with respect to the filtration ( F n ) (\mathcal{F}_n) ( F n ) because
E [ X n + 1 ∣ F n ] = ∑ j = 0 N j p ( X n , j ) = ∑ j = 0 N j ( N j ) ( X n N ) j ( 1 − X n N ) N − j = ( 1 − X n N ) − N ( 1 − X n N ) N X n = X n . \begin{aligned}
\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]=\sum_{j=0}^{N}j\,p(X_n,j) \\
&=\sum_{j=0}^{N}j\binom{N}{j}\left(\frac{X_n}{N}\right)^j\left(1-\frac{X_n}{N}\right)^{N-j} \\
&=\left(1-\frac{X_n}{N}\right)^{-N}\left(1-\frac{X_n}{N}\right)^N X_n=X_n.
\end{aligned} E [ X n + 1 ∣ F n ] = j = 0 ∑ N j p ( X n , j ) = j = 0 ∑ N j ( j N ) ( N X n ) j ( 1 − N X n ) N − j = ( 1 − N X n ) − N ( 1 − N X n ) N X n = X n .
Clearly, as ( X n ) (X_n) ( X n ) is bounded, we have sup n E X n 2 < ∞ \sup_n\mathbb{E}X_n^2<\infty sup n E X n 2 < ∞ . As such, by Theorem 6.7.5 there exists a random variable X ∞ X_\infty X ∞ such that X n → a . s . X ∞ X_n\xrightarrow{\mathrm{a.s.}}X_\infty X n a.s. X ∞ and X n → L 2 X ∞ X_n\xrightarrow{L^2}X_\infty X n L 2 X ∞ . To find the distribution of X ∞ X_\infty X ∞ , we note that both { 0 } \{0\} { 0 } and { N } \{N\} { N } are absorbing states. As there are no other absorbing states, we conclude that ( X n ) (X_n) ( X n ) will eventually end up in one of these two states. Thus, using the martingale property X 0 = E X n X_0=\mathbb{E}X_n X 0 = E X n we find
X 0 = lim n → ∞ E X n = E lim n → ∞ X n = E X ∞ = N ⋅ P ( X ∞ = N ) + 0 ⋅ P ( X ∞ = 0 ) , \begin{aligned}
X_0=\lim_{n\to\infty}\mathbb{E}X_n=\mathbb{E}\lim_{n\to\infty}X_n=\mathbb{E}X_\infty\\
&=N\cdot\mathbb{P}(X_\infty=N)+0\cdot\mathbb{P}(X_\infty=0),
\end{aligned} X 0 = n → ∞ lim E X n = E n → ∞ lim X n = E X ∞ = N ⋅ P ( X ∞ = N ) + 0 ⋅ P ( X ∞ = 0 ) ,
where passing the limit through the expectation is allowed by Lebesgue’s dominated convergence (Theorem 6.3.5 ). From the above computation, we conclude that
X 0 / N = P ( X ∞ = N ) = 1 − P ( X ∞ = 0 ) . X_0/N=\mathbb{P}(X_\infty=N)=1-\mathbb{P}(X_\infty=0). X 0 / N = P ( X ∞ = N ) = 1 − P ( X ∞ = 0 ) .
Uniform Integrability
We begin this section with a motivating example.
Example 6.8.1. Suppose ( Ω , F , P ) = ( [ 0 , 1 ] , B ( [ 0 , 1 ] ) , L e b ) (\Omega,\mathcal{F},\mathbb{P})=([0,1],\mathcal{B}([0,1]),\mathrm{Leb}) ( Ω , F , P ) = ([ 0 , 1 ] , B ([ 0 , 1 ]) , Leb ) so that P ( d ω ) = d ω \mathbb{P}(\mathrm{d}\omega)=\mathrm{d}\omega P ( d ω ) = d ω . Define a sequence of random variables ( X n ) (X_n) ( X n ) by
X n ( ω ) : = n 1 [ 0 , 1 / n ] ( ω ) . X_n(\omega):=n\mathbf{1}_{[0,1/n]}(\omega). X n ( ω ) := n 1 [ 0 , 1/ n ] ( ω ) .
It is easy to see that X n → P 0 X_n\xrightarrow{\mathbb{P}}0 X n P 0 as, for any ε > 0 \varepsilon>0 ε > 0 , we have
lim n → ∞ P ( ∣ X n − 0 ∣ > ε ) = lim n → ∞ ( 1 / n ) = 0. \lim_{n\to\infty}\mathbb{P}(\lvert X_n-0\rvert>\varepsilon)=\lim_{n\to\infty}(1/n)=0. n → ∞ lim P (∣ X n − 0 ∣ > ε ) = n → ∞ lim ( 1/ n ) = 0.
In fact, in this case, we have X n → a . s . 0 X_n \xrightarrow{\mathrm{a.s.}} 0 X n a.s. 0 as well. However, we clearly do not have L 1 L^1 L 1 convergence of X n → 0 X_n \to0 X n → 0 because
lim n → ∞ E ∣ X n − 0 ∣ = lim n → ∞ 1 = 1. \lim_{n\to\infty} \mathbb{E}|X_n-0|=\lim_{n\to\infty}1=1. n → ∞ lim E ∣ X n − 0∣ = n → ∞ lim 1 = 1.
It is natural to ask: are there conditions under which X n → P X ∞ X_n \xrightarrow{\mathrm{P}} X_\infty X n P X ∞ implies X n → L 1 X ∞ X_n \xrightarrow{L^1} X_\infty X n L 1 X ∞ ? It turns out that the answer is “yes.” We introduce these conditions in this section.
Definition 6.8.2. A collection of random variables ( X n ) (X_n) ( X n ) (possibly uncountable) is said to be uniformly integrable (UI) if, for every ε > 0 \varepsilon>0 ε > 0 there exists K ε ∈ [ 0 , ∞ ) K_\varepsilon\in[0,\infty) K ε ∈ [ 0 , ∞ ) such that
sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K ε } < ε . \sup_n \mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K_\varepsilon\}}<\varepsilon. n sup E ∣ X n ∣ 1 { ∣ X n ∣ > K ε } < ε .
Example 6.8.3. Let us return to Example 6.8.1, discussed above. For every n n n we have E ∣ X n ∣ = 1 \mathbb{E}|X_n|=1 E ∣ X n ∣ = 1 . Thus, the collection of random variables ( X n ) (X_n) ( X n ) is integrable. However, we have
E ∣ X n ∣ 1 { ∣ X n ∣ > K } = 1 , ∀ n > K . \mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K\}}=1,\qquad\forall n>K. E ∣ X n ∣ 1 { ∣ X n ∣ > K } = 1 , ∀ n > K . Thus, for a given ε \varepsilon ε there is no K ε ∈ [ 0 , ∞ ) K_\varepsilon\in[0,\infty) K ε ∈ [ 0 , ∞ ) for which sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K ε } < ε \sup_n\mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K_\varepsilon\}}<\varepsilon sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K ε } < ε . As such, we conclude that the collection ( X n ) (X_n) ( X n ) is not UI.
Let us provide some simple conditions which, if satisfied by a collection of random variables, ensure uniform integrability.
Lemma 6.8.4. Suppose a collection of random variables ( X n ) (X_n) ( X n ) satisfies
sup n E ∣ X n ∣ p < ∞ \sup_n \mathbb{E}|X_n|^p<\infty n sup E ∣ X n ∣ p < ∞ for some p > 1 p>1 p > 1 . Then the collection ( X n ) (X_n) ( X n ) is UI.
Proof. Assuming K > 0 K>0 K > 0 and p > 1 p>1 p > 1 we have ∣ X n ∣ 1 − p 1 { ∣ X n ∣ > K } ≤ K 1 − p 1 { ∣ X n ∣ > K } |X_n|^{1-p}\mathbf{1}_{\{|X_n|>K\}}\le K^{1-p}\mathbf{1}_{\{|X_n|>K\}} ∣ X n ∣ 1 − p 1 { ∣ X n ∣ > K } ≤ K 1 − p 1 { ∣ X n ∣ > K } and thus ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ K p − 1 ∣ X n ∣ p 1 { ∣ X n ∣ > K } |X_n|\mathbf{1}_{\{|X_n|>K\}}\le K^{p-1}|X_n|^p\mathbf{1}_{\{|X_n|>K\}} ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ K p − 1 ∣ X n ∣ p 1 { ∣ X n ∣ > K } . Hence
E ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ K 1 − p E ∣ X n ∣ p 1 { ∣ X n ∣ > K } ≤ K 1 − p E ∣ X n ∣ p . \mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K\}}\le K^{1-p}\mathbb{E}|X_n|^p\mathbf{1}_{\{|X_n|>K\}}\le K^{1-p}\mathbb{E}|X_n|^p. E ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ K 1 − p E ∣ X n ∣ p 1 { ∣ X n ∣ > K } ≤ K 1 − p E ∣ X n ∣ p .
Taking sup’s on both sides, we have
sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ K 1 − p sup n E ∣ X n ∣ p = : K 1 − p c , c : = sup n E ∣ X n ∣ p < ∞ . \sup_n\mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K\}}\le K^{1-p}\sup_n\mathbb{E}|X_n|^p=:K^{1-p}c,\qquad c:=\sup_n\mathbb{E}|X_n|^p<\infty. n sup E ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ K 1 − p n sup E ∣ X n ∣ p =: K 1 − p c , c := n sup E ∣ X n ∣ p < ∞.
The right-hand side can be made smaller than ε \varepsilon ε by choosing K > K ε : = ( c / ε ) 1 / ( p − 1 ) K>K_\varepsilon:=(c/\varepsilon)^{1/(p-1)} K > K ε := ( c / ε ) 1/ ( p − 1 ) . Thus, we have that ( X n ) (X_n) ( X n ) is UI. ∎
Lemma 6.8.5. Suppose a collection of random variables ( X n ) (X_n) ( X n ) satisfies
sup n ∣ X n ∣ ≤ Y , E Y < ∞ , \sup_n |X_n| \leq Y, \qquad\mathbb{E}Y < \infty, n sup ∣ X n ∣ ≤ Y , E Y < ∞ , for some random variable Y Y Y . Then the collection ( X n ) (X_n) ( X n ) is UI.
Proof. By assumption ∣ X n ∣ ≤ Y |X_n| \leq Y ∣ X n ∣ ≤ Y , which implies 1 { ∣ X n ∣ > K } ≤ 1 { Y > K } \mathbf{1}_{\{|X_n|>K\}} \leq\mathbf{1}_{\{Y>K\}} 1 { ∣ X n ∣ > K } ≤ 1 { Y > K } . Hence, we have
sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ E Y 1 { Y > K } . \sup_n \mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K\}} \leq\mathbb{E}Y\mathbf{1}_{\{Y>K\}}. n sup E ∣ X n ∣ 1 { ∣ X n ∣ > K } ≤ E Y 1 { Y > K } . Because E Y < ∞ \mathbb{E}Y < \infty E Y < ∞ by assumption, for any ε > 0 \varepsilon> 0 ε > 0 , there exists K ε K_\varepsilon K ε such that E Y 1 { Y > K ε } < ε \mathbb{E}Y\mathbf{1}_{\{Y>K_\varepsilon\}} < \varepsilon E Y 1 { Y > K ε } < ε . Thus, by choosing K > K ε K > K_\varepsilon K > K ε above, we can ensure sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K } < ε \sup_n \mathbb{E}|X_n|\mathbf{1}_{\{|X_n|>K\}} < \varepsilon sup n E ∣ X n ∣ 1 { ∣ X n ∣ > K } < ε . It follows that the collection of random variables ( X n ) (X_n) ( X n ) is UI.
Theorem 6.8.6 (Bounded convergence). Let ( X n ) (X_n) ( X n ) be a sequence of random variables and let X ∞ X_\infty X ∞ be a random variable. Suppose that X n → P X ∞ X_n \xrightarrow{\mathrm{P}} X_\infty X n P X ∞ and that, for some K < ∞ K < \infty K < ∞ we have
sup n ∣ X n ∣ ≤ K . \sup_n |X_n| \leq K. n sup ∣ X n ∣ ≤ K . Then X n → L 1 X ∞ X_n \xrightarrow{L^1} X_\infty X n L 1 X ∞ .
Proof. First, we check that P ( ∣ X ∞ ∣ ≤ K ) = 1 \mathbb{P}(|X_\infty| \leq K) = 1 P ( ∣ X ∞ ∣ ≤ K ) = 1 . For any k ∈ N k \in\mathbb{N} k ∈ N we have
P ( ∣ X ∞ ∣ > K + 1 / k ) ≤ P ( ∣ X ∞ − X n ∣ > 1 / k ) . \mathbb{P}(|X_\infty| > K + 1/k) \leq\mathbb{P}(|X_\infty-X_n| > 1/k). P ( ∣ X ∞ ∣ > K + 1/ k ) ≤ P ( ∣ X ∞ − X n ∣ > 1/ k ) . The right-hand side goes to zero as n → ∞ n \to\infty n → ∞ because (by assumption) X n → P X ∞ X_n \xrightarrow{\mathrm{P}} X_\infty X n P X ∞ . Thus, we have P ( ∣ X ∞ ∣ > K + 1 / k ) = 0 \mathbb{P}(|X_\infty| > K + 1/k) = 0 P ( ∣ X ∞ ∣ > K + 1/ k ) = 0 . Now, observe that
P ( ∣ X ∞ ∣ > K ) = P ( ⋃ k { ∣ X ∞ ∣ > K + 1 / k } ) = lim k → ∞ P ( ∣ X ∞ ∣ > K + 1 / k ) = 0. \mathbb{P}(|X_\infty| > K) = \mathbb{P}\left(\bigcup_k\{|X_\infty| > K + 1/k\}\right) = \lim_{k\to\infty}\mathbb{P}(|X_\infty| > K + 1/k) = 0. P ( ∣ X ∞ ∣ > K ) = P ( k ⋃ { ∣ X ∞ ∣ > K + 1/ k } ) = k → ∞ lim P ( ∣ X ∞ ∣ > K + 1/ k ) = 0. Hence, P ( ∣ X ∞ ∣ ≤ K ) = 1 \mathbb{P}(|X_\infty| \leq K) = 1 P ( ∣ X ∞ ∣ ≤ K ) = 1 . Next, because X n → P X ∞ X_n \xrightarrow{\mathrm{P}} X_\infty X n P X ∞ , for any ε > 0 \varepsilon> 0 ε > 0 we can choose n ε n_\varepsilon n ε such that
P ( ∣ X n − X ∞ ∣ > ε / 3 ) < ε / ( 3 K ) , ∀ n ≥ n ε . \mathbb{P}(|X_n-X_\infty| > \varepsilon/3) < \varepsilon/(3K), \qquad\forall n \geq n_\varepsilon. P ( ∣ X n − X ∞ ∣ > ε /3 ) < ε / ( 3 K ) , ∀ n ≥ n ε . Then, for n ≥ n ε n \geq n_\varepsilon n ≥ n ε , we have
E ∣ X n − X ∞ ∣ = E ∣ X n − X ∞ ∣ 1 { ∣ X n − X ∞ ∣ > ε / 3 } + E ∣ X n − X ∞ ∣ 1 { ∣ X n − X ∞ ∣ ≤ ε / 3 } ≤ 2 K P ( ∣ X n − X ∞ ∣ > ε / 3 ) + ε / 3 ≤ ε , \begin{aligned}
\mathbb{E}|X_n-X_\infty|={}&\mathbb{E}|X_n-X_\infty|\mathbf{1}_{\{|X_n-X_\infty|>\varepsilon/3\}}\\
&+\mathbb{E}|X_n-X_\infty|\mathbf{1}_{\{|X_n-X_\infty|\leq\varepsilon/3\}}\\
\leq{}&2K\mathbb{P}(|X_n-X_\infty|>\varepsilon/3)+\varepsilon/3\leq\varepsilon,
\end{aligned} E ∣ X n − X ∞ ∣ = ≤ E ∣ X n − X ∞ ∣ 1 { ∣ X n − X ∞ ∣ > ε /3 } + E ∣ X n − X ∞ ∣ 1 { ∣ X n − X ∞ ∣ ≤ ε /3 } 2 K P ( ∣ X n − X ∞ ∣ > ε /3 ) + ε /3 ≤ ε , where we have used ∣ X n − X ∞ ∣ ≤ 2 K |X_n-X_\infty| \leq2K ∣ X n − X ∞ ∣ ≤ 2 K . As ε \varepsilon ε was arbitrary, we can make E ∣ X n − X ∞ ∣ \mathbb{E}|X_n-X_\infty| E ∣ X n − X ∞ ∣ as small as we like. Hence, $X_n \xrightarrow{L^1} X_\infty.
We can now state and prove the main result of this section.
Theorem 6.8.7. Let ( X n ) (X_n) ( X n ) be a sequence of random variables and let X ∞ X_\infty X ∞ be a random variable. Suppose that E ∣ X n ∣ < ∞ \mathbb{E}|X_n| < \infty E ∣ X n ∣ < ∞ for every n n n and E ∣ X ∞ ∣ < ∞ \mathbb{E}|X_\infty| < \infty E ∣ X ∞ ∣ < ∞ . Then
X n → L 1 X ∞ ⟺ X n → P X ∞ and ( X n ) is UI . X_n \xrightarrow{L^1} X_\infty\qquad\Longleftrightarrow\qquad X_n \xrightarrow{\mathbb{P}} X_\infty\text{ and } (X_n) \text{ is UI}. X n L 1 X ∞ ⟺ X n P X ∞ and ( X n ) is UI .
Proof. We have already proved the ⇒ \Rightarrow ⇒ part 2 (see Theorem 6.3.1 ). So, we focus now on the ⇐ \Leftarrow ⇐ part. To this end, suppose X n → P X ∞ X_n \xrightarrow{\mathbb{P}} X_\infty X n P X ∞ and ( X n ) (X_n) ( X n ) is UI. For any K ∈ [ 0 , ∞ ) K \in[0,\infty) K ∈ [ 0 , ∞ ) , define φ K : R → [ − K , K ] \varphi_K : \mathbb{R} \to[-K,K] φ K : R → [ − K , K ] as follows
φ K ( x ) : = { − K x < − K , x ∣ x ∣ ≤ K , K x > K . \varphi_K(x) :=
\begin{cases}
-K & x < -K,\\
x & |x| \le K,\\
K & x > K.
\end{cases} φ K ( x ) := ⎩ ⎨ ⎧ − K x K x < − K , ∣ x ∣ ≤ K , x > K .
Observe that
∣ φ K ( x ) − x ∣ = ( ∣ x ∣ − K ) 1 { ∣ x ∣ > K } ≤ ∣ x ∣ 1 { ∣ x ∣ > K } . (73) |\varphi_K(x)-x|=(|x|-K)\mathbf{1}_{\{|x|>K\}}\le|x|\mathbf{1}_{\{|x|>K\}}. \tag*{(73)} ∣ φ K ( x ) − x ∣ = ( ∣ x ∣ − K ) 1 { ∣ x ∣ > K } ≤ ∣ x ∣ 1 { ∣ x ∣ > K } . (73)
Fix ε > 0 \varepsilon> 0 ε > 0 . From (73) and the fact that ( X n ) (X_n) ( X n ) is UI and X ∞ X_\infty X ∞ integrable, there exists a K ε K_\varepsilon K ε such that
sup n E ∣ φ K ε ( X n ) − X n ∣ < ε / 3 , E ∣ φ K ε ( X ∞ ) − X ∞ ∣ < ε / 3. \sup_n \mathbb{E}|\varphi_{K_\varepsilon}(X_n)-X_n|<\varepsilon/3,\qquad\mathbb{E}|\varphi_{K_\varepsilon}(X_\infty)-X_\infty|<\varepsilon/3. n sup E ∣ φ K ε ( X n ) − X n ∣ < ε /3 , E ∣ φ K ε ( X ∞ ) − X ∞ ∣ < ε /3.
Now, X n → P X ∞ X_n \xrightarrow{\mathbb{P}} X_\infty X n P X ∞ implies that φ K ε ( X n ) → P φ K ε ( X ∞ ) \varphi_{K_\varepsilon}(X_n) \xrightarrow{\mathbb{P}} \varphi_{K_\varepsilon}(X_\infty) φ K ε ( X n ) P φ K ε ( X ∞ ) . And furthermore, sup n ∣ φ K ε ( X n ) ∣ ≤ K ε \sup_n|\varphi_{K_\varepsilon}(X_n)|\le K_\varepsilon sup n ∣ φ K ε ( X n ) ∣ ≤ K ε . Thus, it follows from Theorem 6.8.6 that φ K ε ( X n ) → L 1 φ K ε ( X ∞ ) \varphi_{K_\varepsilon}(X_n)\xrightarrow{L^1}\varphi_{K_\varepsilon}(X_\infty) φ K ε ( X n ) L 1 φ K ε ( X ∞ ) . Thus, there exists an n ε n_\varepsilon n ε such that
E ∣ φ K ε ( X n ) − φ K ε ( X ∞ ) ∣ < ε / 3 , ∀ n ≥ n ε . \mathbb{E}|\varphi_{K_\varepsilon}(X_n)-\varphi_{K_\varepsilon}(X_\infty)|<\varepsilon/3,\qquad\forall n\ge n_\varepsilon. E ∣ φ K ε ( X n ) − φ K ε ( X ∞ ) ∣ < ε /3 , ∀ n ≥ n ε .
Combining the above results, we obtain
E ∣ X n − X ∞ ∣ ≤ E ∣ X n − φ K ε ( X n ) ∣ + E ∣ φ K ε ( X n ) − φ K ε ( X ∞ ) ∣ + E ∣ φ K ε ( X ∞ ) − X ∞ ∣ < ε . \mathbb{E}|X_n-X_\infty|\le\mathbb{E}|X_n-\varphi_{K_\varepsilon}(X_n)|+\mathbb{E}|\varphi_{K_\varepsilon}(X_n)-\varphi_{K_\varepsilon}(X_\infty)|+\mathbb{E}|\varphi_{K_\varepsilon}(X_\infty)-X_\infty|<\varepsilon. E ∣ X n − X ∞ ∣ ≤ E ∣ X n − φ K ε ( X n ) ∣ + E ∣ φ K ε ( X n ) − φ K ε ( X ∞ ) ∣ + E ∣ φ K ε ( X ∞ ) − X ∞ ∣ < ε .
As ε \varepsilon ε was arbitrary, it follows that X n → L 1 X ∞ X_n\xrightarrow{L^1}X_\infty X n L 1 X ∞ , as claimed. □
Remark 6.8.8. From Theorem 6.3.1 , we know that X n → a . s . X ∞ X_n \xrightarrow{\mathrm{a.s.}} X_\infty X n a.s. X ∞ implies X n → P X ∞ X_n \xrightarrow{\mathbb{P}} X_\infty X n P X ∞ . From this and Theorem 6.8.7 , it follows that
X n → a . s . X ∞ and ( X n ) is UI ⟹ X n → L 1 X ∞ . X_n \xrightarrow{\mathrm{a.s.}} X_\infty\text{ and } (X_n) \text{ is UI}
\qquad\Longrightarrow\qquad
X_n \xrightarrow{L^1} X_\infty. X n a.s. X ∞ and ( X n ) is UI ⟹ X n L 1 X ∞ .
We will show below that the set of conditional expectations of an integrable random variable form a class of uniformly integrable random variables. To prove this result, we require the following lemma.
Lemma 6.8.9. Suppose E ∣ X ∣ < ∞ \mathbb{E}|X| < \infty E ∣ X ∣ < ∞ . Then, for all ε > 0 \varepsilon> 0 ε > 0 there exists δ ε > 0 \delta_{\varepsilon} > 0 δ ε > 0 such that P ( A ) < δ ε \mathbb{P}(A) < \delta_{\varepsilon} P ( A ) < δ ε implies E ∣ X ∣ 1 A < ε \mathbb{E}|X|\mathbf{1}_A < \varepsilon E ∣ X ∣ 1 A < ε .
Proof. Suppose, by contradiction, that E ∣ X ∣ < ∞ \mathbb{E}|X| < \infty E ∣ X ∣ < ∞ and, for a given ε > 0 \varepsilon> 0 ε > 0 , there exists a sequence of sets ( A n ) (A_n) ( A n ) such that
P ( A n ) < 1 / 2 n , and E ∣ X ∣ 1 A n ≥ ε , ∀ n . \mathbb{P}(A_n) < 1/2^n,\qquad\text{and}\qquad\mathbb{E}|X|\mathbf{1}_{A_n} \ge\varepsilon,\qquad\forall n. P ( A n ) < 1/ 2 n , and E ∣ X ∣ 1 A n ≥ ε , ∀ n .
Define A : = { A n , i . o . } A := \{A_n,\ \mathrm{i.o.}\} A := { A n , i.o. } . As ∑ n = 1 ∞ P ( A n ) < ∞ \sum_{n=1}^{\infty}\mathbb{P}(A_n) < \infty ∑ n = 1 ∞ P ( A n ) < ∞ , we have by the Borel-Cantelli Lemma (see (70) ) that P ( A ) = 0 \mathbb{P}(A) = 0 P ( A ) = 0 and thus E ∣ X ∣ 1 A = 0 \mathbb{E}|X|\mathbf{1}_A = 0 E ∣ X ∣ 1 A = 0 . On the other hand we have
ε ≤ lim sup n → ∞ E ∣ X ∣ 1 A n ≤ E lim sup n → ∞ ∣ X ∣ 1 A n = E ∣ X ∣ 1 A , \varepsilon\le\limsup_{n\to\infty}\mathbb{E}|X|\mathbf{1}_{A_n} \le\mathbb{E}\limsup_{n\to\infty}|X|\mathbf{1}_{A_n} = \mathbb{E}|X|\mathbf{1}_A, ε ≤ n → ∞ lim sup E ∣ X ∣ 1 A n ≤ E n → ∞ lim sup ∣ X ∣ 1 A n = E ∣ X ∣ 1 A ,
which is a contradiction.
Theorem 6.8.10. Let X X X be a random variable on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and suppose E ∣ X ∣ < ∞ \mathbb{E}|X| < \infty E ∣ X ∣ < ∞ . Then the collection C \mathcal{C} C of random variables defined by
C = { E ( X ∣ G ) : G is a sub- σ -algebra of F } , \mathcal{C} = \{\mathbb{E}(X\mid\mathcal{G}) : \mathcal{G}\text{ is a sub-}\sigma\text{-algebra of }\mathcal{F}\}, C = { E ( X ∣ G ) : G is a sub- σ -algebra of F } , is uniformly integrable.
Proof. Fix ε > 0 \varepsilon> 0 ε > 0 . From Lemma 6.8.9 there exists δ ε > 0 \delta_{\varepsilon} > 0 δ ε > 0 such that P ( A ) < δ ε \mathbb{P}(A) < \delta_{\varepsilon} P ( A ) < δ ε implies E ∣ X ∣ 1 A < ε \mathbb{E}|X|\mathbf{1}_A < \varepsilon E ∣ X ∣ 1 A < ε . As E ∣ X ∣ < ∞ \mathbb{E}|X| < \infty E ∣ X ∣ < ∞ we can choose K ε > 0 K_{\varepsilon} > 0 K ε > 0 so that E ∣ X ∣ / K ε < δ ε \mathbb{E}|X|/K_{\varepsilon} < \delta_{\varepsilon} E ∣ X ∣/ K ε < δ ε . Now, for any G ⊆ F \mathcal{G} \subseteq\mathcal{F} G ⊆ F , we can define Y = E ( X ∣ G ) Y = \mathbb{E}(X\mid\mathcal{G}) Y = E ( X ∣ G ) . By Jensen’s inequality (see Theorem 2.3.4 ) we have
∣ Y ∣ = ∣ E ( X ∣ G ) ∣ ≤ E ( ∣ X ∣ ∣ G ) , P –a.s. (74) |Y| = |\mathbb{E}(X\mid\mathcal{G})| \le\mathbb{E}(|X|\mid\mathcal{G}),\qquad\mathbb{P}\text{--a.s.} \tag*{(74)} ∣ Y ∣ = ∣ E ( X ∣ G ) ∣ ≤ E ( ∣ X ∣ ∣ G ) , P –a.s. (74)
Hence, we have E ∣ Y ∣ ≤ E E ( ∣ X ∣ ∣ G ) = E ∣ X ∣ \mathbb{E}|Y| \le\mathbb{E}\mathbb{E}(|X|\mid\mathcal{G}) = \mathbb{E}|X| E ∣ Y ∣ ≤ EE ( ∣ X ∣ ∣ G ) = E ∣ X ∣ and, by the Markov inequality
K ε P ( ∣ Y ∣ > K ε ) ≤ E ∣ Y ∣ ≤ E ∣ X ∣ , K_{\varepsilon}\mathbb{P}(|Y| > K_{\varepsilon}) \le\mathbb{E}|Y| \le\mathbb{E}|X|, K ε P ( ∣ Y ∣ > K ε ) ≤ E ∣ Y ∣ ≤ E ∣ X ∣ ,
so that
P ( ∣ Y ∣ > K ε ) ≤ E ∣ X ∣ / K ε < δ ε . \mathbb{P}(|Y| > K_{\varepsilon}) \le\mathbb{E}|X|/K_{\varepsilon} < \delta_{\varepsilon}. P ( ∣ Y ∣ > K ε ) ≤ E ∣ X ∣/ K ε < δ ε .
Note that { ∣ Y ∣ > K ε } ∈ G \{|Y| > K_{\varepsilon}\} \in\mathcal{G} { ∣ Y ∣ > K ε } ∈ G . Thus, from (74) we have
E ∣ Y ∣ 1 { ∣ Y ∣ > K ε } ≤ E E ( ∣ X ∣ ∣ G ) 1 { ∣ Y ∣ > K ε } = E E ( ∣ X ∣ 1 { ∣ Y ∣ > K ε } ∣ G ) = E ∣ X ∣ 1 { ∣ Y ∣ > K ε } < ε . \mathbb{E}|Y|\mathbf{1}_{\{|Y|>K_{\varepsilon}\}} \le\mathbb{E}\mathbb{E}(|X|\mid\mathcal{G})\mathbf{1}_{\{|Y|>K_{\varepsilon}\}}
= \mathbb{E}\mathbb{E}(|X|\mathbf{1}_{\{|Y|>K_{\varepsilon}\}}\mid\mathcal{G})
= \mathbb{E}|X|\mathbf{1}_{\{|Y|>K_{\varepsilon}\}} < \varepsilon. E ∣ Y ∣ 1 { ∣ Y ∣ > K ε } ≤ EE ( ∣ X ∣ ∣ G ) 1 { ∣ Y ∣ > K ε } = EE ( ∣ X ∣ 1 { ∣ Y ∣ > K ε } ∣ G ) = E ∣ X ∣ 1 { ∣ Y ∣ > K ε } < ε .
As the above inequality holds for any G ⊆ F \mathcal{G} \subseteq\mathcal{F} G ⊆ F , the class C \mathcal{C} C is UI, as claimed.
Example 6.8.11 (Uniformly integrable martingale). Suppose X ∞ X_{\infty} X ∞ is a random variable defined on ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and E ∣ X ∞ ∣ < ∞ \mathbb{E}|X_{\infty}| < \infty E ∣ X ∞ ∣ < ∞ . Suppose ( F n ) (\mathcal{F}_n) ( F n ) is a filtration and define a sequence of random variables ( X n ) (X_n) ( X n ) by X n : = E [ X ∞ ∣ F n ] X_n := \mathbb{E}[X_{\infty}|\mathcal{F}_n] X n := E [ X ∞ ∣ F n ] . Then the process X = ( X n ) X = (X_n) X = ( X n ) is martingale because
E [ X n + 1 ∣ F n ] = E [ E [ X ∞ ∣ F n + 1 ] ∣ F n ] = E [ X ∞ ∣ F n ] = X n , \mathbb{E}[X_{n+1}|\mathcal{F}_n] = \mathbb{E}[\mathbb{E}[X_{\infty}|\mathcal{F}_{n+1}]|\mathcal{F}_n] = \mathbb{E}[X_{\infty}|\mathcal{F}_n] = X_n, E [ X n + 1 ∣ F n ] = E [ E [ X ∞ ∣ F n + 1 ] ∣ F n ] = E [ X ∞ ∣ F n ] = X n , and, moreover, ( X n ) (X_n) ( X n ) is uniformly integrable by Theorem 6.8.10 .
Exercises
Exercise 6.1. Let X 1 , X 2 , X 3 , … X_1, X_2, X_3, \ldots X 1 , X 2 , X 3 , … be a sequence of random variables such that
X n ∼ Geo ( λ / n ) , for n = 1 , 2 , 3 , … X_n \sim\operatorname{Geo}(\lambda/n), \qquad\text{for } n = 1,2,3,\ldots X n ∼ Geo ( λ / n ) , for n = 1 , 2 , 3 , …
where λ > 0 \lambda> 0 λ > 0 is a constant. Define a new sequence as
Y n : = 1 n X n , for n = 1 , 2 , 3 , … Y_n := \frac{1}{n}X_n, \qquad\text{for } n = 1,2,3,\ldots Y n := n 1 X n , for n = 1 , 2 , 3 , …
Show that Y n Y_n Y n converges in distribution to E ( λ ) \mathcal{E}(\lambda) E ( λ ) .
Exercise 6.2. Consider the sample space Ω = [ 0 , 1 ] \Omega= [0,1] Ω = [ 0 , 1 ] with uniform probability distribution, i.e.,
P ( ω ∈ [ a , b ] ) = b − a , ∀ 0 ≤ a ≤ b ≤ 1 \mathbb{P}(\omega\in[a,b]) = b-a, \qquad\forall\ 0 \leq a \leq b \leq1 P ( ω ∈ [ a , b ]) = b − a , ∀ 0 ≤ a ≤ b ≤ 1
Define the sequence { X n , n = 1 , 2 , … } \{X_n, n = 1,2,\ldots\} { X n , n = 1 , 2 , … } as X n ( ω ) = n n + 1 ω + ( 1 − ω ) n X_n(\omega) = \frac{n}{n+1}\omega+ (1-\omega)^n X n ( ω ) = n + 1 n ω + ( 1 − ω ) n . Also, define the random variable X X X on this sample space as X ( ω ) = ω X(\omega) = \omega X ( ω ) = ω . Show that X n → a . s . X X_n \xrightarrow{\mathrm{a.s.}} X X n a.s. X .
Exercise 6.3. Let { X n , n = 1 , 2 , … } \{X_n, n = 1,2,\ldots\} { X n , n = 1 , 2 , … } and { Y n , n = 1 , 2 , … } \{Y_n, n = 1,2,\ldots\} { Y n , n = 1 , 2 , … } be two sequences of random variables, defined on some probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Suppose that we know
X n → a . s . X , Y n → a . s . Y . X_n \xrightarrow{\mathrm{a.s.}} X, \qquad Y_n \xrightarrow{\mathrm{a.s.}} Y. X n a.s. X , Y n a.s. Y .
Prove that X n + Y n → a . s . X + Y X_n + Y_n \xrightarrow{\mathrm{a.s.}} X + Y X n + Y n a.s. X + Y .
Exercise 6.4. Let { X n , n = 1 , 2 , … } \{X_n, n = 1,2,\ldots\} { X n , n = 1 , 2 , … } and { Y n , n = 1 , 2 , … } \{Y_n, n = 1,2,\ldots\} { Y n , n = 1 , 2 , … } be two sequences of random variables, defined on some probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Suppose that we know
X n → P X , Y n → P Y . X_n \xrightarrow{\mathbb{P}} X, \qquad Y_n \xrightarrow{\mathbb{P}} Y. X n P X , Y n P Y .
Prove that X n + Y n → P X + Y X_n + Y_n \xrightarrow{\mathbb{P}} X + Y X n + Y n P X + Y .
Exercises
Exercise 6.5. Show that if X n X_n X n is any sequence of random variables, there are constants c n → ∞ c_n \to\infty c n → ∞ so that X n / c n → a . s . 0 X_n/c_n \xrightarrow{\mathrm{a.s.}} 0 X n / c n a.s. 0 .
Exercise 6.6. Let X 1 , X 2 , … X_1, X_2, \ldots X 1 , X 2 , … be independent with P ( X n = 1 ) = p n \mathrm{P}(X_n = 1) = p_n P ( X n = 1 ) = p n and P ( X n = 0 ) = 1 − p n \mathrm{P}(X_n = 0) = 1 - p_n P ( X n = 0 ) = 1 − p n . Show that
(a) X n → P 0 X_n \xrightarrow{\mathrm{P}} 0 X n P 0 if and only if p n → 0 p_n \to0 p n → 0 .
(b) X n → a . s . 0 X_n \xrightarrow{\mathrm{a.s.}} 0 X n a.s. 0 if and only if ∑ n p n < ∞ \sum_n p_n < \infty ∑ n p n < ∞ .
Exercise 6.7. Suppose that X 1 , X 2 , … X_1, X_2, \ldots X 1 , X 2 , … are independent with P ( X n > x ) = x − 5 \mathrm{P}(X_n > x) = x^{-5} P ( X n > x ) = x − 5 for all x ≥ 1 x \ge1 x ≥ 1 and n = 1 , 2 , … n = 1, 2, \ldots n = 1 , 2 , … Show that lim sup n → ∞ ( log X n ) / log n = c \limsup_{n \to\infty}(\log X_n)/\log n = c lim sup n → ∞ ( log X n ) / log n = c almost surely for some number c c c , and find c c c .
Brownian motion
The notes from this chapter are primarily taken from (Shreve, 2004, Chapter 3). The goals of this chapter are (i) to define what we mean by “Brownian motion” and (ii) to develop important properties of Brownian motion.
Scaled random walks
In order to construct Brownian motion, we will begin with a random walk. We return to the setting of § 4.3.2 . Our sample space Ω \Omega Ω and a generic element ω \omega ω in this sample space are given by
Ω = the set of infinite sequences of Hs and Ts , ω = ω 1 ω 2 ω 3 ⋯ \Omega= \text{the set of infinite sequences of Hs and Ts}, \qquad\omega= \omega_1\omega_2\omega_3\cdots Ω = the set of infinite sequences of Hs and Ts , ω = ω 1 ω 2 ω 3 ⋯
where ω i \omega_i ω i is the result of the i i i th coin toss. We take F n \mathcal{F}_n F n to be the σ \sigma σ -algebra generated by observing the first n n n coin tosses and we set F = σ ( ⋃ n F n ) \mathcal{F} = \sigma\left(\bigcup_n \mathcal{F}_n\right) F = σ ( ⋃ n F n ) . The coin tosses are assumed to be independent and we take P ( ω i = H ) = P ( ω i = T ) = ( 1 / 2 ) \mathbb{P}(\omega_i = \mathrm{H}) = \mathbb{P}(\omega_i = \mathrm{T}) = (1/2) P ( ω i = H ) = P ( ω i = T ) = ( 1/2 ) .
Symmetric random walk
We construct a symmetric random walk M = ( M i ) i ∈ N 0 \mathrm{M} = (\mathrm{M}_i)_{i\in\mathbb{N}_0} M = ( M i ) i ∈ N 0 by
M 0 = 0 , M k = ∑ j = 1 k X j , X i = { + 1 ω i = H , − 1 ω i = T . \mathrm{M}_0 = 0, \qquad\mathrm{M}_k = \sum_{j=1}^{k} X_j, \qquad X_i =
\begin{cases}
+1 & \omega_i = \mathrm{H},\\
-1 & \omega_i = \mathrm{T}.
\end{cases} M 0 = 0 , M k = j = 1 ∑ k X j , X i = { + 1 − 1 ω i = H , ω i = T .
Let k i < k i + 1 k_i < k_{i+1} k i < k i + 1 for all i ∈ N 0 i \in\mathbb{N}_0 i ∈ N 0 . As the ( X i ) i ∈ N (X_i)_{i\in\mathbb{N}} ( X i ) i ∈ N are independent, we clearly have
independent increments: ( M k 2 − M k 1 ) ⊥ ⊥ ( M k 4 − M k 3 ) . \text{independent increments:}\qquad(\mathrm{M}_{k_2}-\mathrm{M}_{k_1}) \perp\!\!\!\perp(\mathrm{M}_{k_4}-\mathrm{M}_{k_3}). independent increments: ( M k 2 − M k 1 ) ⊥ ⊥ ( M k 4 − M k 3 ) .
Moreover, as E X j = 0 \mathbb{E}X_j=0 E X j = 0 and V X j = 1 \mathbb{V}X_j=1 V X j = 1 for all j j j we also have
E ( M k 1 − M k 2 ) = ∑ j = k 1 + 1 k 2 E X j = 0 , V ( M k 2 − M k 1 ) = ∑ j = k 1 + 1 k 2 V X j = k 2 − k 1 . (75) \mathbb{E}(M_{k_1}-M_{k_2})=\sum_{j=k_1+1}^{k_2}\mathbb{E}X_j=0,\qquad\mathbb{V}(M_{k_2}-M_{k_1})=\sum_{j=k_1+1}^{k_2}\mathbb{V}X_j=k_2-k_1. \tag*{(75)} E ( M k 1 − M k 2 ) = j = k 1 + 1 ∑ k 2 E X j = 0 , V ( M k 2 − M k 1 ) = j = k 1 + 1 ∑ k 2 V X j = k 2 − k 1 . (75)
Thus, for the discrete time process M = ( M i ) i ≥ 0 M=(M_i)_{i\geq0} M = ( M i ) i ≥ 0 we see that variance accumulates at a rate of one per unit time.
Next, we note that M M M is a martingale with respect to the filtration F = ( F n ) n ∈ N 0 \mathcal{F}=(\mathcal{F}_n)_{n\in\mathbb{N}_0} F = ( F n ) n ∈ N 0 . To see this, let k ≤ l k\leq l k ≤ l and note that
E [ M l ∣ F k ] = E [ M l − M k + M k ∣ F k ] = E [ M l − M k ∣ F k ] + E [ M k ∣ F k ] = E ( M l − M k ) + M k . = M k , \begin{aligned}
\mathbb{E}[M_l\mid\mathcal{F}_k]=\mathbb{E}[M_l-M_k+M_k\mid\mathcal{F}_k]=\mathbb{E}[M_l-M_k\mid\mathcal{F}_k]+\mathbb{E}[M_k\mid\mathcal{F}_k] \\
&=\mathbb{E}(M_l-M_k)+M_k.=M_k,
\end{aligned} E [ M l ∣ F k ] = E [ M l − M k + M k ∣ F k ] = E [ M l − M k ∣ F k ] + E [ M k ∣ F k ] = E ( M l − M k ) + M k . = M k ,
where we have used the independent increments property ( M l − M k ) ⊥ ⊥ F k (M_l-M_k)\perp\!\!\perp\mathcal{F}_k ( M l − M k ) ⊥ ⊥ F k , equation (75) and the fact that M k ∈ F k M_k\in\mathcal{F}_k M k ∈ F k .
The quadratic variation of the symmetric random walk M M M up to time k k k , denoted [ M , M ] k [M,M]_k [ M , M ] k , is defined as
[ M , M ] k : = ∑ j = 1 k ( M j − M j − 1 ) 2 = ∑ j = 1 k X j 2 = k , [M,M]_k:=\sum_{j=1}^{k}(M_j-M_{j-1})^2=\sum_{j=1}^{k}X_j^2=k, [ M , M ] k := j = 1 ∑ k ( M j − M j − 1 ) 2 = j = 1 ∑ k X j 2 = k ,
where we have used X j 2 = 1 X_j^2=1 X j 2 = 1 . The astute reader will notice that [ M , M ] k = V M k [M,M]_k=\mathbb{V}M_k [ M , M ] k = V M k . However, it is important to note that the computation of variance and the computation of quadratic variation are different!
To see this, note that if P ( ω i = H ) = p ≠ ( 1 / 2 ) \mathbb{P}(\omega_i=\mathrm{H})=p\ne(1/2) P ( ω i = H ) = p = ( 1/2 ) then V X i ≠ 1 \mathbb{V}X_i\ne1 V X i = 1 and thus V M k ≠ k \mathbb{V}M_k\ne k V M k = k . However, since X j 2 = 1 X_j^2=1 X j 2 = 1 is unaffected by the value of p = P ( ω i = H ) p=\mathbb{P}(\omega_i=\mathrm{H}) p = P ( ω i = H ) , the computation of [ M , M ] k [M,M]_k [ M , M ] k is also unaffected by p p p . Another way to see that V M k \mathbb{V}M_k V M k and [ M , M ] k [M,M]_k [ M , M ] k are different is to note that V M k \mathbb{V}M_k V M k is a statistical quantity (i.e., it is an average over all ω \omega ω ) whereas [ M , M ] k [M,M]_k [ M , M ] k is computed ω \omega ω -by-ω \omega ω (it just turns out that, for each ω \omega ω we have [ M , M ] k ( ω ) = k [M,M]_k(\omega)=k [ M , M ] k ( ω ) = k .
Scaled symmetric random walk
We now fix a positive integer n n n . We construct a scaled symmetric random walk W ( n ) = ( W t ( n ) ) t ≥ 0 W^{(n)}=(W_t^{(n)})_{t\geq0} W ( n ) = ( W t ( n ) ) t ≥ 0 by
W t ( n ) = 1 n M n t , if n t ∈ N 0 . W_t^{(n)}=\frac{1}{\sqrt{n}}M_{nt},\qquad\text{if }nt\in\mathbb{N}_0. W t ( n ) = n 1 M n t , if n t ∈ N 0 .
If n t nt n t is not an integer, we define W t ( n ) W_t^{(n)} W t ( n ) to be the linear interpolation of M ⌊ n t ⌋ M_{\lfloor nt\rfloor} M ⌊ n t ⌋ and M ⌈ n t ⌉ M_{\lceil nt\rceil} M ⌈ n t ⌉ . Note that M M M was a discrete time process whereas W ( n ) W^{(n)} W ( n ) is a continuous time process.
Let 0 = t 0 < t 1 < t 2 < … 0=t_0<t_1<t_2<\ldots 0 = t 0 < t 1 < t 2 < … and suppose n t j ∈ N nt_j\in\mathbb{N} n t j ∈ N for every j j j . Then we have
independent increments: ( W t 2 ( n ) − W t 1 ( n ) ) ⊥ ⊥ ( W t 4 ( n ) − W t 3 ( n ) ) , \text{independent increments:}\qquad(W_{t_2}^{(n)}-W_{t_1}^{(n)})\perp\!\!\perp(W_{t_4}^{(n)}-W_{t_3}^{(n)}), independent increments: ( W t 2 ( n ) − W t 1 ( n ) ) ⊥ ⊥ ( W t 4 ( n ) − W t 3 ( n ) ) ,
because non-overlapping increments depend on different coin tosses. For example
W 0.2 ( 100 ) − W 0.0 ( 100 ) W^{(100)}_{0.2}-W^{(100)}_{0.0} W 0.2 ( 100 ) − W 0.0 ( 100 )
depends on coin tosses 1 through 20,
W 0.7 ( 100 ) − W 0.2 ( 100 ) W^{(100)}_{0.7}-W^{(100)}_{0.2} W 0.7 ( 100 ) − W 0.2 ( 100 )
depends on coin tosses 21 through 70.
Next, suppose 0 ≤ s ≤ t 0 \le s \le t 0 ≤ s ≤ t are such that n s ∈ N ns \in\mathbb{N} n s ∈ N and n t ∈ N nt \in\mathbb{N} n t ∈ N . Then
E ( W t ( n ) − W s ( n ) ) = 1 n E ( M n t − M n s ) = 0 , V ( W t ( n ) − W s ( n ) ) = 1 n V ( M n t − M n s ) = t − s . (76) \mathbb{E}\left(W^{(n)}_t-W^{(n)}_s\right)=\frac{1}{\sqrt{n}}\mathbb{E}\left(M_{nt}-M_{ns}\right)=0,\qquad
\mathbb{V}\left(W^{(n)}_t-W^{(n)}_s\right)=\frac{1}{n}\mathbb{V}\left(M_{nt}-M_{ns}\right)=t-s. \tag*{(76)} E ( W t ( n ) − W s ( n ) ) = n 1 E ( M n t − M n s ) = 0 , V ( W t ( n ) − W s ( n ) ) = n 1 V ( M n t − M n s ) = t − s . (76)
Now, let us define the scaled filtration F ( n ) = ( F t ( n ) ) t ≥ 0 \mathbb{F}^{(n)}=\left(\mathcal{F}^{(n)}_t\right)_{t\ge0} F ( n ) = ( F t ( n ) ) t ≥ 0 where
F t ( n ) : = F n t , if n t ∈ N 0 . \mathcal{F}^{(n)}_t:=\mathcal{F}_{nt},\qquad\text{if }nt\in\mathbb{N}_0. F t ( n ) := F n t , if n t ∈ N 0 .
The scaled random walk W ( n ) W^{(n)} W ( n ) , restricted to the set of t t t for which n t ∈ N 0 nt\in\mathbb{N}_0 n t ∈ N 0 , is a martingale with respect to the filtration F ( n ) \mathbb{F}^{(n)} F ( n ) . To see this, assume 0 ≤ s ≤ t 0\le s\le t 0 ≤ s ≤ t are such that n s ∈ N ns\in\mathbb{N} n s ∈ N and n t ∈ N nt\in\mathbb{N} n t ∈ N . Then we have
E [ W t ( n ) ∣ F s ( n ) ] = E [ W t ( n ) − W s ( n ) + W s ( n ) ∣ F s ( n ) ] = E [ W t ( n ) − W s ( n ) ∣ F s ( n ) ] + E [ W s ( n ) ∣ F s ( n ) ] = E ( W t ( n ) − W s ( n ) ) + W s ( n ) = W s ( n ) , \begin{aligned}
\mathbb{E}\left[W^{(n)}_t\mid\mathcal{F}^{(n)}_s\right]
&=\mathbb{E}\left[W^{(n)}_t-W^{(n)}_s+W^{(n)}_s\mid\mathcal{F}^{(n)}_s\right]\\
&=\mathbb{E}\left[W^{(n)}_t-W^{(n)}_s\mid\mathcal{F}^{(n)}_s\right]+\mathbb{E}\left[W^{(n)}_s\mid\mathcal{F}^{(n)}_s\right]\\
&=\mathbb{E}\left(W^{(n)}_t-W^{(n)}_s\right)+W^{(n)}_s=W^{(n)}_s,
\end{aligned} E [ W t ( n ) ∣ F s ( n ) ] = E [ W t ( n ) − W s ( n ) + W s ( n ) ∣ F s ( n ) ] = E [ W t ( n ) − W s ( n ) ∣ F s ( n ) ] + E [ W s ( n ) ∣ F s ( n ) ] = E ( W t ( n ) − W s ( n ) ) + W s ( n ) = W s ( n ) ,
where we have used ( W t ( n ) − W s ( n ) ) ⊥ ⊥ F s ( n ) \left(W^{(n)}_t-W^{(n)}_s\right)\perp\!\!\!\perp\mathcal{F}^{(n)}_s ( W t ( n ) − W s ( n ) ) ⊥ ⊥ F s ( n ) , equation (76) and W s ( n ) ∈ F s ( n ) W^{(n)}_s\in\mathcal{F}^{(n)}_s W s ( n ) ∈ F s ( n ) .
The quadratic variation of the scaled symmetric random walk W ( n ) W^{(n)} W ( n ) up to time t t t , denoted [ W ( n ) , W ( n ) ] t [W^{(n)},W^{(n)}]_t [ W ( n ) , W ( n ) ] t , is defined as
[ W ( n ) , W ( n ) ] t : = ∑ j = 1 n t ( W j / n ( n ) − W ( j − 1 ) / n ( n ) ) 2 = ∑ j = 1 n t ( 1 n X j ) 2 = ∑ j = 1 n t 1 n = t , [W^{(n)},W^{(n)}]_t:=\sum_{j=1}^{nt}\left(W^{(n)}_{j/n}-W^{(n)}_{(j-1)/n}\right)^2
=\sum_{j=1}^{nt}\left(\frac{1}{\sqrt{n}}X_j\right)^2
=\sum_{j=1}^{nt}\frac{1}{n}=t, [ W ( n ) , W ( n ) ] t := j = 1 ∑ n t ( W j / n ( n ) − W ( j − 1 ) / n ( n ) ) 2 = j = 1 ∑ n t ( n 1 X j ) 2 = j = 1 ∑ n t n 1 = t ,
where we again assume n t ∈ N 0 nt\in\mathbb{N}_0 n t ∈ N 0 . Thus, for the scaled symmetric random walk, we see that V W t ( n ) = [ W ( n ) , W ( n ) ] t \mathbb{V}W^{(n)}_t=[W^{(n)},W^{(n)}]_t V W t ( n ) = [ W ( n ) , W ( n ) ] t . However, we emphasize one more time that the computation of variance is a statistical average over all ω \omega ω and the computation of quadratic variation is done ω \omega ω -by-ω \omega ω .
Theorem 7.1.1. Fix t ≥ 0 t\ge0 t ≥ 0 . Then W t ( n ) → D Z W^{(n)}_t\xrightarrow{\mathcal{D}}Z W t ( n ) D Z as n → ∞ n\to\infty n → ∞ , where Z ∼ N ( 0 , t ) Z\sim\mathcal{N}(0,t) Z ∼ N ( 0 , t ) .
Proof. We will use the Continuity Theorem 3.3.8. We have
φ W t ( n ) ( u ) = E exp ( i u W t ( n ) ) = E exp ( i u n ∑ j = 1 n t X j ) = ∏ j = 1 n t E exp ( i u n X j ) = ( 1 2 e i u / n + 1 2 e − i u / n ) n t \begin{aligned}
\varphi_{W^{(n)}_t}(u)
&=\mathbb{E}\exp\left(iuW^{(n)}_t\right)
=\mathbb{E}\exp\left(\frac{iu}{\sqrt{n}}\sum_{j=1}^{nt}X_j\right)\\
&=\prod_{j=1}^{nt}\mathbb{E}\exp\left(\frac{iu}{\sqrt{n}}X_j\right)\\
&=\left(\frac{1}{2}e^{iu/\sqrt{n}}+\frac{1}{2}e^{-iu/\sqrt{n}}\right)^{nt}
\end{aligned} φ W t ( n ) ( u ) = E exp ( i u W t ( n ) ) = E exp ( n i u j = 1 ∑ n t X j ) = j = 1 ∏ n t E exp ( n i u X j ) = ( 2 1 e i u / n + 2 1 e − i u / n ) n t
( 1 2 + i u 2 n − u 2 4 n + O ( 1 n 3 / 2 ) + 1 2 − i u 2 n − u 2 4 n + O ( 1 n 3 / 2 ) ) n t \left(\frac{1}{2}+\frac{\mathrm{i}u}{2\sqrt{n}}-\frac{u^2}{4n}+O\left(\frac{1}{n^{3/2}}\right)+\frac{1}{2}-\frac{\mathrm{i}u}{2\sqrt{n}}-\frac{u^2}{4n}+O\left(\frac{1}{n^{3/2}}\right)\right)^{nt} ( 2 1 + 2 n i u − 4 n u 2 + O ( n 3/2 1 ) + 2 1 − 2 n i u − 4 n u 2 + O ( n 3/2 1 ) ) n t
( 1 − u 2 2 n + O ( 1 n 3 / 2 ) ) n t → e − 1 2 t u 2 , as n → ∞ . \left(1-\frac{u^2}{2n}+O\left(\frac{1}{n^{3/2}}\right)\right)^{nt}\to e^{-\frac{1}{2}tu^2},\qquad\text{as }n\to\infty. ( 1 − 2 n u 2 + O ( n 3/2 1 ) ) n t → e − 2 1 t u 2 , as n → ∞.
From Example 3.3.4, we know that if a random variable Z Z Z is normally distributed Z ∼ N ( 0 , t ) Z\sim\mathcal{N}(0,t) Z ∼ N ( 0 , t ) then its characteristic function φ Z \varphi_Z φ Z is given by
φ Z ( u ) = e − 1 2 t u 2 . \varphi_Z(u)=e^{-\frac{1}{2}tu^2}. φ Z ( u ) = e − 2 1 t u 2 .
As lim n → ∞ φ W t ( n ) → φ Z \lim_{n\to\infty}\varphi_{W_t^{(n)}}\to\varphi_Z lim n → ∞ φ W t ( n ) → φ Z , we have from the Continuity Theorem 3.3.8 that W t ( n ) → D N ( 0 , t ) W_t^{(n)}\xrightarrow{\mathcal{D}}\mathcal{N}(0,t) W t ( n ) D N ( 0 , t ) , as claimed.
Brownian motion
Definition 7.2.1. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space. A Brownian motion is a stochastic process W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 that satisfies:
W 0 = 0 W_0=0 W 0 = 0 .
Independent increments: if 0 ≤ t 0 < t 1 < ⋯ < t n < ∞ 0\leq t_0<t_1<\cdots<t_n<\infty 0 ≤ t 0 < t 1 < ⋯ < t n < ∞ then the increments W t 1 − W t 0 , W t 2 − W t 1 , … , W t n − W t n − 1 W_{t_1}-W_{t_0},W_{t_2}-W_{t_1},\ldots,W_{t_n}-W_{t_{n-1}} W t 1 − W t 0 , W t 2 − W t 1 , … , W t n − W t n − 1 are independent.
If 0 ≤ r < s 0\leq r<s 0 ≤ r < s then W s − W r ∼ N ( 0 , s − r ) W_s-W_r\sim\mathcal{N}(0,s-r) W s − W r ∼ N ( 0 , s − r ) .
The map t ↦ W t t\mapsto W_t t ↦ W t is continuous for every ω \omega ω .
It is clear from the previous sections that we can construct a Brownian motion as a limit of a scaled symmetric random walk. Had we simply given Definition 7.2.1 at the beginning of this chapter with no further introduction, one might have legitimately asked if there exists a process that satisfies the properties of a Brownian motion. There are other methods to prove the existence of Brownian motion. But, the scaled random walk construction is perhaps the most intuitive.
What is Ω \Omega Ω in Definition 7.2.1? It could be an infinite series of Hs and Ts, representing movements up and down of a scaled random walk. Or, it could be Ω = C 0 ( R + ) \Omega=C_0(\mathbb{R}_+) Ω = C 0 ( R + ) , the set of continuous functions on R + \mathbb{R}_+ R + , starting from zero. In this case, an element of Ω \Omega Ω is a continuous function t ↦ ω ( t ) t\mapsto\omega(t) t ↦ ω ( t ) and one could simply take W t ( ω ) = ω ( t ) W_t(\omega)=\omega(t) W t ( ω ) = ω ( t ) . Whatever the sample space, the probability of any single element ω \omega ω is zero: P ( ω ) = 0 \mathbb{P}(\omega)=0 P ( ω ) = 0 , but probabilities such as P ( W t ≤ 0 ) \mathbb{P}(W_t\leq0) P ( W t ≤ 0 ) are well-defined.
It is useful to note that
Cov [ W T , W t ] = E W T W t = E ( W T − W t + W t ) W t = E ( W T − W t ) W t + E W t 2 = E W t E [ W T − W t ∣ W t ] + t = t . (77) \begin{aligned}
\operatorname{Cov}[W_T,W_t]=\mathbb{E}W_TW_t=\mathbb{E}(W_T-W_t+W_t)W_t \\
&=\mathbb{E}(W_T-W_t)W_t+\mathbb{E}W_t^2=\mathbb{E}W_t\mathbb{E}[W_T-W_t\mid W_t]+t=t. \tag*{(77)}
\end{aligned} Cov [ W T , W t ] = E W T W t = E ( W T − W t + W t ) W t = E ( W T − W t ) W t + E W t 2 = E W t E [ W T − W t ∣ W t ] + t = t . (77)
Thus, Cov [ W s , W t ] = s ∧ t \operatorname{Cov}[W_s,W_t]=s\wedge t Cov [ W s , W t ] = s ∧ t .
Definition 7.2.2. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be on probability space on which a Brownian motion W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 is defined. A filtration for the Brownian motion W W W is a collection of σ \sigma σ -algebras F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 satisfying:
Information accumulates: if 0 ≤ s < t 0\leq s<t 0 ≤ s < t then F s ⊂ F t \mathcal{F}_s\subset\mathcal{F}_t F s ⊂ F t .
Adaptivity: for all t ≥ 0 t\geq0 t ≥ 0 , we have W t ∈ F t W_t\in\mathcal{F}_t W t ∈ F t .
Independence of future increments: if u > t ≥ 0 u>t\geq0 u > t ≥ 0 then ( W u − W t ) ⊥ ⊥ F t (W_u-W_t)\perp\!\!\!\perp\mathcal{F}_t ( W u − W t ) ⊥ ⊥ F t .
The most natural choice for this filtration F \mathbb{F} F is the natural filtration for W W W . That is F t = σ ( W u , 0 ≤ u ≤ t ) \mathcal{F}_t=\sigma(W_u,0\leq u\leq t) F t = σ ( W u , 0 ≤ u ≤ t ) . In principle the filtration ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\geq0} ( F t ) t ≥ 0 could contain more than the information obtained by observing W W W . However, the information in the filtration is not allowed to destroy the independence of future increments of Brownian motion.
Not surprisingly, if F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 is a filtration for a Brownian motion W W W then W W W is a martingale with respect to this filtration. We see this, let 0 ≤ s < t 0\leq s<t 0 ≤ s < t and observe that
E [ W t ∣ F s ] = E [ W t − W s + W s ∣ F s ] = E [ W t − W s ∣ F s ] + E [ W s ∣ F s ] = E [ W t − W s ] + W s = W s . \mathbb{E}[W_t\mid\mathcal{F}_s]=\mathbb{E}[W_t-W_s+W_s\mid\mathcal{F}_s]=\mathbb{E}[W_t-W_s\mid\mathcal{F}_s]+\mathbb{E}[W_s\mid\mathcal{F}_s]=\mathbb{E}[W_t-W_s]+W_s=W_s. E [ W t ∣ F s ] = E [ W t − W s + W s ∣ F s ] = E [ W t − W s ∣ F s ] + E [ W s ∣ F s ] = E [ W t − W s ] + W s = W s .
Quadratic variation
In § 18.1 , we computed the quadratic variation of a symmetric random walk M M M and a scaled symmetric random walk W ( n ) W^{(n)} W ( n ) . However, we did not rigorously define the notion of quadratic variation. In this Section we will define what we mean by “quadratic variation” and we will compute this quantity for a Brownian motion W W W . Before defining quadratic variation we first introduce the “first variation.”
Let Π \Pi Π be a partition of [ 0 , T ] [0,T] [ 0 , T ] and define ∥ Π ∥ \|\Pi\| ∥Π∥ as
Π = { t 0 , t 1 , … , t n } , 0 = t 0 < t 1 < … < t n = T , ∥ Π ∥ = max i ( t i + 1 − t i ) . (78) \Pi=\{t_0,t_1,\ldots,t_n\},\qquad0=t_0<t_1<\ldots<t_n=T,\qquad\|\Pi\|=\max_i(t_{i+1}-t_i). \tag*{(78)} Π = { t 0 , t 1 , … , t n } , 0 = t 0 < t 1 < … < t n = T , ∥Π∥ = i max ( t i + 1 − t i ) . (78)
Let f : [ 0 , T ] → R f:[0,T]\to\mathbb{R} f : [ 0 , T ] → R . We define the first variation of f f f up to time T T T , denoted F V T ( f ) \mathrm{FV}_T(f) FV T ( f ) , by
F V T ( f ) : = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ∣ f ( t j + 1 ) − f ( t j ) ∣ . \mathrm{FV}_T(f):=\lim_{\|\Pi\|\to0}\sum_{j=0}^{n-1}|f(t_{j+1})-f(t_j)|. FV T ( f ) := ∥Π∥ → 0 lim j = 0 ∑ n − 1 ∣ f ( t j + 1 ) − f ( t j ) ∣.
Suppose that f ∈ C ( [ 0 , T ] ) f\in C([0,T]) f ∈ C ([ 0 , T ]) and f ′ ( t ) f'(t) f ′ ( t ) exists and is finite for all t ∈ ( 0 , T ) t\in(0,T) t ∈ ( 0 , T ) . Then, by the Mean Value Theorem , there exits t j ∗ ∈ [ t j , t j + 1 ] t_j^*\in[t_j,t_{j+1}] t j ∗ ∈ [ t j , t j + 1 ] such that
f ( t j + 1 ) − f ( t j ) = f ′ ( t j ∗ ) ( t j + 1 − t j ) . f(t_{j+1})-f(t_j)=f'(t_j^*)(t_{j+1}-t_j). f ( t j + 1 ) − f ( t j ) = f ′ ( t j ∗ ) ( t j + 1 − t j ) .
Thus, if f ∈ C ( [ 0 , T ] ) f\in C([0,T]) f ∈ C ([ 0 , T ]) and f ′ ( t ) f'(t) f ′ ( t ) exists and is finite for all t ∈ ( 0 , T ) t\in(0,T) t ∈ ( 0 , T ) , we have
F V T ( f ) : = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ∣ f ′ ( t j ∗ ) ∣ ( t j + 1 − t j ) = ∫ 0 T d t ∣ f ′ ( t ) ∣ . \mathrm{FV}_T(f):=\lim_{\|\Pi\|\to0}\sum_{j=0}^{n-1}|f'(t_j^*)|(t_{j+1}-t_j)=\int_0^T\mathrm{d}t|f'(t)|. FV T ( f ) := ∥Π∥ → 0 lim j = 0 ∑ n − 1 ∣ f ′ ( t j ∗ ) ∣ ( t j + 1 − t j ) = ∫ 0 T d t ∣ f ′ ( t ) ∣.
Definition 7.3.1. Let f : [ 0 , T ] → R f : [0,T] \to\mathbb{R} f : [ 0 , T ] → R . We define the quadratic variation of f f f up to time T T T , denoted [ f , f ] T [f,f]_{T} [ f , f ] T as
[ f , f ] T : = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 [ f ( t j + 1 ) − f ( t j ) ] 2 , [f,f]_{T} := \lim_{\lVert\Pi\rVert\to0} \sum_{j=0}^{n-1} [f(t_{j+1}) - f(t_j)]^2, [ f , f ] T := ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 [ f ( t j + 1 ) − f ( t j ) ] 2 , where Π \Pi Π and ∥ Π ∥ \lVert\Pi\rVert ∥ Π ∥ are as defined in (78) .
Proposition 7.3.2. Suppose f : [ 0 , T ] → R f : [0,T] \to\mathbb{R} f : [ 0 , T ] → R has a continuous first derivative on the closed interval [ 0 , T ] [0,T] [ 0 , T ] : f ∈ C 1 ( [ 0 , T ] ) f \in C^1([0,T]) f ∈ C 1 ([ 0 , T ]) . Then [ f , f ] T = 0 [f,f]_{T} = 0 [ f , f ] T = 0 .
Proof. For any partition Π \Pi Π of [ 0 , T ] [0,T] [ 0 , T ] we have
∑ j = 0 n − 1 [ f ( t j + 1 ) − f ( t j ) ] 2 = ∑ j = 0 n − 1 [ f ′ ( t j ∗ ) ] 2 ( t j + 1 − t j ) 2 ≤ ∥ Π ∥ ∑ j = 0 n − 1 [ f ′ ( t j ∗ ) ] 2 ( t j + 1 − t j ) . \sum_{j=0}^{n-1} [f(t_{j+1}) - f(t_j)]^2 = \sum_{j=0}^{n-1} [f'(t_j^*)]^2(t_{j+1}-t_j)^2 \leq\lVert\Pi\rVert\sum_{j=0}^{n-1} [f'(t_j^*)]^2(t_{j+1}-t_j). j = 0 ∑ n − 1 [ f ( t j + 1 ) − f ( t j ) ] 2 = j = 0 ∑ n − 1 [ f ′ ( t j ∗ ) ] 2 ( t j + 1 − t j ) 2 ≤ ∥ Π ∥ j = 0 ∑ n − 1 [ f ′ ( t j ∗ ) ] 2 ( t j + 1 − t j ) .
Inserting the above inequality into the definition of [ f , f ] T [f,f]_{T} [ f , f ] T , we obtain
[ f , f ] T ≤ lim ∥ Π ∥ → 0 ∥ Π ∥ ⋅ lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 [ f ′ ( t j ∗ ) ] 2 ( t j + 1 − t j ) = 0 ⋅ ∫ 0 T d t ∣ f ′ ( t ) ∣ 2 = 0 , [f,f]_{T} \leq\lim_{\lVert\Pi\rVert\to0} \lVert\Pi\rVert\cdot\lim_{\lVert\Pi\rVert\to0} \sum_{j=0}^{n-1} [f'(t_j^*)]^2(t_{j+1}-t_j) = 0 \cdot\int_0^T \mathrm{d}t\,|f'(t)|^2 = 0, [ f , f ] T ≤ ∥ Π ∥ → 0 lim ∥ Π ∥ ⋅ ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 [ f ′ ( t j ∗ ) ] 2 ( t j + 1 − t j ) = 0 ⋅ ∫ 0 T d t ∣ f ′ ( t ) ∣ 2 = 0 ,
where we have used ∫ 0 T d t ∣ f ′ ( t ) ∣ 2 < ∞ \int_0^T \mathrm{d}t\,|f'(t)|^2 < \infty ∫ 0 T d t ∣ f ′ ( t ) ∣ 2 < ∞ as, by assumption, f ∈ C 1 ( ( 0 , T ) ) f \in C^1((0,T)) f ∈ C 1 (( 0 , T )) .
In ordinary calculus, we typically deal with functions f ∈ C 1 f \in C^1 f ∈ C 1 , and hence [ f , f ] T = 0 [f,f]_{T} = 0 [ f , f ] T = 0 . For this reason, quadratic variation never arises in usual calculus. However, it turns out that for almost every ω ∈ Ω \omega\in\Omega ω ∈ Ω , we have that t ↦ W t ( ω ) t \mapsto W_t(\omega) t ↦ W t ( ω ) is not differentiable. We can see this from the scaled random walk construction of Brownian motion. The slope of the scaled random walk W t ( n ) W_t^{(n)} W t ( n ) at any t t t for which d d t W t ( n ) ( ω ) \frac{\mathrm{d}}{\mathrm{d}t}W_t^{(n)}(\omega) d t d W t ( n ) ( ω ) is defined is
lim ε → 0 W t + ε ( n ) − W t ( n ) ε = ± n → ± ∞ as n → ∞ . \lim_{\varepsilon\to0} \frac{W_{t+\varepsilon}^{(n)} - W_t^{(n)}}{\varepsilon} = \pm\sqrt{n} \to\pm\infty\qquad\text{as } n \to\infty. ε → 0 lim ε W t + ε ( n ) − W t ( n ) = ± n → ± ∞ as n → ∞.
Thus, Brownian motion, which we constructed as a limit of a scaled random walk W ( ω ) : = lim n → ∞ W ( n ) ( ω ) W(\omega) := \lim_{n\to\infty} W^{(n)}(\omega) W ( ω ) := lim n → ∞ W ( n ) ( ω ) is not differentiable at any t t t , P \mathbb{P} P -a.s.. When a function f ∉ C 1 ( ( 0 , T ) ) f \notin C^1((0,T)) f ∈ / C 1 (( 0 , T )) , then Proposition 7.3.2 can fail. Indeed, as we will show, paths of BM have strictly positive quadratic variation. It is for this reason that stochastic calculus is different from ordinary calculus.
Theorem 7.3.3. Let W W W be a Brownian motion. Then, for all T ≥ 0 T \geq0 T ≥ 0 we have [ W , W ] T = T [W,W]_{T} = T [ W , W ] T = T , where the limit in Definition 7.3.1 is taken in L 2 ( Ω , F , P ) L^2(\Omega,\mathcal{F},\mathbb{P}) L 2 ( Ω , F , P ) . Moreover, for any sequence of partitions ( Π k ) k ≥ 1 (\Pi_k)_{k\geq1} ( Π k ) k ≥ 1 of [ 0 , T ] [0,T] [ 0 , T ] satisfying ∥ Π k ∥ → 0 \lVert\Pi_k\rVert\to0 ∥ Π k ∥ → 0 , there exists a subsequence ( Π k j ) j ≥ 1 (\Pi_{k_j})_{j\geq1} ( Π k j ) j ≥ 1 along which the convergence holds P \mathbb{P} P -almost surely.
Proof. For a fixed partition Π = { 0 = t 0 , t 1 , … , t n = T } \Pi= \{0 = t_0, t_1, \ldots, t_n = T\} Π = { 0 = t 0 , t 1 , … , t n = T } we define the sampled quadratic variation of W W W , denoted Q Π Q_{\Pi} Q Π , by
Q Π : = ∑ j = 0 n − 1 ( W t j + 1 − W t j ) 2 . Q_{\Pi} := \sum_{j=0}^{n-1} (W_{t_{j+1}} - W_{t_j})^2. Q Π := j = 0 ∑ n − 1 ( W t j + 1 − W t j ) 2 .
Step 1: L 2 L^2 L 2 convergence. We will show that E Q Π = T \mathbb{E}Q_{\Pi} = T E Q Π = T and V Q Π → 0 \mathop{\mathrm{V}}Q_{\Pi} \to0 V Q Π → 0 as ∥ Π ∥ → 0 \lVert\Pi\rVert\to0 ∥ Π ∥ → 0 , which implies that Q Π → T Q_{\Pi} \to T Q Π → T in L 2 ( Ω , F , P ) L^2(\Omega,\mathcal{F},\mathbb{P}) L 2 ( Ω , F , P ) . To begin, we use the fact that W t j + 1 − W t j ∼ N ( 0 , t j + 1 − t j ) W_{t_{j+1}} - W_{t_j} \sim\mathcal{N}(0,t_{j+1} - t_j) W t j + 1 − W t j ∼ N ( 0 , t j + 1 − t j ) to compute
E ( W t j + 1 − W t j ) 2 = t j + 1 − t j , (79) \mathbb{E}(W_{t_{j+1}} - W_{t_j})^2 = t_{j+1} - t_j, \tag*{(79)} E ( W t j + 1 − W t j ) 2 = t j + 1 − t j , (79)
V ( W t j + 1 − W t j ) 2 = E ( W t j + 1 − W t j ) 4 − ( E ( W t j + 1 − W t j ) 2 ) 2 = 3 ( t j + 1 − t j ) 2 − ( t j + 1 − t j ) 2 = 2 ( t j + 1 − t j ) 2 . (80) \begin{aligned}
\mathop{\mathrm{V}}(W_{t_{j+1}} - W_{t_j})^2
&= \mathbb{E}(W_{t_{j+1}} - W_{t_j})^4 - \left(\mathbb{E}(W_{t_{j+1}} - W_{t_j})^2\right)^2 \\
&= 3(t_{j+1} - t_j)^2 - (t_{j+1} - t_j)^2 = 2(t_{j+1} - t_j)^2. \tag*{(80)}
\end{aligned} V ( W t j + 1 − W t j ) 2 = E ( W t j + 1 − W t j ) 4 − ( E ( W t j + 1 − W t j ) 2 ) 2 = 3 ( t j + 1 − t j ) 2 − ( t j + 1 − t j ) 2 = 2 ( t j + 1 − t j ) 2 . (80)
Next, using (79) , (80) and the independence of the increments of W W W , we obtain
E Q Π = ∑ j = 0 n − 1 E ( W t j + 1 − W t j ) 2 = ∑ j = 0 n − 1 ( t j + 1 − t j ) = T , \mathbb{E}Q_{\Pi} = \sum_{j=0}^{n-1} \mathbb{E}(W_{t_{j+1}} - W_{t_j})^2 = \sum_{j=0}^{n-1}(t_{j+1} - t_j) = T, E Q Π = j = 0 ∑ n − 1 E ( W t j + 1 − W t j ) 2 = j = 0 ∑ n − 1 ( t j + 1 − t j ) = T ,
V Q Π = ∑ j = 0 n − 1 V ( W t j + 1 − W t j ) 2 = ∑ j = 0 n − 1 2 ( t j + 1 − t j ) 2 ≤ ∥ Π ∥ ∑ j = 0 n − 1 2 ( t j + 1 − t j ) = 2 ∥ Π ∥ T . \mathop{\mathrm{V}}Q_{\Pi} = \sum_{j=0}^{n-1} \mathop{\mathrm{V}}(W_{t_{j+1}} - W_{t_j})^2 = \sum_{j=0}^{n-1}2(t_{j+1} - t_j)^2 \le\lVert\Pi\rVert\sum_{j=0}^{n-1}2(t_{j+1} - t_j) = 2\lVert\Pi\rVert T. V Q Π = j = 0 ∑ n − 1 V ( W t j + 1 − W t j ) 2 = j = 0 ∑ n − 1 2 ( t j + 1 − t j ) 2 ≤ ∥ Π ∥ j = 0 ∑ n − 1 2 ( t j + 1 − t j ) = 2 ∥ Π ∥ T .
As E Q Π = T \mathbb{E}Q_{\Pi} = T E Q Π = T , we have
E ( Q Π − T ) 2 = V Q Π ≤ 2 ∥ Π ∥ T → 0 , as ∥ Π ∥ → 0 , (81) \mathbb{E}(Q_{\Pi} - T)^2 = \mathop{\mathrm{V}}Q_{\Pi} \le2\lVert\Pi\rVert T \to0, \qquad\text{as } \lVert\Pi\rVert\to0, \tag*{(81)} E ( Q Π − T ) 2 = V Q Π ≤ 2 ∥ Π ∥ T → 0 , as ∥ Π ∥ → 0 , (81)
which proves that [ W , W ] T = lim ∥ Π ∥ → 0 Q Π = T [W,W]_T = \lim_{\lVert\Pi\rVert\to0} Q_{\Pi} = T [ W , W ] T = lim ∥ Π ∥ → 0 Q Π = T in L 2 ( Ω , F , P ) L^2(\Omega,\mathcal{F},\mathbb{P}) L 2 ( Ω , F , P ) .
Step 2: P \mathbb{P} P -a.s. convergence along a subsequence. Let ( Π k ) k ≥ 1 (\Pi_k)_{k\ge1} ( Π k ) k ≥ 1 be a sequence of partitions of [ 0 , T ] [0,T] [ 0 , T ] with ∥ Π k ∥ → 0 \lVert\Pi_k\rVert\to0 ∥ Π k ∥ → 0 . Choose a subsequence ( Π k j ) j ≥ 1 (\Pi_{k_j})_{j\ge1} ( Π k j ) j ≥ 1 such that ∥ Π k j ∥ ≤ 2 − j \lVert\Pi_{k_j}\rVert\le2^{-j} ∥ Π k j ∥ ≤ 2 − j for all j j j . Fix ε > 0 \varepsilon> 0 ε > 0 . Using Chebyshev’s inequality (i.e., Markov’s inequality (68) applied to ( Q Π − T ) 2 (Q_{\Pi} - T)^2 ( Q Π − T ) 2 ) and (81) , we have
P ( ∣ Q Π k j − T ∣ > ε ) ≤ 1 ε 2 E ( Q Π k j − T ) 2 ≤ 2 T ∥ Π k j ∥ ε 2 ≤ 2 T ε 2 2 − j , \mathbb{P}(|Q_{\Pi_{k_j}} - T| > \varepsilon) \le\frac{1}{\varepsilon^2}\mathbb{E}(Q_{\Pi_{k_j}} - T)^2 \le\frac{2T\lVert\Pi_{k_j}\rVert}{\varepsilon^2} \le\frac{2T}{\varepsilon^2}2^{-j}, P ( ∣ Q Π k j − T ∣ > ε ) ≤ ε 2 1 E ( Q Π k j − T ) 2 ≤ ε 2 2 T ∥ Π k j ∥ ≤ ε 2 2 T 2 − j ,
which is summable in j j j . Thus, by the Borel-Cantelli Lemma 6.6.1, we have P ( ∣ Q Π k j − T ∣ > ε i.o. ) = 0 \mathbb{P}(|Q_{\Pi_{k_j}} - T| > \varepsilon\ \text{i.o.}) = 0 P ( ∣ Q Π k j − T ∣ > ε i.o. ) = 0 . Taking ε = 1 / m \varepsilon= 1/m ε = 1/ m and using the fact that a countable union of null sets is a null set, we obtain
P ( ⋃ m = 1 ∞ { ∣ Q Π k j − T ∣ > 1 / m i.o. } ) = 0 , \mathbb{P}\left(\bigcup_{m=1}^{\infty}\{|Q_{\Pi_{k_j}} - T| > 1/m\ \text{i.o.}\}\right) = 0, P ( m = 1 ⋃ ∞ { ∣ Q Π k j − T ∣ > 1/ m i.o. } ) = 0 ,
which is equivalent to Q Π k j → T Q_{\Pi_{k_j}} \to T Q Π k j → T P \mathbb{P} P -a.s. as $j \to \infty. (The same argument shows that no subsequence is needed if ∑ k ∥ Π k ∥ < ∞ . ) \sum_k \lVert\Pi_k\rVert< \infty.) ∑ k ∥ Π k ∥ < ∞. )
Suppose d t ≪ 1 dt \ll1 d t ≪ 1 and define d W t : = W t + d t − W t dW_t := W_{t+dt} - W_t d W t := W t + d t − W t . The above computations show that E ( ( d W t ) 2 ) = d t \mathbb{E}((dW_t)^2) = dt E (( d W t ) 2 ) = d t and V ( ( d W t ) 2 ) = 2 d t 2 \mathbb{V}((dW_t)^2) = 2dt^2 V (( d W t ) 2 ) = 2 d t 2 . Since d t 2 dt^2 d t 2 is practically zero for d t ≪ 1 dt \ll1 d t ≪ 1 , one can imagine that ( d W t ) 2 (dW_t)^2 ( d W t ) 2 is almost equal to a constant d t dt d t . Informally , we write this as
d W t d W t = d t . (82) dW_t dW_t = dt. \tag*{(82)} d W t d W t = d t . (82)
This informal statement, while not rigorously correct, captures the spirit of the quadratic variation computation for W W W .
Definition 7.3.4. Let f , g : [ 0 , T ] → R f,g : [0,T] \to\mathbb{R} f , g : [ 0 , T ] → R . We define the covariation of f f f and g g g up to time T T T , denoted [ f , g ] T [f,g]_T [ f , g ] T as
[ f , g ] T : = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 [ f ( t j + 1 ) − f ( t j ) ] [ g ( t j + 1 ) − g ( t j ) ] . [f,g]_T := \lim_{\lVert\Pi\rVert\to0} \sum_{j=0}^{n-1} [f(t_{j+1})-f(t_j)] [g(t_{j+1})-g(t_j)]. [ f , g ] T := ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 [ f ( t j + 1 ) − f ( t j )] [ g ( t j + 1 ) − g ( t j )] . where Π \Pi Π and ∥ Π ∥ \lVert\Pi\rVert ∥ Π ∥ are as defined in (78) .
Theorem 7.3.5. Let W W W be a Brownian motion and let Id \operatorname{Id} Id be the identity function: Id ( t ) = t \operatorname{Id}(t)=t Id ( t ) = t . Then, for all T ≥ 0 T \ge0 T ≥ 0 we have [ W , Id ] T = 0 [W,\operatorname{Id}]_T = 0 [ W , Id ] T = 0 almost surely and [ Id , Id ] T = 0 [\operatorname{Id},\operatorname{Id}]_T = 0 [ Id , Id ] T = 0 .
Proof. For a fixed partition Π = { 0 = t 0 , t 1 , … , t n = T } \Pi= \{0=t_0,t_1,\ldots,t_n=T\} Π = { 0 = t 0 , t 1 , … , t n = T } we have
∣ [ W , Id ] T ∣ = ∣ lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ( W t j + 1 − W t j ) ⋅ ( t j + 1 − t j ) ∣ = lim ∥ Π ∥ → 0 ∣ ∑ j = 0 n − 1 ( W t j + 1 − W t j ) ⋅ ( t j + 1 − t j ) ∣ ≤ lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ∣ ( W t j + 1 − W t j ) ⋅ ( t j + 1 − t j ) ∣ ≤ lim ∥ Π ∥ → 0 max 0 ≤ j ≤ n − 1 ∣ W t j + 1 − W t j ∣ ∑ j = 0 n − 1 ( t j + 1 − t j ) = T ⋅ lim ∥ Π ∥ → 0 max 0 ≤ j ≤ n − 1 ∣ W t j + 1 − W t j ∣ = T ⋅ 0 = 0. \begin{aligned}
\left|[W,\operatorname{Id}]_T\right|
&= \left|\lim_{\lVert\Pi\rVert\to0}\sum_{j=0}^{n-1}(W_{t_{j+1}}-W_{t_j})\cdot(t_{j+1}-t_j)\right| \\
&= \lim_{\lVert\Pi\rVert\to0}\left|\sum_{j=0}^{n-1}(W_{t_{j+1}}-W_{t_j})\cdot(t_{j+1}-t_j)\right| \\
&\le\lim_{\lVert\Pi\rVert\to0}\sum_{j=0}^{n-1}\left|(W_{t_{j+1}}-W_{t_j})\cdot(t_{j+1}-t_j)\right| \\
&\le\lim_{\lVert\Pi\rVert\to0}\max_{0\le j\le n-1}|W_{t_{j+1}}-W_{t_j}|\sum_{j=0}^{n-1}(t_{j+1}-t_j) \\
&= T\cdot\lim_{\lVert\Pi\rVert\to0}\max_{0\le j\le n-1}|W_{t_{j+1}}-W_{t_j}| = T\cdot0=0.
\end{aligned} ∣ [ W , Id ] T ∣ = ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 ( W t j + 1 − W t j ) ⋅ ( t j + 1 − t j ) = ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 ( W t j + 1 − W t j ) ⋅ ( t j + 1 − t j ) ≤ ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 ( W t j + 1 − W t j ) ⋅ ( t j + 1 − t j ) ≤ ∥ Π ∥ → 0 lim 0 ≤ j ≤ n − 1 max ∣ W t j + 1 − W t j ∣ j = 0 ∑ n − 1 ( t j + 1 − t j ) = T ⋅ ∥ Π ∥ → 0 lim 0 ≤ j ≤ n − 1 max ∣ W t j + 1 − W t j ∣ = T ⋅ 0 = 0.
where, in the last step we are using the fact that W W W is continuous. To see that [ Id , Id ] T = 0 [\operatorname{Id},\operatorname{Id}]_T=0 [ Id , Id ] T = 0 , simply note that g ∈ C 1 ( [ 0 , T ] ) g\in C^1([0,T]) g ∈ C 1 ([ 0 , T ]) and use Proposition 7.3.2 .
Just as (82) captures the spirit of the computation of [ W , W ] T [W,W]_T [ W , W ] T , the following equations
d W t d t = 0 , d t d t = 0 , dW_tdt=0,\qquad dtdt=0, d W t d t = 0 , d t d t = 0 ,
informally capture the spirit of the [ W , Id ] T [W,\operatorname{Id}]_T [ W , Id ] T and [ Id , Id ] T [\operatorname{Id},\operatorname{Id}]_T [ Id , Id ] T computations.
Markov property of Brownian motion
Theorem 7.4.1. Let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion and let ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\geq0} ( F t ) t ≥ 0 be a filtration for this Brownian motion. Then W W W is a Markov process.
Proof. According to Definition 2.3.6 , we must show that there exists a function g g g such that
E [ f ( W T ) ∣ F t ] = g ( W t ) , \mathbb{E}[f(W_T)\mid\mathcal{F}_t]=g(W_t), E [ f ( W T ) ∣ F t ] = g ( W t ) ,
where T ≥ t ≥ 0 T\geq t\geq0 T ≥ t ≥ 0 . Noting that W t ∈ F t W_t\in\mathcal{F}_t W t ∈ F t and W T − W t ⊥ ⊥ F t W_T-W_t\perp\!\!\!\perp\mathcal{F}_t W T − W t ⊥ ⊥ F t , we have
E [ f ( W T ) ∣ F t ] = E [ f ( W T − W t + W t ) ∣ F t ] = ∫ R d y f ( y + W t ) Γ ( t , 0 ; T , y ) = : g ( W t ) , \mathbb{E}[f(W_T)\mid\mathcal{F}_t]=\mathbb{E}[f(W_T-W_t+W_t)\mid\mathcal{F}_t]=\int_{\mathbb{R}} \mathrm{d}y\,f(y+W_t)\Gamma(t,0;T,y)=:g(W_t), E [ f ( W T ) ∣ F t ] = E [ f ( W T − W t + W t ) ∣ F t ] = ∫ R d y f ( y + W t ) Γ ( t , 0 ; T , y ) =: g ( W t ) ,
where Γ ( t , x ; T , ⋅ ) \Gamma(t,x;T,\cdot) Γ ( t , x ; T , ⋅ ) is the density of a normal random variable with mean x x x and variance T − t T-t T − t .
First hitting time of Brownian motion
Theorem 7.5.1. Let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion with a filtration ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\geq0} ( F t ) t ≥ 0 . Define a process Z = ( Z t ) t ≥ 0 Z=(Z_t)_{t\geq0} Z = ( Z t ) t ≥ 0 by
Z t = e − 1 2 σ 2 t + σ W t . (83) Z_t=e^{-\frac{1}{2}\sigma^2t+\sigma W_t}. \tag*{(83)} Z t = e − 2 1 σ 2 t + σ W t . (83) Then Z Z Z is a martingale with respect to ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\geq0} ( F t ) t ≥ 0 .
Proof. The proof is a simple computation. Fixing T ≥ t T\geq t T ≥ t we have
E [ Z T ∣ F t ] = E [ e − 1 2 σ 2 T + σ W T ∣ F t ] = e − 1 2 σ 2 T + σ W t E [ e σ ( W T − W t ) ∣ F t ] = e − 1 2 σ 2 T + σ W t e 1 2 σ 2 ( T − t ) = e − 1 2 σ 2 t + σ W t = Z t . \begin{aligned}
\mathbb{E}[Z_T\mid\mathcal{F}_t]=\mathbb{E}\left[e^{-\frac{1}{2}\sigma^2T+\sigma W_T}\mid\mathcal{F}_t\right]=e^{-\frac{1}{2}\sigma^2T+\sigma W_t}\mathbb{E}\left[e^{\sigma(W_T-W_t)}\mid\mathcal{F}_t\right] \\
&=e^{-\frac{1}{2}\sigma^2T+\sigma W_t}e^{\frac{1}{2}\sigma^2(T-t)}=e^{-\frac{1}{2}\sigma^2t+\sigma W_t}=Z_t.
\end{aligned} E [ Z T ∣ F t ] = E [ e − 2 1 σ 2 T + σ W T ∣ F t ] = e − 2 1 σ 2 T + σ W t E [ e σ ( W T − W t ) ∣ F t ] = e − 2 1 σ 2 T + σ W t e 2 1 σ 2 ( T − t ) = e − 2 1 σ 2 t + σ W t = Z t .
Thus, Z Z Z is a martingale.
The process Z Z Z is sometimes referred to as an exponential martingale . The exponential martingale will be used to compute the distribution of
τ m : = inf { t ≥ 0 : W t = m } . (84) \tau_m:=\inf\{t\geq0:W_t=m\}. \tag*{(84)} τ m := inf { t ≥ 0 : W t = m } . (84)
We call τ m \tau_m τ m the first hitting time or first passage time of a Brownian motion W W W to level m m m .
Theorem 7.5.2. For any m ∈ R m\in\mathbb{R} m ∈ R and α > 0 \alpha>0 α > 0 we have
E e − α τ m = e − ∣ m ∣ 2 α . (85) \mathbb{E}e^{-\alpha\tau_m}=e^{-|m|\sqrt{2\alpha}}. \tag*{(85)} E e − α τ m = e − ∣ m ∣ 2 α . (85) where τ m \tau_m τ m is defined in (84) .
Proof. First, assume m ≥ 0 m \ge0 m ≥ 0 . Let Z ( m ) = ( Z t ( m ) ) t ≥ 0 Z^{(m)} = (Z_t^{(m)})_{t\ge0} Z ( m ) = ( Z t ( m ) ) t ≥ 0 be given by
Z t ( m ) : = Z t ∧ τ m , Z_t^{(m)} := Z_{t\wedge\tau_m}, Z t ( m ) := Z t ∧ τ m ,
where Z Z Z is given by (83) . We call Z ( m ) Z^{(m)} Z ( m ) a stopped process as it remains at Z τ m Z_{\tau_m} Z τ m forever after W W W hits m m m . As Z Z Z is a martingale it follows that the stopped process Z ( m ) Z^{(m)} Z ( m ) is also a martingale. Thus, we have
1 = Z 0 ( m ) = E Z t ( m ) = E e − 1 2 σ 2 t ∧ τ m + σ W t ∧ τ m . 1 = Z_0^{(m)} = \mathbb{E}Z_t^{(m)} = \mathbb{E}e^{-\frac{1}{2}\sigma^2t\wedge\tau_m+\sigma W_{t\wedge\tau_m}}. 1 = Z 0 ( m ) = E Z t ( m ) = E e − 2 1 σ 2 t ∧ τ m + σ W t ∧ τ m .
Now, it turns out that P ( τ m < ∞ ) = 1 \mathbb{P}(\tau_m < \infty) = 1 P ( τ m < ∞ ) = 1 (a fact that can be proved with relative ease). As a result, we have lim t → ∞ t ∧ τ m = τ m \lim_{t\to\infty}t\wedge\tau_m = \tau_m lim t → ∞ t ∧ τ m = τ m and W τ m = m W_{\tau_m} = m W τ m = m . Thus, we obtain
1 = lim t → ∞ E e − 1 2 σ 2 t ∧ τ m + σ W t ∧ τ m = E lim t → ∞ e − 1 2 σ 2 t ∧ τ m + σ W t ∧ τ m = E e − 1 2 σ 2 τ m + σ m , 1 = \lim_{t\to\infty}\mathbb{E}e^{-\frac{1}{2}\sigma^2t\wedge\tau_m+\sigma W_{t\wedge\tau_m}} = \mathbb{E}\lim_{t\to\infty}e^{-\frac{1}{2}\sigma^2t\wedge\tau_m+\sigma W_{t\wedge\tau_m}} = \mathbb{E}e^{-\frac{1}{2}\sigma^2\tau_m+\sigma m}, 1 = t → ∞ lim E e − 2 1 σ 2 t ∧ τ m + σ W t ∧ τ m = E t → ∞ lim e − 2 1 σ 2 t ∧ τ m + σ W t ∧ τ m = E e − 2 1 σ 2 τ m + σ m ,
where passing the limit through the expectation is allowed because Z t ( m ) ≤ e σ m Z_t^{(m)} \le e^{\sigma m} Z t ( m ) ≤ e σ m (dominated convergence). Setting σ = 2 α \sigma= \sqrt{2\alpha} σ = 2 α we obtain E e − α τ m = e − m 2 α \mathbb{E}e^{-\alpha\tau_m} = e^{-m\sqrt{2\alpha}} E e − α τ m = e − m 2 α , which agrees with (85) for m ≥ 0 m \ge0 m ≥ 0 . To obtain (85) for m < 0 m < 0 m < 0 , simply note that, since Brownian motion is symmetric about zero, the distribution of τ m \tau_m τ m is the same as the distribution of τ − m \tau_{-m} τ − m .
We can compute E τ m \mathbb{E}\tau_m E τ m as follows
E τ m = lim α ↓ 0 − d d α E e − α τ m = lim α ↓ 0 − d d α e − ∣ m ∣ 2 α = lim α ↓ 0 ∣ m ∣ 2 α e − ∣ m ∣ 2 α = ∞ , m ≠ 0. \mathbb{E}\tau_m = \lim_{\alpha\downarrow0}-\frac{\mathrm{d}}{\mathrm{d}\alpha}\mathbb{E}e^{-\alpha\tau_m} = \lim_{\alpha\downarrow0}-\frac{\mathrm{d}}{\mathrm{d}\alpha}e^{-|m|\sqrt{2\alpha}} = \lim_{\alpha\downarrow0}\frac{|m|}{\sqrt{2\alpha}}e^{-|m|\sqrt{2\alpha}} = \infty,\qquad m\ne0. E τ m = α ↓ 0 lim − d α d E e − α τ m = α ↓ 0 lim − d α d e − ∣ m ∣ 2 α = α ↓ 0 lim 2 α ∣ m ∣ e − ∣ m ∣ 2 α = ∞ , m = 0.
Thus, while P ( τ m < ∞ ) = 1 \mathbb{P}(\tau_m < \infty) = 1 P ( τ m < ∞ ) = 1 we have E τ m = ∞ \mathbb{E}\tau_m = \infty E τ m = ∞ .
Reflection principle
For every sample path of Brownian motion that hits a level m > 0 m > 0 m > 0 prior to time t t t and finishes at level W t = w ≤ m W_t = w \le m W t = w ≤ m there is an equally likely path that finishes at a level W t = 2 m − w W_t = 2m-w W t = 2 m − w . In Figure 7.1 these two paths are represented by the solid and dashed lines, respectively. The dotted lines from top to bottom represent the levels 2 m − w 2m-w 2 m − w , m m m and w w w , and τ m \tau_m τ m is the first time the path hits level m m m . It follows that
Figure 7.1. An illustration of the reflection principle for Brownian motion.
P ( τ m ≤ t , W t ≤ w ) = P ( W t ≥ 2 m − w ) , m > 0 , w ≤ m . (86) \mathbb{P}(\tau_m \le t,W_t \le w) = \mathbb{P}(W_t \ge2m-w),\qquad m > 0,\qquad w \le m. \tag*{(86)} P ( τ m ≤ t , W t ≤ w ) = P ( W t ≥ 2 m − w ) , m > 0 , w ≤ m . (86)
We will use this fact to find the density of τ m \tau_m τ m .
Theorem 7.6.1. For all m ≠ 0 m \ne0 m = 0 , the first hitting time τ m \tau_m τ m of Brownian motion to level m m m has a density f τ m f_{\tau_m} f τ m , which is given by
f τ m ( t ) = 1 { t ≥ 0 } ∣ m ∣ t 2 π t e − m 2 / ( 2 t ) . (87) f_{\tau_m}(t) = \mathbf{1}_{\{t\ge0\}}\frac{|m|}{t\sqrt{2\pi t}}e^{-m^2/(2t)}. \tag*{(87)} f τ m ( t ) = 1 { t ≥ 0 } t 2 π t ∣ m ∣ e − m 2 / ( 2 t ) . (87)
Proof. Consider the case m > 0 m > 0 m > 0 . Substitute w = m w = m w = m into (86) to obtain
P ( τ m ≤ t , W t ≤ m ) = P ( W t ≥ m ) . \mathbb{P}(\tau_m \le t, W_t \le m) = \mathbb{P}(W_t \ge m). P ( τ m ≤ t , W t ≤ m ) = P ( W t ≥ m ) .
Now, note that W t ≥ m W_t \ge m W t ≥ m implies that τ m ≤ t \tau_m \le t τ m ≤ t . And thus
P ( τ m ≤ t , W t ≥ m ) = P ( W t ≥ m ) . \mathbb{P}(\tau_m \le t, W_t \ge m) = \mathbb{P}(W_t \ge m). P ( τ m ≤ t , W t ≥ m ) = P ( W t ≥ m ) .
Adding the above two equations, we obtain
2 P ( W t ≥ m ) = P ( τ m ≤ t , W t ≤ m ) + P ( τ m ≤ t , W t ≥ m ) = P ( τ m ≤ t ) . 2\mathbb{P}(W_t \ge m) = \mathbb{P}(\tau_m \le t, W_t \le m) + \mathbb{P}(\tau_m \le t, W_t \ge m) = \mathbb{P}(\tau_m \le t). 2 P ( W t ≥ m ) = P ( τ m ≤ t , W t ≤ m ) + P ( τ m ≤ t , W t ≥ m ) = P ( τ m ≤ t ) .
The density (87) is obtained using
f τ m ( t ) = d d t P ( τ m ≤ t ) = d d t 2 P ( W t ≥ m ) = d d t 2 ∫ m ∞ 1 2 π t exp ( − x 2 2 t ) d x = d d t 2 ∫ m / t ∞ 1 2 π exp ( − y 2 2 ) d y = m t 2 π t e − m 2 / ( 2 t ) . \begin{aligned}
f_{\tau_m}(t) &= \frac{\mathrm{d}}{\mathrm{d}t}\mathbb{P}(\tau_m \le t) = \frac{\mathrm{d}}{\mathrm{d}t}2\mathbb{P}(W_t \ge m) = \frac{\mathrm{d}}{\mathrm{d}t}2\int_m^\infty\frac{1}{\sqrt{2\pi t}}\exp\left(-\frac{x^2}{2t}\right)\,\mathrm{d}x \\
&= \frac{\mathrm{d}}{\mathrm{d}t}2\int_{m/\sqrt{t}}^\infty\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{y^2}{2}\right)\,\mathrm{d}y = \frac{m}{t\sqrt{2\pi t}}e^{-m^2/(2t)}.
\end{aligned} f τ m ( t ) = d t d P ( τ m ≤ t ) = d t d 2 P ( W t ≥ m ) = d t d 2 ∫ m ∞ 2 π t 1 exp ( − 2 t x 2 ) d x = d t d 2 ∫ m / t ∞ 2 π 1 exp ( − 2 y 2 ) d y = t 2 π t m e − m 2 / ( 2 t ) .
The case m ≤ 0 m \le0 m ≤ 0 is proved in a similar manner.
Now, let us define the running maximum , denoted W ‾ = ( W ‾ t ) t ≥ 0 \overline{W} = (\overline{W}_t)_{t\ge0} W = ( W t ) t ≥ 0 , of Brownian motion
W ‾ t = max 0 ≤ s ≤ t W s . \overline{W}_t = \max_{0\le s\le t} W_s. W t = 0 ≤ s ≤ t max W s .
Note that W ‾ t ≥ m \overline{W}_t \ge m W t ≥ m if and only if τ m ≤ t \tau_m \le t τ m ≤ t . Thus, we have from (86) that
P ( W ‾ t ≥ m , W t ≤ w ) = P ( W t ≥ 2 m − w ) , m > 0 , w ≤ m . (88) \mathbb{P}(\overline{W}_t \ge m, W_t \le w) = \mathbb{P}(W_t \ge2m-w), \qquad m > 0,\qquad w \le m. \tag*{(88)} P ( W t ≥ m , W t ≤ w ) = P ( W t ≥ 2 m − w ) , m > 0 , w ≤ m . (88)
We can use (88) to obtain the joint density of ( W t , W ‾ t ) (W_t,\overline{W}_t) ( W t , W t ) .
Theorem 7.6.2. For any t > 0 t > 0 t > 0 , the joint density of Brownian motion W t W_t W t and its running maximum W ‾ t \overline{W}_t W t is
f W t , W ‾ t ( w , m ) = 2 ( 2 m − w ) t 2 π t e − ( 2 m − w ) 2 / ( 2 t ) , m > 0 , w ≤ m . (89) f_{W_t,\overline{W}_t}(w,m) = \frac{2(2m-w)}{t\sqrt{2\pi t}}e^{-(2m-w)^2/(2t)}, \qquad m > 0,\qquad w \le m. \tag*{(89)} f W t , W t ( w , m ) = t 2 π t 2 ( 2 m − w ) e − ( 2 m − w ) 2 / ( 2 t ) , m > 0 , w ≤ m . (89)
Proof. Note that
P ( W t ≤ w , W ‾ t ≤ m ) = P ( W t ≤ w ) − P ( W t ≤ w , W ‾ t ≥ m ) = P ( W t ≤ w ) − P ( W t ≥ 2 m − w ) = P ( W t ≤ w ) − ∫ 2 m − w ∞ d x f W t ( x ) , \begin{aligned}
\mathbb{P}(W_t \le w,\overline{W}_t \le m) = \mathbb{P}(W_t \le w) - \mathbb{P}(W_t \le w,\overline{W}_t \ge m) \\
&= \mathbb{P}(W_t \le w) - \mathbb{P}(W_t \ge2m-w) = \mathbb{P}(W_t \le w) - \int_{2m-w}^{\infty} \mathrm{d}x\,f_{W_t}(x),
\end{aligned} P ( W t ≤ w , W t ≤ m ) = P ( W t ≤ w ) − P ( W t ≤ w , W t ≥ m ) = P ( W t ≤ w ) − P ( W t ≥ 2 m − w ) = P ( W t ≤ w ) − ∫ 2 m − w ∞ d x f W t ( x ) ,
where f W t f_{W_t} f W t is the density of W t W_t W t , which is a N ( 0 , t ) \mathcal{N}(0,t) N ( 0 , t ) random variable. To obtain the density (89) use
f W t , W ‾ t ( w , m ) = ∂ 2 ∂ m ∂ w P ( W t ≤ w , W ‾ t ≤ m ) = − ∂ 2 ∂ m ∂ w ∫ 2 m − w ∞ d x f W t ( x ) . f_{W_t,\overline{W}_t}(w,m) = \frac{\partial^2}{\partial m\partial w}\mathbb{P}(W_t \le w,\overline{W}_t \le m) = -\frac{\partial^2}{\partial m\partial w}\int_{2m-w}^{\infty} \mathrm{d}x\,f_{W_t}(x). f W t , W t ( w , m ) = ∂ m ∂ w ∂ 2 P ( W t ≤ w , W t ≤ m ) = − ∂ m ∂ w ∂ 2 ∫ 2 m − w ∞ d x f W t ( x ) .
The rest of the computation is algebra.
Corollary 7.6.3. For any t > 0 t > 0 t > 0 , the conditional density of W ‾ t \overline{W}_t W t given W t W_t W t is given by
f W ‾ t ∣ W t ( m , w ) = 2 ( 2 m − w ) t e − 2 m ( m − w ) / t , m > 0 , w ≤ m . (90) f_{\overline{W}_t\mid W_t}(m,w) = \frac{2(2m-w)}{t}e^{-2m(m-w)/t}, \qquad m > 0,\qquad w \le m. \tag*{(90)} f W t ∣ W t ( m , w ) = t 2 ( 2 m − w ) e − 2 m ( m − w ) / t , m > 0 , w ≤ m . (90)
Proof. Expression (90) follows directly from
f W ‾ t ∣ W t ( m , w ) = f W t , W ‾ t ( w , m ) f W t ( w ) . f_{\overline{W}_t\mid W_t}(m,w) = \frac{f_{W_t,\overline{W}_t}(w,m)}{f_{W_t}(w)}. f W t ∣ W t ( m , w ) = f W t ( w ) f W t , W t ( w , m ) .
where f W t f_{W_t} f W t is the density of W t W_t W t , which is N ( 0 , t ) \mathcal{N}(0,t) N ( 0 , t ) .
Exercises
Exercise 7.1. Let W W W be a Brownian motion and let F \mathcal{F} F be a filtration for W W W . Show that W t 2 − t W_t^2-t W t 2 − t is a martingale with respect to the filtration F \mathcal{F} F .
Exercise 7.2. Compute the characteristic function of W N t W_{N_t} W N t where N N N is a Poisson process with intensity λ \lambda λ and the Brownian motion W W W is independent of the Poisson process N N N .
Exercise 7.3. The m m m th variation of a function f f f , over the interval [ 0 , T ] [0,T] [ 0 , T ] is defined as
V T ( m , f ) : = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ∣ f ( t j + 1 ) − f ( t j ) ∣ m , Π = { 0 = t 0 , t 1 , … t n = T } , ∥ Π ∥ = max j ( t j + 1 − t j ) . V_T(m,f) := \lim_{\lVert\Pi\rVert\to0}\sum_{j=0}^{n-1}\lvert f(t_{j+1})-f(t_j)\rvert^m,\qquad\Pi=\{0=t_0,t_1,\ldots t_n=T\},\qquad\lVert\Pi\rVert=\max_j(t_{j+1}-t_j). V T ( m , f ) := ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 ∣ f ( t j + 1 ) − f ( t j ) ∣ m , Π = { 0 = t 0 , t 1 , … t n = T } , ∥ Π ∥ = j max ( t j + 1 − t j ) .
Show that V T ( 1 , W ) = ∞ V_T(1,W)=\infty V T ( 1 , W ) = ∞ and V T ( 3 , W ) = 0 V_T(3,W)=0 V T ( 3 , W ) = 0 , where W W W is a Brownian motion.
Exercises
Exercise 7.4. Define
X t = μ t + W t , τ m : = inf { t ≥ 0 : X t = m } . X_t=\mu t+W_t,\qquad\tau_m:=\inf\{t\geq0:X_t=m\}. X t = μ t + W t , τ m := inf { t ≥ 0 : X t = m } .
where W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 is a Brownian motion. Let F = ( F t ) t ≥ 0 \mathbb F=(\mathcal F_t)_{t\geq0} F = ( F t ) t ≥ 0 be a filtration for W W W . Show that Z Z Z is a martingale with respect to F \mathbb F F where
Z t = exp ( σ X t − ( σ μ + σ 2 2 ) t ) . Z_t=\exp\left(\sigma X_t-\left(\sigma\mu+\frac{\sigma^2}{2}\right)t\right). Z t = exp ( σ X t − ( σ μ + 2 σ 2 ) t ) .
Assume μ > 0 \mu>0 μ > 0 and m ≥ 0 m\geq0 m ≥ 0 . Assume further that τ m < ∞ \tau_m<\infty τ m < ∞ with probability one and the stopped process Z t ∧ τ m Z_{t\wedge\tau_m} Z t ∧ τ m is a martingale. Find the Laplace transform E e − α τ m \mathbb E e^{-\alpha\tau_m} E e − α τ m .
Exercise 7.5. Prove that
[ f , W ] T = 0 , [f,W]_T=0, [ f , W ] T = 0 ,
where f ∈ C 1 ( [ 0 , T ] ) f\in C^1([0,T]) f ∈ C 1 ([ 0 , T ]) and W W W is a Brownian motion.
Exercise 7.6. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal F,\mathbb P) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb F=(\mathcal F_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T be a filtration for a Brownian motion W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T . Define a process Z = ( Z t ) 0 ≤ t ≤ T Z=(Z_t)_{0\leq t\leq T} Z = ( Z t ) 0 ≤ t ≤ T by
Z t : = e u W t − u 2 t / 2 , u ∈ R . Z_t:=e^{uW_t-u^2t/2},\qquad u\in\mathbb R. Z t := e u W t − u 2 t /2 , u ∈ R .
(a) Prove that the process Z Z Z is a martingale under P \mathbb P P with respect to the filtration F \mathbb F F .
(b) Using the fact that Z Z Z is a martingale, show that the following processes M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\leq t\leq T} M = ( M t ) 0 ≤ t ≤ T and N = ( N t ) 0 ≤ t ≤ T N=(N_t)_{0\leq t\leq T} N = ( N t ) 0 ≤ t ≤ T are martingales, where
M t : = W t 2 − t , N t = W t 3 − 3 t W t . M_t:=W_t^2-t,\qquad N_t=W_t^3-3tW_t. M t := W t 2 − t , N t = W t 3 − 3 t W t .
Note: if X ∼ N ( μ , σ 2 ) X\sim\mathcal N(\mu,\sigma^2) X ∼ N ( μ , σ 2 ) then E e u X = e μ u + u 2 σ 2 / 2 \mathbb E e^{uX}=e^{\mu u+u^2\sigma^2/2} E e u X = e μu + u 2 σ 2 /2 .
Exercise 7.7. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal F,\mathbb P) ( Ω , F , P ) and a filtration F = ( F t ) 0 ≤ t ≤ T \mathbb F=(\mathcal F_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T . We say that a C \mathbb C C -valued process is a ( P , F ) (\mathbb P,\mathbb F) ( P , F ) -martingale if, for any 0 ≤ s ≤ t ≤ T 0\leq s\leq t\leq T 0 ≤ s ≤ t ≤ T the following holds E ( Z t ∣ F s ) = Z s \mathbb E(Z_t\mid\mathcal F_s)=Z_s E ( Z t ∣ F s ) = Z s . Note that if we decompose Z Z Z into its Real and Imaginary parts Z = X + i Y Z=X+iY Z = X + iY , then we must have E ( X t ∣ F s ) = X s \mathbb E(X_t\mid\mathcal F_s)=X_s E ( X t ∣ F s ) = X s and E ( Y t ∣ F s ) = Y s \mathbb E(Y_t\mid\mathcal F_s)=Y_s E ( Y t ∣ F s ) = Y s . Use this information to prove that the processes M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\leq t\leq T} M = ( M t ) 0 ≤ t ≤ T and N = ( N t ) 0 ≤ t ≤ T N=(N_t)_{0\leq t\leq T} N = ( N t ) 0 ≤ t ≤ T , defined as follows
M t : = e u 2 t / 2 cos ( u W t ) , N t = e u 2 t / 2 sin ( u W t ) , u ∈ R , M_t:=e^{u^2t/2}\cos(uW_t),\qquad N_t=e^{u^2t/2}\sin(uW_t),\qquad u\in\mathbb R, M t := e u 2 t /2 cos ( u W t ) , N t = e u 2 t /2 sin ( u W t ) , u ∈ R ,
are ( P , F ) (\mathbb P,\mathbb F) ( P , F ) -martingales, where W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T is a Brownian motion and F \mathbb F F is a filtration for W W W .
Note: if V ∼ N ( μ , σ 2 ) V\sim\mathcal N(\mu,\sigma^2) V ∼ N ( μ , σ 2 ) then E e i u V = e i μ u − u 2 σ 2 / 2 \mathbb E e^{iuV}=e^{i\mu u-u^2\sigma^2/2} E e i u V = e i μu − u 2 σ 2 /2 .
Exercise 7.8. Fix a probability space ( Ω , F , P ) (\Omega, \mathcal{F}, \mathbb{P}) ( Ω , F , P ) and let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion. Define the running maximum W ‾ = ( W ‾ t ) t ≥ 0 \overline{W}=(\overline{W}_t)_{t\geq0} W = ( W t ) t ≥ 0 of W W W by
W ‾ t : = max 0 ≤ s ≤ t W s . \overline{W}_t:=\max_{0\leq s\leq t}W_s. W t := 0 ≤ s ≤ t max W s .
Recall that, as a consequence of the reflection principle, the joint density of ( W t , W ‾ t ) (W_t,\overline{W}_t) ( W t , W t ) is given by
f W t , W ‾ t ( w , m ) = 2 ( 2 m − w ) t 2 π t e − ( 2 m − w ) 2 / ( 2 t ) , m > 0 , w ≤ m . f_{W_t,\overline{W}_t}(w,m)=\frac{2(2m-w)}{t\sqrt{2\pi t}}e^{-(2m-w)^2/(2t)},\qquad m>0,\qquad w\leq m. f W t , W t ( w , m ) = t 2 π t 2 ( 2 m − w ) e − ( 2 m − w ) 2 / ( 2 t ) , m > 0 , w ≤ m .
(a) Compute the probability density function f W ‾ t f_{\overline{W}_t} f W t of W ‾ t \overline{W}_t W t . Show that W ‾ t \overline{W}_t W t has the same probability density function as ∣ W t ∣ |W_t| ∣ W t ∣ .
(b) Compute E W ‾ t \mathbb{E}\overline{W}_t E W t .
(c) Fix m > 0 m>0 m > 0 and w ≤ m w\leq m w ≤ m . Compute P ( W ‾ t ≥ m ∣ W t = w ) \mathbb{P}(\overline{W}_t\geq m\mid W_t=w) P ( W t ≥ m ∣ W t = w ) .
Note: the density of W t W_t W t is f W t ( w ) = 1 2 π t e − w 2 / ( 2 t ) f_{W_t}(w)=\frac{1}{\sqrt{2\pi t}}e^{-w^2/(2t)} f W t ( w ) = 2 π t 1 e − w 2 / ( 2 t ) .
Stochastic calculus
The notes from this chapter are taken primarily from (Shreve, 2004, Chapters 4 and 5).
Itô Integrals
If a function g : [ 0 , T ] → R g : [0,T] \to\mathbb{R} g : [ 0 , T ] → R satisfies g ∈ C 1 ( [ 0 , T ] ) g \in C^1([0,T]) g ∈ C 1 ([ 0 , T ]) , then we can define
∫ 0 T f ( t ) d g ( t ) : = ∫ 0 T f ( t ) g ′ ( t ) d t . \int_0^T f(t)\,\mathrm{d}g(t) := \int_0^T f(t)g'(t)\,\mathrm{d}t. ∫ 0 T f ( t ) d g ( t ) := ∫ 0 T f ( t ) g ′ ( t ) d t .
where the right-hand side is a Riemann integral with respect to t t t . However, as Brownian motion is not differentiable, this method of integration will not work when we consider integrals of the form ∫ 0 T Δ t d W t \int_0^T \Delta_t \,\mathrm{d}W_t ∫ 0 T Δ t d W t . Our goal is to make sense of ∫ 0 T Δ t d W t \int_0^T \Delta_t \,\mathrm{d}W_t ∫ 0 T Δ t d W t .
Assumption 8.1.1. In what follows W = ( W t ) t ≥ 0 W = (W_t)_{t \ge0} W = ( W t ) t ≥ 0 will always represent a Brownian motion and F = ( F t ) t ≥ 0 \mathbb{F} = (\mathcal{F}_t)_{t \ge0} F = ( F t ) t ≥ 0 will always be a filtration for this Brownian motion. We shall assume the integrand Δ = ( Δ t ) t ≥ 0 \Delta= (\Delta_t)_{t \ge0} Δ = ( Δ t ) t ≥ 0 is adapted to F \mathbb{F} F , meaning Δ t ∈ F t \Delta_t \in\mathcal{F}_t Δ t ∈ F t for all t t t .
Note that the process Δ \Delta Δ can and, in many cases, will be random. However, the information available in F t \mathcal{F}_t F t will always be sufficient to determine the value of Δ t \Delta_t Δ t at time t t t . Also note, since ( W T − W t ) ⊥ ⊥ F t (W_T - W_t) \perp\!\!\!\perp\mathcal{F}_t ( W T − W t ) ⊥ ⊥ F t for T > t T > t T > t , it follows that ( W T − W t ) ⊥ ⊥ Δ t (W_T - W_t) \perp\!\!\!\perp\Delta_t ( W T − W t ) ⊥ ⊥ Δ t . In other words, future increments of Brownian motion are independent of the Δ \Delta Δ process.
Itô integrals for simple integrands
To begin let us assume that Δ \Delta Δ is a simple process , meaning Δ \Delta Δ is of the form
Δ t = ∑ j = 0 n − 1 Δ t j 1 { t j ≤ t < t j + 1 } , 0 = t 0 < t 1 < ⋯ t n = T , Δ t j ∈ F t j . \Delta_t = \sum_{j=0}^{n-1} \Delta_{t_j}\mathbf{1}_{\{t_j \le t < t_{j+1}\}}, \qquad0 = t_0 < t_1 < \cdots t_n = T, \qquad\Delta_{t_j} \in\mathcal{F}_{t_j}. Δ t = j = 0 ∑ n − 1 Δ t j 1 { t j ≤ t < t j + 1 } , 0 = t 0 < t 1 < ⋯ t n = T , Δ t j ∈ F t j .
Since the process Δ \Delta Δ is constant over intervals of the form [ t j , t j + 1 ) [t_j,t_{j+1}) [ t j , t j + 1 ) , it makes sense to define
I T = ∫ 0 T Δ t d W t : = ∑ j = 0 n − 1 Δ t j ( W t j + 1 − W t j ) . (91) I_T=\int_0^T\Delta_t\mathrm{d}W_t:=\sum_{j=0}^{n-1}\Delta_{t_j}(W_{t_{j+1}}-W_{t_j}). \tag*{(91)} I T = ∫ 0 T Δ t d W t := j = 0 ∑ n − 1 Δ t j ( W t j + 1 − W t j ) . (91)
(for Δ \Delta Δ a simple process)
Let us establish some properties of the process I = ( I t ) t ≥ 0 I=(I_t)_{t\geq0} I = ( I t ) t ≥ 0 .
Theorem 8.1.2. The process I = ( I t ) t ≥ 0 I=(I_t)_{t\geq0} I = ( I t ) t ≥ 0 defined in (91) is a martingale with respect to the filtration ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\geq0} ( F t ) t ≥ 0 .
Proof. Without loss of generality assume T = t n T=t_n T = t n and t = t i t=t_i t = t i for some 0 ≤ i ≤ n − 1 0\leq i\leq n-1 0 ≤ i ≤ n − 1 (we can always re-define our time grid so that this is true). Then we have
E [ I T ∣ F t ] = ∑ j = 0 n − 1 E [ Δ t j ( W t j + 1 − W t j ) ∣ F t i ] = ∑ j = 0 i − 1 Δ t j ( W t j + 1 − W t j ) + ∑ j = i n − 1 E [ Δ t j ( W t j + 1 − W t j ) ∣ F t i ] = ∑ j = 0 i − 1 Δ t j ( W t j + 1 − W t j ) + ∑ j = i n − 1 E [ Δ t j E [ W t j + 1 − W t j ∣ F t j ] ∣ F t i ] = ∑ j = 0 i − 1 Δ t j ( W t j + 1 − W t j ) = I t . \begin{aligned}
\mathbb{E}[I_T\mid\mathcal{F}_t]
&=\sum_{j=0}^{n-1}\mathbb{E}[\Delta_{t_j}(W_{t_{j+1}}-W_{t_j})\mid\mathcal{F}_{t_i}]\\
&=\sum_{j=0}^{i-1}\Delta_{t_j}(W_{t_{j+1}}-W_{t_j})+\sum_{j=i}^{n-1}\mathbb{E}[\Delta_{t_j}(W_{t_{j+1}}-W_{t_j})\mid\mathcal{F}_{t_i}]\\
&=\sum_{j=0}^{i-1}\Delta_{t_j}(W_{t_{j+1}}-W_{t_j})+\sum_{j=i}^{n-1}\mathbb{E}[\Delta_{t_j}\mathbb{E}[W_{t_{j+1}}-W_{t_j}\mid\mathcal{F}_{t_j}]\mid\mathcal{F}_{t_i}]\\
&=\sum_{j=0}^{i-1}\Delta_{t_j}(W_{t_{j+1}}-W_{t_j})=I_t.
\end{aligned} E [ I T ∣ F t ] = j = 0 ∑ n − 1 E [ Δ t j ( W t j + 1 − W t j ) ∣ F t i ] = j = 0 ∑ i − 1 Δ t j ( W t j + 1 − W t j ) + j = i ∑ n − 1 E [ Δ t j ( W t j + 1 − W t j ) ∣ F t i ] = j = 0 ∑ i − 1 Δ t j ( W t j + 1 − W t j ) + j = i ∑ n − 1 E [ Δ t j E [ W t j + 1 − W t j ∣ F t j ] ∣ F t i ] = j = 0 ∑ i − 1 Δ t j ( W t j + 1 − W t j ) = I t .
Thus, the process I I I is a martingale, as claimed. □
Note that I 0 = 0 I_0=0 I 0 = 0 and therefore E I t = 0 \mathbb{E}I_t=0 E I t = 0 for all t ≥ 0 t\geq0 t ≥ 0 .
Theorem 8.1.3 (Itô Isometry). The process I = ( I t ) t ≥ 0 I=(I_t)_{t\geq0} I = ( I t ) t ≥ 0 defined in (91) satisfies
V a r I T = E I T 2 = E ∫ 0 T Δ t 2 d t . (92) \mathop{\mathrm{Var}}I_T=\mathbb{E}I_T^2=\mathbb{E}\int_0^T\Delta_t^2\mathrm{d}t. \tag*{(92)} Var I T = E I T 2 = E ∫ 0 T Δ t 2 d t . (92)
Proof. We have
E I T 2 = ∑ j = 0 n − 1 ∑ i = 0 n − 1 E Δ t i Δ t j ( W t i + 1 − W t i ) ( W t j + 1 − W t j ) = ∑ j = 0 n − 1 E Δ t j 2 ( W t j + 1 − W t j ) 2 + 2 ∑ j = 0 n − 1 ∑ i = 0 j − 1 E Δ t i Δ t j ( W t i + 1 − W t i ) ( W t j + 1 − W t j ) = ∑ j = 0 n − 1 E Δ t j 2 E [ ( W t j + 1 − W t j ) 2 ∣ F t j ] + 2 ∑ j = 0 n − 1 ∑ i = 0 j − 1 E [ Δ t i Δ t j ( W t i + 1 − W t i ) E [ ( W t j + 1 − W t j ) ∣ F t j ] ] = ∑ j = 0 n − 1 E Δ t j 2 ( t j + 1 − t j ) = E ∫ 0 T Δ t 2 d t . \begin{aligned}
\mathbb{E}I_T^2
&=\sum_{j=0}^{n-1}\sum_{i=0}^{n-1}\mathbb{E}\Delta_{t_i}\Delta_{t_j}(W_{t_{i+1}}-W_{t_i})(W_{t_{j+1}}-W_{t_j})\\
&=\sum_{j=0}^{n-1}\mathbb{E}\Delta_{t_j}^2(W_{t_{j+1}}-W_{t_j})^2+2\sum_{j=0}^{n-1}\sum_{i=0}^{j-1}\mathbb{E}\Delta_{t_i}\Delta_{t_j}(W_{t_{i+1}}-W_{t_i})(W_{t_{j+1}}-W_{t_j})\\
&=\sum_{j=0}^{n-1}\mathbb{E}\Delta_{t_j}^2\mathbb{E}[(W_{t_{j+1}}-W_{t_j})^2\mid\mathcal{F}_{t_j}]
+2\sum_{j=0}^{n-1}\sum_{i=0}^{j-1}\mathbb{E}[\Delta_{t_i}\Delta_{t_j}(W_{t_{i+1}}-W_{t_i})\mathbb{E}[(W_{t_{j+1}}-W_{t_j})\mid\mathcal{F}_{t_j}]]\\
&=\sum_{j=0}^{n-1}\mathbb{E}\Delta_{t_j}^2(t_{j+1}-t_j)=\mathbb{E}\int_0^T\Delta_t^2\mathrm{d}t.
\end{aligned} E I T 2 = j = 0 ∑ n − 1 i = 0 ∑ n − 1 E Δ t i Δ t j ( W t i + 1 − W t i ) ( W t j + 1 − W t j ) = j = 0 ∑ n − 1 E Δ t j 2 ( W t j + 1 − W t j ) 2 + 2 j = 0 ∑ n − 1 i = 0 ∑ j − 1 E Δ t i Δ t j ( W t i + 1 − W t i ) ( W t j + 1 − W t j ) = j = 0 ∑ n − 1 E Δ t j 2 E [( W t j + 1 − W t j ) 2 ∣ F t j ] + 2 j = 0 ∑ n − 1 i = 0 ∑ j − 1 E [ Δ t i Δ t j ( W t i + 1 − W t i ) E [( W t j + 1 − W t j ) ∣ F t j ]] = j = 0 ∑ n − 1 E Δ t j 2 ( t j + 1 − t j ) = E ∫ 0 T Δ t 2 d t .
□
Theorem 8.1.4. The process I = ( I t ) t ≥ 0 I=(I_t)_{t\ge0} I = ( I t ) t ≥ 0 defined in (91) satisfies
[ I , I ] T = ∫ 0 T Δ t 2 d t . (93) [I,I]_T=\int_0^T \Delta_t^2\,\mathrm{d}t. \tag*{(93)} [ I , I ] T = ∫ 0 T Δ t 2 d t . (93)
Proof. Fix a partition Π \Pi Π of [ t j , t j + 1 ] [t_j,t_{j+1}] [ t j , t j + 1 ]
t j = s 0 < s 1 < … < s m = t j + 1 . t_j=s_0<s_1<\ldots<s_m=t_{j+1}. t j = s 0 < s 1 < … < s m = t j + 1 .
From the definition of Quadratic variation, we compute
[ I , I ] t j + 1 − [ I , I ] t j = lim ∥ Π ∥ → 0 ∑ i = 0 m − 1 ( I s i + 1 − I s i ) 2 = Δ t j 2 lim ∥ Π ∥ → 0 ∑ i = 0 m − 1 ( W s i + 1 − W s i ) 2 = Δ t j 2 ( [ W , W ] t j + 1 − [ W , W ] t j ) = Δ t j 2 ( t j + 1 − t j ) . \begin{aligned}
[I,I]_{t_{j+1}}-[I,I]_{t_j}
=\lim_{\lVert\Pi\rVert\to0}\sum_{i=0}^{m-1}(I_{s_{i+1}}-I_{s_i})^2
=\Delta_{t_j}^2\lim_{\lVert\Pi\rVert\to0}\sum_{i=0}^{m-1}(W_{s_{i+1}}-W_{s_i})^2 \\
&=\Delta_{t_j}^2\left([W,W]_{t_{j+1}}-[W,W]_{t_j}\right)
=\Delta_{t_j}^2(t_{j+1}-t_j).
\end{aligned} [ I , I ] t j + 1 − [ I , I ] t j = ∥ Π ∥ → 0 lim i = 0 ∑ m − 1 ( I s i + 1 − I s i ) 2 = Δ t j 2 ∥ Π ∥ → 0 lim i = 0 ∑ m − 1 ( W s i + 1 − W s i ) 2 = Δ t j 2 ( [ W , W ] t j + 1 − [ W , W ] t j ) = Δ t j 2 ( t j + 1 − t j ) .
Thus, we obtain
[ I , I ] T = ∑ j = 0 n − 1 ( [ I , I ] t j + 1 − [ I , I ] t j ) = ∑ j = 0 n − 1 Δ t j 2 ( t j + 1 − t j ) = ∫ 0 T Δ t 2 d t . [I,I]_T=\sum_{j=0}^{n-1}\left([I,I]_{t_{j+1}}-[I,I]_{t_j}\right)
=\sum_{j=0}^{n-1}\Delta_{t_j}^2\left(t_{j+1}-t_j\right)
=\int_0^T\Delta_t^2\mathrm{d}t. [ I , I ] T = j = 0 ∑ n − 1 ( [ I , I ] t j + 1 − [ I , I ] t j ) = j = 0 ∑ n − 1 Δ t j 2 ( t j + 1 − t j ) = ∫ 0 T Δ t 2 d t .
as claimed.
When we computed V T W V^W_T V T W and [ W , W ] T [W,W]_T [ W , W ] T we found that these two quantities were equal, even though the computations for these quantities were completely different. From (92) and (93) we now see how the variance and quadratic variation of a stochastic process can be different. Note that V T I V^I_T V T I is a non-random constant, whereas [ I , I ] T [I,I]_T [ I , I ] T is random.
Itô integrals for general integrands
Clearly, it is rather restrictive to limit ourselves to integrands Δ \Delta Δ that are simple processes. We now allow the process Δ \Delta Δ to be any process that is adapted to F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\ge0} F = ( F t ) t ≥ 0 and which satisfies the following integrability condition
E ∫ 0 T Δ t 2 d t < ∞ . (94) \mathbb{E}\int_0^T\Delta_t^2\mathrm{d}t<\infty. \tag*{(94)} E ∫ 0 T Δ t 2 d t < ∞. (94)
To construct an Itô integral with Δ \Delta Δ as the integrand, we first approximate Δ \Delta Δ by a simple process
Δ t ≈ Δ t ( n ) : = ∑ j = 0 n − 1 Δ t j 1 { t j ≤ t < t j + 1 } , 0 ≤ t 0 < t 1 < … < t n = T . \Delta_t\approx\Delta_t^{(n)}:=\sum_{j=0}^{n-1}\Delta_{t_j}\mathbf{1}_{\{t_j\le t<t_{j+1}\}},
\qquad0\le t_0<t_1<\ldots<t_n=T. Δ t ≈ Δ t ( n ) := j = 0 ∑ n − 1 Δ t j 1 { t j ≤ t < t j + 1 } , 0 ≤ t 0 < t 1 < … < t n = T .
As n → ∞ n\to\infty n → ∞ the process Δ ( n ) \Delta^{(n)} Δ ( n ) converges to Δ \Delta Δ in the sense that
lim n → ∞ E ∫ 0 T ( Δ t − Δ t ( n ) ) 2 d t = 0. (95) \lim_{n\to\infty}\mathbb{E}\int_0^T\left(\Delta_t-\Delta_t^{(n)}\right)^2\mathrm{d}t=0. \tag*{(95)} n → ∞ lim E ∫ 0 T ( Δ t − Δ t ( n ) ) 2 d t = 0. (95)
We now define the Itô integral for a general integrand Δ \Delta Δ by
I T ≡ ∫ 0 T Δ t d W t ≔ lim n → ∞ ∫ 0 T Δ t ( n ) d W t . (96) I_T \equiv\int_0^T \Delta_t\,\mathrm{d}W_t \coloneqq\lim_{n\to\infty}\int_0^T \Delta_t^{(n)}\,\mathrm{d}W_t. \tag*{(96)} I T ≡ ∫ 0 T Δ t d W t : = n → ∞ lim ∫ 0 T Δ t ( n ) d W t . (96)
Note that the integrals ∫ 0 T Δ t ( n ) d t \int_0^T \Delta_t^{(n)}\,\mathrm{d}t ∫ 0 T Δ t ( n ) d t are well-defined for every n n n , since Δ ( n ) \Delta^{(n)} Δ ( n ) is a simple process. Furthermore, the condition (95) ensures that the limit exists in L 2 ( Ω , F , P ) L^2(\Omega,\mathcal{F},\mathbb{P}) L 2 ( Ω , F , P ) . The Itô integral for general integrands inherits the properties we established for simple integrands.
Theorem 8.1.5. Let W W W be a Brownian motion and let F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 be a filtration for this Brownian motion. Let Δ = ( Δ t ) 0 ≤ t ≤ T \Delta=(\Delta_t)_{0\leq t\leq T} Δ = ( Δ t ) 0 ≤ t ≤ T be adapted to the filtration F \mathbb{F} F and satisfy (95) . Let I = ( I t ) 0 ≤ t ≤ T I=(I_t)_{0\leq t\leq T} I = ( I t ) 0 ≤ t ≤ T be given by I t = ∫ 0 t Δ s d W s I_t=\int_0^t\Delta_s\,\mathrm{d}W_s I t = ∫ 0 t Δ s d W s , where the integral is defined as in (96) . Then the process I I I has the following properties.
The sample paths of I I I are continuous.
The process I I I is adapted to the filtration F \mathbb{F} F . That is, I t ∈ F t I_t\in\mathcal{F}_t I t ∈ F t for all t t t .
If Γ = ( Γ t ) 0 ≤ t ≤ T \Gamma=(\Gamma_t)_{0\leq t\leq T} Γ = ( Γ t ) 0 ≤ t ≤ T satisfies the same conditions as Δ \Delta Δ , then
∫ 0 T ( a Δ t + b Γ t ) d W t = a ∫ 0 T Δ t d W t + b ∫ 0 T Γ t d W t , \int_0^T(a\Delta_t+b\Gamma_t)\,\mathrm{d}W_t
=a\int_0^T\Delta_t\,\mathrm{d}W_t+b\int_0^T\Gamma_t\,\mathrm{d}W_t, ∫ 0 T ( a Δ t + b Γ t ) d W t = a ∫ 0 T Δ t d W t + b ∫ 0 T Γ t d W t , where a a a and b b b are constants.
The process I I I is a martingale with respect to the filtration F \mathbb{F} F .
We have the Itô isometry E I T 2 = E ∫ 0 T Δ t 2 d t \mathbb{E}I_T^2=\mathbb{E}\int_0^T\Delta_t^2\mathrm{d}t E I T 2 = E ∫ 0 T Δ t 2 d t .
The quadratic variation of I I I is given by [ I , I ] T = ∫ 0 T Δ t 2 d t [I,I]_T=\int_0^T\Delta_t^2\mathrm{d}t [ I , I ] T = ∫ 0 T Δ t 2 d t .
Itô formula
If f f f and g g g are differentiable functions then we can compute
d d t f ( g ( t ) ) = f ′ ( g ( t ) ) ⋅ g ′ ( t ) . \frac{\mathrm{d}}{\mathrm{d}t}f(g(t))=f'(g(t))\cdot g'(t). d t d f ( g ( t )) = f ′ ( g ( t )) ⋅ g ′ ( t ) .
Thus, multiplying through by d t \mathrm{d}t d t we obtain
d f ( g ( t ) ) = f ′ ( g ( t ) ) ⋅ g ′ ( t ) d t = f ′ ( g ( t ) ) d g ( t ) , d g ( t ) = g ′ ( t ) d t . \mathrm{d}f(g(t))=f'(g(t))\cdot g'(t)\mathrm{d}t=f'(g(t))\mathrm{d}g(t),\qquad\mathrm{d}g(t)=g'(t)\mathrm{d}t. d f ( g ( t )) = f ′ ( g ( t )) ⋅ g ′ ( t ) d t = f ′ ( g ( t )) d g ( t ) , d g ( t ) = g ′ ( t ) d t .
This formula is correct in the sense that
f ( g ( T ) ) − f ( g ( 0 ) ) = ∫ 0 T d f ( g ( t ) ) = ∫ 0 T f ′ ( g ( t ) ) ⋅ g ′ ( t ) d t = ∫ 0 T f ′ ( g ( t ) ) d g ( t ) . f(g(T))-f(g(0))=\int_0^T\mathrm{d}f(g(t))=\int_0^T f'(g(t))\cdot g'(t)\mathrm{d}t=\int_0^T f'(g(t))\mathrm{d}g(t). f ( g ( T )) − f ( g ( 0 )) = ∫ 0 T d f ( g ( t )) = ∫ 0 T f ′ ( g ( t )) ⋅ g ′ ( t ) d t = ∫ 0 T f ′ ( g ( t )) d g ( t ) .
Although paths of a Brownian motion W W W are not differentiable, we might suspect that
f ( W T ) − f ( W 0 ) = ∫ 0 T d f ( W t ) = ? ∫ 0 T f ′ ( W t ) d W t . (97) f(W_T)-f(W_0)=\int_0^T\mathrm{d}f(W_t)\overset{?}{=}\int_0^T f'(W_t)\mathrm{d}W_t. \tag*{(97)} f ( W T ) − f ( W 0 ) = ∫ 0 T d f ( W t ) = ? ∫ 0 T f ′ ( W t ) d W t . (97)
(Not correct!)
Unfortunately, as indicated above, equation (97) is not correct.
Theorem 8.2.1. Let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion and suppose f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R satisfies f ∈ C 2 ( R ) f\in C^2(\mathbb{R}) f ∈ C 2 ( R ) and E ∫ 0 T ( f ′ ( W t ) ) 2 d t < ∞ \mathbb{E}\int_0^T(f'(W_t))^2\mathrm{d}t<\infty E ∫ 0 T ( f ′ ( W t ) ) 2 d t < ∞ for all T ≥ 0 T\geq0 T ≥ 0 . Then, for any T ≥ 0 T\geq0 T ≥ 0 we have
f ( W T ) − f ( W 0 ) = ∫ 0 T f ′ ( W t ) d W t + 1 2 ∫ 0 T f ′ ′ ( W t ) d t . (98) f(W_T)-f(W_0)=\int_0^T f'(W_t)\mathrm{d}W_t+\frac{1}{2}\int_0^T f''(W_t)\mathrm{d}t. \tag*{(98)} f ( W T ) − f ( W 0 ) = ∫ 0 T f ′ ( W t ) d W t + 2 1 ∫ 0 T f ′′ ( W t ) d t . (98)
Proof. We shall simply sketch the proof of Theorem 8.2.1 . Suppose for simplicity that f f f is analytic (i.e., that f f f is equal to its power series expansion at every point). Let 0 = t 0 < t 1 < … < t n = T 0=t_0<t_1<\ldots<t_n=T 0 = t 0 < t 1 < … < t n = T be a partition Π \Pi Π of [ 0 , T ] [0,T] [ 0 , T ] . Then
∫ 0 T d f ( W t ) = f ( W T ) − f ( W 0 ) = ∑ j = 0 n − 1 ( f ( W t j + 1 ) − f ( W t j ) ) = ∑ j = 0 n − 1 ( A j + B j + C j ) , \int_0^T \mathrm{d}f(W_t)=f(W_T)-f(W_0)=\sum_{j=0}^{n-1}\left(f(W_{t_{j+1}})-f(W_{t_j})\right)=\sum_{j=0}^{n-1}(A_j+B_j+C_j), ∫ 0 T d f ( W t ) = f ( W T ) − f ( W 0 ) = j = 0 ∑ n − 1 ( f ( W t j + 1 ) − f ( W t j ) ) = j = 0 ∑ n − 1 ( A j + B j + C j ) ,
where we have defined
A j : = f ′ ( W t j ) ( W t j + 1 − W t j ) , A_j:=f'(W_{t_j})(W_{t_{j+1}}-W_{t_j}), A j := f ′ ( W t j ) ( W t j + 1 − W t j ) ,
B j : = 1 2 f ′ ′ ( W t j ) ( W t j + 1 − W t j ) 2 , B_j:=\frac{1}{2}f''(W_{t_j})(W_{t_{j+1}}-W_{t_j})^2, B j := 2 1 f ′′ ( W t j ) ( W t j + 1 − W t j ) 2 ,
C j : = 1 3 ! f ′ ′ ′ ( W t j ) ( W t j + 1 − W t j ) 3 + … . C_j:=\frac{1}{3!}f'''(W_{t_j})(W_{t_{j+1}}-W_{t_j})^3+\ldots. C j := 3 ! 1 f ′′′ ( W t j ) ( W t j + 1 − W t j ) 3 + … .
In the limit as ∥ Π ∥ → 0 \lVert\Pi\rVert\to0 ∥ Π ∥ → 0 we have
∑ j = 0 n − 1 A j → ∫ 0 T f ′ ( W t ) d W t , ∑ j = 0 n − 1 B j → ∫ 0 T 1 2 f ′ ′ ( W t ) d t , ∑ j = 0 n − 1 C j → 0. \sum_{j=0}^{n-1}A_j\to\int_0^T f'(W_t)\mathrm{d}W_t,\qquad\sum_{j=0}^{n-1}B_j\to\int_0^T\frac{1}{2}f''(W_t)\mathrm{d}t,\qquad\sum_{j=0}^{n-1}C_j\to0. j = 0 ∑ n − 1 A j → ∫ 0 T f ′ ( W t ) d W t , j = 0 ∑ n − 1 B j → ∫ 0 T 2 1 f ′′ ( W t ) d t , j = 0 ∑ n − 1 C j → 0.
Example 8.2.2. What is ∫ 0 T W t d W t \int_0^T W_t\mathrm{d}W_t ∫ 0 T W t d W t ? To answer this question, consider f ( W t ) f(W_t) f ( W t ) with f ( x ) = x 2 f(x)=x^2 f ( x ) = x 2 . According to equation (98) we have
W T 2 − W 0 2 = ∫ 0 T 2 W t d W t + ∫ 0 T d t . W_T^2-W_0^2=\int_0^T2W_t\mathrm{d}W_t+\int_0^T\mathrm{d}t. W T 2 − W 0 2 = ∫ 0 T 2 W t d W t + ∫ 0 T d t .
where we have used f ′ ( x ) = 2 x f'(x)=2x f ′ ( x ) = 2 x and f ′ ′ ( x ) = 2 f''(x)=2 f ′′ ( x ) = 2 . Noting that W 0 = 0 W_0=0 W 0 = 0 and solving for ∫ 0 T W t d W t \int_0^T W_t\mathrm{d}W_t ∫ 0 T W t d W t we obtain
∫ 0 T W t d W t = 1 2 W T 2 − 1 2 T . \int_0^T W_t\mathrm{d}W_t=\frac{1}{2}W_T^2-\frac{1}{2}T. ∫ 0 T W t d W t = 2 1 W T 2 − 2 1 T .
Not surprisingly, there are stochastic processes that are not adequately described by Brownian motion alone. However, a large class of stochastic processes can be constructed from Brownian motion.
Definition 8.2.3. Let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion and let F = ( F t ) t ≥ 0 \mathcal{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 be a filtration for this Brownian motion. An Itô process is any process X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 of the form
X t = X 0 + ∫ 0 t Θ s d s + ∫ 0 t Δ s d W s . (99) X_t=X_0+\int_0^t\Theta_s\mathrm{d}s+\int_0^t\Delta_s\mathrm{d}W_s. \tag*{(99)} X t = X 0 + ∫ 0 t Θ s d s + ∫ 0 t Δ s d W s . (99)
where Θ = ( Θ t ) t ≥ 0 \Theta=(\Theta_t)_{t\geq0} Θ = ( Θ t ) t ≥ 0 and Δ = ( Δ t ) t ≥ 0 \Delta=(\Delta_t)_{t\geq0} Δ = ( Δ t ) t ≥ 0 are adapted to the filtration F \mathcal{F} F and satisfy
∫ 0 T ∣ Θ t ∣ d t < ∞ , E ∫ 0 T Δ t 2 d t < ∞ , ∀ T ≥ 0 , \int_0^T |\Theta_t|\,\mathrm{d}t < \infty,\qquad\mathbb{E}\int_0^T \Delta_t^2\,\mathrm{d}t < \infty,\qquad\forall T\geq0, ∫ 0 T ∣ Θ t ∣ d t < ∞ , E ∫ 0 T Δ t 2 d t < ∞ , ∀ T ≥ 0 ,
and X 0 ∈ F 0 X_0\in\mathcal{F}_0 X 0 ∈ F 0 .
We sometimes write an Itô process in differential form
d X t = Θ t d t + Δ t d W t . (100) \mathrm{d}X_t=\Theta_t\mathrm{d}t+\Delta_t\mathrm{d}W_t. \tag*{(100)} d X t = Θ t d t + Δ t d W t . (100)
Expression (100) literally means that X X X satisfies (99) . Informally, the differential form can be understood as follows: in a small interval of time δ t \delta t δ t , the process X X X changes according to
X t + δ t − X t ≈ Θ t δ t + Δ t ( W t + δ t − W t ) . (101) X_{t+\delta t}-X_t\approx\Theta_t\delta t+\Delta_t\left(W_{t+\delta t}-W_t\right). \tag*{(101)} X t + δ t − X t ≈ Θ t δ t + Δ t ( W t + δ t − W t ) . (101)
In fact, noting that W t + δ t − W t ∼ N ( 0 , δ t ) W_{t+\delta t}-W_t\sim\mathcal{N}(0,\delta t) W t + δ t − W t ∼ N ( 0 , δ t ) and W t + δ t − W t ⊥ ⊥ F t W_{t+\delta t}-W_t\perp\!\!\perp\mathcal{F}_t W t + δ t − W t ⊥ ⊥ F t , one can use expression (101) to simulate the increment X t + δ t − X t X_{t+\delta t}-X_t X t + δ t − X t . This way of simulating X X X is called the Euler scheme and is the workhorse of many Monte Carlo methods.
Lemma 8.2.4. The quadratic variation [ X , X ] T [X,X]_T [ X , X ] T of an Itô process (99) is given by
[ X , X ] T = ∫ 0 T Δ t 2 d t . [X,X]_T=\int_0^T\Delta_t^2\mathrm{d}t. [ X , X ] T = ∫ 0 T Δ t 2 d t .
Proof. We sketch the proof of Lemma 8.2.4. Let 0 = t 0 < t 1 < … < t n = T 0=t_0<t_1<\ldots<t_n=T 0 = t 0 < t 1 < … < t n = T be a partition Π \Pi Π of [ 0 , T ] [0,T] [ 0 , T ] . By definition we have
[ X , X ] T = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ( X t j + 1 − X t j ) 2 = lim ∥ Π ∥ → 0 ∑ j = 0 n − 1 ( A j + B j + C j ) . [X,X]_T=\lim_{\lVert\Pi\rVert\to0}\sum_{j=0}^{n-1}\left(X_{t_{j+1}}-X_{t_j}\right)^2=\lim_{\lVert\Pi\rVert\to0}\sum_{j=0}^{n-1}\left(A_j+B_j+C_j\right). [ X , X ] T = ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 ( X t j + 1 − X t j ) 2 = ∥ Π ∥ → 0 lim j = 0 ∑ n − 1 ( A j + B j + C j ) .
where we have defined
A j : = ( I t j + 1 − I t j ) 2 , A_j:=\left(I_{t_{j+1}}-I_{t_j}\right)^2, A j := ( I t j + 1 − I t j ) 2 ,
B j : = ( J t j + 1 − J t j ) 2 , B_j:=\left(J_{t_{j+1}}-J_{t_j}\right)^2, B j := ( J t j + 1 − J t j ) 2 ,
C j : = 2 ( I t j + 1 − I t j ) ( J t j + 1 − J t j ) . C_j:=2\left(I_{t_{j+1}}-I_{t_j}\right)\left(J_{t_{j+1}}-J_{t_j}\right). C j := 2 ( I t j + 1 − I t j ) ( J t j + 1 − J t j ) .
with
I t = ∫ 0 t Δ s d W s , J t = ∫ 0 t Θ s d s . I_t=\int_0^t\Delta_s\mathrm{d}W_s,\qquad J_t=\int_0^t\Theta_s\mathrm{d}s. I t = ∫ 0 t Δ s d W s , J t = ∫ 0 t Θ s d s .
In the limit as ∥ Π ∥ → 0 \lVert\Pi\rVert\to0 ∥ Π ∥ → 0 we obtain
∑ j = 0 n − 1 A j → [ I , I ] T , ∑ j = 0 n − 1 B j → 0 , ∑ j = 0 n − 1 C j → 0. \sum_{j=0}^{n-1}A_j\to[I,I]_T,\qquad\sum_{j=0}^{n-1}B_j\to0,\qquad\sum_{j=0}^{n-1}C_j\to0. j = 0 ∑ n − 1 A j → [ I , I ] T , j = 0 ∑ n − 1 B j → 0 , j = 0 ∑ n − 1 C j → 0.
The proof of these limits is similar to the proof of Theorem 8.1.4.
Definition 8.2.5. Let X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 be an Itô process, as described in Definition 8.2.3. Let Γ = ( Γ t ) t ≥ 0 \Gamma=(\Gamma_t)_{t\geq0} Γ = ( Γ t ) t ≥ 0 be adapted to the filtration of the Brownian motion F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 . We define
∫ 0 T Γ t d X t : = ∫ 0 T Γ t Θ t d t + ∫ 0 T Γ t Δ t d W t , \int_0^T \Gamma_t\,\mathrm{d}X_t:=\int_0^T \Gamma_t\Theta_t\,\mathrm{d}t+\int_0^T \Gamma_t\Delta_t\,\mathrm{d}W_t, ∫ 0 T Γ t d X t := ∫ 0 T Γ t Θ t d t + ∫ 0 T Γ t Δ t d W t , where we assume
∫ 0 T ∣ Γ t Θ t ∣ d t < ∞ , E ∫ 0 T ( Γ t Δ t ) 2 d t < ∞ , ∀ T ≥ 0. \int_0^T \lvert\Gamma_t\Theta_t\rvert\,\mathrm{d}t<\infty,\qquad\mathbb{E}\int_0^T(\Gamma_t\Delta_t)^2\,\mathrm{d}t<\infty,\qquad\forall T\geq0. ∫ 0 T ∣ Γ t Θ t ∣ d t < ∞ , E ∫ 0 T ( Γ t Δ t ) 2 d t < ∞ , ∀ T ≥ 0.
Theorem 8.2.6 (Itô formula in one dimension). Let X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 be an Itô process of the form (99) and suppose f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R satisfies f ∈ C 2 ( R ) f\in C^2(\mathbb{R}) f ∈ C 2 ( R ) and E ∫ 0 T ( f ′ ( X t ) Δ t ) 2 d t < ∞ \mathbb{E}\int_0^T(f'(X_t)\Delta_t)^2\,\mathrm{d}t<\infty E ∫ 0 T ( f ′ ( X t ) Δ t ) 2 d t < ∞ for all T ≥ 0 T\geq0 T ≥ 0 . Then, for any T ≥ 0 T\geq0 T ≥ 0 we have
f ( X T ) − f ( X 0 ) = ∫ 0 T f ′ ( X t ) d X t + 1 2 ∫ 0 T f ′ ′ ( X t ) d [ X , X ] t . f(X_T)-f(X_0)=\int_0^T f'(X_t)\,\mathrm{d}X_t+\frac{1}{2}\int_0^T f''(X_t)\,\mathrm{d}[X,X]_t. f ( X T ) − f ( X 0 ) = ∫ 0 T f ′ ( X t ) d X t + 2 1 ∫ 0 T f ′′ ( X t ) d [ X , X ] t .
Proof. The proof of Theorem 8.2.6 is very similar to the proof of Theorem 8.2.1. We outline the proof here. Suppose for simplicity that f f f is analytic (i.e., that f f f is equal to its power series expansion at every point). Let 0 = t 0 < t 1 < … < t n = T 0=t_0<t_1<\ldots<t_n=T 0 = t 0 < t 1 < … < t n = T be a partition Π \Pi Π of [ 0 , T ] [0,T] [ 0 , T ] . Then
∫ 0 T d f ( X t ) = f ( X T ) − f ( X 0 ) = ∑ j = 0 n − 1 ( f ( X t j + 1 ) − f ( X t j ) ) = ∑ j = 0 n − 1 ( A j + B j + C j ) , \int_0^T\mathrm{d}f(X_t)=f(X_T)-f(X_0)=\sum_{j=0}^{n-1}\left(f(X_{t_{j+1}})-f(X_{t_j})\right)=\sum_{j=0}^{n-1}(A_j+B_j+C_j), ∫ 0 T d f ( X t ) = f ( X T ) − f ( X 0 ) = j = 0 ∑ n − 1 ( f ( X t j + 1 ) − f ( X t j ) ) = j = 0 ∑ n − 1 ( A j + B j + C j ) ,
where we have defined
A j : = f ′ ( X t j ) ( X t j + 1 − X t j ) , A_j:=f'(X_{t_j})\left(X_{t_{j+1}}-X_{t_j}\right), A j := f ′ ( X t j ) ( X t j + 1 − X t j ) ,
B j : = 1 2 f ′ ′ ( X t j ) ( X t j + 1 − X t j ) 2 , B_j:=\frac{1}{2}f''(X_{t_j})\left(X_{t_{j+1}}-X_{t_j}\right)^2, B j := 2 1 f ′′ ( X t j ) ( X t j + 1 − X t j ) 2 ,
C j : = 1 3 ! f ′ ′ ′ ( X t j ) ( X t j + 1 − X t j ) 3 + … . C_j:=\frac{1}{3!}f'''(X_{t_j})\left(X_{t_{j+1}}-X_{t_j}\right)^3+\ldots. C j := 3 ! 1 f ′′′ ( X t j ) ( X t j + 1 − X t j ) 3 + … .
In the limit as ∥ Π ∥ → 0 \lVert\Pi\rVert\to0 ∥ Π ∥ → 0 we have
∑ j = 0 n − 1 A j → ∫ 0 T f ′ ( X t ) d X t , ∑ j = 0 n − 1 B j → ∫ 0 T 1 2 f ′ ′ ( X t ) d [ X , X ] t , ∑ j = 0 n − 1 C j → 0. \sum_{j=0}^{n-1}A_j\to\int_0^T f'(X_t)\,\mathrm{d}X_t,\qquad
\sum_{j=0}^{n-1}B_j\to\int_0^T\frac{1}{2}f''(X_t)\,\mathrm{d}[X,X]_t,\qquad
\sum_{j=0}^{n-1}C_j\to0. j = 0 ∑ n − 1 A j → ∫ 0 T f ′ ( X t ) d X t , j = 0 ∑ n − 1 B j → ∫ 0 T 2 1 f ′′ ( X t ) d [ X , X ] t , j = 0 ∑ n − 1 C j → 0.
In differential form, with X X X given by (100) , Itô’s formula becomes
d f ( X t ) = f ′ ( X t ) d X t + 1 2 f ′ ′ ( X t ) d [ X , X ] t = f ′ ( X t ) ( Θ t d t + Δ t d W t ) + 1 2 f ′ ′ ( X t ) Δ t 2 d t , (102) \begin{aligned}
\mathrm{d}f(X_t)&=f'(X_t)\,\mathrm{d}X_t+\frac{1}{2}f''(X_t)\,\mathrm{d}[X,X]_t\\
&=f'(X_t)\left(\Theta_t\,\mathrm{d}t+\Delta_t\,\mathrm{d}W_t\right)+\frac{1}{2}f''(X_t)\Delta_t^2\,\mathrm{d}t, \tag*{(102)}
\end{aligned} d f ( X t ) = f ′ ( X t ) d X t + 2 1 f ′′ ( X t ) d [ X , X ] t = f ′ ( X t ) ( Θ t d t + Δ t d W t ) + 2 1 f ′′ ( X t ) Δ t 2 d t , (102)
where we have used d [ X , X ] t = Δ t 2 d t \mathrm{d}[X,X]_t=\Delta_t^2\mathrm{d}t d [ X , X ] t = Δ t 2 d t . Perhaps the easiest way to remember (102) is to use the following two-step procedure:
Expand f ( X t + d X t ) − f ( X t ) f(X_t+dX_t)-f(X_t) f ( X t + d X t ) − f ( X t ) to second order about the point X t X_t X t
d f ( X t ) = f ( X t + d X t ) − f ( X t ) = f ′ ( X t ) d X t + 1 2 f ′ ′ ( X t ) ( d X t ) 2 , (103) df(X_t)=f(X_t+dX_t)-f(X_t)=f'(X_t)dX_t+\tfrac{1}{2}f''(X_t)(dX_t)^2, \tag*{(103)} df ( X t ) = f ( X t + d X t ) − f ( X t ) = f ′ ( X t ) d X t + 2 1 f ′′ ( X t ) ( d X t ) 2 , (103)
Insert the differential d X t = Θ t d t + Δ t d W t dX_t=\Theta_tdt+\Delta_tdW_t d X t = Θ t d t + Δ t d W t into (103) , expand ( d X t ) 2 (dX_t)^2 ( d X t ) 2 and use the rules
d W t d W t = d t , d W t d t = 0 , d t d t = 0. dW_tdW_t=dt,\qquad dW_tdt=0,\qquad dtdt=0. d W t d W t = d t , d W t d t = 0 , d t d t = 0.
The resulting formula gives the correct expression for d f ( X t ) df(X_t) df ( X t ) .
Example 8.2.7. Let X X X be an Itô process with the following dynamics
d X t = μ t X t d t + σ t X t d W t . (104) dX_t=\mu_tX_tdt+\sigma_tX_tdW_t. \tag*{(104)} d X t = μ t X t d t + σ t X t d W t . (104) Assuming μ = ( μ t ) t ≥ 0 \mu=(\mu_t)_{t\geq0} μ = ( μ t ) t ≥ 0 and σ = ( σ t ) t ≥ 0 \sigma=(\sigma_t)_{t\geq0} σ = ( σ t ) t ≥ 0 are bounded above and below and X 0 > 0 X_0>0 X 0 > 0 , the process X X X remains strictly positive. We call X X X a generalized geometric Brownian motion . The “geometric” part refers to the fact that the relative step size d X t / X t dX_t/X_t d X t / X t has dynamics μ t d t + σ t d W t \mu_tdt+\sigma_tdW_t μ t d t + σ t d W t . The “generalized” part refers to the fact that the processes σ \sigma σ and μ \mu μ are stochastic rather than constant. Define Y t = X t p Y_t=X_t^p Y t = X t p . What is d Y t dY_t d Y t ? Let f ( x ) = x p f(x)=x^p f ( x ) = x p . Then f ′ ( x ) = p x p − 1 f'(x)=px^{p-1} f ′ ( x ) = p x p − 1 and f ′ ′ ( x ) = p ( p − 1 ) x p − 2 f''(x)=p(p-1)x^{p-2} f ′′ ( x ) = p ( p − 1 ) x p − 2 . Thus, we have
d Y t = d f ( X t ) = p X t p − 1 d X t + 1 2 p ( p − 1 ) X t p − 2 ( d X t ) 2 dY_t=df(X_t)=pX_t^{p-1}dX_t+\tfrac{1}{2}p(p-1)X_t^{p-2}(dX_t)^2 d Y t = df ( X t ) = p X t p − 1 d X t + 2 1 p ( p − 1 ) X t p − 2 ( d X t ) 2 = p X t p − 1 ( μ t X t d t + σ t X t d W t ) + 1 2 p ( p − 1 ) X t p − 2 ( μ t X t d t + σ t X t d W t ) 2 =pX_t^{p-1}\left(\mu_tX_tdt+\sigma_tX_tdW_t\right)+\tfrac{1}{2}p(p-1)X_t^{p-2}\left(\mu_tX_tdt+\sigma_tX_tdW_t\right)^2 = p X t p − 1 ( μ t X t d t + σ t X t d W t ) + 2 1 p ( p − 1 ) X t p − 2 ( μ t X t d t + σ t X t d W t ) 2 = p X t p − 1 ( μ t X t d t + σ t X t d W t ) + 1 2 p ( p − 1 ) X t p − 2 σ t 2 X t 2 d t =pX_t^{p-1}\left(\mu_tX_tdt+\sigma_tX_tdW_t\right)+\tfrac{1}{2}p(p-1)X_t^{p-2}\sigma_t^2X_t^2dt = p X t p − 1 ( μ t X t d t + σ t X t d W t ) + 2 1 p ( p − 1 ) X t p − 2 σ t 2 X t 2 d t = ( p μ t + 1 2 p ( p − 1 ) σ t 2 ) X t p d t + p σ t X t p d W t =\left(p\mu_t+\tfrac{1}{2}p(p-1)\sigma_t^2\right)X_t^pdt+p\sigma_tX_t^pdW_t = ( p μ t + 2 1 p ( p − 1 ) σ t 2 ) X t p d t + p σ t X t p d W t = ( p μ t + 1 2 p ( p − 1 ) σ t 2 ) Y t d t + p σ t Y t d W t . =\left(p\mu_t+\tfrac{1}{2}p(p-1)\sigma_t^2\right)Y_tdt+p\sigma_tY_tdW_t. = ( p μ t + 2 1 p ( p − 1 ) σ t 2 ) Y t d t + p σ t Y t d W t . We see from the last line that Y = ( Y t ) t ≥ 0 Y=(Y_t)_{t\geq0} Y = ( Y t ) t ≥ 0 is also a generalized geometric Brownian motion.
Example 8.2.8. Let X X X have generalized geometric Brownian motion dynamics as in (104) . We would like to find an explicit expression for X t X_t X t (i.e., an expression of the form X t = … X_t=\ldots X t = … where … \ldots … does not contain X X X ). To this end, we let Y t = log X t Y_t=\log X_t Y t = log X t . With f ( x ) = log x f(x)=\log x f ( x ) = log x we have f ′ ( x ) = 1 / x f'(x)=1/x f ′ ( x ) = 1/ x and f ′ ′ ( x ) = − 1 / x 2 f''(x)=-1/x^2 f ′′ ( x ) = − 1/ x 2 . Thus, we have
d Y t = 1 X t d X t + 1 − 1 2 X t 2 ( d X t ) 2 = ( μ t − 1 2 σ t 2 ) d t + σ t d W t . dY_t=\frac{1}{X_t}dX_t+\frac{1-1}{2X_t^2}(dX_t)^2=\left(\mu_t-\tfrac{1}{2}\sigma_t^2\right)dt+\sigma_tdW_t. d Y t = X t 1 d X t + 2 X t 2 1 − 1 ( d X t ) 2 = ( μ t − 2 1 σ t 2 ) d t + σ t d W t . Thus, we have
X T = exp ( Y T ) = exp ( Y 0 + ∫ 0 T ( μ t − 1 2 σ t 2 ) d t + ∫ 0 T σ t d W t ) X_T=\exp(Y_T)=\exp\left(Y_0+\int_0^T\left(\mu_t-\tfrac{1}{2}\sigma_t^2\right)dt+\int_0^T\sigma_tdW_t\right) X T = exp ( Y T ) = exp ( Y 0 + ∫ 0 T ( μ t − 2 1 σ t 2 ) d t + ∫ 0 T σ t d W t ) = X 0 exp ( ∫ 0 T ( μ t − 1 2 σ t 2 ) d t + ∫ 0 T σ t d W t ) , (105) =X_0\exp\left(\int_0^T\left(\mu_t-\tfrac{1}{2}\sigma_t^2\right)dt+\int_0^T\sigma_tdW_t\right), \tag*{(105)} = X 0 exp ( ∫ 0 T ( μ t − 2 1 σ t 2 ) d t + ∫ 0 T σ t d W t ) , (105) where we have used Y 0 = log X 0 Y_0=\log X_0 Y 0 = log X 0 .
Proposition 8.2.9. Let W = ( W t ) t ≥ 0 \mathbf{W} = (W_t)_{t\ge0} W = ( W t ) t ≥ 0 be a Brownian motion. Suppose g : R + → R g:\mathbb{R}_{+}\to\mathbb{R} g : R + → R is a deterministic function satisfying ∫ 0 T g 2 ( t ) d t < ∞ \int_{0}^{T}g^{2}(t)\mathrm{d}t<\infty ∫ 0 T g 2 ( t ) d t < ∞ . Then
I T : = ∫ 0 T g ( t ) d W t ∼ N ( 0 , v ( T ) ) , v ( T ) = ∫ 0 T g 2 ( t ) d t . I_T := \int_{0}^{T}g(t)\mathrm{d}W_t \sim\mathcal{N}(0,v(T)), \qquad v(T)=\int_{0}^{T}g^{2}(t)\mathrm{d}t. I T := ∫ 0 T g ( t ) d W t ∼ N ( 0 , v ( T )) , v ( T ) = ∫ 0 T g 2 ( t ) d t . Proof. Set μ t = 0 \mu_t=0 μ t = 0 and σ t = u g ( t ) \sigma_t=ug(t) σ t = ug ( t ) in (104) where u u u is a constant. And suppose X 0 = 1 X_0=1 X 0 = 1 . Then we have
X T = 1 + ∫ 0 T u g ( t ) X t d W t , X_T=1+\int_{0}^{T}ug(t)X_t\mathrm{d}W_t, X T = 1 + ∫ 0 T ug ( t ) X t d W t , which is a martingale since Itô integrals are martingales. From (105) , we know that X T X_T X T can be written explicitly as
X T = exp ( − 1 2 ∫ 0 T u 2 g 2 ( t ) d t + ∫ 0 T u g ( t ) d W t ) = exp ( − u 2 1 2 ∫ 0 T g 2 ( t ) d t + u I T ) . X_T=\exp\left(-\frac{1}{2}\int_{0}^{T}u^{2}g^{2}(t)\mathrm{d}t+\int_{0}^{T}ug(t)\mathrm{d}W_t\right)=\exp\left(-u^{2}\frac{1}{2}\int_{0}^{T}g^{2}(t)\mathrm{d}t+uI_T\right). X T = exp ( − 2 1 ∫ 0 T u 2 g 2 ( t ) d t + ∫ 0 T ug ( t ) d W t ) = exp ( − u 2 2 1 ∫ 0 T g 2 ( t ) d t + u I T ) . Since X X X is a martingale, we have
1 = E X T = E exp ( − u 2 1 2 ∫ 0 T g 2 ( t ) d t + u I T ) ⇒ E exp ( u I T ) = exp ( u 2 1 2 ∫ 0 T g 2 ( t ) d t ) . 1=\mathbb{E}X_T=\mathbb{E}\exp\left(-u^{2}\frac{1}{2}\int_{0}^{T}g^{2}(t)\mathrm{d}t+uI_T\right)
\quad\Rightarrow\quad
\mathbb{E}\exp(uI_T)=\exp\left(u^{2}\frac{1}{2}\int_{0}^{T}g^{2}(t)\mathrm{d}t\right). 1 = E X T = E exp ( − u 2 2 1 ∫ 0 T g 2 ( t ) d t + u I T ) ⇒ E exp ( u I T ) = exp ( u 2 2 1 ∫ 0 T g 2 ( t ) d t ) . Note that E exp ( u I T ) = M I T ( u ) \mathbb{E}\exp(uI_T)=M_{I_T}(u) E exp ( u I T ) = M I T ( u ) is the moment generating function of I T I_T I T and exp ( u 2 1 2 ∫ 0 T g 2 ( t ) d t ) \exp\left(u^{2}\frac{1}{2}\int_{0}^{T}g^{2}(t)\mathrm{d}t\right) exp ( u 2 2 1 ∫ 0 T g 2 ( t ) d t ) is the moment generating function of a normal random variable with mean zero and variance v ( T ) = ∫ 0 T g 2 ( t ) d t v(T)=\int_{0}^{T}g^{2}(t)\mathrm{d}t v ( T ) = ∫ 0 T g 2 ( t ) d t . Thus, I T ∼ N ( 0 , v ( T ) ) I_T\sim\mathcal{N}(0,v(T)) I T ∼ N ( 0 , v ( T )) , as claimed.
Multivariate stochastic calculus
Definition 8.3.1. A d d d -dimensional Brownian motion is a process
W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 \mathbf{W}=(W_t^{1},W_t^{2},\ldots,W_t^{d})_{t\ge0} W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 with the following properties.
Each W i = ( W t i ) t ≥ 0 W^{i}=(W_t^{i})_{t\ge0} W i = ( W t i ) t ≥ 0 , i = 1 , 2 , … , d i=1,2,\ldots,d i = 1 , 2 , … , d , is a one-dimensional Brownian motion.
The processes ( W i ) 1 ≤ i ≤ d (W^{i})_{1\le i\le d} ( W i ) 1 ≤ i ≤ d are independent.
A filtration for W \mathbf{W} W is a collection of σ \sigma σ -algebras F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\ge0} F = ( F t ) t ≥ 0 such that
Information accumulates: F s ⊂ F t \mathcal{F}_s\subset\mathcal{F}_t F s ⊂ F t for all 0 ≤ s < t 0\le s<t 0 ≤ s < t .
Adaptivity: W t ∈ F t W_t\in\mathcal{F}_t W t ∈ F t for all t ≥ 0 t\ge0 t ≥ 0 .
Independent increments: for 0 ≤ s < t 0\le s<t 0 ≤ s < t we have W t − W s ⊥ ⊥ F s W_t-W_s\perp\!\!\!\perp\mathcal{F}_s W t − W s ⊥ ⊥ F s .
Theorem 8.3.2. Let W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 \mathbf{W} = (W_t^1,W_t^2,\ldots,W_t^d)_{t\geq0} W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 be a d d d -dimensional Brownian motion. The covariation of independent components of W \mathbf{W} W is zero: [ W i , W j ] T = 0 [W^i,W^j]_T = 0 [ W i , W j ] T = 0 for all i ≠ j i \ne j i = j and T ≥ 0 T \geq0 T ≥ 0 .
Proof. Let 0 = t 0 < t 1 < … < t n = T 0 = t_0 < t_1 < \ldots< t_n = T 0 = t 0 < t 1 < … < t n = T be a partition Π \Pi Π of [ 0 , T ] [0,T] [ 0 , T ] . The sampled covariation C Π C_\Pi C Π of W i W^i W i and W j W^j W j is given by
C Π = ∑ k = 0 n − 1 ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) C_\Pi= \sum_{k=0}^{n-1} \left(W^i_{t_{k+1}} - W^i_{t_k}\right)\left(W^j_{t_{k+1}} - W^j_{t_k}\right) C Π = k = 0 ∑ n − 1 ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j )
Since E ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) = 0 \mathbb{E}\left(W^i_{t_{k+1}} - W^i_{t_k}\right)\left(W^j_{t_{k+1}} - W^j_{t_k}\right) = 0 E ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) = 0 , we clearly have E C Π = 0 \mathbb{E}C_\Pi= 0 E C Π = 0 . Next, we compute the variance of C Π C_\Pi C Π . We have
V C Π = E C Π 2 = ∑ k = 0 n − 1 ∑ l = 0 n − 1 E ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) ( W t l + 1 i − W t l i ) ( W t l + 1 j − W t l j ) = ∑ k = 0 n − 1 E ( W t k + 1 i − W t k i ) 2 ( W t k + 1 j − W t k j ) 2 + 2 ∑ k = 0 n − 1 ∑ l = 0 k − 1 E ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) ( W t l + 1 i − W t l i ) ( W t l + 1 j − W t l j ) = ∑ k = 0 n − 1 ( t k + 1 − t k ) ( t k + 1 − t k ) ≤ ∥ Π ∥ ∑ k = 0 n − 1 ( t k + 1 − t k ) = ∥ Π ∥ T . \begin{aligned}
\mathop{\mathrm{V}}C_\Pi&= \mathbb{E}C_\Pi^2 = \sum_{k=0}^{n-1}\sum_{l=0}^{n-1}\mathbb{E}\left(W^i_{t_{k+1}}-W^i_{t_k}\right)\left(W^j_{t_{k+1}}-W^j_{t_k}\right)\left(W^i_{t_{l+1}}-W^i_{t_l}\right)\left(W^j_{t_{l+1}}-W^j_{t_l}\right)\\
&= \sum_{k=0}^{n-1}\mathbb{E}\left(W^i_{t_{k+1}}-W^i_{t_k}\right)^2\left(W^j_{t_{k+1}}-W^j_{t_k}\right)^2\\
&\quad+ 2\sum_{k=0}^{n-1}\sum_{l=0}^{k-1}\mathbb{E}\left(W^i_{t_{k+1}}-W^i_{t_k}\right)\left(W^j_{t_{k+1}}-W^j_{t_k}\right)\left(W^i_{t_{l+1}}-W^i_{t_l}\right)\left(W^j_{t_{l+1}}-W^j_{t_l}\right)\\
&= \sum_{k=0}^{n-1}(t_{k+1}-t_k)(t_{k+1}-t_k)\\
&\leq\lVert\Pi\rVert\sum_{k=0}^{n-1}(t_{k+1}-t_k)=\lVert\Pi\rVert T.
\end{aligned} V C Π = E C Π 2 = k = 0 ∑ n − 1 l = 0 ∑ n − 1 E ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) ( W t l + 1 i − W t l i ) ( W t l + 1 j − W t l j ) = k = 0 ∑ n − 1 E ( W t k + 1 i − W t k i ) 2 ( W t k + 1 j − W t k j ) 2 + 2 k = 0 ∑ n − 1 l = 0 ∑ k − 1 E ( W t k + 1 i − W t k i ) ( W t k + 1 j − W t k j ) ( W t l + 1 i − W t l i ) ( W t l + 1 j − W t l j ) = k = 0 ∑ n − 1 ( t k + 1 − t k ) ( t k + 1 − t k ) ≤ ∥ Π ∥ k = 0 ∑ n − 1 ( t k + 1 − t k ) = ∥ Π ∥ T .
Thus, E C Π → 0 \mathbb{E}C_\Pi\to0 E C Π → 0 and V C Π → 0 \mathop{\mathrm{V}}C_\Pi\to0 V C Π → 0 as ∥ Π ∥ → 0 \lVert\Pi\rVert\to0 ∥ Π ∥ → 0 , which proves that [ W i , W j ] T : = lim ∥ Π ∥ → 0 C Π = 0 [W^i,W^j]_T := \lim_{\lVert\Pi\rVert\to0} C_\Pi= 0 [ W i , W j ] T := lim ∥ Π ∥ → 0 C Π = 0 .
Theorem 8.3.2 can be used to derive the covariation of two Itô processes X i X^i X i and X j X^j X j .
Theorem 8.3.3. Let X i = ( X t i ) t ≥ 0 X^i = (X_t^i)_{t\geq0} X i = ( X t i ) t ≥ 0 , i = 1 , 2 , … , n i=1,2,\ldots,n i = 1 , 2 , … , n be the Itô processes given by
d X t i = θ t i d t + ∑ j = 1 d σ t i j d W t j , i = 1 , 2 , … , n , (106) \mathrm{d}X_t^i = \theta_t^i\mathrm{d}t + \sum_{j=1}^{d}\sigma_t^{ij}\mathrm{d}W_t^j,\qquad i=1,2,\ldots,n, \tag*{(106)} d X t i = θ t i d t + j = 1 ∑ d σ t ij d W t j , i = 1 , 2 , … , n , (106) where W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 \mathbf{W} = (W_t^1,W_t^2,\ldots,W_t^d)_{t\geq0} W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 is a d d d -dimensional Brownian motion. Then
d [ X i , X j ] t = ∑ k = 1 d σ t i k σ t j k d t . \mathrm{d}[X^i,X^j]_t = \sum_{k=1}^{d}\sigma_t^{ik}\sigma_t^{jk}\mathrm{d}t. d [ X i , X j ] t = k = 1 ∑ d σ t ik σ t j k d t .
We will not prove Theorem 8.3.3. Rather, we simply remark that it can be obtained informally by writing
d [ X i , X j ] t = d X t i d X t j , (107) \mathrm{d}[X^i,X^j]_t = \mathrm{d}X_t^i\mathrm{d}X_t^j, \tag*{(107)} d [ X i , X j ] t = d X t i d X t j , (107)
inserting expression (106) into (107) and using the multiplication rules
d W t i d W t j = δ i j d t , δ i j = { 1 , i = j , 0 , i ≠ j , d W t j d t = 0 , d t d t = 0. (108) \mathrm{d}W_t^i\mathrm{d}W_t^j=\delta_{ij}\mathrm{d}t,\qquad\delta_{ij}=\begin{cases}1,&i=j,\\0,&i\ne j,\end{cases}\qquad\mathrm{d}W_t^j\mathrm{d}t=0,\qquad\mathrm{d}t\mathrm{d}t=0. \tag*{(108)} d W t i d W t j = δ ij d t , δ ij = { 1 , 0 , i = j , i = j , d W t j d t = 0 , d t d t = 0. (108)
Note that d [ X i , X j ] t = 0 \mathrm{d}[X^i,X^j]_t=0 d [ X i , X j ] t = 0 unless X i X^i X i and X j X^j X j are driven by at least one common one-dimensional Brownian motion.
We can now give a n n n -dimensional version of Itô’s Lemma. We present the formula in differential form, as it is written more compactly in this way.
Theorem 8.3.4 (Itô formula in n n n dimensions). Let X = ( X t 1 , X t 2 , … , X t n ) t ≥ 0 X=(X_t^1,X_t^2,\ldots,X_t^n)_{t\geq0} X = ( X t 1 , X t 2 , … , X t n ) t ≥ 0 be an n n n -dimensional Itô process whose components are of the form (106) , and suppose f : R n → R f:\mathbb{R}^n\to\mathbb{R} f : R n → R satisfies f ∈ C 2 ( R n ) f\in C^2(\mathbb{R}^n) f ∈ C 2 ( R n ) and
E ∫ 0 T ( ∑ i = 1 n ∂ f ( X t ) ∂ x i σ t i j ) 2 d t < ∞ , j = 1 , 2 , … , d , T ≥ 0. \mathbb{E}\int_0^T\left(\sum_{i=1}^n\frac{\partial f(X_t)}{\partial x_i}\sigma_t^{ij}\right)^2\mathrm{d}t<\infty,\qquad j=1,2,\ldots,d,\qquad T\geq0. E ∫ 0 T ( i = 1 ∑ n ∂ x i ∂ f ( X t ) σ t ij ) 2 d t < ∞ , j = 1 , 2 , … , d , T ≥ 0. Then we have
d f ( X t ) = ∑ i = 1 n ∂ f ( X t ) ∂ x i d X t i + 1 2 ∑ i = 1 n ∑ j = 1 n ∂ 2 f ( X t ) ∂ x i ∂ x j d [ X i , X j ] t . \mathrm{d}f(X_t)=\sum_{i=1}^n\frac{\partial f(X_t)}{\partial x_i}\mathrm{d}X_t^i+\frac{1}{2}\sum_{i=1}^n\sum_{j=1}^n\frac{\partial^2f(X_t)}{\partial x_i\partial x_j}\mathrm{d}[X^i,X^j]_t. d f ( X t ) = i = 1 ∑ n ∂ x i ∂ f ( X t ) d X t i + 2 1 i = 1 ∑ n j = 1 ∑ n ∂ x i ∂ x j ∂ 2 f ( X t ) d [ X i , X j ] t .
The proof of Theorem 8.3.4 is a straightforward extension of Theorem 8.2.6 to the n n n -dimensional case and will not be presented here.
To obtain an explicit expression for d f ( X t ) \mathrm{d}f(X_t) d f ( X t ) in terms of d W t 1 , d W t 2 , … , d W t d \mathrm{d}W_t^1,\mathrm{d}W_t^2,\ldots,\mathrm{d}W_t^d d W t 1 , d W t 2 , … , d W t d and d t \mathrm{d}t d t we can repeat the same informal procedure we used in the one-dimensional case.
Expand d f ( X t ) = f ( X t + d X t ) − f ( X t ) \mathrm{d}f(X_t)=f(X_t+\mathrm{d}X_t)-f(X_t) d f ( X t ) = f ( X t + d X t ) − f ( X t ) about the point X t X_t X t to second order
d f ( X t ) = ∑ i = 1 n ∂ f ( X t ) ∂ x i d X t i + 1 2 ∑ i = 1 n ∑ j = 1 n ∂ 2 f ( X t ) ∂ x i ∂ x j d X t i d X t j . (109) \mathrm{d}f(X_t)=\sum_{i=1}^n\frac{\partial f(X_t)}{\partial x_i}\mathrm{d}X_t^i+\frac{1}{2}\sum_{i=1}^n\sum_{j=1}^n\frac{\partial^2f(X_t)}{\partial x_i\partial x_j}\mathrm{d}X_t^i\mathrm{d}X_t^j. \tag*{(109)} d f ( X t ) = i = 1 ∑ n ∂ x i ∂ f ( X t ) d X t i + 2 1 i = 1 ∑ n j = 1 ∑ n ∂ x i ∂ x j ∂ 2 f ( X t ) d X t i d X t j . (109)
Insert expression for d X t i \mathrm{d}X_t^i d X t i into (109) and use the multiplication rules given in (108) .
Example 8.3.5 (Product rule). To compute d ( X t Y t ) \mathrm{d}(X_tY_t) d ( X t Y t ) where X X X and Y Y Y are one-dimensional Itô processes, we define f ( x , y ) = x y f(x,y)=xy f ( x , y ) = x y and use f x = y f_x=y f x = y , f y = x f_y=x f y = x , f x y = 1 f_{xy}=1 f x y = 1 and f x x = f y y = 0 f_{xx}=f_{yy}=0 f xx = f y y = 0 to compute
d ( X t Y t ) = Y t d X t + X t d Y t + d [ X , Y ] t . \mathrm{d}(X_tY_t)=Y_t\mathrm{d}X_t+X_t\mathrm{d}Y_t+\mathrm{d}[X,Y]_t. d ( X t Y t ) = Y t d X t + X t d Y t + d [ X , Y ] t .
Example 8.3.6 (OU process). An Ornstein-Uhlenbeck process (OU process, for short) is an Itô process X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 that satisfies
d X t = κ ( θ − X t ) d t + σ d W t , (110) \mathrm{d}X_t=\kappa(\theta-X_t)\mathrm{d}t+\sigma\mathrm{d}W_t, \tag*{(110)} d X t = κ ( θ − X t ) d t + σ d W t , (110)
where W = ( W t ) t ≥ 0 W = (W_t)_{t \ge0} W = ( W t ) t ≥ 0 is a one-dimensional Brownian motion and κ , θ > 0 \kappa, \theta> 0 κ , θ > 0 . The OU process is mean-reverting in the following sense. If X t > θ X_t > \theta X t > θ then κ ( θ − X t ) < 0 \kappa(\theta- X_t) < 0 κ ( θ − X t ) < 0 and the deterministic part of (110) (i.e., the d t dt d t -term) pushes the process down towards θ \theta θ . If X t < θ X_t < \theta X t < θ then κ ( θ − X t ) > 0 \kappa(\theta- X_t) > 0 κ ( θ − X t ) > 0 and the deterministic part of (110) pushes the process up towards θ \theta θ . The OU process mean-reverts to the long-run mean θ \theta θ . We often call κ \kappa κ the rate of mean reversion , though this nomenclature is somewhat misleading since the instantaneous rate of mean reversion is actually κ ( θ − X t ) \kappa(\theta- X_t) κ ( θ − X t ) .
We will find an explicit expression for X t X_t X t and also compute E X t \operatorname{E}X_t E X t and V X t \operatorname{V}X_t V X t . To this end, let us define Y t = X t − θ Y_t = X_t - \theta Y t = X t − θ so that
d Y t = − κ Y t d t + σ d W t dY_t = -\kappa Y_t dt + \sigma dW_t d Y t = − κ Y t d t + σ d W t
Note that Y Y Y is an OU process that mean-reverts to zero. Next, we define Z t = f ( t , Y t ) = e κ t Y t Z_t = f(t, Y_t) = e^{\kappa t}Y_t Z t = f ( t , Y t ) = e κ t Y t . We can use the two-dimensional Itô formula to compute d Z t dZ_t d Z t . Using f y y = 0 f_{yy} = 0 f y y = 0 and the heuristic rules d t d W t = 0 dt dW_t = 0 d t d W t = 0 and d t d t = 0 dt dt = 0 d t d t = 0 we have
d Z t = f t d t + f y d Y t + 1 2 f y y d [ Y , Y ] t = κ e κ t Y t d t + e κ t d Y t = κ e κ t Y t d t + e κ t ( − κ Y t d t + σ d W t ) = e κ t σ d W t . \begin{aligned}
dZ_t &= f_tdt + f_y dY_t + \frac{1}{2}f_{yy}d[Y,Y]_t \\
&= \kappa e^{\kappa t}Y_tdt + e^{\kappa t}dY_t \\
&= \kappa e^{\kappa t}Y_tdt + e^{\kappa t}(-\kappa Y_tdt + \sigma dW_t) \\
&= e^{\kappa t}\sigma dW_t.
\end{aligned} d Z t = f t d t + f y d Y t + 2 1 f y y d [ Y , Y ] t = κ e κ t Y t d t + e κ t d Y t = κ e κ t Y t d t + e κ t ( − κ Y t d t + σ d W t ) = e κ t σ d W t .
Thus, we have obtained an expression for Z t Z_t Z t :
Z t = Z 0 + ∫ 0 t e κ s σ d W s . Z_t = Z_0 + \int_0^t e^{\kappa s}\sigma dW_s. Z t = Z 0 + ∫ 0 t e κ s σ d W s .
Next, we use Y t = e − κ t Z t Y_t = e^{-\kappa t}Z_t Y t = e − κ t Z t and X t = Y t + θ X_t = Y_t + \theta X t = Y t + θ to obtain
Y t = e − κ t Y 0 + ∫ 0 t e − κ ( t − s ) σ d W s , Y_t = e^{-\kappa t}Y_0 + \int_0^t e^{-\kappa(t-s)}\sigma dW_s, Y t = e − κ t Y 0 + ∫ 0 t e − κ ( t − s ) σ d W s ,
X t = θ + e − κ t ( X 0 − θ ) + ∫ 0 t e − κ ( t − s ) σ d W s . X_t = \theta+ e^{-\kappa t}(X_0 - \theta) + \int_0^t e^{-\kappa(t-s)}\sigma dW_s. X t = θ + e − κ t ( X 0 − θ ) + ∫ 0 t e − κ ( t − s ) σ d W s .
Note that X t X_t X t has a normal distribution at every time t > 0 t > 0 t > 0 because Itô integrals with deterministic integrands are normally distributed random variables; see Proposition 8.2.9. Thus, the distribution of X t X_t X t is completely determined by its mean and variance. We have
E X t = θ + e − κ t ( X 0 − θ ) + E ∫ 0 t e − κ ( t − s ) σ d W s = θ + e − κ t ( X 0 − θ ) , \begin{aligned}
\operatorname{E}X_t = \theta+ e^{-\kappa t}(X_0 - \theta) + \operatorname{E}\int_0^t e^{-\kappa(t-s)}\sigma dW_s \\
&= \theta+ e^{-\kappa t}(X_0 - \theta),
\end{aligned} E X t = θ + e − κ t ( X 0 − θ ) + E ∫ 0 t e − κ ( t − s ) σ d W s = θ + e − κ t ( X 0 − θ ) ,
V X t = V ∫ 0 t e − κ ( t − s ) σ d W s = ∫ 0 t ( e − κ ( t − s ) σ ) 2 d s = σ 2 2 κ ( 1 − e − 2 κ t ) . \begin{aligned}
\operatorname{V}X_t = \operatorname{V}\int_0^t e^{-\kappa(t-s)}\sigma dW_s \\
&= \int_{0}^{t} \left(e^{-\kappa(t-s)}\sigma\right)^2\,\mathrm{d}s = \frac{\sigma^2}{2\kappa}\left(1-e^{-2\kappa t}\right).
\end{aligned} V X t = V ∫ 0 t e − κ ( t − s ) σ d W s = ∫ 0 t ( e − κ ( t − s ) σ ) 2 d s = 2 κ σ 2 ( 1 − e − 2 κ t ) .
where we have used Proposition 8.2.9 .
In some cases, it will be useful to see if a process is in fact a Brownian motion.
Theorem 8.3.7 (Lévy characterization of Brownian motion). Let M = ( M t ) t ≥ 0 M=(M_t)_{t\geq0} M = ( M t ) t ≥ 0 be a martingale with respect to a filtration F = ( F t ) t ≥ 0 \mathcal F=(\mathcal F_t)_{t\geq0} F = ( F t ) t ≥ 0 . Suppose M M M has continuous sample paths and satisfies M 0 = 0 M_0=0 M 0 = 0 and [ M , M ] t = t [M,M]_t=t [ M , M ] t = t for all t ≥ 0 t\geq0 t ≥ 0 . Then M M M is a Brownian motion.
Proof. Let f : R + × R → R f:\mathbb R_+\times\mathbb R\to\mathbb R f : R + × R → R be C 1 , 2 C^{1,2} C 1 , 2 . Note that
d f ( t , M t ) = ∂ t f ( t , M t ) d t + ∂ m f ( t , M t ) d M t + 1 2 ∂ m 2 f ( t , M t ) d [ M , M ] t = ( ∂ t + 1 2 ∂ m 2 ) f ( t , M t ) d t + ∂ m f ( t , M t ) d M t . \begin{aligned}
\mathrm{d}f(t,M_t)=\partial_t f(t,M_t)\,\mathrm{d}t+\partial_m f(t,M_t)\,\mathrm{d}M_t+\frac{1}{2}\partial_m^2 f(t,M_t)\,\mathrm{d}[M,M]_t \\
&=\left(\partial_t+\frac{1}{2}\partial_m^2\right)f(t,M_t)\,\mathrm{d}t+\partial_m f(t,M_t)\,\mathrm{d}M_t.
\end{aligned} d f ( t , M t ) = ∂ t f ( t , M t ) d t + ∂ m f ( t , M t ) d M t + 2 1 ∂ m 2 f ( t , M t ) d [ M , M ] t = ( ∂ t + 2 1 ∂ m 2 ) f ( t , M t ) d t + ∂ m f ( t , M t ) d M t .
In integral form, we have
f ( T , M T ) = f ( t , M t ) + ∫ t T ( ∂ s + 1 2 ∂ m 2 ) f ( s , M s ) d s + ∫ t T ∂ m f ( s , M s ) d M s . f(T,M_T)=f(t,M_t)+\int_t^T\left(\partial_s+\frac{1}{2}\partial_m^2\right)f(s,M_s)\,\mathrm{d}s+\int_t^T\partial_m f(s,M_s)\,\mathrm{d}M_s. f ( T , M T ) = f ( t , M t ) + ∫ t T ( ∂ s + 2 1 ∂ m 2 ) f ( s , M s ) d s + ∫ t T ∂ m f ( s , M s ) d M s .
Although we have not proved it, since M M M is a martingale (by assumption), it follows that any integral of the form I t : = ∫ 0 t Δ s d M s I_t:=\int_0^t\Delta_s\,\mathrm{d}M_s I t := ∫ 0 t Δ s d M s , where Δ = ( Δ t ) t ≥ 0 \Delta=(\Delta_t)_{t\geq0} Δ = ( Δ t ) t ≥ 0 is adapted to ( F t ) t ≥ 0 (\mathcal F_t)_{t\geq0} ( F t ) t ≥ 0 , is a martingale. Thus, using the short-hand E t ⋅ : = E [ ⋅ ∣ F t ] \mathbb E_t\cdot:=\mathbb E[\cdot\mid\mathcal F_t] E t ⋅ := E [ ⋅ ∣ F t ] , we have
E t f ( T , M T ) = f ( t , M t ) + E t ∫ t T ( ∂ s + 1 2 ∂ m 2 ) f ( s , M s ) d s + E t ∫ t T ∂ m f ( s , M s ) d M s = f ( t , M t ) + E t ∫ t T ( ∂ s + 1 2 ∂ m 2 ) f ( s , M s ) d s . \begin{aligned}
\mathbb E_t f(T,M_T)=f(t,M_t)+\mathbb E_t\int_t^T\left(\partial_s+\frac{1}{2}\partial_m^2\right)f(s,M_s)\,\mathrm{d}s+\mathbb E_t\int_t^T\partial_m f(s,M_s)\,\mathrm{d}M_s \\
&=f(t,M_t)+\mathbb E_t\int_t^T\left(\partial_s+\frac{1}{2}\partial_m^2\right)f(s,M_s)\,\mathrm{d}s.
\end{aligned} E t f ( T , M T ) = f ( t , M t ) + E t ∫ t T ( ∂ s + 2 1 ∂ m 2 ) f ( s , M s ) d s + E t ∫ t T ∂ m f ( s , M s ) d M s = f ( t , M t ) + E t ∫ t T ( ∂ s + 2 1 ∂ m 2 ) f ( s , M s ) d s .
Now, let us fix a constant u u u and define
f ( t , m ) = e u m − 1 2 u 2 t . f(t,m)=e^{um-\frac{1}{2}u^2t}. f ( t , m ) = e u m − 2 1 u 2 t .
It is easy to verify that
( ∂ t + 1 2 ∂ m 2 ) f = 0. \left(\partial_t+\frac{1}{2}\partial_m^2\right)f=0. ( ∂ t + 2 1 ∂ m 2 ) f = 0.
It follows that
E t e u M T − 1 2 u 2 T = f ( t , M t ) = e u M t − 1 2 u 2 t ⟹ E t e u ( M T − M t ) = e 1 2 u 2 ( T − t ) . (111) \mathbb E_t e^{uM_T-\frac{1}{2}u^2T}=f(t,M_t)=e^{uM_t-\frac{1}{2}u^2t}
\qquad\Longrightarrow\qquad
\mathbb E_t e^{u(M_T-M_t)}=e^{\frac{1}{2}u^2(T-t)}. \tag*{(111)} E t e u M T − 2 1 u 2 T = f ( t , M t ) = e u M t − 2 1 u 2 t ⟹ E t e u ( M T − M t ) = e 2 1 u 2 ( T − t ) . (111)
By definition, E t e u ( M T − M t ) \mathbb E_t e^{u(M_T-M_t)} E t e u ( M T − M t ) is the F t \mathcal F_t F t -conditional moment generating function of ( M T − M t ) (M_T-M_t) ( M T − M t ) . And e 1 2 u 2 ( T − t ) e^{\frac{1}{2}u^2(T-t)} e 2 1 u 2 ( T − t ) is the moment generating function of a N ( 0 , T − t ) \mathcal N(0,T-t) N ( 0 , T − t ) random variable. It follows that M T − M t ∼ N ( 0 , T − t ) M_T-M_t\sim\mathcal N(0,T-t) M T − M t ∼ N ( 0 , T − t ) for all 0 ≤ t ≤ T < ∞ 0\leq t\leq T<\infty 0 ≤ t ≤ T < ∞ , just like a Brownian motion. Furthermore, we see that M T − M t ⊥ ⊥ F t M_T-M_t\perp\!\!\!\perp\mathcal F_t M T − M t ⊥ ⊥ F t , as the right-hand-side of (111) does not depend on F t \mathcal F_t F t . As M M M satisfies all properties of a Brownian motion, it must be a Brownian motion. ∎
Example 8.3.8 (Correlated Brownian motions). Let B = ( B t ) t ≥ 0 B=(B_t)_{t\geq0} B = ( B t ) t ≥ 0 be given by
B t = ρ W t 1 + ρ ˉ W t 2 , ρ ˉ = 1 − ρ 2 , ρ = [ − 1 , 1 ] , B_t=\rho W_t^1+\bar{\rho}W_t^2,\qquad\bar{\rho}=\sqrt{1-\rho^2},\qquad\rho=[-1,1], B t = ρ W t 1 + ρ ˉ W t 2 , ρ ˉ = 1 − ρ 2 , ρ = [ − 1 , 1 ] , where W = ( W t 1 , W t 2 ) W=(W_t^1,W_t^2) W = ( W t 1 , W t 2 ) is a two-dimensional Brownian motion. We will use Theorem 8.3.7 to show that B B B is a Brownian motion. It is clear that B 0 = 0 B_0=0 B 0 = 0 and B B B has sample paths that are continuous. It is also clear that B B B is a martingale since W 1 W^1 W 1 and W 2 W^2 W 2 are martingales. What remains is to show that [ B , B ] t = t [B,B]_t=t [ B , B ] t = t . We have
[ B , B ] t = [ ρ W 1 + ρ ˉ W 2 , ρ W 1 + ρ ˉ W 2 ] t = ρ 2 [ W 1 , W 1 ] t + 2 ρ ρ ˉ [ W 1 , W 2 ] t + ρ ˉ 2 [ W 2 , W 2 ] t = ρ 2 t + 0 + ρ ˉ 2 t = t . \begin{aligned}
[B,B]_t&=[\rho W^1+\bar{\rho}W^2,\rho W^1+\bar{\rho}W^2]_t\\
&=\rho^2[W^1,W^1]_t+2\rho\bar{\rho}[W^1,W^2]_t+\bar{\rho}^2[W^2,W^2]_t\\
&=\rho^2t+0+\bar{\rho}^2t=t.
\end{aligned} [ B , B ] t = [ ρ W 1 + ρ ˉ W 2 , ρ W 1 + ρ ˉ W 2 ] t = ρ 2 [ W 1 , W 1 ] t + 2 ρ ρ ˉ [ W 1 , W 2 ] t + ρ ˉ 2 [ W 2 , W 2 ] t = ρ 2 t + 0 + ρ ˉ 2 t = t . Alternatively, one can simply compute ( d B t ) 2 = ( ρ d W t 1 + ρ ˉ d W t 2 ) 2 (\mathrm{d}B_t)^2=(\rho\mathrm{d}W_t^1+\bar{\rho}\mathrm{d}W_t^2)^2 ( d B t ) 2 = ( ρ d W t 1 + ρ ˉ d W t 2 ) 2 and use d W t i d W t j = δ i j d t \mathrm{d}W_t^i\mathrm{d}W_t^j=\delta_{ij}\mathrm{d}t d W t i d W t j = δ ij d t in order to show that d B t = d t \mathrm{d}B_t=\mathrm{d}t d B t = d t . Note that
[ B , W 1 ] t = [ ρ W 1 + ρ ˉ W 2 , W 1 ] t = ρ [ W 1 , W 1 ] + ρ ˉ [ W 2 , W 1 ] = ρ t . \begin{aligned}
[B,W^1]_t&=[\rho W^1+\bar{\rho}W^2,W^1]_t\\
&=\rho[W^1,W^1]+\bar{\rho}[W^2,W^1]=\rho t.
\end{aligned} [ B , W 1 ] t = [ ρ W 1 + ρ ˉ W 2 , W 1 ] t = ρ [ W 1 , W 1 ] + ρ ˉ [ W 2 , W 1 ] = ρt . where we have used [ W i , W j ] t = δ i j t [W^i,W^j]_t=\delta_{ij}t [ W i , W j ] t = δ ij t .
Brownian bridge
Definition 8.4.1. A Gaussian process X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 is any process that, for arbitrary times t 1 < t 2 < ⋯ t n t_1<t_2<\cdots t_n t 1 < t 2 < ⋯ t n , the joint distribution of ( X t 1 , X t 2 , … , X t n ) (X_{t_1},X_{t_2},\ldots,X_{t_n}) ( X t 1 , X t 2 , … , X t n ) is jointly normal.
As previously mentioned, the distribution of a normally distributed random vector is uniquely determined by its mean vector m \mathbf{m} m and covariance matrix. Thus, for a Gaussian process, we are interested in
m ( t ) = E X t , c ( s , t ) = E ( X t − m ( t ) ) ( X s − m ( s ) ) . m(t)=\mathbb{E}X_t,\qquad c(s,t)=\mathbb{E}(X_t-m(t))(X_s-m(s)). m ( t ) = E X t , c ( s , t ) = E ( X t − m ( t )) ( X s − m ( s )) .
:::
Example 8.4.2. A Brownian motion W W W is a Gaussian process. To see this, fix an arbitrary sequence of times 0 = t 0 < t 1 < t 2 < ⋯ t n 0=t_0<t_1<t_2<\cdots t_n 0 = t 0 < t 1 < t 2 < ⋯ t n . Note that
W t k = ∑ j = 0 k − 1 ( W t j + 1 − W t j ) . W_{t_k}=\sum_{j=0}^{k-1}(W_{t_{j+1}}-W_{t_j}). W t k = j = 0 ∑ k − 1 ( W t j + 1 − W t j ) . The increments are independent and normally distributed W t j + 1 − W t j ∼ N ( 0 , t j + 1 − t j ) W_{t_{j+1}}-W_{t_j}\sim\mathcal{N}(0,t_{j+1}-t_j) W t j + 1 − W t j ∼ N ( 0 , t j + 1 − t j ) . It follows that the vector ( W t 1 , W t 2 , … , W t n ) (W_{t_1},W_{t_2},\ldots,W_{t_n}) ( W t 1 , W t 2 , … , W t n ) is jointly normal. The mean vector is given by m = ( m ( t 1 ) , m ( t 2 ) , … , m ( t n ) ) = ( 0 , 0 , … , 0 ) \mathbf{m}=(m(t_1),m(t_2),\ldots,m(t_n))=(0,0,\ldots,0) m = ( m ( t 1 ) , m ( t 2 ) , … , m ( t n )) = ( 0 , 0 , … , 0 ) and the covariance matrix C = ( c ( t i , t j ) ) 1 ≤ i , j ≤ n \mathbf{C}=(c(t_i,t_j))_{1\leq i,j\leq n} C = ( c ( t i , t j ) ) 1 ≤ i , j ≤ n has entries c ( t i , t j ) = t i ∧ t j c(t_i,t_j)=t_i\wedge t_j c ( t i , t j ) = t i ∧ t j ; see equation (77) .
Example 8.4.3. A stochastic integral with a deterministic integrand
I t = ∫ 0 t g ( s ) d W s , I_t = \int_{0}^{t} g(s)\,\mathrm{d}W_s, I t = ∫ 0 t g ( s ) d W s ,
is a Gaussian process. To see this, fix an arbitrary sequence of times 0 = t 0 < t 1 < t 2 < ⋯ t n 0 = t_0 < t_1 < t_2 < \cdots t_n 0 = t 0 < t 1 < t 2 < ⋯ t n . Note that
I t k = ∑ j = 0 k − 1 ∫ t j t j + 1 g ( s ) d W s . I_{t_k} = \sum_{j=0}^{k-1} \int_{t_j}^{t_{j+1}} g(s)\,\mathrm{d}W_s. I t k = j = 0 ∑ k − 1 ∫ t j t j + 1 g ( s ) d W s .
The increments are independent and normally distributed with mean zero and variance:
V ( ∫ t j t j + 1 g ( s ) d W s ) = ∫ t j t j + 1 g 2 ( s ) d s \mathop{\mathrm{V}}\left(\int_{t_j}^{t_{j+1}} g(s)\,\mathrm{d}W_s\right) = \int_{t_j}^{t_{j+1}} g^2(s)\,\mathrm{d}s V ( ∫ t j t j + 1 g ( s ) d W s ) = ∫ t j t j + 1 g 2 ( s ) d s
It follows that the vector ( I t 1 , I t 2 , … , I t n ) (I_{t_1}, I_{t_2}, \ldots, I_{t_n}) ( I t 1 , I t 2 , … , I t n ) is jointly normal. The mean vector is given by m = ( m ( t 1 ) , m ( t 2 ) , … , m ( t n ) ) = ( 0 , 0 , … , 0 ) \mathbf{m} = (m(t_1), m(t_2), \ldots, m(t_n)) = (0, 0, \ldots, 0) m = ( m ( t 1 ) , m ( t 2 ) , … , m ( t n )) = ( 0 , 0 , … , 0 ) . To compute the covariance matrix C = ( c ( t i , t j ) ) 1 ≤ i , j ≤ n \mathbf{C} = (c(t_i,t_j))_{1\leq i,j\leq n} C = ( c ( t i , t j ) ) 1 ≤ i , j ≤ n , assume without loss of generality that t i < t j t_i < t_j t i < t j . Then we have
c ( t i , t j ) = E [ ∫ 0 t i g ( s ) d W s ∫ 0 t j g ( u ) d W u ] = E ( ∫ 0 t i g ( s ) d W s ) 2 + E [ ∫ 0 t i g ( s ) d W s ] E [ ∫ t i t j g ( u ) d W u | F t i ] = ∫ 0 t i g 2 ( s ) d s = ∫ 0 t i ∧ t j g 2 ( s ) d s , \begin{aligned}
c(t_i,t_j) &= \mathbb{E}\left[\int_{0}^{t_i} g(s)\,\mathrm{d}W_s \int_{0}^{t_j} g(u)\,\mathrm{d}W_u\right]\\
&= \mathbb{E}\left(\int_{0}^{t_i} g(s)\,\mathrm{d}W_s\right)^2 + \mathbb{E}\left[\int_{0}^{t_i} g(s)\,\mathrm{d}W_s\right]\mathbb{E}\left[\int_{t_i}^{t_j} g(u)\,\mathrm{d}W_u \,\middle|\, \mathcal{F}_{t_i}\right]\\
&= \int_{0}^{t_i} g^2(s)\,\mathrm{d}s = \int_{0}^{t_i\wedge t_j} g^2(s)\,\mathrm{d}s,
\end{aligned} c ( t i , t j ) = E [ ∫ 0 t i g ( s ) d W s ∫ 0 t j g ( u ) d W u ] = E ( ∫ 0 t i g ( s ) d W s ) 2 + E [ ∫ 0 t i g ( s ) d W s ] E [ ∫ t i t j g ( u ) d W u F t i ] = ∫ 0 t i g 2 ( s ) d s = ∫ 0 t i ∧ t j g 2 ( s ) d s ,
where we have used Proposition 8.2.9 .
Definition 8.4.4 (Brownian bridge, version I). Let W = ( W t ) t ≥ 0 \mathbf{W} = (W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion and fix T > 0 T > 0 T > 0 . We define X a → b = ( X t a → b ) 0 ≤ t ≤ T X^{a\to b} = (X_t^{a\to b})_{0\leq t\leq T} X a → b = ( X t a → b ) 0 ≤ t ≤ T , a Brownian bridge from a a a to b b b on [ 0 , T ] [0,T] [ 0 , T ] , by
X t a → b = a + t T ( b − a ) + W t − t T W T , 0 ≤ t ≤ T . (112) X_t^{a\to b} = a + \frac{t}{T}(b-a) + W_t - \frac{t}{T}W_T,\qquad0\leq t\leq T. \tag*{(112)} X t a → b = a + T t ( b − a ) + W t − T t W T , 0 ≤ t ≤ T . (112) Clearly, we have X 0 a → b = a X_0^{a\to b} = a X 0 a → b = a and X T a → b = b X_T^{a\to b} = b X T a → b = b .
Theorem 8.4.5. The Brownian Bridge from a a a to b b b , defined in (112) , is a Gaussian process. The mean vector and covariance matrix have entries given by
m a → b ( t ) = a + t T ( b − a ) , m^{a\to b}(t) = a + \frac{t}{T}(b-a), m a → b ( t ) = a + T t ( b − a ) , c a → b ( t , s ) = t ∧ s − s t T . c^{a\to b}(t,s) = t\wedge s - \frac{st}{T}. c a → b ( t , s ) = t ∧ s − T s t . Proof. Since the sum of two (possibly correlated) normal random variables is again normal, and since for every t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] , the value of X t a → b X_t^{a\to b} X t a → b is a linear combination of W t W_t W t and W T W_T W T , both of which are normal,
it follows that ( X t 1 a → b , X t 2 a → b , … , X t n a → b ) (X_{t_1}^{a\to b},X_{t_2}^{a\to b},\ldots,X_{t_n}^{a\to b}) ( X t 1 a → b , X t 2 a → b , … , X t n a → b ) is jointly normal. To compute the mean and variance, we note that
m a → b ( t ) = E X t a → b = a + t T ( b − a ) + E W t − t T E W T = a + t T ( b − a ) , m^{a\to b}(t)=\mathbb{E}X_t^{a\to b}=a+\frac{t}{T}(b-a)+\mathbb{E}W_t-\frac{t}{T}\mathbb{E}W_T=a+\frac{t}{T}(b-a), m a → b ( t ) = E X t a → b = a + T t ( b − a ) + E W t − T t E W T = a + T t ( b − a ) ,
c a → b ( t , s ) = E ( X t a → b − m a → b ( t ) ) ( X s a → b − m a → b ( s ) ) = E ( W t − t T W T ) ( W s − s T W T ) = E W t W s − t T E W s W T − s T E W t W T + t s T 2 E W T 2 = t ∧ s − s t T − s t T + s t T = t ∧ s − s t T . \begin{aligned}
c^{a\to b}(t,s)&=\mathbb{E}\left(X_t^{a\to b}-m^{a\to b}(t)\right)\left(X_s^{a\to b}-m^{a\to b}(s)\right)\\
&=\mathbb{E}\left(W_t-\frac{t}{T}W_T\right)\left(W_s-\frac{s}{T}W_T\right)\\
&=\mathbb{E}W_tW_s-\frac{t}{T}\mathbb{E}W_sW_T-\frac{s}{T}\mathbb{E}W_tW_T+\frac{ts}{T^2}\mathbb{E}W_T^2\\
&=t\wedge s-\frac{st}{T}-\frac{st}{T}+\frac{st}{T}\\
&=t\wedge s-\frac{st}{T}.
\end{aligned} c a → b ( t , s ) = E ( X t a → b − m a → b ( t ) ) ( X s a → b − m a → b ( s ) ) = E ( W t − T t W T ) ( W s − T s W T ) = E W t W s − T t E W s W T − T s E W t W T + T 2 t s E W T 2 = t ∧ s − T s t − T s t + T s t = t ∧ s − T s t .
Suppose F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 is a filtration for a Brownian motion W W W . Then X a → b X^{a\to b} X a → b is clearly not adapted to F \mathbb{F} F . Indeed, since X t a → b X_t^{a\to b} X t a → b is expressed in terms of W T W_T W T , we require the information in F T \mathcal{F}_T F T in order to write the value of X t a → b X_t^{a\to b} X t a → b . There is an alternative definition of a Brownian bridge, which is adapted to F \mathbb{F} F .
Definition 8.4.6 (Brownian bridge, version II). Let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion and fix T > 0 T>0 T > 0 . We define Y a → b = ( Y t a → b ) 0 ≤ t ≤ T Y^{a\to b}=(Y_t^{a\to b})_{0\leq t\leq T} Y a → b = ( Y t a → b ) 0 ≤ t ≤ T , a Brownian bridge from a a a to b b b on [ 0 , T ] [0,T] [ 0 , T ] , by
Y t a → b = a + t T ( b − a ) + ( T − t ) ∫ 0 t 1 T − s d W s , 0 ≤ t ≤ T . (113) Y_t^{a\to b}=a+\frac{t}{T}(b-a)+(T-t)\int_0^t\frac{1}{T-s}\,\mathrm{d}W_s,\qquad0\leq t\leq T. \tag*{(113)} Y t a → b = a + T t ( b − a ) + ( T − t ) ∫ 0 t T − s 1 d W s , 0 ≤ t ≤ T . (113)
Theorem 8.4.7. The process Y a → b = ( Y t a → b ) 0 ≤ t ≤ T Y^{a\to b}=(Y_t^{a\to b})_{0\leq t\leq T} Y a → b = ( Y t a → b ) 0 ≤ t ≤ T given in Definition 8.4.6 is a Gaussian process. The mean vector and covariance matrix have entries given by
m a → b ( t ) = a + t T ( b − a ) , m^{a\to b}(t)=a+\frac{t}{T}(b-a), m a → b ( t ) = a + T t ( b − a ) , c a → b ( t , s ) = t ∧ s − s t T , c^{a\to b}(t,s)=t\wedge s-\frac{st}{T}, c a → b ( t , s ) = t ∧ s − T s t , just as X a → b = ( X t a → b ) 0 ≤ t ≤ T X^{a\to b}=(X_t^{a\to b})_{0\leq t\leq T} X a → b = ( X t a → b ) 0 ≤ t ≤ T .
Proof. The stochastic integral I t : = ∫ 0 t 1 T − s d W s I_t:=\int_0^t\frac{1}{T-s}\,\mathrm{d}W_s I t := ∫ 0 t T − s 1 d W s in (113) has a deterministic integrand. Therefore, I t I_t I t is Gaussian, as is ( T − t ) I t (T-t)I_t ( T − t ) I t . It follows that the process Y Y Y is a Gaussian process. Next, we compute
m a → b ( t ) = E Y t a → b = a + t T ( b − a ) + ( T − t ) E ∫ 0 t 1 T − s d W s = a + t T ( b − a ) . m^{a\to b}(t)=\mathbb{E}Y_t^{a\to b}=a+\frac{t}{T}(b-a)+(T-t)\mathbb{E}\int_0^t\frac{1}{T-s}\,\mathrm{d}W_s=a+\frac{t}{T}(b-a). m a → b ( t ) = E Y t a → b = a + T t ( b − a ) + ( T − t ) E ∫ 0 t T − s 1 d W s = a + T t ( b − a ) .
Finally, to compute c ( t , s ) c(t,s) c ( t , s ) , assume without loss of generality that s ≤ t s\leq t s ≤ t . Then we have
c a → b ( t , s ) = E ( Y t a → b − m ( t ) ) ( Y s a → b − m ( s ) ) c^{a\to b}(t,s)=\mathbb{E}\left(Y_t^{a\to b}-m(t)\right)\left(Y_s^{a\to b}-m(s)\right) c a → b ( t , s ) = E ( Y t a → b − m ( t ) ) ( Y s a → b − m ( s ) )
= ( T − t ) ( T − s ) E [ ∫ 0 t 1 T − u d W u ⋅ ∫ 0 s 1 T − r d W r ] = ( T − t ) ( T − s ) E [ ( ∫ 0 s 1 T − u d W u ) 2 ] + ( T − t ) ( T − s ) E [ ∫ 0 s 1 T − u d W u E [ ∫ s t 1 T − r d W r ∣ F s ] ] = ( T − t ) ( T − s ) ∫ 0 s ( 1 T − u ) 2 d u = ( T − t ) ( T − s ) ( 1 T − s − 1 T ) = s − s t T = s ∧ t − s t T . \begin{aligned}
&= (T-t)(T-s)\mathbb{E}\left[\int_0^t \frac{1}{T-u}\,\mathrm{d}W_u\cdot\int_0^s \frac{1}{T-r}\,\mathrm{d}W_r\right]\\
&= (T-t)(T-s)\mathbb{E}\left[\left(\int_0^s \frac{1}{T-u}\,\mathrm{d}W_u\right)^2\right]+(T-t)(T-s)\mathbb{E}\left[\int_0^s \frac{1}{T-u}\,\mathrm{d}W_u\mathbb{E}\left[\left.\int_s^t \frac{1}{T-r}\,\mathrm{d}W_r\right|\mathcal{F}_s\right]\right]\\
&= (T-t)(T-s)\int_0^s\left(\frac{1}{T-u}\right)^2\mathrm{d}u\\
&= (T-t)(T-s)\left(\frac{1}{T-s}-\frac{1}{T}\right)=s-\frac{st}{T}=s\wedge t-\frac{st}{T}.
\end{aligned} = ( T − t ) ( T − s ) E [ ∫ 0 t T − u 1 d W u ⋅ ∫ 0 s T − r 1 d W r ] = ( T − t ) ( T − s ) E [ ( ∫ 0 s T − u 1 d W u ) 2 ] + ( T − t ) ( T − s ) E [ ∫ 0 s T − u 1 d W u E [ ∫ s t T − r 1 d W r F s ] ] = ( T − t ) ( T − s ) ∫ 0 s ( T − u 1 ) 2 d u = ( T − t ) ( T − s ) ( T − s 1 − T 1 ) = s − T s t = s ∧ t − T s t .
This completes the proof.
It is interesting to compute the differential of Y a → b Y^{a\to b} Y a → b . For simplicity, consider Y 0 → 0 Y^{0\to0} Y 0 → 0 . We have
d Y t 0 → 0 = ( ∫ 0 t 1 T − s d W s ) d ( T − t ) + ( T − t ) d ∫ 0 t 1 T − s d W s = − ( 1 T − t Y t 0 → 0 ) d t + ( T − t ) 1 T − t d W t = − Y t 0 → 0 T − t d t + d W t . \begin{aligned}
\mathrm{d}Y_t^{0\to0}&=\left(\int_0^t\frac{1}{T-s}\,\mathrm{d}W_s\right)\mathrm{d}(T-t)+(T-t)\mathrm{d}\int_0^t\frac{1}{T-s}\,\mathrm{d}W_s\\
&=-\left(\frac{1}{T-t}Y_t^{0\to0}\right)\mathrm{d}t+(T-t)\frac{1}{T-t}\,\mathrm{d}W_t\\
&=-\frac{Y_t^{0\to0}}{T-t}\,\mathrm{d}t+\mathrm{d}W_t.
\end{aligned} d Y t 0 → 0 = ( ∫ 0 t T − s 1 d W s ) d ( T − t ) + ( T − t ) d ∫ 0 t T − s 1 d W s = − ( T − t 1 Y t 0 → 0 ) d t + ( T − t ) T − t 1 d W t = − T − t Y t 0 → 0 d t + d W t .
Note that Y 0 → 0 Y^{0\to0} Y 0 → 0 mean reverts to zero with an instantaneous rate ∣ Y t 0 → 0 ∣ T − t \frac{\lvert Y_t^{0\to0}\rvert}{T-t} T − t ∣ Y t 0 → 0 ∣ that blows up as t → T t\to T t → T . This blow up guarantees that Y T 0 → 0 = 0 Y_T^{0\to0}=0 Y T 0 → 0 = 0 .
Definition 8.4.8 (Brownian bridge, version III). Let W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 be a Brownian motion and fix T > 0 T>0 T > 0 . We define Z a → b = ( Z t a → b ) 0 ≤ t ≤ T Z^{a\to b}=(Z_t^{a\to b})_{0\leq t\leq T} Z a → b = ( Z t a → b ) 0 ≤ t ≤ T , a *Brownian bridge from a a a to b b b on [ 0 , T ] [0,T] [ 0 , T ] , by
Z t a → b = a + W t ∣ W T = b − a , 0 ≤ t ≤ T . Z_t^{a\to b}=a+W_t\mid W_T=b-a,\qquad0\leq t\leq T. Z t a → b = a + W t ∣ W T = b − a , 0 ≤ t ≤ T .
Theorem 8.4.9. The process Z a → b = ( Z t a → b ) 0 ≤ t ≤ T Z^{a\to b}=(Z_t^{a\to b})_{0\leq t\leq T} Z a → b = ( Z t a → b ) 0 ≤ t ≤ T given in Definition 8.4.8 is a Gaussian process. The mean vector and covariance matrix have entries given by
m a → b ( t ) = a + t T ( b − a ) , m^{a\to b}(t)=a+\frac{t}{T}(b-a), m a → b ( t ) = a + T t ( b − a ) , c a → b ( t , s ) = t ∧ s − s t T , c^{a\to b}(t,s)=t\wedge s-\frac{st}{T}, c a → b ( t , s ) = t ∧ s − T s t , just as X a → b = ( X t a → b ) 0 ≤ t ≤ T X^{a\to b}=(X_t^{a\to b})_{0\leq t\leq T} X a → b = ( X t a → b ) 0 ≤ t ≤ T .
Proof. We will compute the density of Z a → b Z^{a\to b} Z a → b directly. For simplicity, let a = 0 a=0 a = 0 . We have
P ( Z t 0 → b ∈ d x ) = P ( W t ∈ d x ∣ W T = b ) = P ( W t ∈ d x , W T ∈ d b ) P ( W T ∈ d b ) = P ( W t ∈ d x ) P ( W T − W t + x ∈ d b ) P ( W T ∈ d b ) . \begin{aligned}
\mathbb{P}(Z_t^{0\to b}\in\mathrm{d}x)=\mathbb{P}(W_t\in\mathrm{d}x\mid W_T=b)=\frac{\mathbb{P}(W_t\in\mathrm{d}x,W_T\in\mathrm{d}b)}{\mathbb{P}(W_T\in\mathrm{d}b)} \\
&=\frac{\mathbb{P}(W_t\in\mathrm{d}x)\mathbb{P}(W_T-W_t+x\in\mathrm{d}b)}{\mathbb{P}(W_T\in\mathrm{d}b)}.
\end{aligned} P ( Z t 0 → b ∈ d x ) = P ( W t ∈ d x ∣ W T = b ) = P ( W T ∈ d b ) P ( W t ∈ d x , W T ∈ d b ) = P ( W T ∈ d b ) P ( W t ∈ d x ) P ( W T − W t + x ∈ d b ) .
Letting f μ , σ 2 f_{\mu,\sigma^2} f μ , σ 2 denote the density of a N ( μ , σ 2 ) \mathcal{N}(\mu,\sigma^2) N ( μ , σ 2 ) random variable we have
P ( Z t 0 → b ∈ d x ) = f 0 , t ( x ) d x f x , T − t ( b ) d b f 0 , T ( b ) d b = f m 0 → b ( t ) , c 0 → b ( t , T ) ( x ) d x \mathbb{P}(Z_t^{0\to b} \in dx) = \frac{f_{0,t}(x)\,dx\,f_{x,T-t}(b)\,db}{f_{0,T}(b)\,db} = f_{m^{0\to b}(t),c^{0\to b}(t,T)}(x)\,dx P ( Z t 0 → b ∈ d x ) = f 0 , T ( b ) d b f 0 , t ( x ) d x f x , T − t ( b ) d b = f m 0 → b ( t ) , c 0 → b ( t , T ) ( x ) d x
where m a → b ( t ) m^{a\to b}(t) m a → b ( t ) and c a → b ( t , s ) c^{a\to b}(t,s) c a → b ( t , s ) are as given in Theorem 8.4.9 . We can clearly see then, that Z t 0 → b Z_t^{0\to b} Z t 0 → b is normally distributed at every t t t with mean m 0 → b ( t ) m^{0\to b}(t) m 0 → b ( t ) . We leave computation of the entries of the covariance matrix c 0 → b ( t , s ) c^{0\to b}(t,s) c 0 → b ( t , s ) as an exercise for the reader.
Girsanov's Theorem for a single Brownian motion
We briefly recall some results from § 6.3 . Suppose that, on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) we have a random variable Z ≥ 0 Z \ge0 Z ≥ 0 that has expectation E Z = 1 \mathbb{E}Z = 1 E Z = 1 . Then we can define a new probability measure P ~ \widetilde{\mathbb{P}} P via
P ~ ( A ) = E Z 1 A , A ∈ F , \widetilde{\mathbb{P}}(A) = \mathbb{E}Z1_A, \qquad A \in\mathcal{F}, P ( A ) = E Z 1 A , A ∈ F ,
and we call Z = d P ~ d P Z = \frac{d\widetilde{\mathbb{P}}}{d\mathbb{P}} Z = d P d P the Radon-Nikodým derivative of P ~ \widetilde{\mathbb{P}} P with respect to P \mathbb{P} P . If Z Z Z is strictly positive Z > 0 Z > 0 Z > 0 , then we also have
P ( A ) = E ~ 1 Z 1 A , A ∈ F , \mathbb{P}(A) = \widetilde{\mathbb{E}}\frac{1}{Z}1_A, \qquad A \in\mathcal{F}, P ( A ) = E Z 1 1 A , A ∈ F ,
and we call 1 Z = d P d P ~ \frac{1}{Z} = \frac{d\mathbb{P}}{d\widetilde{\mathbb{P}}} Z 1 = d P d P the Radon-Nikodým derivative of P \mathbb{P} P with respect to P ~ \widetilde{\mathbb{P}} P .
In Example 1.7.6, on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) , we defined X ∼ N ( 0 , 1 ) X \sim\mathcal{N}(0,1) X ∼ N ( 0 , 1 ) and a Radon-Nikodým derivative Z = e − θ X − 1 2 θ 2 Z = e^{-\theta X-\frac{1}{2}\theta^2} Z = e − θ X − 2 1 θ 2 . We showed that Y : = X + θ Y := X + \theta Y := X + θ was N ( θ , 1 ) \mathcal{N}(\theta,1) N ( θ , 1 ) under P \mathbb{P} P and N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) under P ~ \widetilde{\mathbb{P}} P . Thus, Z Z Z had the effect of changing the mean of Y Y Y .
We would like to extend this idea from a static to a dynamics setting. Specifically, we would like to find a measure change that modifies the dynamics of a stochastic process X = ( X t ) t ≥ 0 X = (X_t)_{t\ge0} X = ( X t ) t ≥ 0 .
Definition 8.5.1. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F} = (\mathcal{F}_t)_{0\le t\le T} F = ( F t ) 0 ≤ t ≤ T be a filtration on this space. A Radon-Nikodým derivative process ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T is any process of the form
Z t : = E [ Z ∣ F t ] Z_t := \mathbb{E}[Z\mid\mathcal{F}_t] Z t := E [ Z ∣ F t ] where Z Z Z is an F T \mathcal{F}_T F T -measurable random variable satisfying E Z = 1 \mathbb{E}Z = 1 E Z = 1 and Z > 0 Z > 0 Z > 0 .
Note that Z Z Z in Definition 8.5.1 satisfies the conditions of a Radon-Nikodým derivative. As such, one can define a measure change d P ~ d P \frac{d\widetilde{\mathbb{P}}}{d\mathbb{P}} d P d P from Z Z Z .
Lemma 8.5.2. A Radon-Nikodým derivative process Z = ( Z t ) 0 ≤ t ≤ T Z = (Z_t)_{0\le t\le T} Z = ( Z t ) 0 ≤ t ≤ T is a martingale.
Proof. For $0 \le s \le t \le T$ we have
E [ Z t ∣ F s ] = E [ E [ Z ∣ F t ] ∣ F s ] = E [ Z ∣ F s ] = Z s . \mathbb{E}[Z_t\mid\mathcal{F}_s]=\mathbb{E}[\mathbb{E}[Z\mid\mathcal{F}_t]\mid\mathcal{F}_s]=\mathbb{E}[Z\mid\mathcal{F}_s]=Z_s. E [ Z t ∣ F s ] = E [ E [ Z ∣ F t ] ∣ F s ] = E [ Z ∣ F s ] = Z s .
Lemma 8.5.3. Let ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T be a Radon-Nikodým derivative process and define d P ~ d P = Z \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}=Z d P d P = Z . Suppose Y ∈ F s Y\in\mathcal{F}_s Y ∈ F s , where s ∈ [ 0 , T ] s\in[0,T] s ∈ [ 0 , T ] . Then
E ~ Y = E Z s Y . \widetilde{\mathbb{E}}Y=\mathbb{E}Z_sY. E Y = E Z s Y .
Proof. The proof is a simple exercise in iterated conditioning. We have
E ~ Y = E Z Y = E Y E [ Z ∣ F s ] = E Y Z s . \widetilde{\mathbb{E}}Y=\mathbb{E}ZY=\mathbb{E}Y\mathbb{E}[Z\mid\mathcal{F}_s]=\mathbb{E}YZ_s. E Y = E Z Y = E Y E [ Z ∣ F s ] = E Y Z s .
Lemma 8.5.4. Let ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T be a Radon-Nikodým derivative process and define d P ~ d P = Z \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}=Z d P d P = Z . Suppose Y ∈ F t Y\in\mathcal{F}_t Y ∈ F t where 0 ≤ s ≤ t ≤ T 0\le s\le t\le T 0 ≤ s ≤ t ≤ T . Then
E ~ [ Y ∣ F s ] = 1 Z s E [ Z t Y ∣ F s ] . \widetilde{\mathbb{E}}[Y\mid\mathcal{F}_s]=\frac{1}{Z_s}\mathbb{E}[Z_tY\mid\mathcal{F}_s]. E [ Y ∣ F s ] = Z s 1 E [ Z t Y ∣ F s ] . Proof. From Definition 2.3.1 , we recall that a conditional expectation E ~ [ Y ∣ F s ] \widetilde{\mathbb{E}}[Y\mid\mathcal{F}_s] E [ Y ∣ F s ] must satisfy two properties:
(i) E ~ [ Y ∣ F s ] ∈ F s \widetilde{\mathbb{E}}[Y\mid\mathcal{F}_s]\in\mathcal{F}_s E [ Y ∣ F s ] ∈ F s .
(ii) E ~ 1 A E ~ [ Y ∣ F s ] = E ~ 1 A Y \widetilde{\mathbb{E}}1_A\widetilde{\mathbb{E}}[Y\mid\mathcal{F}_s]=\widetilde{\mathbb{E}}1_AY E 1 A E [ Y ∣ F s ] = E 1 A Y for all A ∈ F s A\in\mathcal{F}_s A ∈ F s .
It is clear that property (i) is satisfied since
E ~ [ Y ∣ F s ] = 1 Z s E [ Z t Y ∣ F s ] = 1 E [ Z ∣ F s ] E [ Z t Y ∣ F s ] ∈ F s . \widetilde{\mathbb{E}}[Y\mid\mathcal{F}_s]=\frac{1}{Z_s}\mathbb{E}[Z_tY\mid\mathcal{F}_s]=\frac{1}{\mathbb{E}[Z\mid\mathcal{F}_s]}\mathbb{E}[Z_tY\mid\mathcal{F}_s]\in\mathcal{F}_s. E [ Y ∣ F s ] = Z s 1 E [ Z t Y ∣ F s ] = E [ Z ∣ F s ] 1 E [ Z t Y ∣ F s ] ∈ F s . We must check property (ii). Let A ∈ F s A\in\mathcal{F}_s A ∈ F s . Then
E ~ 1 A E ~ [ Y ∣ F s ] = E ~ 1 A 1 Z s E [ Z t Y ∣ F s ] = E Z s 1 A 1 Z s E [ Z t Y ∣ F s ] (because 1 A 1 Z s E [ Z t Y ∣ F s ] ∈ F s and Lemma 8.5.3) = E 1 A E [ Z t Y ∣ F s ] = E E [ 1 A Z t Y ∣ F s ] = E 1 A Z t Y = E ~ 1 A Y . (because 1 A Y ∈ F t and Lemma 8.5.3) \begin{aligned}
\widetilde{\mathbb{E}}1_A\widetilde{\mathbb{E}}[Y\mid\mathcal{F}_s]
&=\widetilde{\mathbb{E}}1_A\frac{1}{Z_s}\mathbb{E}[Z_tY\mid\mathcal{F}_s]\\
&=\mathbb{E}Z_s1_A\frac{1}{Z_s}\mathbb{E}[Z_tY\mid\mathcal{F}_s]
\quad\text{(because }1_A\frac{1}{Z_s}\mathbb{E}[Z_tY\mid\mathcal{F}_s]\in\mathcal{F}_s\text{ and Lemma 8.5.3)}\\
&=\mathbb{E}1_A\mathbb{E}[Z_tY\mid\mathcal{F}_s]\\
&=\mathbb{E}\mathbb{E}[1_AZ_tY\mid\mathcal{F}_s]\\
&=\mathbb{E}1_AZ_tY\\
&=\widetilde{\mathbb{E}}1_AY.
\quad\text{(because }1_AY\in\mathcal{F}_t\text{ and Lemma 8.5.3)}
\end{aligned} E 1 A E [ Y ∣ F s ] = E 1 A Z s 1 E [ Z t Y ∣ F s ] = E Z s 1 A Z s 1 E [ Z t Y ∣ F s ] (because 1 A Z s 1 E [ Z t Y ∣ F s ] ∈ F s and Lemma 8.5.3) = E 1 A E [ Z t Y ∣ F s ] = EE [ 1 A Z t Y ∣ F s ] = E 1 A Z t Y = E 1 A Y . (because 1 A Y ∈ F t and Lemma 8.5.3)
Theorem 8.5.5 (Girsanov). Let W = ( W t ) 0 ≤ t ≤ T \mathbf{W}=(W_t)_{0\le t\le T} W = ( W t ) 0 ≤ t ≤ T be a Brownian motion on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\le t\le T} F = ( F t ) 0 ≤ t ≤ T be a filtration for W \mathbf{W} W . Suppose Θ = ( Θ t ) 0 ≤ t ≤ T \mathbf{\Theta}=(\Theta_t)_{0\le t\le T} Θ = ( Θ t ) 0 ≤ t ≤ T is adapted to the filtration F \mathbb{F} F . Define ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T and W ~ = ( W ~ t ) 0 ≤ t ≤ T \widetilde{\mathbf{W}}=(\widetilde{W}_t)_{0\le t\le T} W = ( W t ) 0 ≤ t ≤ T by
Z t = exp ( − ∫ 0 t 1 2 Θ s 2 d s − ∫ 0 t Θ s d W s ) , d W ~ t = Θ t d t + d W t , W ~ 0 = 0. Z_t=\exp\left(-\int_0^t \frac{1}{2}\Theta_s^2\,\mathrm{d}s-\int_0^t\Theta_s\,\mathrm{d}W_s\right),\qquad\mathrm{d}\widetilde{W}_t=\Theta_t\,\mathrm{d}t+\mathrm{d}W_t,\qquad\widetilde{W}_0=0. Z t = exp ( − ∫ 0 t 2 1 Θ s 2 d s − ∫ 0 t Θ s d W s ) , d W t = Θ t d t + d W t , W 0 = 0. Assume that
E ∫ 0 T Θ t 2 Z t 2 d t < ∞ . \mathbb{E}\int_0^T\Theta_t^2Z_t^2\,\mathrm{d}t<\infty. E ∫ 0 T Θ t 2 Z t 2 d t < ∞. Define a Radon-Nikodým derivative Z ≡ d P ~ d P : = Z T Z\equiv\frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}:=Z_T Z ≡ d P d P := Z T . Then the process W ~ \widetilde{\mathbf{W}} W is a Brownian motion under P ~ \widetilde{\mathbb{P}} P .
Proof. We will use Lévy’s Theorem 8.3.7 to show that W ~ \widetilde{\mathbf{W}} W is a Brownian motion under P ~ \widetilde{\mathbb{P}} P . By definition, W ~ 0 = 0 \widetilde{W}_0=0 W 0 = 0 . Also, we see that
d [ W ~ , W ~ ] t = ( d W ~ t ) 2 = ( d W t + Θ t d t ) 2 = ( d W t ) 2 + 2 Θ t d W t d t + Θ t 2 ( d t ) 2 = d t . \mathrm{d}[\widetilde{W},\widetilde{W}]_t=(\mathrm{d}\widetilde{W}_t)^2=(\mathrm{d}W_t+\Theta_t\mathrm{d}t)^2=(\mathrm{d}W_t)^2+2\Theta_t\mathrm{d}W_t\mathrm{d}t+\Theta_t^2(\mathrm{d}t)^2=\mathrm{d}t. d [ W , W ] t = ( d W t ) 2 = ( d W t + Θ t d t ) 2 = ( d W t ) 2 + 2 Θ t d W t d t + Θ t 2 ( d t ) 2 = d t . Lastly, we must verify that W ~ \widetilde{\mathbf{W}} W is a martingale under P ~ \widetilde{\mathbb{P}} P . To do this, we first show that ( Z t ) t ≥ 0 (Z_t)_{t\ge0} ( Z t ) t ≥ 0 is a martingale under P \mathbb{P} P . Setting
Z t = e X t , X t = − ∫ 0 t 1 2 Θ s 2 d s − ∫ 0 t Θ s d W s , Z_t=e^{X_t},\qquad X_t=-\int_0^t\frac{1}{2}\Theta_s^2\,\mathrm{d}s-\int_0^t\Theta_s\,\mathrm{d}W_s, Z t = e X t , X t = − ∫ 0 t 2 1 Θ s 2 d s − ∫ 0 t Θ s d W s , we find (using the Itô formula) that
d Z t = e X t d X t + 1 2 e X t d [ X , X ] t = Z t ( − 1 2 Θ t 2 d t − Θ t d W t ) + 1 2 Z t Θ t 2 d t = − Z t Θ t d W t . \begin{aligned}
\mathrm{d}Z_t&=e^{X_t}\mathrm{d}X_t+\frac{1}{2}e^{X_t}\mathrm{d}[X,X]_t\\
&=Z_t\left(-\frac{1}{2}\Theta_t^2\mathrm{d}t-\Theta_t\mathrm{d}W_t\right)+\frac{1}{2}Z_t\Theta_t^2\mathrm{d}t=-Z_t\Theta_t\mathrm{d}W_t.
\end{aligned} d Z t = e X t d X t + 2 1 e X t d [ X , X ] t = Z t ( − 2 1 Θ t 2 d t − Θ t d W t ) + 2 1 Z t Θ t 2 d t = − Z t Θ t d W t . Since Itô integrals are martingales, it follows that ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T is a martingale under P \mathbb{P} P . In particular we have E Z = E Z T = Z 0 = 1 \mathbb{E}Z=\mathbb{E}Z_T=Z_0=1 E Z = E Z T = Z 0 = 1 . We also have
Z t = E [ Z T ∣ F t ] = E [ Z ∣ F t ] , Z_t=\mathbb{E}[Z_T\mid\mathcal{F}_t]=\mathbb{E}[Z\mid\mathcal{F}_t], Z t = E [ Z T ∣ F t ] = E [ Z ∣ F t ] , for all 0 ≤ t ≤ T 0\le t\le T 0 ≤ t ≤ T , which shows that ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T is a Radon-Nikodým derivative process. Next, we show that ( W ~ t Z t ) 0 ≤ t ≤ T (\widetilde{W}_tZ_t)_{0\le t\le T} ( W t Z t ) 0 ≤ t ≤ T is a martingale under P \mathbb{P} P . We have
d ( W ~ t Z t ) = W ~ t d Z t + Z t d W ~ t + d [ W ~ , Z ] t = W ~ t ( − Z t Θ t d W t ) + Z t ( Θ t d t + d W t ) + ( Θ t d t + d W t ) ( − Z t Θ t d W t ) = W ~ t ( − Z t Θ t d W t ) + Z t ( Θ t d t + d W t ) − Z t Θ t d t \begin{aligned}
\mathrm{d}(\widetilde{W}_tZ_t)&=\widetilde{W}_t\mathrm{d}Z_t+Z_t\mathrm{d}\widetilde{W}_t+\mathrm{d}[\widetilde{W},Z]_t\\
&=\widetilde{W}_t(-Z_t\Theta_t\mathrm{d}W_t)+Z_t(\Theta_t\mathrm{d}t+\mathrm{d}W_t)+(\Theta_t\mathrm{d}t+\mathrm{d}W_t)(-Z_t\Theta_t\mathrm{d}W_t)\\
&=\widetilde{W}_t(-Z_t\Theta_t\mathrm{d}W_t)+Z_t(\Theta_t\mathrm{d}t+\mathrm{d}W_t)-Z_t\Theta_t\mathrm{d}t
\end{aligned} d ( W t Z t ) = W t d Z t + Z t d W t + d [ W , Z ] t = W t ( − Z t Θ t d W t ) + Z t ( Θ t d t + d W t ) + ( Θ t d t + d W t ) ( − Z t Θ t d W t ) = W t ( − Z t Θ t d W t ) + Z t ( Θ t d t + d W t ) − Z t Θ t d t
= ( − W ~ t Θ t + 1 ) Z t d W t . =(-\widetilde{W}_t\Theta_t+1)Z_t\,\mathrm{d}W_t. = ( − W t Θ t + 1 ) Z t d W t .
Again, since Itô integrals are martingales, it follows that ( W ~ t Z t ) 0 ≤ t ≤ T (\widetilde{W}_tZ_t)_{0\le t\le T} ( W t Z t ) 0 ≤ t ≤ T is a martingale under P \mathbb{P} P . Finally, we can show that W ~ \widetilde{W} W is a martingale under P ~ \widetilde{\mathbb{P}} P . Assuming 0 ≤ s ≤ t ≤ T 0\le s\le t\le T 0 ≤ s ≤ t ≤ T we have
E ~ [ W ~ t ∣ F s ] = 1 Z s E [ Z t W ~ t ∣ F s ] \widetilde{\mathbb{E}}[\widetilde{W}_t\mid\mathcal{F}_s]=\frac{1}{Z_s}\mathbb{E}[Z_t\widetilde{W}_t\mid\mathcal{F}_s] E [ W t ∣ F s ] = Z s 1 E [ Z t W t ∣ F s ]
(by Lemma 8.5.4)
= 1 Z s Z s W ~ s = W ~ s . =\frac{1}{Z_s}Z_s\widetilde{W}_s=\widetilde{W}_s. = Z s 1 Z s W s = W s .
Thus, W ~ \widetilde{W} W is a martingale, and therefore, a Brownian motion under P ~ \widetilde{\mathbb{P}} P .
Theorem 8.5.6 (Martingale representation). Let W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\le t\le T} W = ( W t ) 0 ≤ t ≤ T be a Brownian motion on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\le t\le T} F = ( F t ) 0 ≤ t ≤ T be a filtration generated by W W W (that is, F t = σ ( W s , 0 ≤ s ≤ t ) \mathcal{F}_t=\sigma(W_s,0\le s\le t) F t = σ ( W s , 0 ≤ s ≤ t ) ). Let M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\le t\le T} M = ( M t ) 0 ≤ t ≤ T be a martingale with respect to the filtration F \mathbb{F} F satisfying E M T 2 < ∞ \mathbb{E}M_T^2<\infty E M T 2 < ∞ . Then there exists a process Γ = ( Γ t ) 0 ≤ t ≤ T \Gamma=(\Gamma_t)_{0\le t\le T} Γ = ( Γ t ) 0 ≤ t ≤ T that is adapted to the filtration F \mathbb{F} F and satisfies E ∫ 0 T Γ t 2 d t < ∞ \mathbb{E}\int_0^T\Gamma_t^2\mathrm{d}t<\infty E ∫ 0 T Γ t 2 d t < ∞ such that
M t = M 0 + ∫ 0 t Γ s d W s , t ∈ [ 0 , T ] . M_t=M_0+\int_0^t\Gamma_s\mathrm{d}W_s,\qquad t\in[0,T]. M t = M 0 + ∫ 0 t Γ s d W s , t ∈ [ 0 , T ] .
The proof of Theorem 8.5.6 is rather involved, and will not be given here. The theorem plays a fundamental role in mathematical finance, where a martingale M M M represents the value of an asset and Γ \Gamma Γ represents a trading strategy that replicates the asset’s value. For more precise details, take a course in mathematical finance!
Girsanov's Theorem for d-dimensional Brownian motion
We conclude this chapter by stating (without proof) Girsanov’s Theorem and the martingale representation theorem for multi-dimensional Brownian motions.
Theorem 8.6.1 (Girsanov). Let W = ( W t 1 , W t 2 , … , W t d ) 0 ≤ t ≤ T W=(W_t^1,W_t^2,\ldots,W_t^d)_{0\le t\le T} W = ( W t 1 , W t 2 , … , W t d ) 0 ≤ t ≤ T be a d d d -dimensional Brownian motion on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\le t\le T} F = ( F t ) 0 ≤ t ≤ T be a filtration for W W W . Suppose Θ = ( Θ t 1 , Θ t 2 , … , Θ t d ) 0 ≤ t ≤ T \Theta=(\Theta_t^1,\Theta_t^2,\ldots,\Theta_t^d)_{0\le t\le T} Θ = ( Θ t 1 , Θ t 2 , … , Θ t d ) 0 ≤ t ≤ T is adapted to the filtration F \mathbb{F} F . Define ( Z t ) 0 ≤ t ≤ T (Z_t)_{0\le t\le T} ( Z t ) 0 ≤ t ≤ T and W ~ = ( W ~ t 1 , W ~ t 2 , … , W ~ t d ) 0 ≤ t ≤ T \widetilde{W}=(\widetilde{W}_t^1,\widetilde{W}_t^2,\ldots,\widetilde{W}_t^d)_{0\le t\le T} W = ( W t 1 , W t 2 , … , W t d ) 0 ≤ t ≤ T by
Z t = exp ( − ∫ 0 t 1 2 ⟨ Θ s , Θ s ⟩ d s − ∫ 0 t ⟨ Θ s , d W s ⟩ ) , d W ~ t = Θ t d t + d W t , W ~ 0 = 0 , Z_t=\exp\left(-\int_0^t\frac{1}{2}\langle\Theta_s,\Theta_s\rangle\mathrm{d}s-\int_0^t\langle\Theta_s,\mathrm{d}W_s\rangle\right),\qquad\mathrm{d}\widetilde{W}_t=\Theta_t\mathrm{d}t+\mathrm{d}W_t,\qquad\widetilde{W}_0=0, Z t = exp ( − ∫ 0 t 2 1 ⟨ Θ s , Θ s ⟩ d s − ∫ 0 t ⟨ Θ s , d W s ⟩ ) , d W t = Θ t d t + d W t , W 0 = 0 , where ⟨ ⋅ , ⋅ ⟩ \langle\cdot,\cdot\rangle ⟨ ⋅ , ⋅ ⟩ denotes a d d d -dimensional Euclidean inner product. Assume that
E ∫ 0 T ⟨ Θ t , Θ t ⟩ Z t 2 d t < ∞ . \mathbb{E}\int_0^T\langle\Theta_t,\Theta_t\rangle Z_t^2\mathrm{d}t<\infty. E ∫ 0 T ⟨ Θ t , Θ t ⟩ Z t 2 d t < ∞.
Define a Radon-Nikodým derivative Z ≡ d P ~ d P : = Z T Z \equiv\frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}} := Z_T Z ≡ d P d P := Z T . Then the process W ~ \widetilde{W} W is a d d d -dimensional Brownian motion under P ~ \widetilde{\mathbb{P}} P .
It is interesting to note that the components of W ~ \widetilde{W} W in Theorem 8.6.1 could be co-dependent under P \mathbb{P} P (as Θ j \Theta_j Θ j could depend on any of W 1 , W 2 , … , W d W^1,W^2,\ldots,W^d W 1 , W 2 , … , W d ). Nevertheless, under P ~ \widetilde{\mathbb{P}} P , the components of W ~ \widetilde{W} W are independent of each other.
Theorem 8.6.2 (Martingale representation). Let W = ( W t 1 , W t 2 , … , W t d ) 0 ≤ t ≤ T W=(W_t^1,W_t^2,\ldots,W_t^d)_{0\leq t\leq T} W = ( W t 1 , W t 2 , … , W t d ) 0 ≤ t ≤ T be a d d d -dimensional Brownian motion on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T be a filtration generated by W W W (that is, F t = σ ( W s , 0 ≤ s ≤ t ) \mathcal{F}_t=\sigma(W_s,0\leq s\leq t) F t = σ ( W s , 0 ≤ s ≤ t ) ). Let M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\leq t\leq T} M = ( M t ) 0 ≤ t ≤ T be a martingale with respect to the filtration F \mathbb{F} F satisfying E M T 2 < ∞ \mathbb{E}M_T^2<\infty E M T 2 < ∞ . Then there exists a process Γ = ( Γ t 1 , Γ t 2 , … , Γ t d ) 0 ≤ t ≤ T \Gamma=(\Gamma_t^1,\Gamma_t^2,\ldots,\Gamma_t^d)_{0\leq t\leq T} Γ = ( Γ t 1 , Γ t 2 , … , Γ t d ) 0 ≤ t ≤ T that is adapted to the filtration F \mathbb{F} F and satisfies E ∫ 0 T ⟨ Γ t , Γ t ⟩ d t < ∞ \mathbb{E}\int_0^T\langle\Gamma_t,\Gamma_t\rangle\mathrm{d}t<\infty E ∫ 0 T ⟨ Γ t , Γ t ⟩ d t < ∞ such that
M t = M 0 + ∫ 0 t ⟨ Γ s , d W s ⟩ , t ∈ [ 0 , T ] . M_t=M_0+\int_0^t\langle\Gamma_s,\mathrm{d}W_s\rangle,\qquad t\in[0,T]. M t = M 0 + ∫ 0 t ⟨ Γ s , d W s ⟩ , t ∈ [ 0 , T ] .
Exercises
Exercise 8.1. Compute d ( W T 4 ) \mathrm{d}(W_T^4) d ( W T 4 ) . Write W T 4 W_T^4 W T 4 as an integral with respect to W W W plus an integral with respect to t t t . Use this representation of W T 4 W_T^4 W T 4 to show that E W T 4 = 3 T 2 \mathbb{E}W_T^4=3T^2 E W T 4 = 3 T 2 . Compute E W T 6 \mathbb{E}W_T^6 E W T 6 using the same technique.
Exercise 8.2. Find an explicit expression for Y T Y_T Y T where
d Y t = r d t + α Y t d W t . \mathrm{d}Y_t=r\mathrm{d}t+\alpha Y_t\mathrm{d}W_t. d Y t = r d t + α Y t d W t .
Hint: compute d ( Y t Z t ) \mathrm{d}(Y_tZ_t) d ( Y t Z t ) where Z t : = exp ( − α W t + 1 2 α 2 t ) Z_t:=\exp(-\alpha W_t+\frac{1}{2}\alpha^2t) Z t := exp ( − α W t + 2 1 α 2 t ) .
Exercise 8.3. Suppose X X X , Δ \Delta Δ and Π \Pi Π are given by
d X t = σ X t d W t , Δ t = ∂ f ∂ x ( t , X t ) , Π t = X t Δ t \mathrm{d}X_t=\sigma X_t\mathrm{d}W_t,\qquad\Delta_t=\frac{\partial f}{\partial x}(t,X_t),\qquad\Pi_t=X_t\Delta_t d X t = σ X t d W t , Δ t = ∂ x ∂ f ( t , X t ) , Π t = X t Δ t
where f f f is some smooth function. Show that if f f f satisfies
( ∂ ∂ t + 1 2 σ 2 x 2 ∂ 2 ∂ x 2 ) f ( t , x ) = 0 , \left(\frac{\partial}{\partial t}+\frac{1}{2}\sigma^2x^2\frac{\partial^2}{\partial x^2}\right)f(t,x)=0, ( ∂ t ∂ + 2 1 σ 2 x 2 ∂ x 2 ∂ 2 ) f ( t , x ) = 0 ,
for all ( t , x ) (t,x) ( t , x ) , then Π \Pi Π is a martingale with respect to a filtration F t \mathcal{F}_t F t for W W W .
Exercise 8.4. Suppose X X X is given by
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . \mathrm{d}X_t=\mu(t,X_t)\mathrm{d}t+\sigma(t,X_t)\mathrm{d}W_t. d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t .
Exercises
For any smooth function f f f define
M t f : = f ( t , X t ) − f ( 0 , X 0 ) − ∫ 0 t ( ∂ ∂ s + μ ( s , X s ) ∂ ∂ x + 1 2 σ 2 ( s , X s ) ∂ 2 ∂ x 2 ) f ( s , X s ) d s . M_t^f := f(t,X_t)-f(0,X_0)-\int_0^t\left(\frac{\partial}{\partial s}+\mu(s,X_s)\frac{\partial}{\partial x}+\frac{1}{2}\sigma^2(s,X_s)\frac{\partial^2}{\partial x^2}\right)f(s,X_s)\,ds. M t f := f ( t , X t ) − f ( 0 , X 0 ) − ∫ 0 t ( ∂ s ∂ + μ ( s , X s ) ∂ x ∂ + 2 1 σ 2 ( s , X s ) ∂ x 2 ∂ 2 ) f ( s , X s ) d s .
Show that M f M^f M f is a martingale with respect to a filtration F t \mathcal{F}_t F t for W W W .
Exercise 8.5. Let X = ( X t ) 0 ≤ t ≤ T X=(X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T be an OU process on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P )
d X t = K ( θ − X t ) d t + σ d W t . dX_t=K(\theta-X_t)\,dt+\sigma dW_t. d X t = K ( θ − X t ) d t + σ d W t .
where W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T is a Brownian motion under probability measure P \mathbb{P} P . Then we can define a new probability measure P ~ \widetilde{\mathbb{P}} P such that the process W ~ = ( W ~ t ) 0 ≤ t ≤ T \widetilde{W}=(\widetilde{W}_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T is a Brownian motion under P ~ \widetilde{\mathbb{P}} P . Then the OU process X = ( X t ) 0 ≤ t ≤ T X=(X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T on the new probability space ( Ω , F , P ~ ) (\Omega,\mathcal{F},\widetilde{\mathbb{P}}) ( Ω , F , P ) will be
d X t = K ( θ ∗ − X t ) d t + σ d W ~ t . dX_t=K(\theta^*-X_t)\,dt+\sigma d\widetilde{W}_t. d X t = K ( θ ∗ − X t ) d t + σ d W t .
Find the Radon-Nikodym derivative d P ~ d P \frac{d\widetilde{\mathbb{P}}}{d\mathbb{P}} d P d P .
Exercise 8.6. Note, to prove that a process is a Brownian Bridge, you simply need to show that it is a Gaussian process and has the correct mean m ( t ) m(t) m ( t ) and covariance c ( t , s ) c(t,s) c ( t , s ) .
(a) Let X X X be a Brownian bridge from zero to zero on the interval t = 0 t=0 t = 0 to t = 1 t=1 t = 1 . Prove that the process X 1 − t X_{1-t} X 1 − t , 0 ≤ t ≤ 1 0\leq t\leq1 0 ≤ t ≤ 1 , is also a Brownian bridge.
(b) Prove that if W W W is a Brownian motion, then the processes ( 1 − t ) W t / ( 1 − t ) (1-t)W_{t/(1-t)} ( 1 − t ) W t / ( 1 − t ) and t W ( 1 / t ) − 1 tW_{(1/t)-1} t W ( 1/ t ) − 1 , 0 ≤ t ≤ 1 0\leq t\leq1 0 ≤ t ≤ 1 , are both Brownian bridges.
(c) If we write t ‾ \overline{t} t for t t t modulo 1 1 1 , prove that the process Y t = X t + s ‾ − X s Y_t=X_{\overline{t+s}}-X_s Y t = X t + s − X s , 0 ≤ t ≤ 1 0\leq t\leq1 0 ≤ t ≤ 1 is a Brownian bridge for every fixed s ∈ ( 0 , 1 ) s\in(0,1) s ∈ ( 0 , 1 ) . This is called the cyclic invariance of Brownian bridge.
Exercise 8.7. Suppose that the dynamics of X X X are given by
d X t = μ t d t + σ t d W t , dX_t=\mu_tdt+\sigma_tdW_t, d X t = μ t d t + σ t d W t ,
where W W W is a Brownian motion under P \mathbb{P} P . Find a change of measure d P ~ d P \frac{d\widetilde{\mathbb{P}}}{d\mathbb{P}} d P d P so that
d X t = μ ~ t d t + σ t d W ~ t , dX_t=\widetilde{\mu}_tdt+\sigma_td\widetilde{W}_t, d X t = μ t d t + σ t d W t ,
where W ~ \widetilde{W} W is a Brownian motion under P ~ \widetilde{\mathbb{P}} P . You may assume σ t ≠ 0 \sigma_t\ne0 σ t = 0 for all t t t and that any process you introduce satisfies the conditions needed for Girsanov’s Theorem to apply.
Exercise 8.8. Consider a market with stochastic interest rates R = ( R t ) 0 ≤ t ≤ T R=(R_t)_{0\leq t\leq T} R = ( R t ) 0 ≤ t ≤ T . The Cox-Ingersoll-Ross (CIR) model for R R R is
d R t = κ ( θ − R t ) d t + δ R t d W t , dR_t=\kappa(\theta-R_t)\,dt+\delta\sqrt{R_t}dW_t, d R t = κ ( θ − R t ) d t + δ R t d W t ,
where W W W is a Brownian motion under the real-world probability measure P \mathbb{P} P . Note that, in this model, R t ≥ 0 R_t \ge0 R t ≥ 0 for all t ∈ [ 0 , T ] t \in[0,T] t ∈ [ 0 , T ] . Show that, under the following change of measure
d P ~ d P = exp ( − 1 2 ∫ 0 T λ 2 R t d t − ∫ 0 T λ R t d W t ) , \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}=\exp\left(-\frac{1}{2}\int_0^T\lambda^2R_t\mathrm{d}t-\int_0^T\lambda\sqrt{R_t}\mathrm{d}W_t\right), d P d P = exp ( − 2 1 ∫ 0 T λ 2 R t d t − ∫ 0 T λ R t d W t ) ,
the dynamics of R R R have the form
d R t = κ ~ ( θ ~ − R t ) d t + δ R t d W ~ t , \mathrm{d}R_t=\widetilde{\kappa}(\widetilde{\theta}-R_t)\mathrm{d}t+\delta\sqrt{R_t}\mathrm{d}\widetilde{W}_t, d R t = κ ( θ − R t ) d t + δ R t d W t ,
and find κ ~ \widetilde{\kappa} κ and θ ~ \widetilde{\theta} θ .
Exercise 8.9. Suppose that X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 and Z = ( Z t ) t ≥ 0 Z=(Z_t)_{t\ge0} Z = ( Z t ) t ≥ 0 satisfy
d X t = 1 X t d t + α X t d W t , X 0 = x > 0 , d Z t = − α Z t d W t , Z 0 = 1 , \mathrm{d}X_t=\frac{1}{X_t}\mathrm{d}t+\alpha X_t\mathrm{d}W_t,\qquad X_0=x>0,\qquad\mathrm{d}Z_t=-\alpha Z_t\mathrm{d}W_t,\qquad Z_0=1, d X t = X t 1 d t + α X t d W t , X 0 = x > 0 , d Z t = − α Z t d W t , Z 0 = 1 ,
where W = ( W t ) t ≥ 0 W=(W_t)_{t\ge0} W = ( W t ) t ≥ 0 is a Brownian motion.
(a) Compute d ( X t Z t ) \mathrm{d}(X_tZ_t) d ( X t Z t ) .
(b) Compute d ( ( X t Z t ) 2 ) \mathrm{d}((X_tZ_t)^2) d (( X t Z t ) 2 ) .
(c) Find an explicit expression for Z t Z_t Z t .
(d) Find an explicit solution for X t X_t X t . Hint: the solution to
d Y t = ( G t − D Y t ) d t , Y 0 = y , \mathrm{d}Y_t=(G_t-DY_t)\mathrm{d}t,\qquad Y_0=y, d Y t = ( G t − D Y t ) d t , Y 0 = y ,
is
Y t = e − D t y + ∫ 0 t e − D ( t − s ) G s d s . (114) Y_t=e^{-Dt}y+\int_0^t e^{-D(t-s)}G_s\mathrm{d}s. \tag*{(114)} Y t = e − D t y + ∫ 0 t e − D ( t − s ) G s d s . (114)
Exercise 8.10. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and a filtration F = ( F t ) t ∈ [ 0 , T ] \mathbb{F}=(\mathcal{F}_t)_{t\in[0,T]} F = ( F t ) t ∈ [ 0 , T ] for a Brownian motion W = ( W t ) t ∈ [ 0 , T ] W=(W_t)_{t\in[0,T]} W = ( W t ) t ∈ [ 0 , T ] . Define the process Z = ( Z t ) t ∈ [ 0 , T ] Z=(Z_t)_{t\in[0,T]} Z = ( Z t ) t ∈ [ 0 , T ] by
Z t = exp ( − 1 2 ∫ 0 t σ s 2 d s − ∫ 0 t σ s d W s ) , Z_t=\exp\left(-\frac{1}{2}\int_0^t\sigma_s^2\mathrm{d}s-\int_0^t\sigma_s\mathrm{d}W_s\right), Z t = exp ( − 2 1 ∫ 0 t σ s 2 d s − ∫ 0 t σ s d W s ) ,
where σ = ( σ t ) t ∈ [ 0 , T ] \sigma=(\sigma_t)_{t\in[0,T]} σ = ( σ t ) t ∈ [ 0 , T ] is F \mathbb{F} F -adapted and bounded and introduce the Radon-Nikodým derivative
d P ~ d P : = Z T \frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}:=Z_T d P d P := Z T
Recall that the process W ~ = ( W ~ t ) t ∈ [ 0 , T ] \widetilde{W}=(\widetilde{W}_t)_{t\in[0,T]} W = ( W t ) t ∈ [ 0 , T ] defined as follows
W ~ t : = ∫ 0 t σ s d s + W t , W ~ 0 = 0 \widetilde{W}_t:=\int_0^t\sigma_s\mathrm{d}s+W_t,\qquad\widetilde{W}_0=0 W t := ∫ 0 t σ s d s + W t , W 0 = 0
is a Brownian motion under P ~ \widetilde{\mathbb{P}} P . Show that
E ( Z t log Z t ) = E ~ ( 1 2 ∫ 0 t σ s 2 d s ) , \mathbb{E}(Z_t\log Z_t)=\widetilde{\mathbb{E}}\left(\frac{1}{2}\int_0^t\sigma_s^2\mathrm{d}s\right), E ( Z t log Z t ) = E ( 2 1 ∫ 0 t σ s 2 d s ) ,
where E ~ \widetilde{\mathbb{E}} E denotes expectation under P ~ \widetilde{\mathbb{P}} P .
Exercises
Exercise 8.11. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T be a filtration for a Brownian motion W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T . Define
Z = d P ~ d P : = e a W T E e a W T , a ∈ R . Z=\frac{\mathrm{d}\widetilde{\mathbb{P}}}{\mathrm{d}\mathbb{P}}:=\frac{e^{aW_T}}{\mathbb{E}e^{aW_T}},\qquad a\in\mathbb{R}. Z = d P d P := E e a W T e a W T , a ∈ R .
(a) Show that Z Z Z satisfies the properties of a Radon-Nikodým derivative (i.e., a valid change of measure).
(b) Compute E ~ W T \widetilde{\mathbb{E}}W_T E W T , where E ~ \widetilde{\mathbb{E}} E denotes expectation under P ~ \widetilde{\mathbb{P}} P .
(c) Compute E ~ W t \widetilde{\mathbb{E}}W_t E W t , where 0 ≤ t ≤ T 0\leq t\leq T 0 ≤ t ≤ T .
Note: if X ∼ N ( μ , σ 2 ) X\sim\mathcal{N}(\mu,\sigma^2) X ∼ N ( μ , σ 2 ) then E e b X = e μ b + σ 2 b 2 / 2 \mathbb{E}e^{bX}=e^{\mu b+\sigma^2b^2/2} E e b X = e μ b + σ 2 b 2 /2 .
Exercise 8.12. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and a filtration F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T . Suppose W W W and B B B are independent Brownian motions. Define processes X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 , Y = ( Y t ) t ≥ 0 Y=(Y_t)_{t\geq0} Y = ( Y t ) t ≥ 0 , R = ( R t ) t ≥ 0 R=(R_t)_{t\geq0} R = ( R t ) t ≥ 0 and Θ = ( Θ t ) t ≥ 0 \Theta=(\Theta_t)_{t\geq0} Θ = ( Θ t ) t ≥ 0 as follows
d X t = d W t , d Y t = d B t , R t : = X t 2 + Y t 2 , Θ t = arctan ( Y t / X t ) . \mathrm{d}X_t=\mathrm{d}W_t,\qquad\mathrm{d}Y_t=\mathrm{d}B_t,\qquad R_t:=\sqrt{X_t^2+Y_t^2},\qquad\Theta_t=\arctan(Y_t/X_t). d X t = d W t , d Y t = d B t , R t := X t 2 + Y t 2 , Θ t = arctan ( Y t / X t ) .
(a) Derive the dynamics of R R R , i.e., d R t = … d W t + … d B t + … d t \mathrm{d}R_t=\ldots\mathrm{d}W_t+\ldots\mathrm{d}B_t+\ldots\mathrm{d}t d R t = … d W t + … d B t + … d t where … \ldots … is written in terms of ( R t , Θ t ) (R_t,\Theta_t) ( R t , Θ t ) and not ( X t , Y t ) (X_t,Y_t) ( X t , Y t ) .
(b) Derive the dynamics of Θ \Theta Θ (i.e., d Θ t = … \mathrm{d}\Theta_t=\ldots d Θ t = … ). Your expression should not involve X t X_t X t or Y t Y_t Y t .
In case you forgot, ∂ z arctan ( z ) = 1 / ( 1 + z 2 ) \partial_z\arctan(z)=1/(1+z^2) ∂ z arctan ( z ) = 1/ ( 1 + z 2 ) .
Exercise 8.13. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T be a filtration for a Brownian motion W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T . Consider two processes S = ( S t ) 0 ≤ t ≤ T S=(S_t)_{0\leq t\leq T} S = ( S t ) 0 ≤ t ≤ T and M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\leq t\leq T} M = ( M t ) 0 ≤ t ≤ T whose dynamics are
d S t = μ S t d t + σ S t d W t , d M t = r M t d t , \mathrm{d}S_t=\mu S_t\mathrm{d}t+\sigma S_t\mathrm{d}W_t,\qquad\mathrm{d}M_t=rM_t\mathrm{d}t, d S t = μ S t d t + σ S t d W t , d M t = r M t d t ,
where μ \mu μ , r r r and σ > 0 \sigma>0 σ > 0 are constants. (In finance, this is the Black-Scholes model, in which S S S is the price of a stock and M M M is the value of a money market account.) Let Δ = ( Δ t ) 0 ≤ t ≤ T \Delta=(\Delta_t)_{0\leq t\leq T} Δ = ( Δ t ) 0 ≤ t ≤ T be an F \mathbb{F} F -adapted process and let X = ( X t ) 0 ≤ t ≤ T X=(X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T satisfy
d X t = Δ t d S t + X t − Δ t S t M t d M t . \mathrm{d}X_t=\Delta_t\mathrm{d}S_t+\frac{X_t-\Delta_tS_t}{M_t}\mathrm{d}M_t. d X t = Δ t d S t + M t X t − Δ t S t d M t .
(In finance, X X X is the value of a self-financing portfolio that holds Δ t \Delta_t Δ t shares of S S S at time t t t and invests the rest of its value in M M M .)
(a) Find a change of measure d P ^ / d P \mathrm{d}\widehat{\mathbb{P}}/\mathrm{d}\mathbb{P} d P / d P under which X / S X/S X / S is a ( P ^ , F ) (\widehat{\mathbb{P}},\mathbb{F}) ( P , F ) martingale for every process Δ \Delta Δ .
(b) What are the dynamics of M / S M/S M / S under P ^ \widehat{\mathbb{P}} P ? That is, find Θ t \Theta_t Θ t and Σ t \Sigma_t Σ t such that d ( M t / S t ) = Θ t d t + Σ t d W ^ t \mathrm{d}(M_t/S_t)=\Theta_t\mathrm{d}t+\Sigma_t\mathrm{d}\widehat{W}_t d ( M t / S t ) = Θ t d t + Σ t d W t where W ^ \widehat{W} W is a ( P ^ , F ) (\widehat{\mathbb{P}},\mathbb{F}) ( P , F ) Brownian motion.
SDEs and PDEs
The notes from this chapter are taken primarily from (Shreve, 2004, Chapter 6), (Øksendal, 2005, Chapters 5, 7, 8 and 9) and Linetsky (2007). Another good reference is (Karlin and Taylor, 1981, Chapter 15).
Stochastic differential equations
We will begin this chapter by focusing on scalar stochastic differential equations driven by a single Brownian motion. The extension to d d d -dimensional stochastic differential equations driven by m m m Brownian motions will be given in § 26.3 .
Definition 9.1.1. A stochastic differential equation (SDE) is an equation of the form
d X s = μ ( s , X s ) d s + σ ( s , X s ) d W s , X t = x , (115) \mathrm{d}X_s = \mu(s, X_s)\mathrm{d}s + \sigma(s, X_s)\mathrm{d}W_s,\qquad X_t = x, \tag*{(115)} d X s = μ ( s , X s ) d s + σ ( s , X s ) d W s , X t = x , (115) We call functions μ \mu μ and σ \sigma σ the drift and diffusion , respectively, and we call X t = x X_t = x X t = x the initial condition . A (strong) solution of an SDE is a stochastic process X = ( X s ) s ≥ t X = (X_s)_{s \ge t} X = ( X s ) s ≥ t such that
X T = x + ∫ t T μ ( s , X s ) d s + ∫ t T σ ( s , X s ) d W s , (116) X_T = x + \int_t^T \mu(s, X_s)\mathrm{d}s + \int_t^T \sigma(s, X_s)\mathrm{d}W_s, \tag*{(116)} X T = x + ∫ t T μ ( s , X s ) d s + ∫ t T σ ( s , X s ) d W s , (116) for all T ≥ t T \ge t T ≥ t .
One way to envision a strong solution of an SDE is as follows: think of a sample path W ⋅ ( ω ) : [ t , ∞ ) → R W_\cdot(\omega) : [t,\infty) \to\mathbb{R} W ⋅ ( ω ) : [ t , ∞ ) → R as input. From this input, we can construct a unique sample path X ⋅ ( ω ) : [ t , ∞ ) → R X_\cdot(\omega) : [t,\infty) \to\mathbb{R} X ⋅ ( ω ) : [ t , ∞ ) → R .
Ideally, we would like to write X T X_T X T as an explicit functional of the Brownian path ( W s ) s ≥ t (W_s)_{s \ge t} ( W s ) s ≥ t . Unfortunately, this is typically not possible. Still, it will help to build intuition if we see some explicitly solvable examples.
Example 9.1.2 (Geometric Brownian motion). A geometric Brownian motion is a process Z = ( Z t ) t ≥ 0 Z=(Z_t)_{t\geq0} Z = ( Z t ) t ≥ 0 that satisfies
d Z t = μ ( t ) Z t d t + σ ( t ) Z t d W t , Z 0 = z . \mathrm{d}Z_t=\mu(t)Z_t\mathrm{d}t+\sigma(t)Z_t\mathrm{d}W_t,\qquad Z_0=z. d Z t = μ ( t ) Z t d t + σ ( t ) Z t d W t , Z 0 = z . where μ \mu μ and σ \sigma σ are deterministic functions of t t t . To solve this SDE, we consider X t = log Z t X_t=\log Z_t X t = log Z t . Using the Itô formula, we obtain
d X t = d log Z t = 1 Z t d Z t + 1 2 ( − 1 Z t 2 ) d [ Z , Z ] t = ( μ ( t ) − 1 2 σ 2 ( t ) ) d t + σ ( t ) d W t . \begin{aligned}
\mathrm{d}X_t&=\mathrm{d}\log Z_t=\frac{1}{Z_t}\mathrm{d}Z_t+\frac{1}{2}\left(\frac{-1}{Z_t^2}\right)\mathrm{d}[Z,Z]_t\\
&=\left(\mu(t)-\frac{1}{2}\sigma^2(t)\right)\mathrm{d}t+\sigma(t)\mathrm{d}W_t.
\end{aligned} d X t = d log Z t = Z t 1 d Z t + 2 1 ( Z t 2 − 1 ) d [ Z , Z ] t = ( μ ( t ) − 2 1 σ 2 ( t ) ) d t + σ ( t ) d W t . Integrating from 0 0 0 to T T T , we obtain
X T = x + ∫ 0 T ( μ ( t ) − 1 2 σ 2 ( t ) ) d t + ∫ 0 T σ ( t ) d W t , x = log z . X_T=x+\int_0^T\left(\mu(t)-\frac{1}{2}\sigma^2(t)\right)\mathrm{d}t+\int_0^T\sigma(t)\mathrm{d}W_t,\qquad x=\log z. X T = x + ∫ 0 T ( μ ( t ) − 2 1 σ 2 ( t ) ) d t + ∫ 0 T σ ( t ) d W t , x = log z . Finally, we obtain our expression for Z T Z_T Z T .
Z T = exp ( X T ) = z exp ( ∫ 0 T ( μ ( t ) − 1 2 σ 2 ( t ) ) d t + ∫ 0 T σ ( t ) d W t ) . Z_T=\exp(X_T)=z\exp\left(\int_0^T\left(\mu(t)-\frac{1}{2}\sigma^2(t)\right)\mathrm{d}t+\int_0^T\sigma(t)\mathrm{d}W_t\right). Z T = exp ( X T ) = z exp ( ∫ 0 T ( μ ( t ) − 2 1 σ 2 ( t ) ) d t + ∫ 0 T σ ( t ) d W t ) .
Example 9.1.3 (Linear SDE). Consider the following SDE with linear coefficients
d X t = ( b ( t ) + μ ( t ) X t ) d t + ( a ( t ) + σ ( t ) X t ) d W t , X 0 = x . (117) \mathrm{d}X_t=(b(t)+\mu(t)X_t)\,\mathrm{d}t+(a(t)+\sigma(t)X_t)\,\mathrm{d}W_t,\qquad X_0=x. \tag*{(117)} d X t = ( b ( t ) + μ ( t ) X t ) d t + ( a ( t ) + σ ( t ) X t ) d W t , X 0 = x . (117) Define two processes Z = ( Z t ) t ≥ 0 Z=(Z_t)_{t\geq0} Z = ( Z t ) t ≥ 0 and Y = ( Y t ) t ≥ 0 Y=(Y_t)_{t\geq0} Y = ( Y t ) t ≥ 0 by
d Z t = μ ( t ) Z t d t + σ ( t ) Z t d W t , Z 0 = 1 , \mathrm{d}Z_t=\mu(t)Z_t\mathrm{d}t+\sigma(t)Z_t\mathrm{d}W_t,\qquad Z_0=1, d Z t = μ ( t ) Z t d t + σ ( t ) Z t d W t , Z 0 = 1 , d Y t = b ( t ) − σ ( t ) a ( t ) Z t d t + a ( t ) Z t d W t , Y 0 = x . \mathrm{d}Y_t=\frac{b(t)-\sigma(t)a(t)}{Z_t}\mathrm{d}t+\frac{a(t)}{Z_t}\mathrm{d}W_t,\qquad Y_0=x. d Y t = Z t b ( t ) − σ ( t ) a ( t ) d t + Z t a ( t ) d W t , Y 0 = x . We will show that X t = Y t Z t X_t=Y_tZ_t X t = Y t Z t solves (117) . Using the product rule, we have
d X t = d ( Y t Z t ) = Y t d Z t + Z t d Y t + d [ Y , Z ] t = ( μ ( t ) Y t Z t d t + σ ( t ) Y t Z t d W t ) + ( ( b ( t ) − σ ( t ) a ( t ) ) d t + a ( t ) d W t ) + a ( t ) σ ( t ) d t = ( b ( t ) + μ ( t ) Y t Z t ) d t + ( a ( t ) + σ ( t ) Y t Z t ) d W t = ( b ( t ) + μ ( t ) X t ) d t + ( a ( t ) + σ ( t ) X t ) d W t . \begin{aligned}
\mathrm{d}X_t&=\mathrm{d}(Y_tZ_t)\\
&=Y_t\mathrm{d}Z_t+Z_t\mathrm{d}Y_t+\mathrm{d}[Y,Z]_t\\
&=\left(\mu(t)Y_tZ_t\mathrm{d}t+\sigma(t)Y_tZ_t\mathrm{d}W_t\right)+\left((b(t)-\sigma(t)a(t))\mathrm{d}t+a(t)\mathrm{d}W_t\right)+a(t)\sigma(t)\mathrm{d}t\\
&=(b(t)+\mu(t)Y_tZ_t)\mathrm{d}t+(a(t)+\sigma(t)Y_tZ_t)\mathrm{d}W_t\\
&=(b(t)+\mu(t)X_t)\mathrm{d}t+(a(t)+\sigma(t)X_t)\mathrm{d}W_t.
\end{aligned} d X t = d ( Y t Z t ) = Y t d Z t + Z t d Y t + d [ Y , Z ] t = ( μ ( t ) Y t Z t d t + σ ( t ) Y t Z t d W t ) + ( ( b ( t ) − σ ( t ) a ( t )) d t + a ( t ) d W t ) + a ( t ) σ ( t ) d t = ( b ( t ) + μ ( t ) Y t Z t ) d t + ( a ( t ) + σ ( t ) Y t Z t ) d W t = ( b ( t ) + μ ( t ) X t ) d t + ( a ( t ) + σ ( t ) X t ) d W t . We also have X 0 = Y 0 Z 0 = x ⋅ 1 = x X_0=Y_0Z_0=x\cdot1=x X 0 = Y 0 Z 0 = x ⋅ 1 = x . Thus, we have shown that Y Z YZ Y Z solves (117) . Now, we note that
Y T = x + ∫ 0 T b ( t ) − σ ( t ) a ( t ) Z t d t + ∫ 0 T a ( t ) Z t d W t , Y_T=x+\int_0^T\frac{b(t)-\sigma(t)a(t)}{Z_t}\mathrm{d}t+\int_0^T\frac{a(t)}{Z_t}\mathrm{d}W_t, Y T = x + ∫ 0 T Z t b ( t ) − σ ( t ) a ( t ) d t + ∫ 0 T Z t a ( t ) d W t ,
and thus
X T = Y T Z T = x Z T + Z T ∫ 0 T b ( t ) − σ ( t ) a ( t ) Z t d t + Z T ∫ 0 T a ( t ) Z t d W t . X_T = Y_TZ_T = xZ_T + Z_T \int_0^T \frac{b(t)-\sigma(t)a(t)}{Z_t}\,\mathrm{d}t + Z_T \int_0^T \frac{a(t)}{Z_t}\,\mathrm{d}W_t. X T = Y T Z T = x Z T + Z T ∫ 0 T Z t b ( t ) − σ ( t ) a ( t ) d t + Z T ∫ 0 T Z t a ( t ) d W t .
where, from Example 9.1.2 , we have
Z t = exp ( ∫ 0 t ( μ ( s ) − 1 2 σ 2 ( s ) ) d s + ∫ 0 t σ ( s ) d W s ) . Z_t = \exp\left(\int_0^t \left(\mu(s)-\frac{1}{2}\sigma^2(s)\right)\,\mathrm{d}s+\int_0^t \sigma(s)\,\mathrm{d}W_s\right). Z t = exp ( ∫ 0 t ( μ ( s ) − 2 1 σ 2 ( s ) ) d s + ∫ 0 t σ ( s ) d W s ) .
Although we have mostly thrown mathematical rigor out the window in these notes, in an effort to be responsible mathematicians, we should at least state (even if we do not prove) an existence and uniqueness result.
Theorem 9.1.4 (Existence and Uniqueness of SDEs). Consider the following SDE
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . (118) \mathrm{d}X_t=\mu(t,X_t)\,\mathrm{d}t+\sigma(t,X_t)\,\mathrm{d}W_t. \tag*{(118)} d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . (118) Suppose the functions μ : R + × R → R \mu:\mathbb{R}_{+}\times\mathbb{R}\to\mathbb{R} μ : R + × R → R and σ : R + × R → R \sigma:\mathbb{R}_{+}\times\mathbb{R}\to\mathbb{R} σ : R + × R → R satisfy
At most linear growth : ∣ μ ( t , x ) ∣ + ∣ σ ( t , x ) ∣ ≤ C 1 ( 1 + ∣ x ∣ ) , ∀ t , x , Lipschitz continuity : ∣ μ ( t , x ) − μ ( t , y ) ∣ + ∣ σ ( t , x ) − σ ( t , y ) ∣ ≤ C 2 ∣ x − y ∣ , ∀ t , x , y . (119) \begin{aligned}
\text{At most linear growth}: \quad&|\mu(t,x)|+|\sigma(t,x)|\le C_1(1+|x|), &&\forall\,t,x, \\
\text{Lipschitz continuity}: \quad&|\mu(t,x)-\mu(t,y)|+|\sigma(t,x)-\sigma(t,y)|\le C_2|x-y|, &&\forall\,t,x,y. \tag*{(119)}
\end{aligned} At most linear growth : Lipschitz continuity : ∣ μ ( t , x ) ∣ + ∣ σ ( t , x ) ∣ ≤ C 1 ( 1 + ∣ x ∣ ) , ∣ μ ( t , x ) − μ ( t , y ) ∣ + ∣ σ ( t , x ) − σ ( t , y ) ∣ ≤ C 2 ∣ x − y ∣ , ∀ t , x , ∀ t , x , y . (119) for some constants C 1 , C 2 < ∞ C_1,C_2<\infty C 1 , C 2 < ∞ . Then, for any initial condition X 0 = x ∈ R X_0=x\in\mathbb{R} X 0 = x ∈ R , SDE (9.4) has a unique solution, which is adapted to the filtration F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\ge0} F = ( F t ) t ≥ 0 generated by W = ( W t ) t ≥ 0 W=(W_t)_{t\ge0} W = ( W t ) t ≥ 0 and satisfies E ∫ 0 T X t 2 d t < ∞ \mathbb{E}\int_0^T X_t^2\,\mathrm{d}t<\infty E ∫ 0 T X t 2 d t < ∞ for all T < ∞ T<\infty T < ∞ .
Remark 9.1.5. Theorem 9.1.4 actually refers to a strong solution of an SDE. There is another notion of a solution of an SDE called a weak solution . We will not discuss weak solutions here.
We will not prove Theorem 9.1.4. We refer the interested reader, instead, to (Øksendal, 2005, Theorem 5.21). However, we will illustrate with two examples what can go wrong if equations (119) and (9.6) are not satisfied.
Example 9.1.6. Let X X X satisfy
d X t = X t 2 d t , X 0 = 1. (120) \mathrm{d}X_t=X_t^2\,\mathrm{d}t,\qquad X_0=1. \tag*{(120)} d X t = X t 2 d t , X 0 = 1. (120) We identify μ ( t , x ) = x 2 \mu(t,x)=x^2 μ ( t , x ) = x 2 , which does not satisfy the linear growth condition (119) . The unique solution to (120) is
X t = 1 1 − t , 0 ≤ t < 1. X_t=\frac{1}{1-t},\qquad0\le t<1. X t = 1 − t 1 , 0 ≤ t < 1. Note that X t X_t X t blows up as t → 1 t\to1 t → 1 and that X t X_t X t is not defined for t ≥ 1 t\ge1 t ≥ 1 . Thus, it is impossible to find a solution that is defined for all t ≥ 0 t\ge0 t ≥ 0 .
Example 9.1.7. Let X X X satisfy
d X t = 3 X t 2 / 3 d t , X 0 = 0. (121) \mathrm{d}X_t = 3X_t^{2/3}\,\mathrm{d}t, \qquad X_0 = 0. \tag*{(121)} d X t = 3 X t 2/3 d t , X 0 = 0. (121) We identify μ ( t , x ) = 3 x 2 / 3 \mu(t,x)=3x^{2/3} μ ( t , x ) = 3 x 2/3 , which does not satisfy the Lipschitz condition (9.6) at x = 0 x=0 x = 0 . One can check directly that any X ( a ) X^{(a)} X ( a ) of the form
X t ( a ) = { 0 t ≤ a , ( t − a ) 3 t > a , X_t^{(a)} =
\begin{cases}
0 & t \le a,\\
(t-a)^3 & t>a,
\end{cases} X t ( a ) = { 0 ( t − a ) 3 t ≤ a , t > a , satisfies (121) . Thus, solutions of (121) are not unique.
Theorem 9.1.8 (Markov property of solutions of an SDE). Let X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 be the unique solution of an SDE of the form (115) whose coefficients satisfy the conditions of Theorem 9.1.4 . Then X X X is a Markov process. That is, for t ≤ T t\le T t ≤ T and for any bounded measurable function φ \varphi φ , there exists a function g g g (which depends on t t t , T T T and φ \varphi φ ) such that
E [ φ ( X T ) ∣ F t ] = g ( X t ) , \mathbb{E}[\varphi(X_T)\mid\mathcal{F}_t] = g(X_t), E [ φ ( X T ) ∣ F t ] = g ( X t ) , where F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\ge0} F = ( F t ) t ≥ 0 is a filtration for the Brownian motion W W W (see Definition 7.2.2 ).
The proof of Theorem 9.1.8 is somewhat technical and will not be given here. But, the intuitive idea for why the theorem is true is rather simple. From (116) , we see that the value of X T X_T X T depends only on the path of the Brownian motion over the interval [ t , T ] [t,T] [ t , T ] and the initial value X t = x X_t=x X t = x . The path that X X X took to arrive at X t = x X_t=x X t = x plays no role. In other words, given the present X t = x X_t=x X t = x , the future ( X T ) T > t (X_T)_{T>t} ( X T ) T > t is independent of the past F t \mathcal{F}_t F t . With this in mind, the process X X X should admit a transition density
P ( X T ∈ d y ∣ X t = x ) = Γ ( t , x ; T , y ) d y , \mathbb{P}(X_T\in\mathrm{d}y\mid X_t=x)=\Gamma(t,x;T,y)\,\mathrm{d}y, P ( X T ∈ d y ∣ X t = x ) = Γ ( t , x ; T , y ) d y ,
and thus, the function g g g should be given by
g ( X t ) = E [ φ ( X T ) ∣ F t ] = E [ φ ( X T ) ∣ X t ] = ∫ d y Γ ( t , X t ; T , y ) φ ( y ) . g(X_t)=\mathbb{E}[\varphi(X_T)\mid\mathcal{F}_t]
=\mathbb{E}[\varphi(X_T)\mid X_t]
=\int\mathrm{d}y\,\Gamma(t,X_t;T,y)\varphi(y). g ( X t ) = E [ φ ( X T ) ∣ F t ] = E [ φ ( X T ) ∣ X t ] = ∫ d y Γ ( t , X t ; T , y ) φ ( y ) .
Of course, finding an explicit representation of the transition density Γ \Gamma Γ may not be possible.
Relation between SDEs and Langevin Equations
Suppose we formally divide both sides of SDE (118) by d t \mathrm{d}t d t . Then we obtain
d X t d t = μ ( t , X t ) + σ ( t , X t ) W ˙ t , W ˙ t : = d W t d t . (122) \frac{\mathrm{d}X_t}{\mathrm{d}t}
=\mu(t,X_t)+\sigma(t,X_t)\dot{W}_t,
\qquad
\dot{W}_t:=\frac{\mathrm{d}W_t}{\mathrm{d}t}. \tag*{(122)} d t d X t = μ ( t , X t ) + σ ( t , X t ) W ˙ t , W ˙ t := d t d W t . (122)
where we have introduced white noise W ˙ \dot{W} W ˙ , the time derivative for Brownian motion. Ignore, for the moment, that paths of Brownian motion are not differentiable and, thus, that d W t / d t \mathrm{d}W_t/\mathrm{d}t d W t / d t is therefore not well-defined. Equations of the form (122) are known as Langevin equations , in honor of the French physicist Paul Langevin. You can view a Langevin equation as an ODE with a random source term.
As physicists often prefer to describe random systems using Langevin equations rather than SDEs, it is worth taking a moment to learn the mathematical formalism of Langevin equations. Introducing the physicists’ notation ⟨ ⋅ ⟩ : = E ⋅ \langle\cdot\rangle:= \mathbb{E}\cdot ⟨ ⋅ ⟩ := E ⋅ , we note that
⟨ W ˙ t ⟩ = d d t E W t = 0. (123) \langle\dot{W}_t\rangle= \frac{\mathrm{d}}{\mathrm{d}t}\mathbb{E}W_t = 0. \tag*{(123)} ⟨ W ˙ t ⟩ = d t d E W t = 0. (123)
Next, denoting by Θ \Theta Θ the Heaviside Step Function , we have
⟨ W ˙ t W ˙ T ⟩ = ∂ 2 ∂ t ∂ T E W t W T = ∂ 2 ∂ t ∂ T ( t ∧ T ) = ∂ 2 ∂ t ∂ T ( t Θ ( T − t ) + T Θ ( t − T ) ) = ∂ ∂ t ( t δ ( T − t ) + Θ ( t − T ) − T δ ( t − T ) ) = ∂ ∂ t Θ ( t − T ) = δ ( t − T ) , (124) \begin{aligned}
\langle\dot{W}_t\dot{W}_T\rangle
&= \frac{\partial^2}{\partial t\partial T}\mathbb{E}W_tW_T
= \frac{\partial^2}{\partial t\partial T}(t\wedge T) \\
&= \frac{\partial^2}{\partial t\partial T}\bigl(t\Theta(T-t)+T\Theta(t-T)\bigr) \\
&= \frac{\partial}{\partial t}\bigl(t\delta(T-t)+\Theta(t-T)-T\delta(t-T)\bigr) \\
&= \frac{\partial}{\partial t}\Theta(t-T)=\delta(t-T), \tag*{(124)}
\end{aligned} ⟨ W ˙ t W ˙ T ⟩ = ∂ t ∂ T ∂ 2 E W t W T = ∂ t ∂ T ∂ 2 ( t ∧ T ) = ∂ t ∂ T ∂ 2 ( t Θ ( T − t ) + T Θ ( t − T ) ) = ∂ t ∂ ( t δ ( T − t ) + Θ ( t − T ) − T δ ( t − T ) ) = ∂ t ∂ Θ ( t − T ) = δ ( t − T ) , (124)
where we have used the facts that t δ ( T − t ) = T δ ( t − T ) t\delta(T-t)=T\delta(t-T) t δ ( T − t ) = T δ ( t − T ) and Θ ′ = δ \Theta'=\delta Θ ′ = δ . We can use (123) and (124) to recapture properties we have already established about Itô integrals.
For example, suppose that
d I t d t = σ ( t ) W ˙ t , I 0 = 0 , so that I T = ∫ 0 T σ ( t ) W ˙ t d t . \frac{\mathrm{d}I_t}{\mathrm{d}t}=\sigma(t)\dot{W}_t,\qquad I_0=0,\qquad\text{so that}\qquad I_T=\int_0^T \sigma(t)\dot{W}_t\,\mathrm{d}t. d t d I t = σ ( t ) W ˙ t , I 0 = 0 , so that I T = ∫ 0 T σ ( t ) W ˙ t d t .
where σ \sigma σ is a deterministic (i.e., non-random) function of time. Then we have
⟨ I T ⟩ = ∫ 0 T σ ( t ) ⟨ W ˙ t ⟩ d t = 0. \langle I_T\rangle=\int_0^T \sigma(t)\langle\dot{W}_t\rangle\,\mathrm{d}t=0. ⟨ I T ⟩ = ∫ 0 T σ ( t ) ⟨ W ˙ t ⟩ d t = 0.
Thus, we recover the fact that Itô integrals have expectation zero. We can also recover the Itô isometry
⟨ I T 2 ⟩ = ∫ 0 T ∫ 0 T σ ( t ) σ ( s ) ⟨ W ˙ t W ˙ s ⟩ d s d t = ∫ 0 T ∫ 0 T σ ( t ) σ ( s ) δ ( t − s ) d s d t = ∫ 0 T σ 2 ( t ) d t . \begin{aligned}
\langle I_T^2\rangle
&=\int_0^T\int_0^T \sigma(t)\sigma(s)\langle\dot{W}_t\dot{W}_s\rangle\,\mathrm{d}s\mathrm{d}t \\
&=\int_0^T\int_0^T \sigma(t)\sigma(s)\delta(t-s)\,\mathrm{d}s\mathrm{d}t \\
&=\int_0^T \sigma^2(t)\,\mathrm{d}t.
\end{aligned} ⟨ I T 2 ⟩ = ∫ 0 T ∫ 0 T σ ( t ) σ ( s ) ⟨ W ˙ t W ˙ s ⟩ d s d t = ∫ 0 T ∫ 0 T σ ( t ) σ ( s ) δ ( t − s ) d s d t = ∫ 0 T σ 2 ( t ) d t .
The above results can be extenuated to the case where the diffusion coefficient σ \sigma σ is a function of t t t and X t X_t X t , but this is a bit more involved.
Connection to partial differential equations
In Chapter 5, we showed that, when X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 is a continuous time Markov chain, its semigroup P = ( P t ) t ≥ 0 \mathbf{P}=(P_t)_{t\geq0} P = ( P t ) t ≥ 0 satisfies coupled ODEs (see Theorem 5.2.5 ). Similarly, when X X X is the solution of an SDE, conditional expectations of the path of X X X can be related to certain PDEs. We discuss these relations below.
Theorem 9.3.1 (Kolmogorov Backward equation). Let X X X be the solution of SDE (115) . For some suitable function φ \varphi φ , define
u ( t , X t ) : = E [ φ ( X T ) ∣ F t ] . (125) u(t,X_t):=\mathbb{E}[\varphi(X_T)|\mathcal{F}_t]. \tag*{(125)} u ( t , X t ) := E [ φ ( X T ) ∣ F t ] . (125) If the function u ∈ C 1 , 2 u\in C^{1,2} u ∈ C 1 , 2 , then it satisfies the Kolmogorov Backward Equation (KBE), a linear PDE of the form
( ∂ t + A ( t ) ) u ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ , A ( t ) = μ ( t , x ) ∂ x + 1 2 σ 2 ( t , x ) ∂ x 2 . (126) (\partial_t+\mathcal{A}(t))u(t,\cdot)=0,\qquad u(T,\cdot)=\varphi,\qquad\mathcal{A}(t)=\mu(t,x)\partial_x+\frac{1}{2}\sigma^2(t,x)\partial_x^2. \tag*{(126)} ( ∂ t + A ( t )) u ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ , A ( t ) = μ ( t , x ) ∂ x + 2 1 σ 2 ( t , x ) ∂ x 2 . (126)
Proof. First, we note that the process ( u ( t , X t ) ) 0 ≤ t ≤ T (u(t,X_t))_{0\leq t\leq T} ( u ( t , X t ) ) 0 ≤ t ≤ T is a martingale since, for any 0 ≤ s ≤ t ≤ T 0\leq s\leq t\leq T 0 ≤ s ≤ t ≤ T , we have
E [ u ( t , X t ) ∣ F s ] = E [ E [ φ ( X T ) ∣ F t ] ∣ F s ] = E [ φ ( X T ) ∣ F s ] = u ( s , X s ) . \mathbb{E}[u(t,X_t)|\mathcal{F}_s]=\mathbb{E}[\mathbb{E}[\varphi(X_T)|\mathcal{F}_t]|\mathcal{F}_s]=\mathbb{E}[\varphi(X_T)|\mathcal{F}_s]=u(s,X_s). E [ u ( t , X t ) ∣ F s ] = E [ E [ φ ( X T ) ∣ F t ] ∣ F s ] = E [ φ ( X T ) ∣ F s ] = u ( s , X s ) .
Next we take the differential of u ( t , X t ) u(t,X_t) u ( t , X t ) and find using the Itô formula that
d u ( t , X t ) = ∂ t u ( t , X t ) d t + ∂ x u ( t , X t ) d X t + 1 2 ∂ x 2 u ( t , X t ) d [ X , X ] t = ( ∂ t + A ( t ) ) u ( t , X t ) d t + σ ( t , X t ) ∂ x u ( t , X t ) d W t . \begin{aligned}
du(t,X_t)&=\partial_tu(t,X_t)dt+\partial_xu(t,X_t)dX_t+\frac{1}{2}\partial_x^2u(t,X_t)d[X,X]_t\\
&=(\partial_t+\mathcal{A}(t))u(t,X_t)dt+\sigma(t,X_t)\partial_xu(t,X_t)dW_t.
\end{aligned} d u ( t , X t ) = ∂ t u ( t , X t ) d t + ∂ x u ( t , X t ) d X t + 2 1 ∂ x 2 u ( t , X t ) d [ X , X ] t = ( ∂ t + A ( t )) u ( t , X t ) d t + σ ( t , X t ) ∂ x u ( t , X t ) d W t .
The only way for ( u ( t , X t ) ) 0 ≤ t ≤ T (u(t,X_t))_{0\leq t\leq T} ( u ( t , X t ) ) 0 ≤ t ≤ T to be a martingale is for the d t dt d t term to be zero for every possible path of X X X . Thus, the function u u u must satisfy PDE (126) . To see why u ( T , x ) = φ ( x ) u(T,x)=\varphi(x) u ( T , x ) = φ ( x ) simply use the fact that φ ( X T ) ∈ F T \varphi(X_T)\in\mathcal{F}_T φ ( X T ) ∈ F T to write
u ( T , X T ) = E [ φ ( X T ) ∣ F T ] = φ ( X T ) , u(T,X_T)=\mathbb{E}[\varphi(X_T)|\mathcal{F}_T]=\varphi(X_T), u ( T , X T ) = E [ φ ( X T ) ∣ F T ] = φ ( X T ) ,
which must hold for all possible X T X_T X T .
Theorem 9.3.1 tells us that the function u u u defined in (125) satisfies a PDE (126) . Alternatively, the Feynman-Kac formula says that the solution u u u of the PDE (126) has the stochastic representation
u ( t , x ) = E [ φ ( x ) ∣ X t = x ] . u(t,x)=\mathbb{E}[\varphi(x)|X_t=x]. u ( t , x ) = E [ φ ( x ) ∣ X t = x ] .
The methods outlined in the proof above can be applied more generally to find PDE representations for more complicated functionals of the path of X X X . The basic steps are as follows
Find a martingale process.
Take the differential of it.
Set the d t dt d t term to zero.
We illustrate this method with the following Theorem
Theorem 9.3.2. Define
u ( t , X t ) : = E [ e − A ( t , T ) φ ( X T ) + ∫ t T d s e − A ( t , s ) g ( s , X s ) | F t ] , A ( t , s ) : = ∫ t s d u γ ( u , X u ) . u(t,X_t) := \mathbb{E}\left[e^{-A(t,T)}\varphi(X_T) + \int_t^T \mathrm{d}s\,e^{-A(t,s)}g(s,X_s)\,\middle|\,\mathcal{F}_t\right], \qquad A(t,s) := \int_t^s \mathrm{d}u\,\gamma(u,X_u). u ( t , X t ) := E [ e − A ( t , T ) φ ( X T ) + ∫ t T d s e − A ( t , s ) g ( s , X s ) F t ] , A ( t , s ) := ∫ t s d u γ ( u , X u ) . Then u u u solves the following PDE
( ∂ t + A ) u + g = 0 , u ( T , ⋅ ) = φ , A = 1 2 σ 2 ( t , x ) ∂ x 2 + μ ( t , x ) ∂ x − γ ( t , x ) . (127) (\partial_t+\mathcal{A})u+g=0,\qquad u(T,\cdot)=\varphi,\qquad\mathcal{A}=\frac{1}{2}\sigma^2(t,x)\partial_x^2+\mu(t,x)\partial_x-\gamma(t,x). \tag*{(127)} ( ∂ t + A ) u + g = 0 , u ( T , ⋅ ) = φ , A = 2 1 σ 2 ( t , x ) ∂ x 2 + μ ( t , x ) ∂ x − γ ( t , x ) . (127)
Proof. Note that the function u u u is not a martingale as, for 0 ≤ s ≤ t ≤ T 0\leq s\leq t\leq T 0 ≤ s ≤ t ≤ T we have
E [ u ( t , X t ) ∣ F s ] = E [ E [ e − A ( t , T ) φ ( X T ) + ∫ t T d r e − A ( t , r ) g ( r , X r ) | F t ] | F s ] = E [ e − A ( t , T ) φ ( X T ) + ∫ t T d r e − A ( t , r ) g ( r , X r ) | F s ] ≠ E [ e − A ( s , T ) φ ( X T ) + ∫ s T d r e − A ( s , r ) g ( r , X r ) | F s ] = u ( s , X s ) . \begin{aligned}
\mathbb{E}[u(t,X_t)|\mathcal{F}_s] = \mathbb{E}\left[\mathbb{E}\left[e^{-A(t,T)}\varphi(X_T)+\int_t^T \mathrm{d}r\,e^{-A(t,r)}g(r,X_r)\middle|\mathcal{F}_t\right]\middle|\mathcal{F}_s\right] \\
&= \mathbb{E}\left[e^{-A(t,T)}\varphi(X_T)+\int_t^T \mathrm{d}r\,e^{-A(t,r)}g(r,X_r)\middle|\mathcal{F}_s\right] \\
&\ne\mathbb{E}\left[e^{-A(s,T)}\varphi(X_T)+\int_s^T \mathrm{d}r\,e^{-A(s,r)}g(r,X_r)\middle|\mathcal{F}_s\right] = u(s,X_s).
\end{aligned} E [ u ( t , X t ) ∣ F s ] = E [ E [ e − A ( t , T ) φ ( X T ) + ∫ t T d r e − A ( t , r ) g ( r , X r ) F t ] F s ] = E [ e − A ( t , T ) φ ( X T ) + ∫ t T d r e − A ( t , r ) g ( r , X r ) F s ] = E [ e − A ( s , T ) φ ( X T ) + ∫ s T d r e − A ( s , r ) g ( r , X r ) F s ] = u ( s , X s ) .
However, the process M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\leq t\leq T} M = ( M t ) 0 ≤ t ≤ T , defined by
M t : = e − A ( 0 , t ) u ( t , X t ) + ∫ 0 t d s e − A ( 0 , s ) g ( s , X s ) , = E [ e − A ( 0 , T ) φ ( X T ) + ∫ 0 T d s e − A ( 0 , s ) g ( s , X s ) | F t ] \begin{aligned}
M_t := e^{-A(0,t)}u(t,X_t)+\int_0^t \mathrm{d}s\,e^{-A(0,s)}g(s,X_s), \\
&= \mathbb{E}\left[e^{-A(0,T)}\varphi(X_T)+\int_0^T \mathrm{d}s\,e^{-A(0,s)}g(s,X_s)\middle|\mathcal{F}_t\right]
\end{aligned} M t := e − A ( 0 , t ) u ( t , X t ) + ∫ 0 t d s e − A ( 0 , s ) g ( s , X s ) , = E [ e − A ( 0 , T ) φ ( X T ) + ∫ 0 T d s e − A ( 0 , s ) g ( s , X s ) F t ]
is a martingale as, for 0 ≤ s ≤ t ≤ T 0\leq s\leq t\leq T 0 ≤ s ≤ t ≤ T we have
E [ M t ∣ F s ] = E [ E [ e − A ( 0 , T ) φ ( X T ) + ∫ 0 T d r e − A ( 0 , r ) g ( r , X r ) | F t ] | F s ] = E [ e − A ( 0 , T ) φ ( X T ) + ∫ 0 T d r e − A ( 0 , r ) g ( r , X r ) | F s ] = M s . \begin{aligned}
\mathbb{E}[M_t|\mathcal{F}_s] = \mathbb{E}\left[\mathbb{E}\left[e^{-A(0,T)}\varphi(X_T)+\int_0^T \mathrm{d}r\,e^{-A(0,r)}g(r,X_r)\middle|\mathcal{F}_t\right]\middle|\mathcal{F}_s\right] \\
&= \mathbb{E}\left[e^{-A(0,T)}\varphi(X_T)+\int_0^T \mathrm{d}r\,e^{-A(0,r)}g(r,X_r)\middle|\mathcal{F}_s\right] = M_s.
\end{aligned} E [ M t ∣ F s ] = E [ E [ e − A ( 0 , T ) φ ( X T ) + ∫ 0 T d r e − A ( 0 , r ) g ( r , X r ) F t ] F s ] = E [ e − A ( 0 , T ) φ ( X T ) + ∫ 0 T d r e − A ( 0 , r ) g ( r , X r ) F s ] = M s .
Now, we take the differential of M M M . We compute
d M t = u ( t , X t ) d e − A ( 0 , t ) + e − A ( 0 , t ) d u ( t , X t ) + d ∫ 0 t d r e − A ( 0 , r ) g ( r , X r ) = − e − A ( 0 , t ) γ ( t , X t ) u ( t , X t ) d t + e − A ( 0 , t ) ( ∂ t + μ ( t , X t ) ∂ x + 1 2 σ 2 ( t , X t ) ∂ x 2 ) u ( t , X t ) d t + e − A ( 0 , t ) σ ( t , X ) ∂ x u ( t , X t ) d W t + e − A ( 0 , t ) g ( t , X t ) d t = e − A ( 0 , t ) ( ( ∂ t + A ) u ( t , X t ) + g ( t , X t ) ) d t + e − A ( 0 , t ) σ ( t , X ) ∂ x u ( t , X t ) d W t . \begin{aligned}
\mathrm{d}M_t &= u(t,X_t)\mathrm{d}e^{-A(0,t)}+e^{-A(0,t)}\mathrm{d}u(t,X_t)+\mathrm{d}\int_0^t \mathrm{d}r\,e^{-A(0,r)}g(r,X_r)\\
&=-e^{-A(0,t)}\gamma(t,X_t)u(t,X_t)\mathrm{d}t+e^{-A(0,t)}\left(\partial_t+\mu(t,X_t)\partial_x+\frac{1}{2}\sigma^2(t,X_t)\partial_x^2\right)u(t,X_t)\mathrm{d}t\\
&\quad+e^{-A(0,t)}\sigma(t,X)\partial_xu(t,X_t)\mathrm{d}W_t+e^{-A(0,t)}g(t,X_t)\mathrm{d}t\\
&=e^{-A(0,t)}\left((\partial_t+\mathcal{A})u(t,X_t)+g(t,X_t)\right)\mathrm{d}t+e^{-A(0,t)}\sigma(t,X)\partial_xu(t,X_t)\mathrm{d}W_t.
\end{aligned} d M t = u ( t , X t ) d e − A ( 0 , t ) + e − A ( 0 , t ) d u ( t , X t ) + d ∫ 0 t d r e − A ( 0 , r ) g ( r , X r ) = − e − A ( 0 , t ) γ ( t , X t ) u ( t , X t ) d t + e − A ( 0 , t ) ( ∂ t + μ ( t , X t ) ∂ x + 2 1 σ 2 ( t , X t ) ∂ x 2 ) u ( t , X t ) d t + e − A ( 0 , t ) σ ( t , X ) ∂ x u ( t , X t ) d W t + e − A ( 0 , t ) g ( t , X t ) d t = e − A ( 0 , t ) ( ( ∂ t + A ) u ( t , X t ) + g ( t , X t ) ) d t + e − A ( 0 , t ) σ ( t , X ) ∂ x u ( t , X t ) d W t .
Setting the d t dt d t term equal to zero, we obtain (127) . The terminal condition is obtained using
u ( T , X T ) = E [ e − A ( T , T ) φ ( X T ) + ∫ T T d s e − A ( t , s ) g ( s , X s ) ∣ F T ] = E [ φ ( X T ) ∣ F T ] = φ ( X T ) , \begin{aligned}
u(T,X_T)=\mathbb{E}\left[e^{-A(T,T)}\varphi(X_T)+\int_T^T \mathrm{d}s\,e^{-A(t,s)}g(s,X_s)\mid\mathcal{F}_T\right] \\
&=\mathbb{E}\left[\varphi(X_T)\mid\mathcal{F}_T\right]=\varphi(X_T),
\end{aligned} u ( T , X T ) = E [ e − A ( T , T ) φ ( X T ) + ∫ T T d s e − A ( t , s ) g ( s , X s ) ∣ F T ] = E [ φ ( X T ) ∣ F T ] = φ ( X T ) ,
completing the proof.
Killing a diffusion
On a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) , consider the following model for a diffusion X X X and a random time τ \tau τ
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t , X 0 = x , (128) \mathrm{d}X_t=\mu(t,X_t)\mathrm{d}t+\sigma(t,X_t)\mathrm{d}W_t,\qquad X_0=x, \tag*{(128)} d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t , X 0 = x , (128)
τ = inf { t ≥ 0 : ∫ 0 t γ ( s , X s ) d s ≥ E } , E ∼ E ( 1 ) , E ⊥ ⊥ X . (129) \tau=\inf\left\{t\geq0:\int_0^t\gamma(s,X_s)\mathrm{d}s\geq\mathcal{E}\right\},\qquad\mathcal{E}\sim\mathcal{E}(1),\qquad\mathcal{E}\perp\!\!\!\perp X. \tag*{(129)} τ = inf { t ≥ 0 : ∫ 0 t γ ( s , X s ) d s ≥ E } , E ∼ E ( 1 ) , E ⊥ ⊥ X . (129)
where E \mathcal{E} E exponentially distributed and independent of the X X X . Here, we assume γ : R + × R → R + \gamma:\mathbb{R}_{+}\times\mathbb{R}\to\mathbb{R}_{+} γ : R + × R → R + so that the integral ∫ 0 t γ ( s , X s ) d s \int_0^t\gamma(s,X_s)\mathrm{d}s ∫ 0 t γ ( s , X s ) d s is non-decreasing. Note that τ \tau τ is the first time the integral ∫ 0 t γ ( s , X s ) d s \int_0^t\gamma(s,X_s)\mathrm{d}s ∫ 0 t γ ( s , X s ) d s equals or exceeds E \mathcal{E} E . The integral depends on the path of X X X , and therefore is stochastic. The exponentially distributed random variable E \mathcal{E} E is also random (obviously). For these reasons, we say that τ \tau τ is doubly stochastic . The random time τ \tau τ called the killing time as it sometimes is used to model the lifetime of the process X X X .
Let F X = ( F t X ) t ≥ 0 \mathbb{F}^{X}=(\mathcal{F}^{X}_{t})_{t\geq0} F X = ( F t X ) t ≥ 0 be the filtration generated by observing the X X X process. Note that
1 { τ > t } ∉ F t X . 1_{\{\tau>t\}}\notin\mathcal{F}^{X}_{t}. 1 { τ > t } ∈ / F t X .
In order to keep track of the information obtained by observing τ \tau τ , we introduce an auxiliary process D as follows
D t = 1 { τ ≤ t } . D_t=1_{\{\tau\leq t\}}. D t = 1 { τ ≤ t } .
Note that D t = 0 D_t=0 D t = 0 if τ > t \tau>t τ > t and D t = 1 D_t=1 D t = 1 if τ ≤ t \tau\leq t τ ≤ t . As a result
1 { τ > t } ∈ F t D , 1_{\{\tau>t\}}\in\mathcal{F}^{D}_{t}, 1 { τ > t } ∈ F t D ,
where F D = ( F t D ) t ≥ 0 \mathbb{F}^{D}=(\mathcal{F}^{D}_{t})_{t\geq0} F D = ( F t D ) t ≥ 0 is the filtration generated by observing D. We denote by F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_{t})_{t\geq0} F = ( F t ) t ≥ 0 the filtration generated by observing both D and X; this is sometimes written F = F X ∨ F D \mathbb{F}=\mathbb{F}^{X}\vee\mathbb{F}^{D} F = F X ∨ F D , which means F t = F t X ∨ F t D : = σ ( F t X ∪ F t D ) \mathcal{F}_{t}=\mathcal{F}^{X}_{t}\vee\mathcal{F}^{D}_{t}:=\sigma(\mathcal{F}^{X}_{t}\cup\mathcal{F}^{D}_{t}) F t = F t X ∨ F t D := σ ( F t X ∪ F t D ) .
Theorem 9.3.3. Let X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 and τ \tau τ be as given in (128) and (129) . Then
E [ 1 { τ > T } φ ( X T ) ∣ F t ] = 1 { τ > t } E [ e − ∫ t T γ ( s , X s ) d s φ ( X T ) ∣ F t ] , (130) \mathbb{E}\left[1_{\{\tau>T\}}\varphi(X_T)\mid\mathcal{F}_t\right]
=
1_{\{\tau>t\}}\mathbb{E}\left[e^{-\int_t^T\gamma(s,X_s)\mathrm{d}s}\varphi(X_T)\mid\mathcal{F}_t\right], \tag*{(130)} E [ 1 { τ > T } φ ( X T ) ∣ F t ] = 1 { τ > t } E [ e − ∫ t T γ ( s , X s ) d s φ ( X T ) ∣ F t ] , (130) where T ≥ t T\geq t T ≥ t .
Proof. Noting that 1 = 1 { τ ≤ t } + 1 { τ > t } 1 = \mathbf{1}_{\{\tau\le t\}} + \mathbf{1}_{\{\tau> t\}} 1 = 1 { τ ≤ t } + 1 { τ > t } and 1 { τ ≤ t } 1 { τ > T } = 0 \mathbf{1}_{\{\tau\le t\}}\mathbf{1}_{\{\tau> T\}} = 0 1 { τ ≤ t } 1 { τ > T } = 0 , we have
E [ 1 { τ > T } φ ( X T ) ∣ F t ] = E [ 1 { τ > t } 1 { τ > T } φ ( X T ) ∣ F t ] + E [ 1 { τ ≤ t } 1 { τ > T } φ ( X T ) ∣ F t ] = E [ φ ( X T ) 1 { τ > t } E [ 1 { τ > T } ∣ F T X , F t ] ∣ F t ] . (131) \begin{aligned}
\mathbb{E}[\mathbf{1}_{\{\tau>T\}}\varphi(X_T)\mid\mathcal{F}_t]
&=\mathbb{E}[\mathbf{1}_{\{\tau>t\}}\mathbf{1}_{\{\tau>T\}}\varphi(X_T)\mid\mathcal{F}_t]
+\mathbb{E}[\mathbf{1}_{\{\tau\le t\}}\mathbf{1}_{\{\tau>T\}}\varphi(X_T)\mid\mathcal{F}_t]\\
&=\mathbb{E}[\varphi(X_T)\mathbf{1}_{\{\tau>t\}}\mathbb{E}[\mathbf{1}_{\{\tau>T\}}\mid\mathcal{F}^{X}_T,\mathcal{F}_t]\mid\mathcal{F}_t]. \tag*{(131)}
\end{aligned} E [ 1 { τ > T } φ ( X T ) ∣ F t ] = E [ 1 { τ > t } 1 { τ > T } φ ( X T ) ∣ F t ] + E [ 1 { τ ≤ t } 1 { τ > T } φ ( X T ) ∣ F t ] = E [ φ ( X T ) 1 { τ > t } E [ 1 { τ > T } ∣ F T X , F t ] ∣ F t ] . (131)
Now, noting that P ( E > x ) = e − x \mathbb{P}(E>x)=e^{-x} P ( E > x ) = e − x , we compute
P ( τ > t ∣ F t X ) = P ( E > ∫ 0 t γ ( s , X s ) d s | F t X ) = e − ∫ 0 t γ ( s , X s ) d s . \mathbb{P}(\tau>t\mid\mathcal{F}^{X}_t)
=\mathbb{P}\left(E>\int_0^t\gamma(s,X_s)\,\mathrm{d}s\middle|\mathcal{F}^{X}_t\right)
=e^{-\int_0^t\gamma(s,X_s)\,\mathrm{d}s}. P ( τ > t ∣ F t X ) = P ( E > ∫ 0 t γ ( s , X s ) d s F t X ) = e − ∫ 0 t γ ( s , X s ) d s .
From this, we find
1 { τ > t } E [ 1 { τ > T } ∣ F T X , F t ] = 1 { τ > t } P ( τ > T ∣ F T X , τ > t ) = 1 { τ > t } P ( E > ∫ 0 T γ ( s , X s ) d s | E > ∫ 0 t γ ( s , X s ) d s , F T X ) = 1 { τ > t } e − ∫ t T γ ( s , X s ) d s . (132) \begin{aligned}
\mathbf{1}_{\{\tau>t\}}\mathbb{E}[\mathbf{1}_{\{\tau>T\}}\mid\mathcal{F}^{X}_T,\mathcal{F}_t]
&=\mathbf{1}_{\{\tau>t\}}\mathbb{P}(\tau>T\mid\mathcal{F}^{X}_T,\tau>t)\\
&=\mathbf{1}_{\{\tau>t\}}\mathbb{P}\left(E>\int_0^T\gamma(s,X_s)\,\mathrm{d}s\middle|E>\int_0^t\gamma(s,X_s)\,\mathrm{d}s,\mathcal{F}^{X}_T\right)\\
&=\mathbf{1}_{\{\tau>t\}}e^{-\int_t^T\gamma(s,X_s)\,\mathrm{d}s}. \tag*{(132)}
\end{aligned} 1 { τ > t } E [ 1 { τ > T } ∣ F T X , F t ] = 1 { τ > t } P ( τ > T ∣ F T X , τ > t ) = 1 { τ > t } P ( E > ∫ 0 T γ ( s , X s ) d s E > ∫ 0 t γ ( s , X s ) d s , F T X ) = 1 { τ > t } e − ∫ t T γ ( s , X s ) d s . (132)
where, in the last equality, we have used the memorylessness property of the exponential distribution. Inserting (132) into (131) we obtain (130) .
Kolmogorov forward and backward Equations
Let X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 be the solution of the following SDE
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . \mathrm{d}X_t=\mu(t,X_t)\,\mathrm{d}t+\sigma(t,X_t)\,\mathrm{d}W_t. d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t .
Let Γ \Gamma Γ denote the transition density of the process X X X .
Γ ( t , x ; T , y ) d y : = P ( X T ∈ d y ∣ X t = x ) . \Gamma(t,x;T,y)\,\mathrm{d}y:=\mathbb{P}(X_T\in\mathrm{d}y\mid X_t=x). Γ ( t , x ; T , y ) d y := P ( X T ∈ d y ∣ X t = x ) .
In this Section, we will derive two PDEs which are satisfied by the transition density Γ \Gamma Γ : the Kolmogorov Backward Equation (KBE), which is a PDE in the backward variables ( t , x ) (t,x) ( t , x ) , and the Kolmogorov Forward Equation (KFE), which is a PDE satisfied by the forward variables ( T , y ) (T,y) ( T , y ) . Physicists and biologists sometimes call the KFE the Fokker-Planck Equation .
We have already seen the KFE and KBE in the continuous time Markov chain setting, discussed in Section § 13.1 . The development of the KFE and KBE for diffusion processes is remarkably similar.
Definition 9.4.1. The two-parameter semigroup ( P ( t , T ) ) 0 ≤ t ≤ T < ∞ (\mathcal{P}(t,T))_{0\le t\le T<\infty} ( P ( t , T ) ) 0 ≤ t ≤ T < ∞ , of a Markov diffusion X X X , is defined as
P ( t , T ) φ ( x ) = E [ φ ( X T ) ∣ X t = x ] = ∫ d y Γ ( t , x ; T , y ) φ ( y ) . \mathcal{P}(t,T)\varphi(x)=\mathbb{E}[\varphi(X_T)\mid X_t=x]=\int\mathrm{d}y\,\Gamma(t,x;T,y)\varphi(y). P ( t , T ) φ ( x ) = E [ φ ( X T ) ∣ X t = x ] = ∫ d y Γ ( t , x ; T , y ) φ ( y ) . where φ \varphi φ is integrable with respect to the transition density Γ \Gamma Γ .
Theorem 9.4.2. The semigroup satisfies
P ( t , t ) = I , P ( t , s ) P ( s , T ) = P ( t , T ) , 0 ≤ t ≤ s ≤ T . \mathcal{P}(t,t)=I,\qquad\mathcal{P}(t,s)\mathcal{P}(s,T)=\mathcal{P}(t,T),\qquad0\leq t\leq s\leq T. P ( t , t ) = I , P ( t , s ) P ( s , T ) = P ( t , T ) , 0 ≤ t ≤ s ≤ T . Proof. To see that P ( t , t ) = I \mathcal{P}(t,t)=I P ( t , t ) = I simply observe that
P ( t , t ) φ ( x ) = E [ φ ( X t ) ∣ X t = x ] = φ ( x ) = I φ ( x ) . \mathcal{P}(t,t)\varphi(x)=\mathbb{E}[\varphi(X_t)|X_t=x]=\varphi(x)=I\varphi(x). P ( t , t ) φ ( x ) = E [ φ ( X t ) ∣ X t = x ] = φ ( x ) = I φ ( x ) . To see the semigroup property P ( t , s ) P ( s , T ) = P ( t , T ) \mathcal{P}(t,s)\mathcal{P}(s,T)=\mathcal{P}(t,T) P ( t , s ) P ( s , T ) = P ( t , T ) , note that
P ( t , s ) P ( s , T ) φ ( x ) = P ( t , s ) E [ φ ( X T ) ∣ X s = x ] = E [ E [ φ ( X T ) ∣ X s ] ∣ X t = x ] \mathcal{P}(t,s)\mathcal{P}(s,T)\varphi(x)=\mathcal{P}(t,s)\mathbb{E}[\varphi(X_T)|X_s=x]=\mathbb{E}[\mathbb{E}[\varphi(X_T)|X_s]|X_t=x] P ( t , s ) P ( s , T ) φ ( x ) = P ( t , s ) E [ φ ( X T ) ∣ X s = x ] = E [ E [ φ ( X T ) ∣ X s ] ∣ X t = x ] = E [ φ ( X T ) ∣ X t = x ] = P ( t , T ) φ ( x ) . =\mathbb{E}[\varphi(X_T)|X_t=x]=\mathcal{P}(t,T)\varphi(x). = E [ φ ( X T ) ∣ X t = x ] = P ( t , T ) φ ( x ) . The semigroup property can, alternatively, be derived from the Chapman-Kolmogorov equation
Γ ( t , x ; T , y ) = ∫ d z Γ ( t , x ; s , z ) Γ ( s , z ; T , y ) , 0 ≤ t ≤ s ≤ T < ∞ . \Gamma(t,x;T,y)=\int dz\,\Gamma(t,x;s,z)\Gamma(s,z;T,y),\qquad0\leq t\leq s\leq T<\infty. Γ ( t , x ; T , y ) = ∫ d z Γ ( t , x ; s , z ) Γ ( s , z ; T , y ) , 0 ≤ t ≤ s ≤ T < ∞. We have
P ( t , s ) P ( s , T ) φ ( x ) = P ( t , s ) ∫ d y Γ ( s , x ; T , y ) φ ( y ) = ∫ d z Γ ( t , x ; s , z ) ∫ d y Γ ( s , z ; T , y ) φ ( y ) = ∫ d y Γ ( t , x ; T , y ) φ ( y ) = P ( t , T ) φ ( x ) . \begin{aligned}
\mathcal{P}(t,s)\mathcal{P}(s,T)\varphi(x)&=\mathcal{P}(t,s)\int dy\,\Gamma(s,x;T,y)\varphi(y)\\
&=\int dz\,\Gamma(t,x;s,z)\int dy\,\Gamma(s,z;T,y)\varphi(y)\\
&=\int dy\,\Gamma(t,x;T,y)\varphi(y)=\mathcal{P}(t,T)\varphi(x).
\end{aligned} P ( t , s ) P ( s , T ) φ ( x ) = P ( t , s ) ∫ d y Γ ( s , x ; T , y ) φ ( y ) = ∫ d z Γ ( t , x ; s , z ) ∫ d y Γ ( s , z ; T , y ) φ ( y ) = ∫ d y Γ ( t , x ; T , y ) φ ( y ) = P ( t , T ) φ ( x ) .
We associate with a semigroup, its generator.
Definition 9.4.3. The infinitesimal generator or simply the generator , of a semigroup of operators ( P ( t , s ) ) 0 ≤ t ≤ T < ∞ (\mathcal{P}(t,s))_{0\leq t\leq T<\infty} ( P ( t , s ) ) 0 ≤ t ≤ T < ∞ is defined as
A ( t ) φ ( x ) : = lim s ↓ t 1 s − t ( P ( t , s ) φ ( x ) − φ ( x ) ) \mathcal{A}(t)\varphi(x):=\lim_{s\downarrow t}\frac{1}{s-t}\left(\mathcal{P}(t,s)\varphi(x)-\varphi(x)\right) A ( t ) φ ( x ) := s ↓ t lim s − t 1 ( P ( t , s ) φ ( x ) − φ ( x ) ) = lim s ↓ t 1 s − t ( E [ φ ( X s ) ∣ X t = x ] − φ ( x ) ) =\lim_{s\downarrow t}\frac{1}{s-t}\left(\mathbb{E}[\varphi(X_s)|X_t=x]-\varphi(x)\right) = s ↓ t lim s − t 1 ( E [ φ ( X s ) ∣ X t = x ] − φ ( x ) ) where φ \varphi φ is any function for which the limit exists.
Theorem 9.4.4. If φ ∈ C 0 2 \varphi\in C_0^2 φ ∈ C 0 2 (bounded and twice differentiable ), then and X X X is the solution of
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t dX_t=\mu(t,X_t)dt+\sigma(t,X_t)dW_t d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t then the generator A ( t ) \mathcal{A}(t) A ( t ) of the semigroup ( P ( t , s ) ) 0 ≤ t ≤ s < ∞ (\mathcal{P}(t,s))_{0\leq t\leq s<\infty} ( P ( t , s ) ) 0 ≤ t ≤ s < ∞ of X X X is given by
A ( t ) = μ ( t , x ) ∂ x + 1 2 σ 2 ( t , x ) ∂ x 2 . \mathcal{A}(t)=\mu(t,x)\partial_x+\frac{1}{2}\sigma^2(t,x)\partial_x^2. A ( t ) = μ ( t , x ) ∂ x + 2 1 σ 2 ( t , x ) ∂ x 2 .
Proof. From the Itô formula we have
d φ ( X t ) = ∂ x φ ( X t ) d X t + 1 2 ∂ x 2 φ ( X t ) d [ X , X ] t = ( μ ( t , X t ) ∂ x φ ( X t ) + 1 2 σ 2 ( t , X t ) ∂ x 2 φ ( X t ) ) d t + σ ( t , X t ) ∂ x φ ( X t ) d W t = A ( t ) φ ( X t ) d t + σ ( t , X t ) ∂ x φ ( X t ) d W t . \begin{aligned}
\mathrm{d}\varphi(X_t) &= \partial_x\varphi(X_t)\mathrm{d}X_t + \frac{1}{2}\partial_x^2\varphi(X_t)\mathrm{d}[X,X]_t \\
&= \left(\mu(t,X_t)\partial_x\varphi(X_t) + \frac{1}{2}\sigma^2(t,X_t)\partial_x^2\varphi(X_t)\right)\mathrm{d}t + \sigma(t,X_t)\partial_x\varphi(X_t)\mathrm{d}W_t \\
&= \mathcal{A}(t)\varphi(X_t)\mathrm{d}t + \sigma(t,X_t)\partial_x\varphi(X_t)\mathrm{d}W_t.
\end{aligned} d φ ( X t ) = ∂ x φ ( X t ) d X t + 2 1 ∂ x 2 φ ( X t ) d [ X , X ] t = ( μ ( t , X t ) ∂ x φ ( X t ) + 2 1 σ 2 ( t , X t ) ∂ x 2 φ ( X t ) ) d t + σ ( t , X t ) ∂ x φ ( X t ) d W t = A ( t ) φ ( X t ) d t + σ ( t , X t ) ∂ x φ ( X t ) d W t .
Therefore, we have
φ ( X s ) = φ ( X t ) + ∫ t s A ( r ) φ ( X r ) d r + ∫ t s σ ( r , X r ) φ ( X r ) d W r , \varphi(X_s) = \varphi(X_t) + \int_t^s \mathcal{A}(r)\varphi(X_r)\mathrm{d}r + \int_t^s \sigma(r,X_r)\varphi(X_r)\mathrm{d}W_r, φ ( X s ) = φ ( X t ) + ∫ t s A ( r ) φ ( X r ) d r + ∫ t s σ ( r , X r ) φ ( X r ) d W r ,
E [ φ ( X s ) ∣ X t = x ] = φ ( x ) + ∫ t s E [ A ( r ) φ ( X r ) ∣ X t = x ] d r . \mathbb{E}[\varphi(X_s)\mid X_t=x] = \varphi(x) + \int_t^s \mathbb{E}[\mathcal{A}(r)\varphi(X_r)\mid X_t=x]\mathrm{d}r. E [ φ ( X s ) ∣ X t = x ] = φ ( x ) + ∫ t s E [ A ( r ) φ ( X r ) ∣ X t = x ] d r .
Finally,
lim s ↘ t 1 s − t ( E [ φ ( X s ) ∣ X t = x ] − φ ( x ) ) = lim s ↘ t 1 s − t ∫ t s E [ A ( r ) φ ( X r ) ∣ X t = x ] d r = A ( t ) φ ( x ) . \lim_{s\searrow t}\frac{1}{s-t}\left(\mathbb{E}[\varphi(X_s)\mid X_t=x]-\varphi(x)\right)
=\lim_{s\searrow t}\frac{1}{s-t}\int_t^s\mathbb{E}[\mathcal{A}(r)\varphi(X_r)\mid X_t=x]\mathrm{d}r
=\mathcal{A}(t)\varphi(x). s ↘ t lim s − t 1 ( E [ φ ( X s ) ∣ X t = x ] − φ ( x ) ) = s ↘ t lim s − t 1 ∫ t s E [ A ( r ) φ ( X r ) ∣ X t = x ] d r = A ( t ) φ ( x ) .
□ \square □
Remark 9.4.5. For all intents and purposes, when X is the solution of an SDE the generator A ( t ) \mathcal{A}(t) A ( t ) is simply the operator that acts on φ \varphi φ in the d t \mathrm{d}t d t term of d φ ( X t ) \mathrm{d}\varphi(X_t) d φ ( X t ) . Thus, we can write the Itô formula more compactly as
d φ ( X t ) = A ( t ) φ ( X t ) d t + σ ( t , X t ) ∂ x φ ( X t ) d W t . \mathrm{d}\varphi(X_t)=\mathcal{A}(t)\varphi(X_t)\mathrm{d}t+\sigma(t,X_t)\partial_x\varphi(X_t)\mathrm{d}W_t. d φ ( X t ) = A ( t ) φ ( X t ) d t + σ ( t , X t ) ∂ x φ ( X t ) d W t .
Of course, if X is not the solution of an SDE, we cannot write the differential d φ ( X t ) \mathrm{d}\varphi(X_t) d φ ( X t ) in this more compact form.
We can now state and prove the KBE and KFE
Theorem 9.4.6. Let X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 be the unique solution (assumed to exist) of
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . \mathrm{d}X_t=\mu(t,X_t)\mathrm{d}t+\sigma(t,X_t)\mathrm{d}W_t. d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . Let Γ \Gamma Γ denote the transition density of X X X
Γ ( t , x ; T , y ) d y = P ( X T ∈ d y ∣ X t = x ) . \Gamma(t,x;T,y)\mathrm{d}y=\mathbb{P}(X_T\in\mathrm{d}y\mid X_t=x). Γ ( t , x ; T , y ) d y = P ( X T ∈ d y ∣ X t = x ) . Then, seen as a function of the backwards variables ( t , x ) (t,x) ( t , x ) , the transition density Γ ( ⋅ , ⋅ ; T , y ) \Gamma(\cdot,\cdot;T,y) Γ ( ⋅ , ⋅ ; T , y ) satisfies
K B E : ( ∂ t + A ( t ) ) Γ ( t , ⋅ ; T , y ) = 0 , Γ ( T , ⋅ , T , y ) = δ y , \mathrm{KBE}: \qquad(\partial_t+\mathcal{A}(t))\Gamma(t,\cdot;T,y)=0,\qquad\Gamma(T,\cdot,T,y)=\delta_y, KBE : ( ∂ t + A ( t )) Γ ( t , ⋅ ; T , y ) = 0 , Γ ( T , ⋅ , T , y ) = δ y ,
where A ( t ) \mathcal{A}(t) A ( t ) is the infinitesimal generator of X X X
A ( t ) = μ ( t , x ) ∂ x + 1 2 σ 2 ( t , x ) ∂ x 2 \mathcal{A}(t)=\mu(t,x)\partial_x+\frac{1}{2}\sigma^2(t,x)\partial_x^2 A ( t ) = μ ( t , x ) ∂ x + 2 1 σ 2 ( t , x ) ∂ x 2
Seen as a function of the forward variables ( T , y ) (T,y) ( T , y ) , the transition density Γ ( t , x ; ⋅ , ⋅ ) \Gamma(t,x;\cdot,\cdot) Γ ( t , x ; ⋅ , ⋅ ) satisfies
K F E : ( − ∂ T + A ∗ ( T ) ) Γ ( t , x ; T , ⋅ ) = 0 , Γ ( t , x ; t , ⋅ ) = δ x \mathrm{KFE:}\qquad(-\partial_T+\mathcal{A}^*(T))\Gamma(t,x;T,\cdot)=0,\qquad\Gamma(t,x;t,\cdot)=\delta_x KFE : ( − ∂ T + A ∗ ( T )) Γ ( t , x ; T , ⋅ ) = 0 , Γ ( t , x ; t , ⋅ ) = δ x
where A ∗ ( T ) \mathcal{A}^*(T) A ∗ ( T ) is the L 2 ( d x ) L^2(dx) L 2 ( d x ) adjoint of A ( T ) \mathcal{A}(T) A ( T )
A ∗ ( T ) = − ∂ y μ ( T , y ) + 1 2 ∂ y 2 σ 2 ( T , y ) \mathcal{A}^*(T)=-\partial_y\mu(T,y)+\frac{1}{2}\partial_y^2\sigma^2(T,y) A ∗ ( T ) = − ∂ y μ ( T , y ) + 2 1 ∂ y 2 σ 2 ( T , y )
Proof. We have already established in Theorem 9.3.1 that
u ( t , x ) : = E [ φ ( X T ) ∣ X t = x ] = ∫ d y Γ ( t , x ; T , y ) φ ( y ) u(t,x):=\mathbb{E}[\varphi(X_T)|X_t=x]=\int dy\,\Gamma(t,x;T,y)\varphi(y) u ( t , x ) := E [ φ ( X T ) ∣ X t = x ] = ∫ d y Γ ( t , x ; T , y ) φ ( y )
satisfies (126) . Thus, we must have
0 = ( ∂ t + A ( t ) ) u ( t , ⋅ ) = ∫ d y φ ( y ) ( ∂ t + A ( t ) ) Γ ( t , ⋅ ; T , y ) , 0=(\partial_t+\mathcal{A}(t))u(t,\cdot)=\int dy\,\varphi(y)(\partial_t+\mathcal{A}(t))\Gamma(t,\cdot;T,y), 0 = ( ∂ t + A ( t )) u ( t , ⋅ ) = ∫ d y φ ( y ) ( ∂ t + A ( t )) Γ ( t , ⋅ ; T , y ) ,
φ = u ( T , ⋅ ) = ∫ d y φ ( y ) Γ ( T , ⋅ ; T , y ) . \varphi=u(T,\cdot)=\int dy\,\varphi(y)\Gamma(T,\cdot;T,y). φ = u ( T , ⋅ ) = ∫ d y φ ( y ) Γ ( T , ⋅ ; T , y ) .
The above equations must hold for all functions φ \varphi φ . It follows that
( ∂ t + A ( t ) ) Γ ( ⋅ , t ; T , y ) = 0 , Γ ( T , ⋅ , T , y ) = δ y (\partial_t+\mathcal{A}(t))\Gamma(\cdot,t;T,y)=0,\qquad\Gamma(T,\cdot,T,y)=\delta_y ( ∂ t + A ( t )) Γ ( ⋅ , t ; T , y ) = 0 , Γ ( T , ⋅ , T , y ) = δ y
which is precisely the KBE for Γ \Gamma Γ .
Proving the KFE requires a little more effort. First, we note that, by definition, the generator A ( t ) \mathcal{A}(t) A ( t ) and its L 2 ( d x ) L^2(dx) L 2 ( d x ) adjoint satisfy
∫ d y f ( y ) A ( t ) g ( y ) = ∫ d y g ( y ) A ∗ ( t ) f ( y ) , \int dy\,f(y)\mathcal{A}(t)g(y)=\int dy\,g(y)\mathcal{A}^*(t)f(y), ∫ d y f ( y ) A ( t ) g ( y ) = ∫ d y g ( y ) A ∗ ( t ) f ( y ) ,
where, both A ( t ) \mathcal{A}(t) A ( t ) and A ∗ ( t ) \mathcal{A}^*(t) A ∗ ( t ) act on the y y y variable and f , g → 0 f,g\to0 f , g → 0 as y → ± ∞ y\to\pm\infty y → ± ∞ . Now, observe that
∫ d y φ ( y ) Γ ( t , x ; T , y ) = E [ φ ( X T ) ∣ X t = x ] = φ ( x ) + ∫ t T d s E [ A ( s ) φ ( X s ) ∣ X t = x ] + E [ ∫ t T σ ( s , X s ) ∂ x φ ( X s ) d W s | X t = x ] = φ ( x ) + ∫ t T d s ∫ d y Γ ( t , x ; s , y ) A ( s ) φ ( y ) = φ ( x ) + ∫ t T d s ∫ d y φ ( y ) A ∗ ( s ) Γ ( t , x ; s , y ) . \begin{aligned}
\int dy\,\varphi(y)\Gamma(t,x;T,y)&=\mathbb{E}[\varphi(X_T)|X_t=x]\\
&=\varphi(x)+\int_t^T ds\,\mathbb{E}[\mathcal{A}(s)\varphi(X_s)|X_t=x]+\mathbb{E}\left[\int_t^T\sigma(s,X_s)\partial_x\varphi(X_s)dW_s\middle|X_t=x\right]\\
&=\varphi(x)+\int_t^T ds\int dy\,\Gamma(t,x;s,y)\mathcal{A}(s)\varphi(y)\\
&=\varphi(x)+\int_t^T ds\int dy\,\varphi(y)\mathcal{A}^*(s)\Gamma(t,x;s,y).
\end{aligned} ∫ d y φ ( y ) Γ ( t , x ; T , y ) = E [ φ ( X T ) ∣ X t = x ] = φ ( x ) + ∫ t T d s E [ A ( s ) φ ( X s ) ∣ X t = x ] + E [ ∫ t T σ ( s , X s ) ∂ x φ ( X s ) d W s X t = x ] = φ ( x ) + ∫ t T d s ∫ d y Γ ( t , x ; s , y ) A ( s ) φ ( y ) = φ ( x ) + ∫ t T d s ∫ d y φ ( y ) A ∗ ( s ) Γ ( t , x ; s , y ) .
Now, we differentiate the above expression with respect to T T T and find
∫ d y φ ( y ) ∂ T Γ ( t , x ; T , y ) = ∫ d y φ ( y ) A ∗ ( T ) Γ ( t , x ; T , y ) . \int\mathrm{d}y\,\varphi(y)\partial_T\Gamma(t,x;T,y)=\int\mathrm{d}y\,\varphi(y)\mathcal{A}^{*}(T)\Gamma(t,x;T,y). ∫ d y φ ( y ) ∂ T Γ ( t , x ; T , y ) = ∫ d y φ ( y ) A ∗ ( T ) Γ ( t , x ; T , y ) .
We also have
∫ d y φ ( y ) Γ ( t , x ; t , y ) = E [ φ ( X t ) ∣ X t = x ] = φ ( x ) . \int\mathrm{d}y\,\varphi(y)\Gamma(t,x;t,y)=\mathbb{E}[\varphi(X_t)\mid X_t=x]=\varphi(x). ∫ d y φ ( y ) Γ ( t , x ; t , y ) = E [ φ ( X t ) ∣ X t = x ] = φ ( x ) .
Again, the above expressions must hold for all φ \varphi φ . Thus, we see that
0 = ( − ∂ T + A ∗ ( T ) ) Γ ( t , x ; T , ⋅ ) , Γ ( t , x ; t , ⋅ ) = δ x . 0=(-\partial_T+\mathcal{A}^{*}(T))\Gamma(t,x;T,\cdot),\qquad\Gamma(t,x;t,\cdot)=\delta_x. 0 = ( − ∂ T + A ∗ ( T )) Γ ( t , x ; T , ⋅ ) , Γ ( t , x ; t , ⋅ ) = δ x .
which is exactly the KFE.
Example 9.4.7. Let us check that the KBE and KFE are satisfied in the following simple case:
d X t = σ d W t . \mathrm{d}X_t=\sigma\mathrm{d}W_t. d X t = σ d W t . Since ( X T ∣ X t = x ) ∼ N ( x , σ 2 ( T − t ) ) (X_T\mid X_t=x)\sim\mathcal{N}(x,\sigma^2(T-t)) ( X T ∣ X t = x ) ∼ N ( x , σ 2 ( T − t )) we have
Γ ( t , x ; T , y ) = 1 2 π σ 2 ( T − t ) exp ( − ( y − x ) 2 2 σ 2 ( T − t ) ) . \Gamma(t,x;T,y)=\frac{1}{\sqrt{2\pi\sigma^2(T-t)}}\exp\left(\frac{-(y-x)^2}{2\sigma^2(T-t)}\right). Γ ( t , x ; T , y ) = 2 π σ 2 ( T − t ) 1 exp ( 2 σ 2 ( T − t ) − ( y − x ) 2 ) . Once can check by direct computation that
( ∂ t + A ) Γ ( t , ⋅ ; T , y ) = 0 , lim t ↗ T Γ ( t , ⋅ ; T , y ) = δ y , (\partial_t+\mathcal{A})\Gamma(t,\cdot;T,y)=0,\qquad\lim_{t\nearrow T}\Gamma(t,\cdot;T,y)=\delta_y, ( ∂ t + A ) Γ ( t , ⋅ ; T , y ) = 0 , t ↗ T lim Γ ( t , ⋅ ; T , y ) = δ y , ( − ∂ T + A ∗ ) Γ ( t , x ; T , ⋅ ) = 0 , lim T ↘ t Γ ( t , x ; T , ⋅ ) = δ x , (-\partial_T+\mathcal{A}^{*})\Gamma(t,x;T,\cdot)=0,\qquad\lim_{T\searrow t}\Gamma(t,x;T,\cdot)=\delta_x, ( − ∂ T + A ∗ ) Γ ( t , x ; T , ⋅ ) = 0 , T ↘ t lim Γ ( t , x ; T , ⋅ ) = δ x , where A = 1 2 σ 2 ∂ x 2 \mathcal{A}=\frac{1}{2}\sigma^2\partial_x^2 A = 2 1 σ 2 ∂ x 2 and A ∗ = 1 2 σ 2 ∂ y 2 \mathcal{A}^{*}=\frac{1}{2}\sigma^2\partial_y^2 A ∗ = 2 1 σ 2 ∂ y 2 .
Poisson equations
The KFE and KBE are parabolic PDEs (think heat-type equations). It turns out we can also relate the expected value of a functional of X X X to an elliptic PDE (think, potential equations). Consider a one-dimensional time-homogeneous diffusion X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 that lives on some interval I = ( l , r ) I=(l,r) I = ( l , r ) , where − ∞ ≤ l < r ≤ ∞ -\infty\leq l<r\leq\infty − ∞ ≤ l < r ≤ ∞ . We suppose X X X satisfies
d X t = μ ( X t ) d t + σ ( X t ) d W t . (133) \mathrm{d}X_t=\mu(X_t)\mathrm{d}t+\sigma(X_t)\mathrm{d}W_t. \tag*{(133)} d X t = μ ( X t ) d t + σ ( X t ) d W t . (133)
Let us define the hitting time
τ = inf { t ≥ 0 : X t ∉ I } . (134) \tau=\inf\{t\geq0:X_t\notin I\}. \tag*{(134)} τ = inf { t ≥ 0 : X t ∈ / I } . (134)
where ∂ I = { l } ∪ { r } \partial I=\{l\}\cup\{r\} ∂ I = { l } ∪ { r } denotes the boundary of $I.
Theorem 9.5.1. Let X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 and τ \tau τ be given by (133) and (134) . Assume P ( τ < ∞ ) = 1 \mathbb{P}(\tau<\infty)=1 P ( τ < ∞ ) = 1 . Let λ ≥ 0 \lambda\ge0 λ ≥ 0 , and let φ : ∂ I → R \varphi:\partial I\to\mathbb{R} φ : ∂ I → R and g : I → R g:I\to\mathbb{R} g : I → R be bounded measurable functions. For any t ≥ 0 t\ge0 t ≥ 0 and x ∈ I x\in I x ∈ I , define
u ( x ) : = E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ X t = x ] , t < τ . (135) u(x):=\mathbb{E}\left[e^{-\lambda(\tau-t)}\varphi(X_\tau)+\int_t^\tau e^{-\lambda(s-t)}g(X_s)\,\mathrm{d}s\mid X_t=x\right], \qquad t<\tau. \tag*{(135)} u ( x ) := E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ X t = x ] , t < τ . (135) Then u ( x ) u(x) u ( x ) does not depend on t t t (by time-homogeneity of X X X ). Moreover, if u ∈ C 2 ( I ) u\in C^2(I) u ∈ C 2 ( I ) and u u u is continuous on I ∪ ∂ I I\cup\partial I I ∪ ∂ I , then the function u u u satisfies
( A − λ ) u + g = 0 (\mathcal{A}-\lambda)u+g=0 ( A − λ ) u + g = 0 in I I I , {#eq:9.23}
on ∂ I \partial I ∂ I , {#eq:9.24}
where A = μ ( x ) ∂ x + 1 2 σ 2 ( x ) ∂ x 2 \mathcal{A}=\mu(x)\partial_x+\frac{1}{2}\sigma^2(x)\partial_x^2 A = μ ( x ) ∂ x + 2 1 σ 2 ( x ) ∂ x 2 is the infinitesimal generator of X X X .
Proof. We first verify that u ( x ) u(x) u ( x ) is independent of t t t . By the strong Markov property and time-homogeneity of X X X , conditioning on X t = x X_t=x X t = x is equivalent to restarting the process at time t t t from position x x x , and the future evolution is identical in distribution to starting at time 0 0 0 from position x x x . More precisely, for any 0 ≤ s < t < τ 0\le s<t<\tau 0 ≤ s < t < τ , we have
E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( r − t ) g ( X r ) d r ∣ X t = x ] = E [ e − λ ( τ − s ) φ ( X τ ) + ∫ s τ e − λ ( r − s ) g ( X r ) d r ∣ X s = x ] . \mathbb{E}\left[e^{-\lambda(\tau-t)}\varphi(X_\tau)+\int_t^\tau e^{-\lambda(r-t)}g(X_r)\,\mathrm{d}r\mid X_t=x\right]
=
\mathbb{E}\left[e^{-\lambda(\tau-s)}\varphi(X_\tau)+\int_s^\tau e^{-\lambda(r-s)}g(X_r)\,\mathrm{d}r\mid X_s=x\right]. E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( r − t ) g ( X r ) d r ∣ X t = x ] = E [ e − λ ( τ − s ) φ ( X τ ) + ∫ s τ e − λ ( r − s ) g ( X r ) d r ∣ X s = x ] .
Thus, u ( x ) u(x) u ( x ) is well-defined independently of t t t .
Now, let F = ( F t ) 0 ≤ t < τ \mathbb{F}=(\mathcal{F}_t)_{0\le t<\tau} F = ( F t ) 0 ≤ t < τ be the filtration generated by X X X . The process M = ( M t ) 0 ≤ t < τ M=(M_t)_{0\le t<\tau} M = ( M t ) 0 ≤ t < τ , defined by
M t : = e − λ t u ( X t ) + ∫ 0 t e − λ s g ( X s ) d s M_t:=e^{-\lambda t}u(X_t)+\int_0^t e^{-\lambda s}g(X_s)\,\mathrm{d}s M t := e − λ t u ( X t ) + ∫ 0 t e − λ s g ( X s ) d s
is a martingale. To see this, first observe that
M t = e − λ t E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ X t ] + ∫ 0 t e − λ s g ( X s ) d s = e − λ t E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ F t ] + ∫ 0 t e − λ s g ( X s ) d s = E [ e − λ τ φ ( X τ ) + ∫ t τ e − λ s g ( X s ) d s ∣ F t ] + ∫ 0 t e − λ s g ( X s ) d s = E [ e − λ τ φ ( X τ ) + ∫ 0 τ e − λ s g ( X s ) d s ∣ F t ] . \begin{aligned}
M_t&=e^{-\lambda t}\mathbb{E}\left[e^{-\lambda(\tau-t)}\varphi(X_\tau)+\int_t^\tau e^{-\lambda(s-t)}g(X_s)\,\mathrm{d}s\mid X_t\right]+\int_0^t e^{-\lambda s}g(X_s)\,\mathrm{d}s\\
&=e^{-\lambda t}\mathbb{E}\left[e^{-\lambda(\tau-t)}\varphi(X_\tau)+\int_t^\tau e^{-\lambda(s-t)}g(X_s)\,\mathrm{d}s\mid\mathcal{F}_t\right]+\int_0^t e^{-\lambda s}g(X_s)\,\mathrm{d}s\\
&=\mathbb{E}\left[e^{-\lambda\tau}\varphi(X_\tau)+\int_t^\tau e^{-\lambda s}g(X_s)\,\mathrm{d}s\mid\mathcal{F}_t\right]+\int_0^t e^{-\lambda s}g(X_s)\,\mathrm{d}s\\
&=\mathbb{E}\left[e^{-\lambda\tau}\varphi(X_\tau)+\int_0^\tau e^{-\lambda s}g(X_s)\,\mathrm{d}s\mid\mathcal{F}_t\right].
\end{aligned} M t = e − λ t E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ X t ] + ∫ 0 t e − λ s g ( X s ) d s = e − λ t E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ F t ] + ∫ 0 t e − λ s g ( X s ) d s = E [ e − λ τ φ ( X τ ) + ∫ t τ e − λ s g ( X s ) d s ∣ F t ] + ∫ 0 t e − λ s g ( X s ) d s = E [ e − λ τ φ ( X τ ) + ∫ 0 τ e − λ s g ( X s ) d s ∣ F t ] .
Thus, for 0 ≤ s ≤ t < τ 0\le s\le t<\tau 0 ≤ s ≤ t < τ we have
E [ M t ∣ F s ] = E [ E [ e − λ τ φ ( X τ ) + ∫ 0 τ e − λ r g ( X r ) d r ∣ F t ] ∣ F s ] = E [ e − λ τ φ ( X τ ) + ∫ 0 τ e − λ r g ( X r ) d r ∣ F s ] = M s , \begin{aligned}
\mathbb{E}[M_t\mid\mathcal{F}_s]=\mathbb{E}\left[\mathbb{E}\left[e^{-\lambda\tau}\varphi(X_\tau)+\int_0^\tau e^{-\lambda r}g(X_r)\,\mathrm{d}r\mid\mathcal{F}_t\right]\mid\mathcal{F}_s\right] \\
&= \mathbb{E}\left[e^{-\lambda\tau}\varphi(X_\tau)+\int_0^\tau e^{-\lambda r}g(X_r)\,dr\mid\mathcal{F}_s\right]=M_s,
\end{aligned} E [ M t ∣ F s ] = E [ E [ e − λ τ φ ( X τ ) + ∫ 0 τ e − λ r g ( X r ) d r ∣ F t ] ∣ F s ] = E [ e − λ τ φ ( X τ ) + ∫ 0 τ e − λ r g ( X r ) d r ∣ F s ] = M s ,
which proves that M M M is a martingale.
Assuming u ∈ C 2 ( I ) u\in C^2(I) u ∈ C 2 ( I ) , we take the differential of M M M and obtain
d M t = d ( e − λ t u ( X t ) ) + d ∫ 0 t e − λ s g ( X s ) d s = e − λ t d u ( X t ) + u ( X t ) d e − λ t + e − λ t g ( X t ) d t = e − λ t [ ∂ x u ( X t ) d X t + 1 2 ∂ x 2 u ( X t ) d [ X , X ] t ] − λ e − λ t u ( X t ) d t + e − λ t g ( X t ) d t = e − λ t [ μ ( X t ) ∂ x u ( X t ) d t + σ ( X t ) ∂ x u ( X t ) d W t + 1 2 σ 2 ( X t ) ∂ x 2 u ( X t ) d t ] − λ e − λ t u ( X t ) d t + e − λ t g ( X t ) d t = e − λ t ( A u ( X t ) − λ u ( X t ) + g ( X t ) ) d t + e − λ t σ ( X t ) ∂ x u ( X t ) d W t . \begin{aligned}
dM_t=d\left(e^{-\lambda t}u(X_t)\right)+d\int_0^t e^{-\lambda s}g(X_s)\,ds \\
&=e^{-\lambda t}du(X_t)+u(X_t)de^{-\lambda t}+e^{-\lambda t}g(X_t)\,dt \\
&=e^{-\lambda t}\left[\partial_xu(X_t)dX_t+\frac{1}{2}\partial_x^2u(X_t)d[X,X]_t\right]-\lambda e^{-\lambda t}u(X_t)\,dt+e^{-\lambda t}g(X_t)\,dt \\
&=e^{-\lambda t}\left[\mu(X_t)\partial_xu(X_t)\,dt+\sigma(X_t)\partial_xu(X_t)\,dW_t+\frac{1}{2}\sigma^2(X_t)\partial_x^2u(X_t)\,dt\right]
-\lambda e^{-\lambda t}u(X_t)\,dt+e^{-\lambda t}g(X_t)\,dt \\
&=e^{-\lambda t}\left(\mathcal{A}u(X_t)-\lambda u(X_t)+g(X_t)\right)\,dt+e^{-\lambda t}\sigma(X_t)\partial_xu(X_t)\,dW_t.
\end{aligned} d M t = d ( e − λ t u ( X t ) ) + d ∫ 0 t e − λ s g ( X s ) d s = e − λ t d u ( X t ) + u ( X t ) d e − λ t + e − λ t g ( X t ) d t = e − λ t [ ∂ x u ( X t ) d X t + 2 1 ∂ x 2 u ( X t ) d [ X , X ] t ] − λ e − λ t u ( X t ) d t + e − λ t g ( X t ) d t = e − λ t [ μ ( X t ) ∂ x u ( X t ) d t + σ ( X t ) ∂ x u ( X t ) d W t + 2 1 σ 2 ( X t ) ∂ x 2 u ( X t ) d t ] − λ e − λ t u ( X t ) d t + e − λ t g ( X t ) d t = e − λ t ( A u ( X t ) − λ u ( X t ) + g ( X t ) ) d t + e − λ t σ ( X t ) ∂ x u ( X t ) d W t .
Since M M M is a martingale, the d t dt d t term must be zero for all X t ∈ I X_t\in I X t ∈ I . Thus, we have
( A − λ ) u + g = 0 , in I , (\mathcal{A}-\lambda)u+g=0,\qquad\text{in }I, ( A − λ ) u + g = 0 , in I ,
in agreement with (9.23).
The boundary condition (9.24) follows from observing that if X t ∈ ∂ I X_t\in\partial I X t ∈ ∂ I (i.e., X t = l X_t=l X t = l or X t = r X_t=r X t = r ), then τ = t \tau=t τ = t since the process has already exited I I I . Thus,
u ( x ) = E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ X t = x ∈ ∂ I ] = E [ e λ ( t − t ) φ ( x ) + ∫ t t e − λ ( s − t ) g ( X s ) d s ∣ X t = x ] = φ ( x ) . \begin{aligned}
u(x)=\mathbb{E}\left[e^{-\lambda(\tau-t)}\varphi(X_\tau)+\int_t^\tau e^{-\lambda(s-t)}g(X_s)\,ds\mid X_t=x\in\partial I\right] \\
&=\mathbb{E}\left[e^{\lambda(t-t)}\varphi(x)+\int_t^t e^{-\lambda(s-t)}g(X_s)\,ds\mid X_t=x\right]=\varphi(x).
\end{aligned} u ( x ) = E [ e − λ ( τ − t ) φ ( X τ ) + ∫ t τ e − λ ( s − t ) g ( X s ) d s ∣ X t = x ∈ ∂ I ] = E [ e λ ( t − t ) φ ( x ) + ∫ t t e − λ ( s − t ) g ( X s ) d s ∣ X t = x ] = φ ( x ) .
This completes the proof.
Remark 9.5.2. The key insight is that for a time-homogeneous diffusion (where μ \mu μ and σ \sigma σ do not depend on t t t ), the function u ( x ) u(x) u ( x ) defined in (135) is independent of t t t . This is because the future evolution of X X X starting from position x x x at time t t t is identical in distribution to starting from position x x x at time 0. This allows us to work with u u u as a function of x x x alone, making the PDE elliptic rather than parabolic.
Remark 9.5.3. We can generalize Theorem 9.5.1 as follows: if we replace λ ( τ − t ) \lambda(\tau-t) λ ( τ − t ) with ∫ t τ λ ( X s ) d s \int_t^\tau\lambda(X_s)\,ds ∫ t τ λ ( X s ) d s (and λ ( s − t ) \lambda(s-t) λ ( s − t ) with ∫ t s λ ( X r ) d r \int_t^s\lambda(X_r)\,dr ∫ t s λ ( X r ) d r ) then u u u solves ( A − λ ( x ) ) u ( x ) + g ( x ) = 0 (\mathcal{A}-\lambda(x))u(x)+g(x)=0 ( A − λ ( x )) u ( x ) + g ( x ) = 0 . We leave the proof of this as an exercise for the reader.
Corollary 9.5.4. The Laplace transform of τ \tau τ , given by
u ( x ) : = E [ e − λ τ ∣ X 0 = x ] , u(x):=\mathbb{E}[e^{-\lambda\tau}\mid X_0=x], u ( x ) := E [ e − λ τ ∣ X 0 = x ] ,
satisfies
( A − λ ) u = 0 , in I , u = 1 , on ∂ I . (\mathcal{A}-\lambda)u=0,\qquad\text{in }I,\qquad u=1,\qquad\text{on }\partial I. ( A − λ ) u = 0 , in I , u = 1 , on ∂ I .
Proof. Take φ ( x ) = 1 \varphi(x)=1 φ ( x ) = 1 , g ( x ) = 0 g(x)=0 g ( x ) = 0 and t = 0 t=0 t = 0 in Theorem 9.5.1 .
Example 9.5.5 (First hitting time of Brownian motion). Let us define a process X X X and a hitting time τ \tau τ by
d X t = d W t , X 0 = x < r < ∞ , τ = inf { t ≥ 0 : X t ∉ ( − ∞ , r ) } . \mathrm{d}X_t=\mathrm{d}W_t,\qquad X_0=x<r<\infty,\qquad\tau=\inf\{t\geq0:X_t\notin(-\infty,r)\}. d X t = d W t , X 0 = x < r < ∞ , τ = inf { t ≥ 0 : X t ∈ / ( − ∞ , r )} . According to Corollary 9.5.4 , the function
u ( x ) : = E [ e − λ τ ∣ X 0 = x ] , u(x):=\mathbb{E}[\mathrm{e}^{-\lambda\tau}\mid X_0=x], u ( x ) := E [ e − λ τ ∣ X 0 = x ] , should satisfy
( A − λ ) u = 0 , in ( − ∞ , r ) , A = 1 2 ∂ x 2 , u ( r ) = 1. (136) (\mathcal{A}-\lambda)u=0,\qquad\text{in }(-\infty,r),\qquad\mathcal{A}=\frac{1}{2}\partial_x^2,\qquad u(r)=1. \tag*{(136)} ( A − λ ) u = 0 , in ( − ∞ , r ) , A = 2 1 ∂ x 2 , u ( r ) = 1. (136) Let us check that this is the case. We showed in § 18.5 that
E e − λ τ m = e − ∣ m ∣ 2 λ , τ m : = inf { t ≥ 0 : W t = m } , \mathbb{E}\mathrm{e}^{-\lambda\tau_m}=\mathrm{e}^{-|m|\sqrt{2\lambda}},\qquad\tau_m:=\inf\{t\geq0:W_t=m\}, E e − λ τ m = e − ∣ m ∣ 2 λ , τ m := inf { t ≥ 0 : W t = m } , where W 0 = 0 W_0=0 W 0 = 0 by definition. Since Brownian motion is invariant to translations, it follows that
u ( x ) = e − ( r − x ) 2 λ . (137) u(x)=\mathrm{e}^{-(r-x)\sqrt{2\lambda}}. \tag*{(137)} u ( x ) = e − ( r − x ) 2 λ . (137) One can easily see that (137) satisfies (136) , since
u ( r ) = 1 , 1 2 ∂ x 2 u ( x ) = λ u ( x ) . u(r)=1,\qquad\frac{1}{2}\partial_x^2u(x)=\lambda u(x). u ( r ) = 1 , 2 1 ∂ x 2 u ( x ) = λ u ( x ) .
In depth look: scalar time-homogeneous diffusions
In this section we will specialize to time-homogeneous scalar diffusions. Specifically, we consider a diffusion X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 that lives on some interval I I I with endpoints l l l and r r r , where − ∞ ≤ l < r ≤ ∞ -\infty\leq l<r\leq\infty − ∞ ≤ l < r ≤ ∞ . We suppose X X X satisfies
d X t = μ ( X t ) d t + σ ( X t ) d W t . \mathrm{d}X_t=\mu(X_t)\mathrm{d}t+\sigma(X_t)\mathrm{d}W_t. d X t = μ ( X t ) d t + σ ( X t ) d W t .
The end-points l l l and r r r may or may not be part of the interval I I I . The generator A \mathcal{A} A of X X X is given by
A = μ ( x ) ∂ x + 1 2 σ 2 ( x ) ∂ x 2 . \mathcal{A}=\mu(x)\partial_x+\frac{1}{2}\sigma^2(x)\partial_x^2. A = μ ( x ) ∂ x + 2 1 σ 2 ( x ) ∂ x 2 .
We can alternatively write A \mathcal{A} A as
A = 1 m ( x ) ∂ x ( 1 s ( x ) ∂ x ) , (138) \mathcal{A}=\frac{1}{m(x)}\partial_x\left(\frac{1}{s(x)}\partial_x\right), \tag*{(138)} A = m ( x ) 1 ∂ x ( s ( x ) 1 ∂ x ) , (138)
where s s s and m m m , called the scale and speed densities, respectively, are given by
s ( x ) = exp ( − ∫ d x 2 μ ( x ) σ 2 ( x ) ) , m ( x ) = 2 σ 2 ( x ) s ( x ) = 2 σ 2 ( x ) exp ( ∫ d x 2 μ ( x ) σ 2 ( x ) ) . (139) s(x)=\exp\left(-\int\mathrm{d}x\frac{2\mu(x)}{\sigma^2(x)}\right),\qquad m(x)=\frac{2}{\sigma^2(x)s(x)}=\frac{2}{\sigma^2(x)}\exp\left(\int\mathrm{d}x\frac{2\mu(x)}{\sigma^2(x)}\right). \tag*{(139)} s ( x ) = exp ( − ∫ d x σ 2 ( x ) 2 μ ( x ) ) , m ( x ) = σ 2 ( x ) s ( x ) 2 = σ 2 ( x ) 2 exp ( ∫ d x σ 2 ( x ) 2 μ ( x ) ) . (139)
The constant of integration is arbitrary.
Note that, if the constant of integration in the indefinite integral in m m m can be chose such that
∫ l r d y m ( y ) = 1 , \int_l^r \mathrm{d}y\,m(y)=1, ∫ l r d y m ( y ) = 1 ,
then m m m is a time-homogeneous solution of the KFE. To see this simply observe that
( − ∂ T + A ∗ ) m ( y ) = A ∗ m ( y ) = ∂ y ( − μ ( y ) m ( y ) + ∂ y σ 2 ( y ) 2 m ( y ) ) = ∂ y ( − 2 μ ( y ) σ 2 ( y ) exp ( ∫ d y 2 μ ( y ) σ 2 ( y ) ) + ∂ y exp ( ∫ d y 2 μ ( y ) σ 2 ( y ) ) ) = 0. \begin{aligned}
(-\partial_T+\mathcal{A}^*)m(y)=\mathcal{A}^*m(y)=\partial_y\left(-\mu(y)m(y)+\partial_y\frac{\sigma^2(y)}{2}m(y)\right) \\
&=\partial_y\left(-\frac{2\mu(y)}{\sigma^2(y)}\exp\left(\int\mathrm{d}y\frac{2\mu(y)}{\sigma^2(y)}\right)+\partial_y\exp\left(\int\mathrm{d}y\frac{2\mu(y)}{\sigma^2(y)}\right)\right)=0.
\end{aligned} ( − ∂ T + A ∗ ) m ( y ) = A ∗ m ( y ) = ∂ y ( − μ ( y ) m ( y ) + ∂ y 2 σ 2 ( y ) m ( y ) ) = ∂ y ( − σ 2 ( y ) 2 μ ( y ) exp ( ∫ d y σ 2 ( y ) 2 μ ( y ) ) + ∂ y exp ( ∫ d y σ 2 ( y ) 2 μ ( y ) ) ) = 0.
Thus, m m m is a stationary density for X.
Useful results from functional analysis
Before continuing further, it will be convenient to recall a few definitions from functional analysis.
Definition 9.6.1. Let L 2 ( E , ρ ) L^2(E,\rho) L 2 ( E , ρ ) denote the set of square-integrable functions on E E E weighted by ρ \rho ρ
L 2 ( E , ρ ) : = { f : ⟨ f , f ⟩ ρ < ∞ } , ⟨ f , g ⟩ ρ : = ∫ E d x ρ ( x ) f ( x ) g ( x ) . L^2(E,\rho):=\{f:\langle f,f\rangle_\rho<\infty\},\qquad\langle f,g\rangle_\rho:=\int_E \mathrm{d}x\,\rho(x)f(x)g(x). L 2 ( E , ρ ) := { f : ⟨ f , f ⟩ ρ < ∞ } , ⟨ f , g ⟩ ρ := ∫ E d x ρ ( x ) f ( x ) g ( x ) . The adjoint of L \mathcal{L} L , is the operator L ∗ \mathcal{L}^* L ∗ that satisfies
⟨ f , L g ⟩ ρ = ⟨ L ∗ f , g ⟩ ρ , ∀ g ∈ dom ( L ) , ∀ f ∈ dom ( L ∗ ) . \langle f,\mathcal{L}g\rangle_\rho=\langle\mathcal{L}^*f,g\rangle_\rho,\qquad\forall g\in\operatorname{dom}(\mathcal{L}),\qquad\forall f\in\operatorname{dom}(\mathcal{L}^*). ⟨ f , L g ⟩ ρ = ⟨ L ∗ f , g ⟩ ρ , ∀ g ∈ dom ( L ) , ∀ f ∈ dom ( L ∗ ) . We say that L \mathcal{L} L is self-adjoint in L 2 ( E , ρ ) L^2(E,\rho) L 2 ( E , ρ ) if L = L ∗ \mathcal{L}=\mathcal{L}^* L = L ∗ and dom ( L ) = dom ( L ∗ ) \operatorname{dom}(\mathcal{L})=\operatorname{dom}(\mathcal{L}^*) dom ( L ) = dom ( L ∗ ) .
Expression (138) is called the self-adjoint form of A \mathcal{A} A as, for functions f f f and g g g that satisfy appropriate boundary conditions (to be determined below) we have
⟨ f , A g ⟩ m = ⟨ A f , g ⟩ m , ⟨ f , g ⟩ m : = ∫ l r d x m ( x ) f ( x ) g ( x ) . \langle f,\mathcal{A}g\rangle_m=\langle\mathcal{A}f,g\rangle_m,\qquad\langle f,g\rangle_m:=\int_l^r \mathrm{d}x\,m(x)f(x)g(x). ⟨ f , A g ⟩ m = ⟨ A f , g ⟩ m , ⟨ f , g ⟩ m := ∫ l r d x m ( x ) f ( x ) g ( x ) .
To see this, simply note that
⟨ f , A g ⟩ m = ∫ I d x m ( x ) f ( x ) ⋅ 1 m ( x ) ∂ x ( 1 s ( x ) ∂ x g ( x ) ) = ∫ I d x f ( x ) ⋅ ∂ x ( 1 s ( x ) ∂ x g ( x ) ) = − ∫ I d x ( 1 s ( x ) ∂ x g ( x ) ) ⋅ ∂ x f ( x ) + [ f ( x ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) ] ∂ I = − ∫ I d x ( ∂ x g ( x ) ) ⋅ 1 s ( x ) ∂ x f ( x ) + [ f ( x ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) ] ∂ I = ∫ I d x g ( x ) ⋅ ∂ x ( 1 s ( x ) ∂ x f ( x ) ) + [ f ( x ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) − g ( x ) ⋅ ( 1 s ( x ) ∂ x f ( x ) ) ] ∂ I = ∫ I d x m ( x ) g ( x ) ⋅ 1 m ( x ) ∂ x ( 1 s ( x ) ∂ x f ( x ) ) + [ f ( x ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) − g ( x ) ⋅ ( 1 s ( x ) ∂ x f ( x ) ) ] ∂ I = ⟨ A f , g ⟩ m + [ f ( x ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) − g ( x ) ⋅ ( 1 s ( x ) ∂ x f ( x ) ) ] ∂ I . \begin{aligned}
\langle f,\mathcal{A}g\rangle_m
&=\int_I \mathrm{d}x\,m(x)f(x)\cdot\frac{1}{m(x)}\partial_x\left(\frac{1}{s(x)}\partial_x g(x)\right)\\
&=\int_I \mathrm{d}x\,f(x)\cdot\partial_x\left(\frac{1}{s(x)}\partial_x g(x)\right)\\
&=-\int_I \mathrm{d}x\,\left(\frac{1}{s(x)}\partial_xg(x)\right)\cdot\partial_xf(x)+\left[f(x)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)\right]_{\partial I}\\
&=-\int_I \mathrm{d}x\,\left(\partial_xg(x)\right)\cdot\frac{1}{s(x)}\partial_xf(x)+\left[f(x)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)\right]_{\partial I}\\
&=\int_I \mathrm{d}x\,g(x)\cdot\partial_x\left(\frac{1}{s(x)}\partial_xf(x)\right)+\left[f(x)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)-g(x)\cdot\left(\frac{1}{s(x)}\partial_xf(x)\right)\right]_{\partial I}\\
&=\int_I \mathrm{d}x\,m(x)g(x)\cdot\frac{1}{m(x)}\partial_x\left(\frac{1}{s(x)}\partial_xf(x)\right)+\left[f(x)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)-g(x)\cdot\left(\frac{1}{s(x)}\partial_xf(x)\right)\right]_{\partial I}\\
&=\langle\mathcal{A}f,g\rangle_m+\left[f(x)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)-g(x)\cdot\left(\frac{1}{s(x)}\partial_xf(x)\right)\right]_{\partial I}.
\end{aligned} ⟨ f , A g ⟩ m = ∫ I d x m ( x ) f ( x ) ⋅ m ( x ) 1 ∂ x ( s ( x ) 1 ∂ x g ( x ) ) = ∫ I d x f ( x ) ⋅ ∂ x ( s ( x ) 1 ∂ x g ( x ) ) = − ∫ I d x ( s ( x ) 1 ∂ x g ( x ) ) ⋅ ∂ x f ( x ) + [ f ( x ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) ] ∂ I = − ∫ I d x ( ∂ x g ( x ) ) ⋅ s ( x ) 1 ∂ x f ( x ) + [ f ( x ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) ] ∂ I = ∫ I d x g ( x ) ⋅ ∂ x ( s ( x ) 1 ∂ x f ( x ) ) + [ f ( x ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) − g ( x ) ⋅ ( s ( x ) 1 ∂ x f ( x ) ) ] ∂ I = ∫ I d x m ( x ) g ( x ) ⋅ m ( x ) 1 ∂ x ( s ( x ) 1 ∂ x f ( x ) ) + [ f ( x ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) − g ( x ) ⋅ ( s ( x ) 1 ∂ x f ( x ) ) ] ∂ I = ⟨ A f , g ⟩ m + [ f ( x ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) − g ( x ) ⋅ ( s ( x ) 1 ∂ x f ( x ) ) ] ∂ I .
Thus, we have
⟨ f , A g ⟩ m = ⟨ A f , g ⟩ m , assuming [ f ( x ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) − g ( x ) ⋅ ( 1 s ( x ) ∂ x f ( x ) ) ] ∂ I = 0. \langle f,\mathcal{A}g\rangle_m=\langle\mathcal{A}f,g\rangle_m,\qquad\text{assuming}\qquad\left[f(x)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)-g(x)\cdot\left(\frac{1}{s(x)}\partial_xf(x)\right)\right]_{\partial I}=0. ⟨ f , A g ⟩ m = ⟨ A f , g ⟩ m , assuming [ f ( x ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) − g ( x ) ⋅ ( s ( x ) 1 ∂ x f ( x ) ) ] ∂ I = 0.
If we require that A \mathcal{A} A acts only on functions h h h that satisfy
h ( l ) + α h ′ ( l ) s ( l ) = 0 , h ( r ) + γ h ′ ( r ) s ( r ) = 0 , for some constants α , γ ∈ R ∪ { ∞ } , (140) h(l)+\alpha\frac{h'(l)}{s(l)}=0,\qquad h(r)+\gamma\frac{h'(r)}{s(r)}=0,\qquad\text{for some constants }\alpha,\gamma\in\mathbb{R}\cup\{\infty\}, \tag*{(140)} h ( l ) + α s ( l ) h ′ ( l ) = 0 , h ( r ) + γ s ( r ) h ′ ( r ) = 0 , for some constants α , γ ∈ R ∪ { ∞ } , (140)
then the boundary terms will vanish because we have
f ( l ) g ′ ( l ) s ( l ) − g ( l ) f ′ ( l ) s ( l ) = f ( l ) g ( l ) ( g ′ ( l ) s ( l ) g ( l ) − f ′ ( l ) s ( l ) f ( l ) ) = f ( l ) g ( l ) ( − 1 α − − 1 α ) = 0 , f(l)\frac{g'(l)}{s(l)}-g(l)\frac{f'(l)}{s(l)}
=f(l)g(l)\left(\frac{g'(l)}{s(l)g(l)}-\frac{f'(l)}{s(l)f(l)}\right)
=f(l)g(l)\left(-\frac{1}{\alpha}-\frac{-1}{\alpha}\right)=0, f ( l ) s ( l ) g ′ ( l ) − g ( l ) s ( l ) f ′ ( l ) = f ( l ) g ( l ) ( s ( l ) g ( l ) g ′ ( l ) − s ( l ) f ( l ) f ′ ( l ) ) = f ( l ) g ( l ) ( − α 1 − α − 1 ) = 0 ,
and likewise at x = r x=r x = r . Thus, the operator A \mathcal{A} A with domain
dom ( A ) = { h ∈ L 2 ( I , m ) : A h ∈ L 2 ( I , m ) and (9.29) holds } . \operatorname{dom}(\mathcal{A})=\{h\in L^2(I,m):\mathcal{A}h\in L^2(I,m)\text{ and (9.29) holds}\}. dom ( A ) = { h ∈ L 2 ( I , m ) : A h ∈ L 2 ( I , m ) and (9.29) holds } .
is self-adjoint in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) . Note that A \mathcal{A} A is not self-adjoint in L 2 ( I , d x ) L^2(I,\mathrm{d}x) L 2 ( I , d x ) . Also note, when we talk about the operator A \mathcal{A} A we are really talking about the pair ( A , dom ( A ) ) (\mathcal{A},\operatorname{dom}(\mathcal{A})) ( A , dom ( A )) . The domain of A \mathcal{A} A includes boundary conditions that must be satisfied in order of A \mathcal{A} A to be self-adjoint in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) .
Boundary classification and boundary conditions
The endpoints l l l and r r r of the interval I I I fall in to one of four categories, which we describe below. In order to determine which of these four categories the end-points belong to we introduce S S S , the scale measure
S [ x , y ] : = ∫ x y d z s ( z ) , x , y ∈ ( l , r ) , S ( l , y ] = lim x ↘ l S [ x , y ] , S [ x , r ) = lim y ↗ r S [ x , y ] . S[x,y]:=\int_x^y \mathrm{d}z\,s(z),\qquad x,y\in(l,r),\qquad S(l,y]=\lim_{x\searrow l}S[x,y],\qquad S[x,r)=\lim_{y\nearrow r}S[x,y]. S [ x , y ] := ∫ x y d z s ( z ) , x , y ∈ ( l , r ) , S ( l , y ] = x ↘ l lim S [ x , y ] , S [ x , r ) = y ↗ r lim S [ x , y ] .
Next we introduce four useful integrals related to the scale measure and the speed density m m m given in (139) .
I l = ∫ l x d z m ( z ) S ( l , z ] , I r = ∫ x r d z m ( z ) S [ z , r ) , J l = ∫ l x d z m ( z ) S [ z , x ] , J r = ∫ x r d z m ( z ) S [ x , z ] . (141) \begin{aligned}
I_l &= \int_l^x \mathrm{d}z\,m(z)S(l,z], & I_r &= \int_x^r \mathrm{d}z\,m(z)S[z,r),\\
J_l &= \int_l^x \mathrm{d}z\,m(z)S[z,x], & J_r &= \int_x^r \mathrm{d}z\,m(z)S[x,z]. \tag*{(141)}
\end{aligned} I l J l = ∫ l x d z m ( z ) S ( l , z ] , = ∫ l x d z m ( z ) S [ z , x ] , I r J r = ∫ x r d z m ( z ) S [ z , r ) , = ∫ x r d z m ( z ) S [ x , z ] . (141)
Here, x x x is any point in the open interval ( l , r ) (l,r) ( l , r ) .
I l = ∫ l x d z m ( z ) S ( l , z ] , I r = ∫ x r d z m ( z ) S [ z , r ) , J l = ∫ l x d z m ( z ) S [ z , x ] , J r = ∫ x r d z m ( z ) S [ x , z ] . (142) \begin{aligned}
I_l &= \int_l^x \mathrm{d}z\,m(z)S(l,z], & I_r &= \int_x^r \mathrm{d}z\,m(z)S[z,r),\\
J_l &= \int_l^x \mathrm{d}z\,m(z)S[z,x], & J_r &= \int_x^r \mathrm{d}z\,m(z)S[x,z]. \tag*{(142)}
\end{aligned} I l J l = ∫ l x d z m ( z ) S ( l , z ] , = ∫ l x d z m ( z ) S [ z , x ] , I r J r = ∫ x r d z m ( z ) S [ z , r ) , = ∫ x r d z m ( z ) S [ x , z ] . (142)
Definition 9.6.2. Let X X X be a time-homogeneous scalar diffusion, which lives on an interval I I I with endpoints l l l and r r r . An endpoint l l l or r r r is said to be
regular if I < ∞ I < \infty I < ∞ and J < ∞ J < \infty J < ∞ .
exit if I < ∞ I < \infty I < ∞ and J = ∞ J = \infty J = ∞ .
entrance if I = ∞ I = \infty I = ∞ and J < ∞ J < \infty J < ∞ .
natural if I = ∞ I = \infty I = ∞ and J = ∞ J = \infty J = ∞ .
where I I I and J J J are given in (141) and (142) .
The definition above, while precise, gives us very little intuition as to what the four different boundary classifications mean. Thus, we elaborate a bit.
Regular boundary The process X X X can be started from a regular boundary and X X X can reach a regular boundary in finite time.
Exit boundary: The process X X X cannot be started from an exit boundary but X X X can reach an exit boundary in finite time. If X X X reaches an exit boundary, it does not return.
Entrance boundary: The process X X X can be started from an entrance boundary but X X X cannot reach an entrance boundary in finite time.
Natural boundary: The process X X X cannot be started from a natural boundary nor can X X X reach a natural boundary in finite time.
We can further classify the boundaries as follows:
Unattainable: I = ∞ I = \infty I = ∞ . Entrance and natural boundaries are unattainable. A diffusion starting in ( l , r ) (l,r) ( l , r ) cannot reach an unattainable boundary in finite time.
Attainable: I < ∞ I < \infty I < ∞ . Regular and exit boundaries are attainable. A a diffusion starting in ( l , r ) (l,r) ( l , r ) can reach an attainable boundary in finite time.
We must specify the behavior of X X X at a regular boundary. Different boundary behaviors correspond to different boundary conditions for dom ( A ) \operatorname{dom}(\mathcal{A}) dom ( A ) . The two most common behaviors are killing and reflecting.
If a regular boundary is specified as killing then the process X X X is killed as soon as it hits this boundary and it cannot return to the state space I I I . Thus, if l l l and r r r are regular killing boundaries we clearly have
P ( X T ∈ d y ∣ X t = l ) ≡ Γ ( t , l ; T , y ) d y = 0 , P ( X T ∈ d y ∣ X t = r ) ≡ Γ ( t , r ; T , y ) d y = 0. \mathbb{P}(X_T \in dy \mid X_t=l) \equiv\Gamma(t,l;T,y)dy=0,\qquad\mathbb{P}(X_T \in dy \mid X_t=r) \equiv\Gamma(t,r;T,y)dy=0. P ( X T ∈ d y ∣ X t = l ) ≡ Γ ( t , l ; T , y ) d y = 0 , P ( X T ∈ d y ∣ X t = r ) ≡ Γ ( t , r ; T , y ) d y = 0.
Since Γ ( t , ⋅ ; T , y ) ∈ dom ( A ) \Gamma(t,\cdot;T,y)\in\operatorname{dom}(\mathcal{A}) Γ ( t , ⋅ ; T , y ) ∈ dom ( A ) , we clearly want A \mathcal{A} A to act on functions f f f that satisfy f ( l ) = f ( r ) = 0 f(l)=f(r)=0 f ( l ) = f ( r ) = 0 .
If a regular boundary is specified as reflecting , then the process X X X is instantaneously reflected back into I I I if it hits this boundary. Thus, if l l l and r r r are regular reflecting boundaries we clearly have
1 = ∫ I d y Γ ( t , x ; T , y ) , 1=\int_I dy\,\Gamma(t,x;T,y), 1 = ∫ I d y Γ ( t , x ; T , y ) ,
0 = ∫ I d y ∂ T Γ ( t , x ; T , y ) = ∫ I d y A ∗ Γ ( t , x ; T , y ) = ∫ I d y ∂ y ( − μ ( y ) Γ ( t , x ; T , y ) + 1 2 ∂ y σ 2 ( y ) Γ ( t , x ; T , y ) ) = [ − μ ( y ) Γ ( t , x ; T , y ) + 1 2 ∂ y σ 2 ( y ) Γ ( t , x ; T , y ) ] ∂ I = [ − μ ( y ) m ( y ) Γ ( t , x ; T , y ) m ( y ) + 1 2 ∂ y σ 2 ( y ) m ( y ) Γ ( t , x ; T , y ) m ( y ) ] ∂ I = [ − 2 μ ( y ) σ 2 ( y ) exp ( ∫ d y 2 μ ( y ) σ 2 ( y ) ) Γ ( t , x ; T , y ) m ( y ) + ∂ y exp ( ∫ d y 2 μ ( y ) σ 2 ( y ) ) Γ ( t , x ; T , y ) m ( y ) ] ∂ I = [ exp ( ∫ d y 2 μ ( y ) σ 2 ( y ) ) ∂ y Γ ( t , x ; T , y ) m ( y ) ] ∂ I = [ 1 s ( y ) ∂ y Γ ( t , x ; T , y ) m ( y ) ] ∂ I . \begin{aligned}
0=\int_I dy\,\partial_T\Gamma(t,x;T,y) \\
&=\int_I dy\,\mathcal{A}^{*}\Gamma(t,x;T,y) \\
&=\int_I dy\,\partial_y\left(-\mu(y)\Gamma(t,x;T,y)+\frac{1}{2}\partial_y\sigma^2(y)\Gamma(t,x;T,y)\right) \\
&=\left[-\mu(y)\Gamma(t,x;T,y)+\frac{1}{2}\partial_y\sigma^2(y)\Gamma(t,x;T,y)\right]_{\partial I} \\
&=\left[-\mu(y)m(y)\frac{\Gamma(t,x;T,y)}{m(y)}+\frac{1}{2}\partial_y\sigma^2(y)m(y)\frac{\Gamma(t,x;T,y)}{m(y)}\right]_{\partial I} \\
&=\left[-\frac{2\mu(y)}{\sigma^2(y)}\exp\left(\int dy\frac{2\mu(y)}{\sigma^2(y)}\right)\frac{\Gamma(t,x;T,y)}{m(y)}+\partial_y\exp\left(\int dy\frac{2\mu(y)}{\sigma^2(y)}\right)\frac{\Gamma(t,x;T,y)}{m(y)}\right]_{\partial I} \\
&=\left[\exp\left(\int dy\frac{2\mu(y)}{\sigma^2(y)}\right)\partial_y\frac{\Gamma(t,x;T,y)}{m(y)}\right]_{\partial I} \\
&=\left[\frac{1}{s(y)}\partial_y\frac{\Gamma(t,x;T,y)}{m(y)}\right]_{\partial I}.
\end{aligned} 0 = ∫ I d y ∂ T Γ ( t , x ; T , y ) = ∫ I d y A ∗ Γ ( t , x ; T , y ) = ∫ I d y ∂ y ( − μ ( y ) Γ ( t , x ; T , y ) + 2 1 ∂ y σ 2 ( y ) Γ ( t , x ; T , y ) ) = [ − μ ( y ) Γ ( t , x ; T , y ) + 2 1 ∂ y σ 2 ( y ) Γ ( t , x ; T , y ) ] ∂ I = [ − μ ( y ) m ( y ) m ( y ) Γ ( t , x ; T , y ) + 2 1 ∂ y σ 2 ( y ) m ( y ) m ( y ) Γ ( t , x ; T , y ) ] ∂ I = [ − σ 2 ( y ) 2 μ ( y ) exp ( ∫ d y σ 2 ( y ) 2 μ ( y ) ) m ( y ) Γ ( t , x ; T , y ) + ∂ y exp ( ∫ d y σ 2 ( y ) 2 μ ( y ) ) m ( y ) Γ ( t , x ; T , y ) ] ∂ I = [ exp ( ∫ d y σ 2 ( y ) 2 μ ( y ) ) ∂ y m ( y ) Γ ( t , x ; T , y ) ] ∂ I = [ s ( y ) 1 ∂ y m ( y ) Γ ( t , x ; T , y ) ] ∂ I .
Since Γ ( t , x ; T , ⋅ ) ∈ dom ( A ∗ ) \Gamma(t,x;T,\cdot)\in\operatorname{dom}(\mathcal{A}^{*}) Γ ( t , x ; T , ⋅ ) ∈ dom ( A ∗ ) , it follows that A ∗ \mathcal{A}^{*} A ∗ acts on functions f f f that satisfy ∂ y ( f ( y ) / m ( y ) ) = 0 \partial_y(f(y)/m(y))=0 ∂ y ( f ( y ) / m ( y )) = 0 at y = l y=l y = l and y = r y=r y = r . But we are interested in the domain of A \mathcal{A} A . Recall that A ∗ \mathcal{A}^{*} A ∗ is the L 2 ( I , d x ) L^2(I,dx) L 2 ( I , d x ) adjoint of A \mathcal{A} A – not the L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) adjoint of A \mathcal{A} A . Thus, we seek boundary conditions for A \mathcal{A} A such that ⟨ f , A g ⟩ = ⟨ A ∗ f , g ⟩ \langle f,\mathcal{A}g\rangle=\langle\mathcal{A}^{*}f,g\rangle ⟨ f , A g ⟩ = ⟨ A ∗ f , g ⟩ . Note that
⟨ f , A g ⟩ = ∫ I d x f ( x ) 1 m ( x ) ∂ x ( 1 s ( x ) ∂ x g ( x ) ) = − ∫ I d x ( ∂ x f ( x ) m ( x ) ) ⋅ ( 1 s ( x ) ∂ x g ( x ) ) + [ f ( x ) m ( x ) 1 s ( x ) ∂ x g ( x ) ] ∂ I = ∫ I d x ∂ x ( 1 s ( x ) ∂ x f ( x ) m ( x ) ) ⋅ g ( x ) + [ f ( x ) m ( x ) 1 s ( x ) ∂ x g ( x ) − g ( x ) s ( x ) ∂ x f ( x ) m ( x ) ] ∂ I = ⟨ A ∗ f , g ⟩ + [ f ( x ) m ( x ) 1 s ( x ) ∂ x g ( x ) − g ( x ) s ( x ) ∂ x f ( x ) m ( x ) ] ∂ I . \begin{aligned}
\langle f,\mathcal{A}g\rangle=\int_I dx\,f(x)\frac{1}{m(x)}\partial_x\left(\frac{1}{s(x)}\partial_xg(x)\right) \\
&=-\int_I dx\left(\partial_x\frac{f(x)}{m(x)}\right)\cdot\left(\frac{1}{s(x)}\partial_xg(x)\right)+\left[\frac{f(x)}{m(x)}\frac{1}{s(x)}\partial_xg(x)\right]_{\partial I} \\
&=\int_I dx\,\partial_x\left(\frac{1}{s(x)}\partial_x\frac{f(x)}{m(x)}\right)\cdot g(x)+\left[\frac{f(x)}{m(x)}\frac{1}{s(x)}\partial_xg(x)-\frac{g(x)}{s(x)}\partial_x\frac{f(x)}{m(x)}\right]_{\partial I} \\
&=\langle\mathcal{A}^{*}f,g\rangle+\left[\frac{f(x)}{m(x)}\frac{1}{s(x)}\partial_xg(x)-\frac{g(x)}{s(x)}\partial_x\frac{f(x)}{m(x)}\right]_{\partial I}.
\end{aligned} ⟨ f , A g ⟩ = ∫ I d x f ( x ) m ( x ) 1 ∂ x ( s ( x ) 1 ∂ x g ( x ) ) = − ∫ I d x ( ∂ x m ( x ) f ( x ) ) ⋅ ( s ( x ) 1 ∂ x g ( x ) ) + [ m ( x ) f ( x ) s ( x ) 1 ∂ x g ( x ) ] ∂ I = ∫ I d x ∂ x ( s ( x ) 1 ∂ x m ( x ) f ( x ) ) ⋅ g ( x ) + [ m ( x ) f ( x ) s ( x ) 1 ∂ x g ( x ) − s ( x ) g ( x ) ∂ x m ( x ) f ( x ) ] ∂ I = ⟨ A ∗ f , g ⟩ + [ m ( x ) f ( x ) s ( x ) 1 ∂ x g ( x ) − s ( x ) g ( x ) ∂ x m ( x ) f ( x ) ] ∂ I .
Since
[ ∂ x f ( x ) m ( x ) ] ∂ I = 0 , ∀ f ∈ dom ( A ∗ ) , \left[\partial_x\frac{f(x)}{m(x)}\right]_{\partial I}=0,\qquad\forall f\in\operatorname{dom}(\mathcal{A}^*), [ ∂ x m ( x ) f ( x ) ] ∂ I = 0 , ∀ f ∈ dom ( A ∗ ) ,
in order for ⟨ f , A g ⟩ = ⟨ A ∗ f , g ⟩ \langle f,\mathcal{A}g\rangle=\langle\mathcal{A}^*f,g\rangle ⟨ f , A g ⟩ = ⟨ A ∗ f , g ⟩ , we must have
[ 1 s ( x ) ∂ x g ( x ) ] ∂ I = 0 , ∀ g ∈ dom ( A ) . \left[\frac{1}{s(x)}\partial_xg(x)\right]_{\partial I}=0,\qquad\forall g\in\operatorname{dom}(\mathcal{A}). [ s ( x ) 1 ∂ x g ( x ) ] ∂ I = 0 , ∀ g ∈ dom ( A ) .
To summarize: If the endpoints of I I I are regular, we must specify if the boundary is killing or reflecting. The correct boundary conditions to impose for the generator A \mathcal{A} A are
killing: [ f ( x ) ] ∂ I = 0 , reflecting: [ 1 s ( x ) ∂ x f ( x ) ] ∂ I = 0. \text{killing:}\qquad[f(x)]_{\partial I}=0,\qquad
\text{reflecting:}\qquad\left[\frac{1}{s(x)}\partial_xf(x)\right]_{\partial I}=0. killing: [ f ( x ) ] ∂ I = 0 , reflecting: [ s ( x ) 1 ∂ x f ( x ) ] ∂ I = 0.
Although we will not derive it, the dom ( A ) \operatorname{dom}(\mathcal{A}) dom ( A ) should also satisfy certain boundary conditions at exit and entrance boundaries
exit: [ f ( x ) ] ∂ I = 0 , entrance: [ 1 s ( x ) ∂ x f ( x ) ] ∂ I = 0. \text{exit:}\qquad[f(x)]_{\partial I}=0,\qquad
\text{entrance:}\qquad\left[\frac{1}{s(x)}\partial_xf(x)\right]_{\partial I}=0. exit: [ f ( x ) ] ∂ I = 0 , entrance: [ s ( x ) 1 ∂ x f ( x ) ] ∂ I = 0.
No boundary conditions are required for natural boundaries.
Eigenfunction expansions
Consider the following eigenvalue equation
A ψ n = λ n ψ n , ψ n ∈ dom ( A ) . \mathcal{A}\psi_n=\lambda_n\psi_n,\qquad\psi_n\in\operatorname{dom}(\mathcal{A}). A ψ n = λ n ψ n , ψ n ∈ dom ( A ) .
Here, and throughout this subsection, we have implicitly assumed that the spectrum of A \mathcal{A} A is discrete and simple (i.e., no repeated eigenvalues).
Theorem 9.6.3. Eigenfunctions corresponding to different eigenvalues of A \mathcal{A} A are orthogonal in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) .
Proof. Let n ≠ k n\ne k n = k . Then
λ n ⟨ ψ n , ψ k ⟩ m = ⟨ A ψ n , ψ k ⟩ m = ⟨ ψ n , A ψ k ⟩ m = λ k ⟨ ψ n , ψ k ⟩ m . \lambda_n\langle\psi_n,\psi_k\rangle_m=\langle\mathcal{A}\psi_n,\psi_k\rangle_m=\langle\psi_n,\mathcal{A}\psi_k\rangle_m=\lambda_k\langle\psi_n,\psi_k\rangle_m. λ n ⟨ ψ n , ψ k ⟩ m = ⟨ A ψ n , ψ k ⟩ m = ⟨ ψ n , A ψ k ⟩ m = λ k ⟨ ψ n , ψ k ⟩ m .
It follows that ⟨ ψ n , ψ k ⟩ m = 0 \langle\psi_n,\psi_k\rangle_m=0 ⟨ ψ n , ψ k ⟩ m = 0 .
Theorem 9.6.4. Suppose the eigenfunctions of A \mathcal{A} A are complete in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) and ϕ ∈ L 2 ( I , m ) \phi\in L^2(I,m) ϕ ∈ L 2 ( I , m ) . Then the function u ( t , x ) : = E [ ϕ ( X T ) ∣ X t = x ] u(t,x):=\mathbb{E}[\phi(X_T)\mid X_t=x] u ( t , x ) := E [ ϕ ( X T ) ∣ X t = x ] is given by
u ( t , x ) = ∑ n e ( T − t ) λ n ⟨ ψ n , ϕ ⟩ m ψ n ( x ) , (143) u(t,x)=\sum_n e^{(T-t)\lambda_n}\langle\psi_n,\phi\rangle_m\psi_n(x), \tag*{(143)} u ( t , x ) = n ∑ e ( T − t ) λ n ⟨ ψ n , ϕ ⟩ m ψ n ( x ) , (143) where A ψ n = λ n ψ n \mathcal{A}\psi_n=\lambda_n\psi_n A ψ n = λ n ψ n , the eigenfunctions are normalized (i.e., ⟨ ψ n , ψ n ⟩ m = 1 \langle\psi_n,\psi_n\rangle_m=1 ⟨ ψ n , ψ n ⟩ m = 1 for all n n n ) and m m m is the speed measure of A \mathcal{A} A .
Proof. We must show that the function u u u satisfies the KBE. We have
( ∂ t + A ) u ( t , x ) = ∑ n ⟨ ψ n , ϕ ⟩ m ( ∂ t e ( T − t ) λ n ψ n ( x ) + e ( T − t ) λ n A ψ n ( x ) ) = ∑ n ⟨ ψ n , ϕ ⟩ m ( − λ n e ( T − t ) λ n ψ n ( x ) + e ( T − t ) λ n λ n ψ n ( x ) ) = 0. \begin{aligned}
(\partial_t+\mathcal{A})u(t,x)=\sum_n\langle\psi_n,\phi\rangle_m\left(\partial_t e^{(T-t)\lambda_n}\psi_n(x)+e^{(T-t)\lambda_n}\mathcal{A}\psi_n(x)\right) \\
&=\sum_n\langle\psi_n,\phi\rangle_m\left(-\lambda_n e^{(T-t)\lambda_n}\psi_n(x)+e^{(T-t)\lambda_n}\lambda_n\psi_n(x)\right)=0.
\end{aligned} ( ∂ t + A ) u ( t , x ) = n ∑ ⟨ ψ n , ϕ ⟩ m ( ∂ t e ( T − t ) λ n ψ n ( x ) + e ( T − t ) λ n A ψ n ( x ) ) = n ∑ ⟨ ψ n , ϕ ⟩ m ( − λ n e ( T − t ) λ n ψ n ( x ) + e ( T − t ) λ n λ n ψ n ( x ) ) = 0.
u ( T , x ) = ∑ n ⟨ ψ n , ϕ ⟩ m ψ n ( x ) = ϕ ( x ) . u(T,x)=\sum_n\langle\psi_n,\phi\rangle_m\psi_n(x)=\phi(x). u ( T , x ) = n ∑ ⟨ ψ n , ϕ ⟩ m ψ n ( x ) = ϕ ( x ) .
where, to establish the terminal condition u ( T , x ) = ϕ ( x ) u(T,x)=\phi(x) u ( T , x ) = ϕ ( x ) we have used the fact that the eigenfunctions ( ψ n ) n ≥ 0 (\psi_n)_{n\geq0} ( ψ n ) n ≥ 0 are complete in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) .
Corollary 9.6.5. Under the conditions of Theorem 9.6.4 , the transition density Γ ( t , x ; T , y ) \Gamma(t,x;T,y) Γ ( t , x ; T , y ) of X X X has the following eigenfunction expansion
Γ ( t , x ; T , y ) = m ( y ) ∑ n e ( T − t ) λ n ψ n ( y ) ψ n ( x ) . (144) \Gamma(t,x;T,y)=m(y)\sum_n e^{(T-t)\lambda_n}\psi_n(y)\psi_n(x). \tag*{(144)} Γ ( t , x ; T , y ) = m ( y ) n ∑ e ( T − t ) λ n ψ n ( y ) ψ n ( x ) . (144)
Proof. The transition density Γ ( t , x ; T , y ) \Gamma(t,x;T,y) Γ ( t , x ; T , y ) satisfies the KBE with a terminal condition Γ ( T , x ; T , y ) = δ y ( x ) \Gamma(T,x;T,y)=\delta_y(x) Γ ( T , x ; T , y ) = δ y ( x ) . Setting ϕ = δ y \phi=\delta_y ϕ = δ y in (143) , we obtain
Γ ( t , x ; T , y ) = ∑ n e ( T − t ) λ n ⟨ ψ n , δ y ⟩ m ψ n ( x ) = m ( y ) ∑ n e ( T − t ) λ n ψ n ( y ) ψ n ( x ) , \begin{aligned}
\Gamma(t,x;T,y)=\sum_n e^{(T-t)\lambda_n}\langle\psi_n,\delta_y\rangle_m\psi_n(x) \\
&=m(y)\sum_n e^{(T-t)\lambda_n}\psi_n(y)\psi_n(x),
\end{aligned} Γ ( t , x ; T , y ) = n ∑ e ( T − t ) λ n ⟨ ψ n , δ y ⟩ m ψ n ( x ) = m ( y ) n ∑ e ( T − t ) λ n ψ n ( y ) ψ n ( x ) ,
where we have used
⟨ ψ n , δ y ⟩ m = ∫ I d x ψ n ( x ) δ y ( x ) m ( x ) = ψ n ( y ) m ( y ) . \langle\psi_n,\delta_y\rangle_m=\int_I \mathrm{d}x\,\psi_n(x)\delta_y(x)m(x)=\psi_n(y)m(y). ⟨ ψ n , δ y ⟩ m = ∫ I d x ψ n ( x ) δ y ( x ) m ( x ) = ψ n ( y ) m ( y ) .
Example 9.6.6 (Brownian motion in a finite interval). Consider a diffusion X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 that lives on a finite interval ( l , r ) (l,r) ( l , r ) and satisfies the SDE
d X t = d W t . \mathrm{d}X_t=\mathrm{d}W_t. d X t = d W t . We identify the drift μ ( x ) = 0 \mu(x)=0 μ ( x ) = 0 and the diffusion coefficient σ ( x ) = 1 \sigma(x)=1 σ ( x ) = 1 . The generator A \mathcal{A} A of X X X and the speed density m m m are given by
A = 1 2 ∂ x 2 , m ( x ) = 1 r − l . \mathcal{A}=\frac{1}{2}\partial_x^2,\qquad m(x)=\frac{1}{r-l}. A = 2 1 ∂ x 2 , m ( x ) = r − l 1 . One can easily check that the endpoints l l l and r r r are regular. As such, we must specify the behavior at the endpoints as either killing or reflecting. In the case of killing boundaries we have
killing: A ψ n = λ n ψ n , ψ n ( l ) = ψ n ( r ) = 0 , \text{killing:}\qquad\mathcal{A}\psi_n=\lambda_n\psi_n,\qquad\psi_n(l)=\psi_n(r)=0, killing: A ψ n = λ n ψ n , ψ n ( l ) = ψ n ( r ) = 0 ,
ψ n ( x ) = 2 sin ( n π r − l ( x − l ) ) , λ n = − 1 2 ( n π r − l ) 2 , n = 1 , 2 , … , \psi_n(x)=\sqrt{2}\sin\left(\frac{n\pi}{r-l}(x-l)\right), \qquad\lambda_n=-\frac{1}{2}\left(\frac{n\pi}{r-l}\right)^2, \qquad n=1,2,\ldots, ψ n ( x ) = 2 sin ( r − l nπ ( x − l ) ) , λ n = − 2 1 ( r − l nπ ) 2 , n = 1 , 2 , … ,
where we have normalized the eigenfunctions so that ⟨ ψ n , ψ k ⟩ m = δ n , k \langle\psi_n,\psi_k\rangle_m=\delta_{n,k} ⟨ ψ n , ψ k ⟩ m = δ n , k . In the case of reflecting boundaries we have
reflecting: A ψ n = λ n ψ n , ψ n ′ ( l ) = ψ n ′ ( r ) = 0 , \text{reflecting:}\qquad\mathcal{A}\psi_n=\lambda_n\psi_n,\qquad\psi_n'(l)=\psi_n'(r)=0, reflecting: A ψ n = λ n ψ n , ψ n ′ ( l ) = ψ n ′ ( r ) = 0 ,
ψ n ( x ) = 2 cos ( n π r − l ( x − l ) ) , λ n = − 1 2 ( n π r − l ) 2 , n = 0 , 1 , 2 , … , \psi_n(x)=\sqrt{2}\cos\left(\frac{n\pi}{r-l}(x-l)\right), \qquad\lambda_n=-\frac{1}{2}\left(\frac{n\pi}{r-l}\right)^2, \qquad n=0,1,2,\ldots, ψ n ( x ) = 2 cos ( r − l nπ ( x − l ) ) , λ n = − 2 1 ( r − l nπ ) 2 , n = 0 , 1 , 2 , … ,
where, again, we have normalized the eigenfunctions. One can immediately write down the transition density Γ ( t , x ; T , y ) \Gamma(t,x;T,y) Γ ( t , x ; T , y ) using (144) . Of course, the sin and cos expansions should be familiar to anybody that has solved the heat equation on a finite domain.
Example 9.6.7 (Ornstein-Uhlenbeck process). Consider now an OU process
d X t = − X t d t + d W t , \mathrm{d}X_t=-X_t\mathrm{d}t+\mathrm{d}W_t, d X t = − X t d t + d W t , which has a state space I = R I=\mathbb{R} I = R . We identify the drift μ ( x ) = − x \mu(x)=-x μ ( x ) = − x and volatility coefficient σ ( x ) = 1 \sigma(x)=1 σ ( x ) = 1 . The generator and speed density of X X X are
A = − x ∂ x + 1 2 ∂ x 2 , m ( x ) = e − x 2 . \mathcal{A}=-x\partial_x+\frac{1}{2}\partial_x^2,\qquad m(x)=e^{-x^2}. A = − x ∂ x + 2 1 ∂ x 2 , m ( x ) = e − x 2 . One can easily verify that the endpoints l = − ∞ l=-\infty l = − ∞ and r = + ∞ r=+\infty r = + ∞ are natural. As such, we do not need to specify any boundary conditions. The eigenfunctions and eigenvalues of A \mathcal{A} A are given by
ψ n ( x ) = H n ( x ) , λ n = − n , n = 0 , 1 , 2 , … , \psi_n(x)=H_n(x),\qquad\lambda_n=-n,\qquad n=0,1,2,\ldots, ψ n ( x ) = H n ( x ) , λ n = − n , n = 0 , 1 , 2 , … , where ( H n ) n ≥ 0 (H_n)_{n\geq0} ( H n ) n ≥ 0 are the Hermite polynomials, properly normalized so that ⟨ ψ n , ψ k ⟩ m = δ n , k \langle\psi_n,\psi_k\rangle_m=\delta_{n,k} ⟨ ψ n , ψ k ⟩ m = δ n , k .
Heuristic computations
We have used the fact that A \mathcal{A} A is self-adjoint in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) in order to derive to write an eigenfunction expansion for u ( t , x ) = E [ ϕ ( X T ) ∣ X t = x ] u(t,x)=\mathbb{E}[\phi(X_T)\mid X_t=x] u ( t , x ) = E [ ϕ ( X T ) ∣ X t = x ] , the solution of the KBE. In fact, this is a special case of a more general method of solving PDEs and ODEs involving a self-adjoint operator A \mathcal{A} A . In what follows, we continue to assume that the generator A \mathcal{A} A of a diffusion X X X has a discrete spectrum, and that the eigenfunctions of A \mathcal{A} A are a complete basis in L 2 ( I , m ) L^2(I,m) L 2 ( I , m ) .
For any Borel-measurable function g : R → R g:\mathbb{R}\to\mathbb{R} g : R → R we define the operator
g ( A ) ⋅ = ∑ n g ( λ n ) ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( g ( A ) ) : = { f ∈ L 2 ( I , m ) : ∑ n g 2 ( λ n ) ⟨ ψ n , f ⟩ m 2 < ∞ } . (145) g(\mathcal{A})\cdot=\sum_n g(\lambda_n)\langle\psi_n,\cdot\rangle_m\psi_n,\qquad\operatorname{dom}(g(\mathcal{A})):=\left\{f\in L^2(I,m):\sum_n g^2(\lambda_n)\langle\psi_n,f\rangle_m^2<\infty\right\}. \tag*{(145)} g ( A ) ⋅ = n ∑ g ( λ n ) ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( g ( A )) := { f ∈ L 2 ( I , m ) : n ∑ g 2 ( λ n ) ⟨ ψ n , f ⟩ m 2 < ∞ } . (145)
We call (145) the eigenfunction or spectral representation of g ( A ) g(\mathcal{A}) g ( A ) . For example, taking g ( λ ) = 1 g(\lambda)=1 g ( λ ) = 1 gives the spectral representation of the identity operator
I ⋅ = ∑ n ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( I ) : = { f ∈ L 2 ( I , m ) : ∑ n ⟨ ψ n , f ⟩ m 2 < ∞ } . \mathbf{I}\cdot=\sum_n\langle\psi_n,\cdot\rangle_m\psi_n,\qquad\operatorname{dom}(\mathbf{I}):=\{f\in L^2(I,m):\sum_n\langle\psi_n,f\rangle_m^2<\infty\}. I ⋅ = n ∑ ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( I ) := { f ∈ L 2 ( I , m ) : n ∑ ⟨ ψ n , f ⟩ m 2 < ∞ } .
Similarly, taking g ( λ ) = ( λ − μ ) − 1 g(\lambda)=(\lambda-\mu)^{-1} g ( λ ) = ( λ − μ ) − 1 and defining R μ : = g ( A ) \mathcal{R}_\mu:=g(\mathcal{A}) R μ := g ( A ) gives
R μ ⋅ = ∑ n 1 λ n − μ ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( R μ ) : = { f ∈ L 2 ( I , m ) : ∑ n ( 1 λ n − μ ) 2 ⟨ ψ n , f ⟩ m 2 < ∞ } . \mathcal{R}_\mu\cdot=\sum_n\frac{1}{\lambda_n-\mu}\langle\psi_n,\cdot\rangle_m\psi_n,\qquad\operatorname{dom}(\mathcal{R}_\mu):=\{f\in L^2(I,m):\sum_n\left(\frac{1}{\lambda_n-\mu}\right)^2\langle\psi_n,f\rangle_m^2<\infty\}. R μ ⋅ = n ∑ λ n − μ 1 ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( R μ ) := { f ∈ L 2 ( I , m ) : n ∑ ( λ n − μ 1 ) 2 ⟨ ψ n , f ⟩ m 2 < ∞ } .
and taking g ( λ ) = e ( T − t ) λ g(\lambda)=e^{(\mathrm{T}-t)\lambda} g ( λ ) = e ( T − t ) λ and defining P ( t , T ) : = g ( A ) \mathcal{P}(t,\mathrm{T}):=g(\mathcal{A}) P ( t , T ) := g ( A ) gives
P ( t , T ) ⋅ = ∑ n e ( T − t ) λ n ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( P ( t , T ) ) : = { f ∈ L 2 ( I , m ) : ∑ n e ( T − t ) λ n ⟨ ψ n , f ⟩ m 2 < ∞ } . \mathcal{P}(t,\mathrm{T})\cdot=\sum_n e^{(\mathrm{T}-t)\lambda_n}\langle\psi_n,\cdot\rangle_m\psi_n,\qquad\operatorname{dom}(\mathcal{P}(t,\mathrm{T})):=\{f\in L^2(I,m):\sum_n e^{(\mathrm{T}-t)\lambda_n}\langle\psi_n,f\rangle_m^2<\infty\}. P ( t , T ) ⋅ = n ∑ e ( T − t ) λ n ⟨ ψ n , ⋅ ⟩ m ψ n , dom ( P ( t , T )) := { f ∈ L 2 ( I , m ) : n ∑ e ( T − t ) λ n ⟨ ψ n , f ⟩ m 2 < ∞ } .
Note that P ( t , T ) \mathcal{P}(t,\mathrm{T}) P ( t , T ) as defined above is in fact the semigroup operator we introduced previously. To see this note that
P ( t , T ) φ ( x ) = ∑ n e ( T − t ) λ n ⟨ ψ n , φ ⟩ m ψ n ( x ) = ∑ n e ( T − t ) λ n ∫ I d y ψ n ( y ) φ ( y ) m ( y ) ψ n ( x ) = ∫ I d y ( m ( y ) ∑ n e ( T − t ) λ n ψ n ( y ) ψ n ( x ) ) φ ( y ) = ∫ I d y Γ ( t , x ; T , y ) φ ( y ) , \begin{aligned}
\mathcal{P}(t,\mathrm{T})\varphi(x)&=\sum_n e^{(\mathrm{T}-t)\lambda_n}\langle\psi_n,\varphi\rangle_m\psi_n(x)\\
&=\sum_n e^{(\mathrm{T}-t)\lambda_n}\int_I dy\,\psi_n(y)\varphi(y)m(y)\psi_n(x)\\
&=\int_I dy\left(m(y)\sum_n e^{(\mathrm{T}-t)\lambda_n}\psi_n(y)\psi_n(x)\right)\varphi(y)\\
&=\int_I dy\,\Gamma(t,x;\mathrm{T},y)\varphi(y),
\end{aligned} P ( t , T ) φ ( x ) = n ∑ e ( T − t ) λ n ⟨ ψ n , φ ⟩ m ψ n ( x ) = n ∑ e ( T − t ) λ n ∫ I d y ψ n ( y ) φ ( y ) m ( y ) ψ n ( x ) = ∫ I d y ( m ( y ) n ∑ e ( T − t ) λ n ψ n ( y ) ψ n ( x ) ) φ ( y ) = ∫ I d y Γ ( t , x ; T , y ) φ ( y ) ,
where we have used (144) .
The usefulness in defining operators of the form g ( A ) g(\mathcal{A}) g ( A ) is as follows. Consider a linear ODE or PDE for a function u u u involving the operator A \mathcal{A} A . One can obtain a solution to this ODE or PDE as follows:
Solve the ODE or PDE assuming A \mathcal{A} A is a constant; the solution will involve terms of the form g ( A ) f g(\mathcal{A})f g ( A ) f .
Replace g ( A ) f g(\mathcal{A})f g ( A ) f with its spectral representation.
Let us see how this works with a few examples.
Example 9.6.8. Let us check that the above two-step method works for the KBE. We want to solve
( ∂ t + A ) u ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ , (146) (\partial_t+\mathcal{A})u(t,\cdot)=0,\qquad u(\mathrm{T},\cdot)=\varphi, \tag*{(146)} ( ∂ t + A ) u ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ , (146) where φ ∈ L 2 ( I , m ) \varphi\in L^2(I,m) φ ∈ L 2 ( I , m ) . Treating A \mathcal{A} A like a constant, we have an ODE in t t t . Solving this ODE and then using the spectral representation any expression involving A \mathcal{A} A we obtain
u ( t , ⋅ ) = e ( T − t ) A φ = ∑ n e ( T − t ) λ n ⟨ ψ n , φ ⟩ m ψ n , u(t,\cdot)=e^{(\mathrm{T}-t)\mathcal{A}}\varphi=\sum_n e^{(\mathrm{T}-t)\lambda_n}\langle\psi_n,\varphi\rangle_m\psi_n, u ( t , ⋅ ) = e ( T − t ) A φ = n ∑ e ( T − t ) λ n ⟨ ψ n , φ ⟩ m ψ n , which agrees with the expression given in (143) . Noting that e ( T − t ) A = P ( t , T ) e^{(\mathrm{T}-t)\mathcal{A}}=\mathcal{P}(t,\mathrm{T}) e ( T − t ) A = P ( t , T ) we can write the solution to (146) compactly as u ( t , ⋅ ) = P ( t , T ) φ u(t,\cdot)=\mathcal{P}(t,\mathrm{T})\varphi u ( t , ⋅ ) = P ( t , T ) φ .
Example 9.6.9. Consider the following PDE involving A \mathcal{A} A
( ∂ t + A ) u ( t , ⋅ ) + g ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ (147) (\partial_t+\mathcal{A})u(t,\cdot)+g(t,\cdot)=0,\qquad u(T,\cdot)=\varphi \tag*{(147)} ( ∂ t + A ) u ( t , ⋅ ) + g ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ (147)
where φ ∈ L 2 ( I , m ) \varphi\in L^2(I,m) φ ∈ L 2 ( I , m ) and g ( s , ⋅ ) ∈ L 2 ( I , m ) g(s,\cdot)\in L^2(I,m) g ( s , ⋅ ) ∈ L 2 ( I , m ) for all s ∈ [ 0 , T ] s\in[0,T] s ∈ [ 0 , T ] . Treating A \mathcal{A} A as a constant, we have an ODE in t t t . The solution is given by
u ( t , ⋅ ) = e ( T − t ) A φ + ∫ t T d s e ( s − t ) A g ( s , ⋅ ) . u(t,\cdot)=e^{(T-t)\mathcal{A}}\varphi+\int_t^T\mathrm{d}s\,e^{(s-t)\mathcal{A}}g(s,\cdot). u ( t , ⋅ ) = e ( T − t ) A φ + ∫ t T d s e ( s − t ) A g ( s , ⋅ ) .
Using the spectral representation of e ( s − t ) A e^{(s-t)\mathcal{A}} e ( s − t ) A we obtain
u ( t , ⋅ ) = ∑ n e ( T − t ) λ n ⟨ ψ n , φ ⟩ m ψ n + ∫ t T d s ∑ n e ( s − t ) λ n ⟨ ψ n , g ( s , ⋅ ) ⟩ m ψ n . u(t,\cdot)=\sum_n e^{(T-t)\lambda_n}\langle\psi_n,\varphi\rangle_m\psi_n+\int_t^T\mathrm{d}s\sum_n e^{(s-t)\lambda_n}\langle\psi_n,g(s,\cdot)\rangle_m\psi_n. u ( t , ⋅ ) = n ∑ e ( T − t ) λ n ⟨ ψ n , φ ⟩ m ψ n + ∫ t T d s n ∑ e ( s − t ) λ n ⟨ ψ n , g ( s , ⋅ ) ⟩ m ψ n .
Again, recognizing that e ( s − t ) A = P ( t , s ) e^{(s-t)\mathcal{A}}=\mathcal{P}(t,s) e ( s − t ) A = P ( t , s ) we can write the solution of (147) as
u ( t , ⋅ ) = P ( t , T ) φ + ∫ t T d s P ( t , s ) g ( s , ⋅ ) . u(t,\cdot)=\mathcal{P}(t,T)\varphi+\int_t^T\mathrm{d}s\,\mathcal{P}(t,s)g(s,\cdot). u ( t , ⋅ ) = P ( t , T ) φ + ∫ t T d s P ( t , s ) g ( s , ⋅ ) .
Example 9.6.10. Consider one last example. We wish to solve
( A − μ ) u = g . (148) (\mathcal{A}-\mu)u=g. \tag*{(148)} ( A − μ ) u = g . (148)
Treating A \mathcal{A} A as a constant, we have an algebraic equation for u u u . Solving this, and using the spectral representation for any expression involving A \mathcal{A} A we obtain
u = g A − μ = ∑ n 1 λ n − μ ⟨ ψ n , g ⟩ m ψ n . u=\frac{g}{\mathcal{A}-\mu}=\sum_n\frac{1}{\lambda_n-\mu}\langle\psi_n,g\rangle_m\psi_n. u = A − μ g = n ∑ λ n − μ 1 ⟨ ψ n , g ⟩ m ψ n .
Noting that 1 A − μ = R μ \frac{1}{\mathcal{A}-\mu}=\mathcal{R}_\mu A − μ 1 = R μ we can write the solution u u u of (148) compactly as u = R μ g u=\mathcal{R}_\mu g u = R μ g . Note that his solution makes sense only if g ∈ dom ( R μ ) g\in\operatorname{dom}(\mathcal{R}_\mu) g ∈ dom ( R μ ) . In particular, this means that we must have λ n ≠ μ \lambda_n\ne\mu λ n = μ for all n n n .
The meaning of “scale” and “speed”
From the scale density s s s we can define a scale function S S S , which is given by
S ( x ) : = ∫ d x s ( x ) . S(x):=\int\mathrm{d}x\,s(x). S ( x ) := ∫ d x s ( x ) .
By construction, the scale function S S S is one-to-one. It is interesting to note that the process S ( X ) = ( S ( X t ) ) t ≥ 0 S(X)=(S(X_t))_{t\geq0} S ( X ) = ( S ( X t ) ) t ≥ 0 is a martingale, as the following computation shows
d S ( X t ) = A S ( X t ) d t + σ ( X t ) ∂ x S ( X t ) d W t = ( μ ( X t ) s ( X t ) + 1 2 σ 2 ( X t ) ∂ x s ( X t ) ) d t + σ ( X t ) s ( X t ) d W t = ( μ ( X t ) s ( X t ) + 1 2 σ 2 ( X t ) ( − 2 μ ( X t ) σ 2 ( X t ) s ( X t ) ) ) d t + σ ( X t ) s ( X t ) d W t = σ ( X t ) s ( X t ) d W t . \begin{aligned}
\mathrm{d}S(X_t)=\mathcal{A}S(X_t)\mathrm{d}t+\sigma(X_t)\partial_xS(X_t)\mathrm{d}W_t \\
&=\left(\mu(X_t)s(X_t)+\frac{1}{2}\sigma^2(X_t)\partial_xs(X_t)\right)\mathrm{d}t+\sigma(X_t)s(X_t)\mathrm{d}W_t \\
&= \left(\mu(X_t)s(X_t) + \frac{1}{2}\sigma^2(X_t)\left(\frac{-2\mu(X_t)}{\sigma^2(X_t)}s(X_t)\right)\right)\,dt + \sigma(X_t)s(X_t)\,dW_t \\
&= \sigma(X_t)s(X_t)\,dW_t.
\end{aligned} d S ( X t ) = A S ( X t ) d t + σ ( X t ) ∂ x S ( X t ) d W t = ( μ ( X t ) s ( X t ) + 2 1 σ 2 ( X t ) ∂ x s ( X t ) ) d t + σ ( X t ) s ( X t ) d W t = ( μ ( X t ) s ( X t ) + 2 1 σ 2 ( X t ) ( σ 2 ( X t ) − 2 μ ( X t ) s ( X t ) ) ) d t + σ ( X t ) s ( X t ) d W t = σ ( X t ) s ( X t ) d W t .
Now, consider two regular points { l } \{l\} { l } and { r } \{r\} { r } of a scalar diffusion X X X . Let τ l \tau_l τ l and τ r \tau_r τ r be the first hitting times of X X X to l l l and r r r , respectively
τ l : = inf { t ≥ 0 : X t = l } , τ r : = inf { t ≥ 0 : X t = r } . \tau_l := \inf\{t \ge0 : X_t = l\}, \qquad\tau_r := \inf\{t \ge0 : X_t = r\}. τ l := inf { t ≥ 0 : X t = l } , τ r := inf { t ≥ 0 : X t = r } .
We wish to compute P ( τ l < τ r ∣ X 0 = x ) \mathbb{P}(\tau_l < \tau_r \mid X_0 = x) P ( τ l < τ r ∣ X 0 = x ) , the probability that X X X hits l l l prior to hitting r r r . Let us define
τ : = τ l ∧ τ r . \tau:= \tau_l \wedge\tau_r. τ := τ l ∧ τ r .
As the process S ( X ) S(X) S ( X ) is a martingale it follows from Theorems 2.4.3 and 2.4.5 that
S ( x ) = E S ( X τ ) = P ( τ l < τ r ∣ X 0 = x ) S ( l ) + P ( τ r < τ l ∣ X 0 = x ) S ( r ) . S(x) = \mathbb{E}S(X_\tau) = \mathbb{P}(\tau_l < \tau_r \mid X_0 = x)S(l) + \mathbb{P}(\tau_r < \tau_l \mid X_0 = x)S(r). S ( x ) = E S ( X τ ) = P ( τ l < τ r ∣ X 0 = x ) S ( l ) + P ( τ r < τ l ∣ X 0 = x ) S ( r ) .
Also, because { τ l < τ r } ∪ { τ l < τ r } = Ω \{\tau_l < \tau_r\} \cup\{\tau_l < \tau_r\} = \Omega { τ l < τ r } ∪ { τ l < τ r } = Ω we have trivially that
1 = P ( τ l < τ r ∣ X 0 = x ) + P ( τ r < τ l ∣ X 0 = x ) . 1 = \mathbb{P}(\tau_l < \tau_r \mid X_0 = x) + \mathbb{P}(\tau_r < \tau_l \mid X_0 = x). 1 = P ( τ l < τ r ∣ X 0 = x ) + P ( τ r < τ l ∣ X 0 = x ) .
Using the above two equations, we find
P ( τ l < τ r ∣ X 0 = x ) = S ( r ) − S ( x ) S ( r ) − S ( l ) , P ( τ r < τ l ∣ X 0 = x ) = S ( x ) − S ( l ) S ( r ) − S ( l ) . (149) \mathbb{P}(\tau_l < \tau_r \mid X_0 = x) = \frac{S(r) - S(x)}{S(r) - S(l)}, \qquad\mathbb{P}(\tau_r < \tau_l \mid X_0 = x) = \frac{S(x) - S(l)}{S(r) - S(l)}. \tag*{(149)} P ( τ l < τ r ∣ X 0 = x ) = S ( r ) − S ( l ) S ( r ) − S ( x ) , P ( τ r < τ l ∣ X 0 = x ) = S ( r ) − S ( l ) S ( x ) − S ( l ) . (149)
If we set Y = S ( X ) Y = S(X) Y = S ( X ) , let L = S ( l ) L = S(l) L = S ( l ) and R = S ( r ) R = S(r) R = S ( r ) , and define
τ L : = inf { t ≥ 0 : Y t = L } , τ R = inf { t ≥ 0 : Y t = R } , (150) \tau_L := \inf\{t \ge0 : Y_t = L\}, \qquad\tau_R = \inf\{t \ge0 : Y_t = R\}, \tag*{(150)} τ L := inf { t ≥ 0 : Y t = L } , τ R = inf { t ≥ 0 : Y t = R } , (150)
then equation (149) becomes
P ( τ L < τ R ∣ Y 0 = y ) = R − y R − L , P ( τ R < τ L ∣ Y 0 = y ) = y − L R − L . (151) \mathbb{P}(\tau_L < \tau_R \mid Y_0 = y) = \frac{R - y}{R - L}, \qquad\mathbb{P}(\tau_R < \tau_L \mid Y_0 = y) = \frac{y - L}{R - L}. \tag*{(151)} P ( τ L < τ R ∣ Y 0 = y ) = R − L R − y , P ( τ R < τ L ∣ Y 0 = y ) = R − L y − L . (151)
Any process that satisfies (150) -(151) is said to be on a natural scale . Thus, given a scalar diffusion X X X we can always associate with it a diffusion S ( X ) S(X) S ( X ) that is on a natural scale.
To do: meaning of speed .
Extensions to higher dimensions
Throughout this Chapter, we have mostly dealt with Scalar SDEs and PDEs with one spacial dimension. To conclude this chapter, we simply restate some of the main results in higher dimensions, without providing the conditions under which they hold rigorously.
Throughout this section, we consider d d d -dimensional diffusion process X = ( X t ) t ≥ 0 \mathbf{X} = (X_t)_{t\geq0} X = ( X t ) t ≥ 0 that satisfies the following SDE
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t , (152) dX_t = \mu(t,X_t)dt + \sigma(t,X_t)dW_t, \tag*{(152)} d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t , (152)
where W W W is an m m m -dimensional Brownian motion and
μ : R + × R d → R d , σ : R + × R d → R + d × m . \mu: \mathbb{R}_{+} \times\mathbb{R}^{d} \to\mathbb{R}^{d}, \qquad\sigma: \mathbb{R}_{+} \times\mathbb{R}^{d} \to\mathbb{R}_{+}^{d\times m}. μ : R + × R d → R d , σ : R + × R d → R + d × m .
SDE (9.41) has a unique strong solution, which is square integrable for all t t t (i.e., E X t 2 < ∞ \mathbb{E}X_t^2 < \infty E X t 2 < ∞ ) and adapted to the filtration F = ( F t ) t ≥ 0 \mathbb{F} = (\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 generated by W W W (i.e., X t ∈ F t X_t \in\mathcal{F}_t X t ∈ F t ) if the following are satisfies
Linear Growth :
∣ μ ( t , x ) ∣ + ∣ σ ( t , x ) ∣ ≤ C 1 ( 1 + ∣ x ∣ ) , (153) |\mu(t,x)| + |\sigma(t,x)| \leq C_1(1 + |x|), \tag*{(153)} ∣ μ ( t , x ) ∣ + ∣ σ ( t , x ) ∣ ≤ C 1 ( 1 + ∣ x ∣ ) , (153)
Lipschitz Continuous :
∣ μ ( t , x ) − μ ( t , y ) ∣ + ∣ σ ( t , x ) − σ ( t , y ) ∣ ≤ C 2 ∣ x − y ∣ , (154) |\mu(t,x) - \mu(t,y)| + |\sigma(t,x) - \sigma(t,y)| \leq C_2|x-y|, \tag*{(154)} ∣ μ ( t , x ) − μ ( t , y ) ∣ + ∣ σ ( t , x ) − σ ( t , y ) ∣ ≤ C 2 ∣ x − y ∣ , (154)
for some constants C 1 C_1 C 1 and C 2 C_2 C 2 , independent of t t t , x x x , y y y , where
∣ σ ∣ 2 : = ∑ i = 1 d ∑ j = 1 m σ i j 2 , ∣ μ ∣ 2 : = ∑ i = 1 d μ i 2 , ∣ x ∣ 2 : = ∑ i = 1 d x i 2 . |\sigma|^2 := \sum_{i=1}^{d}\sum_{j=1}^{m}\sigma_{ij}^2,\qquad|\mu|^2 := \sum_{i=1}^{d}\mu_i^2,\qquad|x|^2 := \sum_{i=1}^{d}x_i^2. ∣ σ ∣ 2 := i = 1 ∑ d j = 1 ∑ m σ ij 2 , ∣ μ ∣ 2 := i = 1 ∑ d μ i 2 , ∣ x ∣ 2 := i = 1 ∑ d x i 2 .
For any continuous-time Markov process X \mathbf{X} X that lives in R d \mathbb{R}^{d} R d , we can define an infinitesimal generator A ( t ) \mathcal{A}(t) A ( t ) as follows
A ( t ) φ ( x ) : = lim T ↓ t 1 T − t ( E ( φ ( X T ) ∣ X t = x ) − φ ( x ) ) . \mathcal{A}(t)\varphi(x) := \lim_{T\downarrow t}\frac{1}{T-t}\left(\mathbb{E}(\varphi(X_T)\mid X_t=x)-\varphi(x)\right). A ( t ) φ ( x ) := T ↓ t lim T − t 1 ( E ( φ ( X T ) ∣ X t = x ) − φ ( x ) ) .
where φ : R d → R \varphi: \mathbb{R}^{d} \to\mathbb{R} φ : R d → R may be any function for which the limit exists. When X X X is given by (9.41), the generator A ( t ) \mathcal{A}(t) A ( t ) and its L 2 ( R d , d x ) L^2(\mathbb{R}^{d},dx) L 2 ( R d , d x ) adjoint are given by
A ( t ) = ∑ i = 1 d μ i ( t , x ) ∂ x i + 1 2 ∑ i = 1 d ∑ j = 1 d ( σ σ T ) i , j ( t , x ) ∂ x i ∂ x j , \mathcal{A}(t) = \sum_{i=1}^{d}\mu_i(t,x)\partial_{x_i} + \frac{1}{2}\sum_{i=1}^{d}\sum_{j=1}^{d}(\sigma\sigma^{\mathsf{T}})_{i,j}(t,x)\partial_{x_i}\partial_{x_j}, A ( t ) = i = 1 ∑ d μ i ( t , x ) ∂ x i + 2 1 i = 1 ∑ d j = 1 ∑ d ( σ σ T ) i , j ( t , x ) ∂ x i ∂ x j ,
A ∗ ( t ) = − ∑ i = 1 d ∂ y i μ i ( t , y ) + 1 2 ∑ i = 1 d ∑ j = 1 d ∂ y i ∂ y j ( σ σ T ) i , j ( t , y ) . \mathcal{A}^{*}(t) = -\sum_{i=1}^{d}\partial_{y_i}\mu_i(t,y) + \frac{1}{2}\sum_{i=1}^{d}\sum_{j=1}^{d}\partial_{y_i}\partial_{y_j}(\sigma\sigma^{\mathsf{T}})_{i,j}(t,y). A ∗ ( t ) = − i = 1 ∑ d ∂ y i μ i ( t , y ) + 2 1 i = 1 ∑ d j = 1 ∑ d ∂ y i ∂ y j ( σ σ T ) i , j ( t , y ) .
We can write the Itô Formula most compactly as
d φ ( X t ) = A ( t ) φ ( X t ) d t + ( ∇ φ ) ( X t ) σ ( t , X t ) d W t . d\varphi(X_t) = \mathcal{A}(t)\varphi(X_t)dt + (\nabla\varphi)(X_t)\sigma(t,X_t)dW_t. d φ ( X t ) = A ( t ) φ ( X t ) d t + ( ∇ φ ) ( X t ) σ ( t , X t ) d W t .
Thus, we can think of A ( t ) \mathcal{A}(t) A ( t ) as the operator that appears in the d t dt d t -term when we take the differential d φ ( X t ) d\varphi(X_t) d φ ( X t ) .
The transition density Γ \Gamma Γ , defined by Γ ( t , x ; T , y ) d y = P ( X T ∈ d y ∣ X t = x ) \Gamma(t,x;T,y)\,dy = \mathbb{P}(X_T \in dy|X_t=x) Γ ( t , x ; T , y ) d y = P ( X T ∈ d y ∣ X t = x ) satisfies the KBE in the backwards variables ( t , x ) (t,x) ( t , x ) and the KFE in the forward variables ( T , y ) (T,y) ( T , y )
K B E : ( ∂ t + A ( t ) ) Γ ( t , ⋅ ; T , y ) = 0 , Γ ( T , ⋅ ; T , y ) = δ y , K F E : ( − ∂ T + A ∗ ( T ) ) Γ ( t , x ; T , ⋅ ) = 0 , Γ ( t , x ; t , ⋅ ) = δ x . \begin{aligned}
\mathrm{KBE}: \qquad&(\partial_t+\mathcal{A}(t))\Gamma(t,\cdot;T,y)=0, && \Gamma(T,\cdot;T,y)=\delta_y,\\
\mathrm{KFE}: \qquad&(-\partial_T+\mathcal{A}^{*}(T))\Gamma(t,x;T,\cdot)=0, && \Gamma(t,x;t,\cdot)=\delta_x.
\end{aligned} KBE : KFE : ( ∂ t + A ( t )) Γ ( t , ⋅ ; T , y ) = 0 , ( − ∂ T + A ∗ ( T )) Γ ( t , x ; T , ⋅ ) = 0 , Γ ( T , ⋅ ; T , y ) = δ y , Γ ( t , x ; t , ⋅ ) = δ x .
If we define a function u : R + × R → R u:\mathbb{R}_{+}\times\mathbb{R}\to\mathbb{R} u : R + × R → R by
u ( t , x ) : = E [ e − A ( t , T ) φ ( X T ) + ∫ t T e − A ( t , s ) g ( s , X s ) d s ∣ X t = x ] , u(t,x):=\mathbb{E}\left[\mathrm{e}^{-A(t,T)}\varphi(X_T)+\int_t^T\mathrm{e}^{-A(t,s)}g(s,X_s)\,ds\mathrel{\Big|}X_t=x\right], u ( t , x ) := E [ e − A ( t , T ) φ ( X T ) + ∫ t T e − A ( t , s ) g ( s , X s ) d s X t = x ] ,
where A ( t , T ) A(t,T) A ( t , T ) is given by
A ( t , T ) = ∫ t T γ ( s , X s ) d s , γ : R + × R d → R + . A(t,T)=\int_t^T\gamma(s,X_s)\,ds,\qquad\gamma:\mathbb{R}_{+}\times\mathbb{R}^{d}\to\mathbb{R}_{+}. A ( t , T ) = ∫ t T γ ( s , X s ) d s , γ : R + × R d → R + .
Then u u u satisfies
( ∂ t − γ ( t , ⋅ ) + A ( t ) ) u ( t , ⋅ ) + g ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ . \left(\partial_t-\gamma(t,\cdot)+\mathcal{A}(t)\right)u(t,\cdot)+g(t,\cdot)=0,\qquad u(T,\cdot)=\varphi. ( ∂ t − γ ( t , ⋅ ) + A ( t ) ) u ( t , ⋅ ) + g ( t , ⋅ ) = 0 , u ( T , ⋅ ) = φ .
Lastly, consider the time-homogeneous diffusion: μ ( t , x ) = μ ( x ) \mu(t,x)=\mu(x) μ ( t , x ) = μ ( x ) and σ ( t , x ) = σ ( x ) \sigma(t,x)=\sigma(x) σ ( t , x ) = σ ( x ) . Let D ⊂ R d D\subset\mathbb{R}^{d} D ⊂ R d be an open, connect set. Denote by ∂ D \partial D ∂ D the boundary of D D D . Assume X 0 ∈ D X_0\in D X 0 ∈ D and define the hitting time
τ D : = inf { t ≥ 0 : X t ∉ D } . \tau_D:=\inf\{t\geq0:X_t\notin D\}. τ D := inf { t ≥ 0 : X t ∈ / D } .
Assume P ( τ D < ∞ ) = 1 \mathbb{P}(\tau_D<\infty)=1 P ( τ D < ∞ ) = 1 and let
u ( x ) = E [ e − λ ( τ D − t ) φ ( X τ D ) + ∫ t τ D e − λ ( s − t ) g ( X s ) d s ∣ X t = x ] , t < τ D . u(x)=\mathbb{E}\left[\mathrm{e}^{-\lambda(\tau_D-t)}\varphi(X_{\tau_D})+\int_t^{\tau_D}\mathrm{e}^{-\lambda(s-t)}g(X_s)\,ds\mathrel{\Big|}X_t=x\right],\qquad t<\tau_D. u ( x ) = E [ e − λ ( τ D − t ) φ ( X τ D ) + ∫ t τ D e − λ ( s − t ) g ( X s ) d s X t = x ] , t < τ D .
Then u u u satisfies
( A − λ ) u + g = 0 , in D , (\mathcal{A}-\lambda)u+g=0,\qquad\text{in }D, ( A − λ ) u + g = 0 , in D ,
u = φ , on ∂ D . u=\varphi,\qquad\text{on }\partial D. u = φ , on ∂ D .
Exercises
Exercise 9.1. A (one-dimensional) backward stochastic differential equation (BSDE), defined on a probability space filtered probability space ( Ω , F , F = ( F t ) t ≥ 0 , P ) (\Omega,\mathcal{F},\mathbb{F}=(\mathcal{F}_t)_{t\geq0},\mathbb{P}) ( Ω , F , F = ( F t ) t ≥ 0 , P ) , is an equation of the form
d Y t = − f ( ω , t , Y t , Z t ) d t + Z t d W t , Y T = ξ , (155) dY_t=-f(\omega,t,Y_t,Z_t)\,dt+Z_t\,dW_t,\qquad Y_T=\xi, \tag*{(155)} d Y t = − f ( ω , t , Y t , Z t ) d t + Z t d W t , Y T = ξ , (155)
where W = ( W t ) 0 ≤ t ≤ T W=(W_t)_{0\leq t\leq T} W = ( W t ) 0 ≤ t ≤ T is an ( P , F ) (\mathbb{P},\mathbb{F}) ( P , F ) -Brownian motion, the process Y = ( Y t ) 0 ≤ t ≤ T Y=(Y_t)_{0\leq t\leq T} Y = ( Y t ) 0 ≤ t ≤ T lives in R \mathbb{R} R , the process Z = ( Z t ) 0 ≤ t ≤ T Z=(Z_t)_{0\leq t\leq T} Z = ( Z t ) 0 ≤ t ≤ T lives in R \mathbb{R} R , the driver f : Ω × [ 0 , T ] × R × R → R f:\Omega\times[0,T]\times\mathbb{R}\times\mathbb{R}\to\mathbb{R} f : Ω × [ 0 , T ] × R × R → R , satisfies f ( ω , t , Y t , Z t ) ∈ F t f(\omega,t,Y_t,Z_t)\in\mathcal{F}_t f ( ω , t , Y t , Z t ) ∈ F t for all
Exercises
t ∈ [ 0 , T ] t \in[0,T] t ∈ [ 0 , T ] and the random variable ξ \xi ξ is F T \mathcal{F}_T F T -measurable. A solution of BSDE (155) is any pair of F \mathcal{F} F -adapted processes ( Y , Z ) (Y,Z) ( Y , Z ) such that the terminal condition Y T = ξ Y_T=\xi Y T = ξ is satisfied. A forward-backward stochastic differential equation (FBSDE), is a BSDE of the form
d X t = μ ( t , X t , Y t , Z t ) d t + σ ( t , X t , Y t ) d W t , X 0 = x , d Y t = − f ( t , X t , Y t , Z t ) d t + Z t d W t , Y T = φ ( X T ) ∈ R , (156) \begin{aligned}
\mathrm{d}X_t &= \mu(t,X_t,Y_t,Z_t)\mathrm{d}t+\sigma(t,X_t,Y_t)\mathrm{d}W_t,\qquad X_0=x,\\
\mathrm{d}Y_t &= -f(t,X_t,Y_t,Z_t)\mathrm{d}t+Z_t\mathrm{d}W_t,\qquad Y_T=\varphi(X_T)\in\mathbb{R}, \tag*{(156)}
\end{aligned} d X t d Y t = μ ( t , X t , Y t , Z t ) d t + σ ( t , X t , Y t ) d W t , X 0 = x , = − f ( t , X t , Y t , Z t ) d t + Z t d W t , Y T = φ ( X T ) ∈ R , (156)
where the processes W W W , Y Y Y , and Z Z Z are as described above, the process X = ( X t ) 0 ≤ t ≤ T X=(X_t)_{0\leq t\leq T} X = ( X t ) 0 ≤ t ≤ T lives in R \mathbb{R} R , and the functions μ \mu μ , σ \sigma σ , f f f and φ \varphi φ are maps
μ : [ 0 , T ] × R × R × R → R , σ : [ 0 , T ] × R × R → R , f : [ 0 , T ] × R × R × R → R , φ : R → R . \begin{aligned}
\mu&: [0,T]\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\to\mathbb{R},\qquad
\sigma: [0,T]\times\mathbb{R}\times\mathbb{R}\to\mathbb{R},\\
f &: [0,T]\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\to\mathbb{R},\qquad
\varphi: \mathbb{R}\to\mathbb{R}.
\end{aligned} μ f : [ 0 , T ] × R × R × R → R , σ : [ 0 , T ] × R × R → R , : [ 0 , T ] × R × R × R → R , φ : R → R .
In general, the coefficient σ \sigma σ in (156) could depend on Z Z Z as well. However, for simplicity, we do not consider this case here. FBSDEs naturally arise in mathematical finance where components of X X X are assets in a market, components of Y Y Y are values of hedging portfolios, and components Z Z Z are the associated hedges.
We wish to solve FBSDE (156) – meaning we wish to find a pair of F \mathcal{F} F -adapted processes ( Y , Z ) (Y,Z) ( Y , Z ) such that Y T = φ ( X T ) Y_T=\varphi(X_T) Y T = φ ( X T ) . Let us supposed that Y t = u ( t , X t ) Y_t=u(t,X_t) Y t = u ( t , X t ) for some function u : [ 0 , T ] × R → R u:[0,T]\times\mathbb{R}\to\mathbb{R} u : [ 0 , T ] × R → R , which is to be determined. Let us further suppose that u ∈ C 1 , 2 u\in C^{1,2} u ∈ C 1 , 2 . Compute d u ( t , X t ) \mathrm{d}u(t,X_t) d u ( t , X t ) and compare your result to the expression given for d Y t \mathrm{d}Y_t d Y t in (156) . Conclude that, if the function u u u satisfies the semilinear PDE
0 = ( ∂ t + A u ( t ) ) u ( t ) + f u ( t ) , u ( T ) = φ , 0=(\partial_t+\mathcal{A}^u(t))u(t)+f^u(t),\qquad u(T)=\varphi, 0 = ( ∂ t + A u ( t )) u ( t ) + f u ( t ) , u ( T ) = φ ,
where A u \mathcal{A}^u A u and f u f^u f u are given by
A u ( t ) = μ ( t , x , u ( t , x ) , σ ( t , x ) ∂ x u ( t , x ) ) ∂ x + 1 2 σ 2 ( t , x , u ( t , x ) ) ∂ x 2 , \mathcal{A}^u(t)=\mu(t,x,u(t,x),\sigma(t,x)\partial_xu(t,x))\partial_x+\frac{1}{2}\sigma^2(t,x,u(t,x))\partial_x^2, A u ( t ) = μ ( t , x , u ( t , x ) , σ ( t , x ) ∂ x u ( t , x )) ∂ x + 2 1 σ 2 ( t , x , u ( t , x )) ∂ x 2 ,
f u ( t ) = f ( t , x , u ( t , x ) , σ ( t , x ) ∂ x u ( t , x ) ) , f^u(t)=f(t,x,u(t,x),\sigma(t,x)\partial_xu(t,x)), f u ( t ) = f ( t , x , u ( t , x ) , σ ( t , x ) ∂ x u ( t , x )) ,
then the pair ( Y , Z ) (Y,Z) ( Y , Z ) defined by
Y t : = u ( t , X t ) , Z t : = σ ( t , X t ) ∂ x u ( t , X t ) , Y_t:=u(t,X_t),\qquad Z_t:=\sigma(t,X_t)\partial_xu(t,X_t), Y t := u ( t , X t ) , Z t := σ ( t , X t ) ∂ x u ( t , X t ) ,
is (at least formally) a solution of FBSDE (156) .
Exercise 9.2. Let W W W be a Brownian motion on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathcal{F}=(\mathcal{F}_t)_{0\leq t\leq T} F = ( F t ) 0 ≤ t ≤ T be a filtration for W W W . Let X X X be the solution of the following SDE
d X t = κ ( θ − X t ) d t + δ X t d W t . \mathrm{d}X_t=\kappa(\theta-X_t)\mathrm{d}t+\delta\sqrt{X_t}\mathrm{d}W_t. d X t = κ ( θ − X t ) d t + δ X t d W t .
The process X X X is called a Cox-Ingersoll-Ross (CIR) process. Define
u ( t , x ) : = E [ exp ( − ∫ t T X s d s ) | X t = x ] . u(t,x):=\mathbb{E}\left[\exp\left(-\int_t^T X_s\mathrm{d}s\right)\middle|X_t=x\right]. u ( t , x ) := E [ exp ( − ∫ t T X s d s ) X t = x ] .
(In quantitative finance, the CIR process is often used to model a stochastic short-term interest rate, in which case u ( t , X t ) u(t,X_t) u ( t , X t ) is the price at time t t t of a zero-coupon bond that pays one unit of currency at time T T T .)
(a) Derive a PDE and terminal condition for the function u u u .
(b) To solve the PDE for u u u , try a solution of the form
u ( t , x ) = exp ( − x A ( t ) − B ( t ) ) , (157) u(t,x)=\exp(-xA(t)-B(t)), \tag*{(157)} u ( t , x ) = exp ( − x A ( t ) − B ( t )) , (157)
where A A A and B B B are deterministic functions of t t t . Show that A A A and B B B must satisfy a pair of coupled ODEs. Write these ODEs as well as their terminal conditions at time T T T . Hint: insert the expression (157) into the PDE for u u u and collect terms of like powers in x x x . As the PDE for u u u must hold for all values of x x x , what does this tell you about the terms that are of order x 0 x^0 x 0 , x 1 x^1 x 1 , etc.?
(c) Bonus question: solve the ODEs (it may be helpful to note that one of the ODEs is a Riccati equation).
Exercise 9.3. For i = 1 , 2 , … , d i=1,2,\ldots,d i = 1 , 2 , … , d , let X ( i ) X^{(i)} X ( i ) satisfy
d X t ( i ) = − b 2 X t ( i ) d t + 1 2 σ d W t ( i ) . \mathrm{d}X_t^{(i)}=-\frac{b}{2}X_t^{(i)}\mathrm{d}t+\frac{1}{2}\sigma\mathrm{d}W_t^{(i)}. d X t ( i ) = − 2 b X t ( i ) d t + 2 1 σ d W t ( i ) .
where the ( W ( i ) ) i = 1 d (W^{(i)})_{i=1}^d ( W ( i ) ) i = 1 d are independent Brownian motions. Define
R t : = ∑ i = 1 d ( X t ( i ) ) 2 , B t : = ∑ i = 1 d ∫ 0 t 1 R s X s ( i ) d W s ( i ) . R_t:=\sum_{i=1}^{d}\left(X_t^{(i)}\right)^2,\qquad B_t:=\sum_{i=1}^{d}\int_0^t\frac{1}{\sqrt{R_s}}X_s^{(i)}\mathrm{d}W_s^{(i)}. R t := i = 1 ∑ d ( X t ( i ) ) 2 , B t := i = 1 ∑ d ∫ 0 t R s 1 X s ( i ) d W s ( i ) .
Show that B B B is a Brownian motion. Derive an SDE for R R R that involves only d t \mathrm{d}t d t and d B t \mathrm{d}B_t d B t terms (i.e., no d W t ( i ) \mathrm{d}W_t^{(i)} d W t ( i ) terms should appear).
Exercise 9.4. Consider the following 2-dimensional SDE
d X t = − 1 2 Z t d t + Z t d W t , \mathrm{d}X_t=-\frac{1}{2}Z_t\mathrm{d}t+\sqrt{Z_t}\mathrm{d}W_t, d X t = − 2 1 Z t d t + Z t d W t ,
d Z t = κ ( θ − Z t ) d t + σ Z t ( ρ d W t + 1 − ρ 2 d B t ) , \mathrm{d}Z_t=\kappa(\theta-Z_t)\mathrm{d}t+\sigma\sqrt{Z_t}\left(\rho\mathrm{d}W_t+\sqrt{1-\rho^2}\mathrm{d}B_t\right), d Z t = κ ( θ − Z t ) d t + σ Z t ( ρ d W t + 1 − ρ 2 d B t ) ,
where W W W and B B B are independent Brownian motions on a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) . Define
u ( t , x , z ) : = E [ ϕ ( X T ) ∣ X t = x , Z t = z ] . u(t,x,z):=\mathbb{E}[\phi(X_T)\mid X_t=x,Z_t=z]. u ( t , x , z ) := E [ ϕ ( X T ) ∣ X t = x , Z t = z ] .
Exercises
(In quantitative finance, the process ( X , Z ) (X, Z) ( X , Z ) is known as the Heston model . In this model, e X \mathrm{e}^{X} e X represents the price of a stock and Z Z Z represents its stochastic variance. If interest rates are zero and P \mathbb{P} P is the market’s pricing measure, then u ( t , X t , Z t ) u(t, X_t, Z_t) u ( t , X t , Z t ) is the price at time t t t of an option that pays φ ( X T ) \varphi(X_T) φ ( X T ) at time T T T .)
(a) Derive a PDE and terminal condition for the function u u u .
(b) Let u ^ \hat{u} u ^ be the Fourier transform of u u u in the x x x variable
u ^ ( t , ξ , z ) = ∫ R d x e − i ξ x u ( t , x , z ) . \hat{u}(t, \xi, z) = \int_{\mathbb{R}} \mathrm{d}x\, \mathrm{e}^{-\mathrm{i}\xi x}u(t, x, z). u ^ ( t , ξ , z ) = ∫ R d x e − i ξ x u ( t , x , z ) .
Show that u ^ \hat{u} u ^ satisfies a PDE in ( t , z ) (t,z) ( t , z ) with a terminal condition u ^ ( T , ξ , z ) = φ ^ ( ξ ) \hat{u}(T,\xi,z)=\hat{\varphi}(\xi) u ^ ( T , ξ , z ) = φ ^ ( ξ ) where φ ^ \hat{\varphi} φ ^ is the Fourier transform of φ \varphi φ . Hint: using integration by parts, we have
∫ R d x e − i ξ x ∂ x u ( t , x , z ) = − ∫ R d x u ( t , x , z ) ∂ x e − i ξ x = i ξ ∫ R d x u ( t , x , z ) e − i ξ x = i ξ u ^ ( t , ξ , z ) . \int_{\mathbb{R}} \mathrm{d}x\,\mathrm{e}^{-\mathrm{i}\xi x}\partial_xu(t,x,z)=-\int_{\mathbb{R}}\mathrm{d}x\,u(t,x,z)\partial_x\mathrm{e}^{-\mathrm{i}\xi x}=\mathrm{i}\xi\int_{\mathbb{R}}\mathrm{d}x\,u(t,x,z)\mathrm{e}^{-\mathrm{i}\xi x}=\mathrm{i}\xi\hat{u}(t,\xi,z). ∫ R d x e − i ξ x ∂ x u ( t , x , z ) = − ∫ R d x u ( t , x , z ) ∂ x e − i ξ x = i ξ ∫ R d x u ( t , x , z ) e − i ξ x = i ξ u ^ ( t , ξ , z ) .
(c) To solve the PDE from part (b) assume that u ^ \hat{u} u ^ is of the form
u ^ ( t , ξ , z ) = φ ^ ( ξ ) e A ( t , ξ ) + z B ( t , ξ ) . \hat{u}(t,\xi,z)=\hat{\varphi}(\xi)\mathrm{e}^{A(t,\xi)+zB(t,\xi)}. u ^ ( t , ξ , z ) = φ ^ ( ξ ) e A ( t , ξ ) + z B ( t , ξ ) .
where A A A and B B B are deterministic functions of t t t and ξ \xi ξ . Show that A A A and B B B satisfy a pair of coupled ODEs in t t t ; write these ODEs as well as their terminal conditions at time T T T (it may be a good idea to look at the hint given in Exercise 9.2). Note, if one solves the ODEs for A A A and B B B , then the function u u u can be obtained by taking the inverse Fourier transform of u ^ \hat{u} u ^ , that is
u ( t , x , z ) = 1 2 π ∫ R d ξ e i ξ x u ^ ( t , ξ , z ) . u(t,x,z)=\frac{1}{2\pi}\int_{\mathbb{R}}\mathrm{d}\xi\,\mathrm{e}^{\mathrm{i}\xi x}\hat{u}(t,\xi,z). u ( t , x , z ) = 2 π 1 ∫ R d ξ e i ξ x u ^ ( t , ξ , z ) .
Typically, this integral must be computed numerically.
(d) Bonus question: solve the ODEs (it may be helpful to note that one of the ODEs is a Riccati equation).
Exercise 9.5. Consider a diffusion X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 that lives on a finite interval ( l , r ) (l,r) ( l , r ) , 0 < l < r < ∞ 0<l<r<\infty 0 < l < r < ∞ and satisfies the SDE
d X t = μ X t d t + σ X t d W t \mathrm{d}X_t=\mu X_t\mathrm{d}t+\sigma X_t\mathrm{d}W_t d X t = μ X t d t + σ X t d W t
One can easily check that the endpoints l l l and r r r are regular (you do not have to prove it here). Assume both endpoints are killing. Find the transition density Γ ( t , x ; T , y ) \Gamma(t,x;T,y) Γ ( t , x ; T , y ) of X X X .
Exercise 9.6. Consider a two-dimensional diffusion processes X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 and Y = ( Y t ) t ≥ 0 Y=(Y_t)_{t\geq0} Y = ( Y t ) t ≥ 0 that satisfy the SDEs
d X t = d W t 1 d Y t = d W t 2 \begin{aligned}
\mathrm{d}X_t&=\mathrm{d}W_t^1\\
\mathrm{d}Y_t&=\mathrm{d}W_t^2
\end{aligned} d X t d Y t = d W t 1 = d W t 2
where W t 1 W_t^1 W t 1 and W t 2 W_t^2 W t 2 are two independent Brownian motions. Define a function u u u as follows
u ( x , y ) = E [ ϕ ( X τ ) ∣ X t = x , Y t = y ] , u(x,y)=\mathbb{E}[\phi(X_\tau)\mid X_t=x,Y_t=y], u ( x , y ) = E [ ϕ ( X τ ) ∣ X t = x , Y t = y ] ,
τ = inf { s ≥ t : Y s = a } \tau=\inf\{s\ge t:Y_s=a\} τ = inf { s ≥ t : Y s = a }
(1) State a PDE and boundary conditions satisfied by the function u u u .
(2) Let us define the Fourier transform and inverse Fourier transform, respectively, as follows
Fourier Transform:
f ^ ( ω ) : = ∫ e − i ω x f ( x ) d x , \widehat{f}(\omega):=\int e^{-i\omega x}f(x)\,\mathrm{d}x, f ( ω ) := ∫ e − iω x f ( x ) d x ,
Inverse Transform:
f ( x ) : = 1 2 π ∫ e i ω x f ^ ( ω ) d ω . f(x):=\frac{1}{2\pi}\int e^{i\omega x}\widehat{f}(\omega)\,\mathrm{d}\omega. f ( x ) := 2 π 1 ∫ e iω x f ( ω ) d ω .
Use Fourier transforms and a conditioning argument to derive an expression for u ( x , y ) u(x,y) u ( x , y ) as an inverse Fourier transform. Use this result to derive an explicit form for P ( X τ ∈ d z ∣ X t = x , Y t = y ) \mathbb{P}(X_\tau\in\mathrm{d}z\mid X_t=x,Y_t=y) P ( X τ ∈ d z ∣ X t = x , Y t = y ) (i.e., an expression involving no integrals).
(3) Show the expression you derived in part (2) for u ( x , y ) u(x,y) u ( x , y ) satisfies the PDE and BCs you stated in part (1).
Exercise 9.7. Suppose W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 W=(W_t^1,W_t^2,\ldots,W_t^d)_{t\ge0} W = ( W t 1 , W t 2 , … , W t d ) t ≥ 0 is a d d d -dimensional Brownian motion. Define
R t = ( ∑ i = 1 d ( W t i ) 2 ) 1 / 2 . R_t=\left(\sum_{i=1}^{d}(W_t^i)^2\right)^{1/2}. R t = ( i = 1 ∑ d ( W t i ) 2 ) 1/2 .
Clearly, R R R lives in an interval I I I with endpoints { 0 } \{0\} { 0 } and { ∞ } \{\infty\} { ∞ } . Show that, when d = 1 d=1 d = 1 the origin is a regular endpoint. Show that when d ≥ 2 d\ge2 d ≥ 2 , the origin is an entrance point.
Exercise 9.8. Fix a probability space ( Ω , F , P ~ ) (\Omega,\mathcal{F},\widetilde{\mathbb{P}}) ( Ω , F , P ) and let F = ( F t ) 0 ≤ t ≤ T \mathbb{F}=(\mathcal{F}_t)_{0\le t\le T} F = ( F t ) 0 ≤ t ≤ T be a filtration for a Brownian motion W ~ = ( W ~ t ) 0 ≤ t ≤ T \widetilde{W}=(\widetilde{W}_t)_{0\le t\le T} W = ( W t ) 0 ≤ t ≤ T . Let S = ( S t ) 0 ≤ t ≤ T S=(S_t)_{0\le t\le T} S = ( S t ) 0 ≤ t ≤ T satisfy
d S t = σ S t d W ~ t , σ > 0. \mathrm{d}S_t=\sigma S_t\mathrm{d}\widetilde{W}_t,\qquad\sigma>0. d S t = σ S t d W t , σ > 0.
(In finance, this is the Black-Scholes model for a stock S S S under the risk-neutral measure P ~ \widetilde{\mathbb{P}} P when the risk-free rate of interest is zero.) Fix constants p , q ∈ R p,q\in\mathbb{R} p , q ∈ R and define
u ( t , S t ) : = E ~ ( p S T q ∣ F t ) . u(t,S_t):=\widetilde{\mathbb{E}}(pS_T^q\mid\mathcal{F}_t). u ( t , S t ) := E ( p S T q ∣ F t ) .
(In finance, u ( t , S t ) u(t,S_t) u ( t , S t ) is the value at time t t t of a power option , which pays p S T q pS_T^q p S T q at time $T.)
(a) Find the function u u u by solving the SDE for S S S and computing the expectation above directly. Hint: once you solve the SDE for S S S , use that fact that W ~ T − W ~ t ∼ N ( 0 , T − t ) \widetilde{W}_T-\widetilde{W}_t\sim N(0,T-t) W T − W t ∼ N ( 0 , T − t ) under P ~ \widetilde{\mathbb{P}} P . It may be helpful to use
E ~ e a Z = e μ a + 1 2 σ 2 a 2 , Z ∼ N ( μ , σ 2 ) . \widetilde{\mathbb{E}}e^{aZ}=e^{\mu a+\frac{1}{2}\sigma^2a^2},\qquad Z\sim N(\mu,\sigma^2). E e a Z = e μ a + 2 1 σ 2 a 2 , Z ∼ N ( μ , σ 2 ) .
(b) Derive a PDE and terminal condition for the function u u u . Show that the expression you obtained for u u u in part (a) satisfies this PDE and terminal condition.
Exercises
Exercise 9.9. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 be a filtration for a Brownian motion W W W . Let X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 be the unique strong solution of the following SDE
d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . \mathrm{d}X_t=\mu(t,X_t)\mathrm{d}t+\sigma(t,X_t)\mathrm{d}W_t. d X t = μ ( t , X t ) d t + σ ( t , X t ) d W t . Define
u ( t , X t ) : = E ( e − λ ( T − t ) h ( X T ) + ∫ t T e − λ ( s − t ) g ( X s ) d s ∣ F t ) . u(t,X_t):=\mathbb{E}\left(\left.e^{-\lambda(T-t)}h(X_T)+\int_t^T e^{-\lambda(s-t)}g(X_s)\mathrm{d}s\right|\mathcal{F}_t\right). u ( t , X t ) := E ( e − λ ( T − t ) h ( X T ) + ∫ t T e − λ ( s − t ) g ( X s ) d s F t ) . where λ ≥ 0 \lambda\geq0 λ ≥ 0 is a constant.
(a) Define M = ( M t ) t ≥ 0 M=(M_t)_{t\geq0} M = ( M t ) t ≥ 0 by
M t = e − λ t u ( t , X t ) + ∫ 0 t e − λ s g ( X s ) d s . M_t=e^{-\lambda t}u(t,X_t)+\int_0^t e^{-\lambda s}g(X_s)\mathrm{d}s. M t = e − λ t u ( t , X t ) + ∫ 0 t e − λ s g ( X s ) d s . Show that M M M is a martingale.
(b) Derive a partial differential equation (PDE) and terminal condition (TC) for the function u u u .
Exercise 9.10. (a) Suppose g , h : R + → R g,h:\mathbb{R}_+\to\mathbb{R} g , h : R + → R are deterministic functions. Define
X : = ∫ 0 T g ( t ) d W t , Y : = ∫ 0 T h ( t ) d W t . X:=\int_0^T g(t)\mathrm{d}W_t,\qquad Y:=\int_0^T h(t)\mathrm{d}W_t. X := ∫ 0 T g ( t ) d W t , Y := ∫ 0 T h ( t ) d W t . Compute the mean vector μ \mu μ and the covariance matrix Σ \Sigma Σ for ( X , Y ) (X,Y) ( X , Y ) .
(b) Consider the process ( X , Y ) = ( X t , Y t ) t ≥ 0 (X,Y)=(X_t,Y_t)_{t\geq0} ( X , Y ) = ( X t , Y t ) t ≥ 0 , defined by the following SDE
d X t = d W t , d Y t = X t d g ( t ) , \mathrm{d}X_t=\mathrm{d}W_t,\qquad\mathrm{d}Y_t=X_t\mathrm{d}g(t), d X t = d W t , d Y t = X t d g ( t ) , where g : R + → R g:\mathbb{R}_+\to\mathbb{R} g : R + → R is a deterministic function with a continuous first derivative. Solve this SDE. That is, give explicit expressions for X T X_T X T and Y T Y_T Y T in terms of X t X_t X t , Y t Y_t Y t and some functional of the Brownian motion W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 . Your expressions for X T X_T X T and Y T Y_T Y T should not involve ( X s , Y s ) t < s ≤ T (X_s,Y_s)_{t<s\leq T} ( X s , Y s ) t < s ≤ T .
(c) Define the marginal transition density of the ( X , Y ) (X,Y) ( X , Y ) process
Γ ( t , x , y ; T , η ) d η : = P ( Y T ∈ d η ∣ X t = x , Y t = y ) . \Gamma(t,x,y;T,\eta)\mathrm{d}\eta:=\mathbb{P}(Y_T\in\mathrm{d}\eta\mid X_t=x,Y_t=y). Γ ( t , x , y ; T , η ) d η := P ( Y T ∈ d η ∣ X t = x , Y t = y ) . Write the Kolmogorov backward equation for Γ \Gamma Γ and give an explicit expression for Γ \Gamma Γ .
Exercise 9.11. Fix a probability space ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) and let F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 be a filtration for a scalar Brownian motion W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 . Consider a scalar diffusion X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 , which is the unique strong solution of the following SDE
d X t = μ ( X t ) d t + σ ( X t ) d W t . \mathrm{d}X_t=\mu(X_t)\mathrm{d}t+\sigma(X_t)\mathrm{d}W_t. d X t = μ ( X t ) d t + σ ( X t ) d W t .
Assume that the state space of X X X is R \mathbb{R} R . Let Y = ( Y t ) t ≥ 0 Y=(Y_t)_{t\geq0} Y = ( Y t ) t ≥ 0 be given by d Y t = d t \mathrm{d}Y_t=\mathrm{d}t d Y t = d t . Define
p ( x , y ; t ) d t : = P ( Y τ ∈ d t ∣ X 0 = x , Y 0 = y ) , τ : = inf { t ≥ 0 : X t = m } . p(x,y;t)\mathrm{d}t:=\mathbb{P}(Y_\tau\in\mathrm{d}t\mid X_0=x,Y_0=y),\qquad\tau:=\inf\{t\geq0:X_t=m\}. p ( x , y ; t ) d t := P ( Y τ ∈ d t ∣ X 0 = x , Y 0 = y ) , τ := inf { t ≥ 0 : X t = m } .
(a) Derive a PDE and boundary condition for p ( x , y ; t ) p(x,y;t) p ( x , y ; t ) .
(b) Suppose μ ( x ) = 0 \mu(x)=0 μ ( x ) = 0 and σ ( x ) = 1 \sigma(x)=1 σ ( x ) = 1 . Find p ( x , y ; t ) p(x,y;t) p ( x , y ; t ) .
Exercise 9.12. Let X X X and Y Y Y be given by
d X t = d W t , d Y t = d B t , 0 < a < X 0 2 + Y 0 2 < b < ∞ , \mathrm{d}X_t=\mathrm{d}W_t,\qquad\mathrm{d}Y_t=\mathrm{d}B_t,\qquad0<a<\sqrt{X_0^2+Y_0^2}<b<\infty, d X t = d W t , d Y t = d B t , 0 < a < X 0 2 + Y 0 2 < b < ∞ , where W W W and B B B are independent Brownian motions. Define R R R and Φ \Phi Φ as follows
R t : = X t 2 + Y t 2 , Φ t : = tan − 1 ( Y t / X t ) . R_t:=\sqrt{X_t^2+Y_t^2},\qquad\Phi_t:=\tan^{-1}(Y_t/X_t). R t := X t 2 + Y t 2 , Φ t := tan − 1 ( Y t / X t ) . (a) Find an expression for d R t \mathrm{d}R_t d R t and d Φ t \mathrm{d}\Phi_t d Φ t that does not involve X t X_t X t or Y t Y_t Y t .
(b) Define the first exit time τ \tau τ as follows
τ : = inf { t ≥ 0 : R t ∉ ( a , b ) } , \tau:=\inf\{t\geq0:R_t\notin(a,b)\}, τ := inf { t ≥ 0 : R t ∈ / ( a , b )} , Note that P ( τ < ∞ ) = 1 \mathbb{P}(\tau<\infty)=1 P ( τ < ∞ ) = 1 . Find P ( R τ = a ∣ R 0 = r , Φ 0 = φ ) \mathbb{P}(R_\tau=a\mid R_0=r,\Phi_0=\varphi) P ( R τ = a ∣ R 0 = r , Φ 0 = φ ) .
(c) Prove the following
E r , φ f ( R τ , Φ τ ) = f ( r , φ ) + E r , φ ∫ 0 τ ( 1 2 r ∂ r + 1 2 ∂ r 2 + 1 2 r 2 ∂ φ 2 ) f ( R t , Φ t ) d t . (158) \mathbb{E}_{r,\varphi}f(R_\tau,\Phi_\tau)=f(r,\varphi)+\mathbb{E}_{r,\varphi}\int_0^\tau\left(\frac{1}{2r}\partial_r+\frac{1}{2}\partial_r^2+\frac{1}{2r^2}\partial_\varphi^2\right)f(R_t,\Phi_t)\mathrm{d}t. \tag*{(158)} E r , φ f ( R τ , Φ τ ) = f ( r , φ ) + E r , φ ∫ 0 τ ( 2 r 1 ∂ r + 2 1 ∂ r 2 + 2 r 2 1 ∂ φ 2 ) f ( R t , Φ t ) d t . (158) where we have introduced the short-hand notation E r , φ ⋅ = E ( ⋅ ∣ R 0 = r , Φ 0 = φ ) \mathbb{E}_{r,\varphi}\cdot=\mathbb{E}(\cdot\mid R_0=r,\Phi_0=\varphi) E r , φ ⋅ = E ( ⋅ ∣ R 0 = r , Φ 0 = φ ) .
(d) Use (158) to find E r , φ τ \mathbb{E}_{r,\varphi}\tau E r , φ τ .
Jump diffusions
Notes from this chapter are taken primarily from (Øksendal and Sulem, 2005, Chapter 1). Notes for Section 10.5 on Hawkes processes follow Hawkes (1971).
Basic definitions and results on Lévy processes
Definition 10.1.1. A d d d -dimensional stochastic process η = ( η t ) t ≥ 0 \eta= (\eta_t)_{t \ge0} η = ( η t ) t ≥ 0 , defined on a probability space ( Ω , F , P ) (\Omega, \mathcal{F}, \mathbb{P}) ( Ω , F , P ) , is called a Lévy process if it satisfies
η 0 = 0 \eta_0 = 0 η 0 = 0 .
Independent increments: for any 0 ≤ t 0 < t 1 < ⋯ < t n < ∞ 0 \le t_0 < t_1 < \cdots< t_n < \infty 0 ≤ t 0 < t 1 < ⋯ < t n < ∞ , the increments η t 1 − η t 0 , η t 2 − η t 1 , … , η t n − η t n − 1 \eta_{t_1} - \eta_{t_0}, \eta_{t_2} - \eta_{t_1}, \ldots, \eta_{t_n} - \eta_{t_{n-1}} η t 1 − η t 0 , η t 2 − η t 1 , … , η t n − η t n − 1 are independent.
Stationary increments: for any 0 ≤ t 1 < t 2 < ∞ 0 \le t_1 < t_2 < \infty 0 ≤ t 1 < t 2 < ∞ , we have η t 2 − η t 1 ∼ η t 2 − t 1 \eta_{t_2} - \eta_{t_1} \sim\eta_{t_2-t_1} η t 2 − η t 1 ∼ η t 2 − t 1 .
Continuity in probability: for any ε > 0 \varepsilon> 0 ε > 0 and t ≥ 0 t \ge0 t ≥ 0 , we have lim s ↘ 0 P ( ∣ η t + s − η t ∣ > ε ) = 0 \lim_{s \searrow0} \mathbb{P}(|\eta_{t+s} - \eta_t| > \varepsilon) = 0 lim s ↘ 0 P ( ∣ η t + s − η t ∣ > ε ) = 0 .
Note, Item 4 in Definition 10.1.1 does not mean that a Lévy process cannot jump. For example, consider a Poisson process N = ( N t ) t ≥ 0 N = (N_t)_{t \ge0} N = ( N t ) t ≥ 0 with intensity λ \lambda λ . A Poisson process is a jump process and yet it is easy to see that it is continuous in probability as, for any ε ∈ ( 0 , 1 ) \varepsilon\in(0,1) ε ∈ ( 0 , 1 ) , we have
P ( ∣ N t + s − N t ∣ > ε ) = P ( N t + s − N t ≥ 1 ) = 1 − P ( N t + s − N t = 0 ) = 1 − e − λ s → 0 as s ↘ 0. \mathbb{P}(|N_{t+s} - N_t| > \varepsilon) = \mathbb{P}(N_{t+s} - N_t \ge1) = 1 - \mathbb{P}(N_{t+s} - N_t = 0) = 1 - e^{-\lambda s} \to0 \quad\text{as } s \searrow0. P ( ∣ N t + s − N t ∣ > ε ) = P ( N t + s − N t ≥ 1 ) = 1 − P ( N t + s − N t = 0 ) = 1 − e − λ s → 0 as s ↘ 0.
Item 4 simply means that, at a fixed t t t , the probability that a Lévy process has a discontinuity at t t t is zero.
We can and do assume that any Lévy process is right-continuous with left limits (RCLL). That is
lim s ↘ 0 η t + s = η t , ∀ t ≥ 0. \lim_{s \searrow0} \eta_{t+s} = \eta_t, \qquad\forall t \ge0. s ↘ 0 lim η t + s = η t , ∀ t ≥ 0.
A process that is RCLL is sometimes called càdlàg (for those who speak French: continue à droite, limite à gauche ). We have already encountered two examples of Lévy processes.
Example 10.1.2. A Brownian motion W = ( W t ) t ≥ 0 W = (W_t)_{t \ge0} W = ( W t ) t ≥ 0 is a continuous Lévy process with W t + s − W t ∼ N ( 0 , s ) W_{t+s} - W_t \sim\mathcal{N}(0,s) W t + s − W t ∼ N ( 0 , s ) .
Example 10.1.3. A Poisson process N = ( N t ) t ≥ 0 N = (N_t)_{t \ge0} N = ( N t ) t ≥ 0 with intensity λ \lambda λ is a pure jump Lévy process with N t + s − N t ∼ Poi ( λ s ) N_{t+s} - N_t \sim\operatorname{Poi}(\lambda s) N t + s − N t ∼ Poi ( λ s ) .
A filtration for a Lévy process is defined just as a filtration for a Brownian motion.
Definition 10.1.4. Let ( Ω , F , P ) (\Omega,\mathcal{F},\mathbb{P}) ( Ω , F , P ) be a probability space on which a Lévy process η = ( η t ) t ≥ 0 \eta= (\eta_t)_{t \ge0} η = ( η t ) t ≥ 0 is defined. A filtration for the Lévy process η \eta η is a collection of σ \sigma σ -algebras F = ( F t ) t ≥ 0 \mathbb{F} = (\mathcal{F}_t)_{t \ge0} F = ( F t ) t ≥ 0 satisfying:
Information accumulates: if 0 ≤ s < t 0 \le s < t 0 ≤ s < t then F s ⊂ F t \mathcal{F}_s \subset\mathcal{F}_t F s ⊂ F t .
Adaptivity: for all t ≥ 0 t \ge0 t ≥ 0 , we have η t ∈ F t \eta_t \in\mathcal{F}_t η t ∈ F t .
Independence of future increments: if u > t ≥ 0 u > t \ge0 u > t ≥ 0 then ( η u − η t ) ⊥ ⊥ F t (\eta_u - \eta_t) \perp\!\!\!\perp\mathcal{F}_t ( η u − η t ) ⊥ ⊥ F t .
The most natural choice for this filtration F \mathbb{F} F is (not surprisingly) the natural filtration for η \eta η , that is, F t = σ ( η s , 0 ≤ s ≤ t ) \mathcal{F}_t = \sigma(\eta_s, 0 \le s \le t) F t = σ ( η s , 0 ≤ s ≤ t ) . In principle the filtration F \mathbb{F} F could contain more than the information obtained by observing η \eta η . However, the information in the filtration is not allowed to destroy the independence of future increments of the Lévy process.
Definition 10.1.5. The jump of a Lévy process η \eta η at time t ≥ 0 t \ge0 t ≥ 0 is defined as
Δ η t : = η t − η t − , η t − : = lim s ↑ t η s . \Delta\eta_t := \eta_t - \eta_{t-}, \qquad\eta_{t-} := \lim_{s \uparrow t} \eta_s. Δ η t := η t − η t − , η t − := s ↑ t lim η s . Obviously, if η \eta η does not experience a jump at time t t t , then Δ η t = 0 \Delta\eta_t = 0 Δ η t = 0 .
Definition 10.1.6. We define N : R + × B 0 d × Ω → N 0 N : \mathbb{R}_{+} \times\mathcal{B}^{d}_{0} \times\Omega\to\mathbb{N}_{0} N : R + × B 0 d × Ω → N 0 the Poisson random measure of η \eta η by
N ( t , U , ω ) = ∑ s : 0 < s ≤ t 1 { Δ η s ( ω ) ∈ U } , U ∈ B 0 d : = { B ∈ B ( R d ) : 0 ∉ B ‾ } . N(t,U,\omega) = \sum_{s:0<s\le t} \mathbf{1}_{\{\Delta\eta_s(\omega)\in U\}}, \qquad U \in\mathcal{B}^{d}_{0} := \{B \in\mathcal{B}(\mathbb{R}^{d}) : 0 \notin\overline{B}\}. N ( t , U , ω ) = s : 0 < s ≤ t ∑ 1 { Δ η s ( ω ) ∈ U } , U ∈ B 0 d := { B ∈ B ( R d ) : 0 ∈ / B } . where B ‾ \overline{B} B denotes the closure of B B B . That is, the sets U ∈ B 0 d U \in\mathcal{B}^{d}_{0} U ∈ B 0 d are bounded away from the origin.
As with most random variables and stochastic processes, we will typically omit the dependence of N N N on ω \omega ω , writing simply N ( t , U ) N(t,U) N ( t , U ) as opposed to N ( t , U , ω ) N(t,U,\omega) N ( t , U , ω ) . The Poisson random measure N ( t , U ) N(t,U) N ( t , U ) counts the number of jumps of size Δ η s ∈ U \Delta\eta_s \in U Δ η s ∈ U prior to time t t t . It will be convenient to introduce the following differential form
N ( d t , d z ) = ∑ s > 0 δ s ( d t ) δ Δ η s ( d z ) , so that N ( t , U ) = ∫ 0 t ∫ U N ( d s , d z ) . N(dt,dz) = \sum_{s>0} \delta_s(dt)\delta_{\Delta\eta_s}(dz),
\qquad\text{so that} \qquad
N(t,U) = \int_0^t \int_U N(ds,dz). N ( d t , d z ) = s > 0 ∑ δ s ( d t ) δ Δ η s ( d z ) , so that N ( t , U ) = ∫ 0 t ∫ U N ( d s , d z ) . Roughly speaking, N ( d t , d z ) N(dt,dz) N ( d t , d z ) counts the number of jumps of size d z dz d z over the time interval d t dt d t .
Definition 10.1.7. Let N N N be the Poisson random measure of a Lévy process η \eta η . We define ν : B 0 d → R + \nu:\mathcal{B}^{d}_{0}\to\mathbb{R}_{+} ν : B 0 d → R + , the Lévy measure of η \eta η , as follows
ν ( U ) : = E N ( 1 , U ) , U ∈ B 0 d . \nu(U):=\mathbb{E}N(1,U), \qquad U\in\mathcal{B}^{d}_{0}. ν ( U ) := E N ( 1 , U ) , U ∈ B 0 d .
Theorem 10.1.8. Let U ∈ B 0 d U\in\mathcal{B}^{d}_{0} U ∈ B 0 d . Then the process ( N ( t , U ) ) t ≥ 0 (N(t,U))_{t\geq0} ( N ( t , U ) ) t ≥ 0 is a Poisson process with intensity ν ( U ) \nu(U) ν ( U ) .
Because N ( t , U ) N(t,U) N ( t , U ) is a Poisson process with intensity ν ( U ) \nu(U) ν ( U ) , we have
E N ( t , U ) = ν ( U ) t , ⇒ 1 t E N ( t , U ) = ν ( U ) . \mathbb{E}N(t,U)=\nu(U)t,
\qquad\Rightarrow\qquad
\frac{1}{t}\mathbb{E}N(t,U)=\nu(U). E N ( t , U ) = ν ( U ) t , ⇒ t 1 E N ( t , U ) = ν ( U ) .
Thus, ν ( U ) \nu(U) ν ( U ) is the expected number of times η \eta η has a jump of size z ∈ U z\in U z ∈ U per unit time. Note that
ν ( U ) d t = E N ( d t , U ) = ∑ k = 0 ∞ k P ( N ( t , U ) = k ) = P ( N ( d t , U ) = 1 ) + O ( d t 2 ) , as d t → 0. \nu(U)dt=\mathbb{E}N(dt,U)=\sum_{k=0}^{\infty}k\mathbb{P}(N(t,U)=k)=\mathbb{P}(N(dt,U)=1)+O(dt^{2}),
\qquad\text{as }dt\to0. ν ( U ) d t = E N ( d t , U ) = k = 0 ∑ ∞ k P ( N ( t , U ) = k ) = P ( N ( d t , U ) = 1 ) + O ( d t 2 ) , as d t → 0.
Hence, for small d t dt d t , one can think of ν ( U ) d t \nu(U)dt ν ( U ) d t as the probability that η \eta η experiences a jump of size z ∈ U z\in U z ∈ U in the time interval d t dt d t .
Example 10.1.9 (Compound Poisson process). Let ( X n ) n ∈ N (X_n)_{n\in\mathbb{N}} ( X n ) n ∈ N be a sequence of iid random vectors in R d \mathbb{R}^{d} R d with distribution F X F_X F X . Let ( P t ) t ≥ 0 (P_t)_{t\geq0} ( P t ) t ≥ 0 be a Poisson process with intensity λ \lambda λ . Assume ( P t ) t ≥ 0 ⊥ ⊥ ( X n ) n ∈ N (P_t)_{t\geq0}\perp\!\!\!\perp(X_n)_{n\in\mathbb{N}} ( P t ) t ≥ 0 ⊥ ⊥ ( X n ) n ∈ N . We define a compound Poisson process η = ( η t ) t ≥ 0 \eta=(\eta_t)_{t\geq0} η = ( η t ) t ≥ 0 by
η t = ∑ n = 1 P t X n . \eta_t=\sum_{n=1}^{P_t}X_n. η t = n = 1 ∑ P t X n . The increments of η \eta η are given by
η t − η s = ∑ n = P s + 1 P t X n . \eta_t-\eta_s=\sum_{n=P_s+1}^{P_t}X_n. η t − η s = n = P s + 1 ∑ P t X n . The distribution F η t − η s F_{\eta_t-\eta_s} F η t − η s depends only on ( t − s ) (t-s) ( t − s ) and F X F_X F X . As such we see that η \eta η has stationary increments. Also, non-overlapping increments of η \eta η are clearly independent, as they depend on different ( X i ) (X_i) ( X i ) . Thus, η \eta η is a Lévy process in R d \mathbb{R}^{d} R d . Let us find the Lévy measure ν \nu ν corresponding to η \eta η . For any U ∈ B 0 d U\in\mathcal{B}^{d}_{0} U ∈ B 0 d we have
ν ( U ) = E N ( 1 , U ) = E ∑ 0 < s ≤ 1 1 { Δ η s ∈ U } \nu(U)=\mathbb{E}N(1,U)=\mathbb{E}\sum_{0<s\leq1}\mathbf{1}_{\{\Delta\eta_s\in U\}} ν ( U ) = E N ( 1 , U ) = E 0 < s ≤ 1 ∑ 1 { Δ η s ∈ U } = E ∑ n = 1 P 1 1 { X n ∈ U } = E ∑ n = 1 P 1 E [ 1 { X n ∈ U } ∣ P 1 ] =\mathbb{E}\sum_{n=1}^{P_1}\mathbf{1}_{\{X_n\in U\}}
=\mathbb{E}\sum_{n=1}^{P_1}\mathbb{E}\left[\mathbf{1}_{\{X_n\in U\}}\mid P_1\right] = E n = 1 ∑ P 1 1 { X n ∈ U } = E n = 1 ∑ P 1 E [ 1 { X n ∈ U } ∣ P 1 ] = E ∑ n = 1 P 1 E [ 1 { X n ∈ U } ] = E ∑ n = 1 P 1 P ( X n ∈ U ) =\mathbb{E}\sum_{n=1}^{P_1}\mathbb{E}\left[\mathbf{1}_{\{X_n\in U\}}\right]
=\mathbb{E}\sum_{n=1}^{P_1}\mathbb{P}(X_n\in U) = E n = 1 ∑ P 1 E [ 1 { X n ∈ U } ] = E n = 1 ∑ P 1 P ( X n ∈ U ) = F X ( U ) E P 1 = λ F X ( U ) . =F_X(U)\mathbb{E}P_1=\lambda F_X(U). = F X ( U ) E P 1 = λ F X ( U ) . Thus, the Lévy measure of η \eta η is ν = λ F X \nu=\lambda F_X ν = λ F X .
A pure-jump Lévy process η \eta η has a finite Lévy measure ν ( R d ) < ∞ \nu(\mathbb{R}^{d})<\infty ν ( R d ) < ∞ if and only if it can be represented by a compound Poisson process. In this case, we can express η \eta η in one of two ways
η t = ∫ R d z N ( t , d z ) , or η t = ∑ n = 1 P t X n . (159) \eta_t=\int_{\mathbb{R}^{d}}zN(t,dz),
\qquad\text{or}\qquad
\eta_t=\sum_{n=1}^{P_t}X_n. \tag*{(159)} η t = ∫ R d z N ( t , d z ) , or η t = n = 1 ∑ P t X n . (159)
However, there exist Lévy processes for which ν ( R d ) = ∞ \nu(\mathbb{R}^{d})=\infty ν ( R d ) = ∞ . We call a Lévy process for which ν ( R d ) = ∞ \nu(\mathbb{R}^{d})=\infty ν ( R d ) = ∞ an infinite activity Lévy process. For an infinite activity Lévy process, neither of the representations in (10.1) make sense. To write the most general form of a Lévy process, we introduce the compensated Poisson random measure.
Definition 10.1.10. Let N N N be a Poisson random measure with associated Lévy measure ν \nu ν . The compensated Poisson random measure , denoted N ~ \widetilde{N} N is defined as
N ~ ( t , A ) : = N ( t , A ) − ν ( A ) t . \widetilde{N}(t,A):=N(t,A)-\nu(A)t. N ( t , A ) := N ( t , A ) − ν ( A ) t . In differential form, we have
N ~ ( d t , d z ) : = N ( d t , d z ) − ν ( d z ) d t . \widetilde{N}(dt,dz):=N(dt,dz)-\nu(dz)dt. N ( d t , d z ) := N ( d t , d z ) − ν ( d z ) d t . For any A ∈ B 0 d A\in\mathcal{B}_{0}^{d} A ∈ B 0 d the process ( N ~ ( t , A ) ) t ≥ 0 (\widetilde{N}(t,A))_{t\geq0} ( N ( t , A ) ) t ≥ 0 is a martingale (with respect to its own filtration) as
E [ N ~ ( t , A ) ∣ F s ] = N ~ ( s , A ) + E [ N ~ ( t , A ) − N ~ ( s , A ) ∣ F s ] \mathbb{E}[\widetilde{N}(t,A)\mid\mathcal{F}_{s}]
=\widetilde{N}(s,A)+\mathbb{E}[\widetilde{N}(t,A)-\widetilde{N}(s,A)\mid\mathcal{F}_{s}] E [ N ( t , A ) ∣ F s ] = N ( s , A ) + E [ N ( t , A ) − N ( s , A ) ∣ F s ] = N ~ ( s , A ) + E [ N ( t , A ) − N ( s , A ) ∣ F s ] − ν ( A ) ( t − s ) =\widetilde{N}(s,A)+\mathbb{E}[N(t,A)-N(s,A)\mid\mathcal{F}_{s}]-\nu(A)(t-s) = N ( s , A ) + E [ N ( t , A ) − N ( s , A ) ∣ F s ] − ν ( A ) ( t − s ) = N ~ ( s , A ) + ν ( A ) ( t − s ) − ν ( A ) ( t − s ) = N ~ ( s , A ) , =\widetilde{N}(s,A)+\nu(A)(t-s)-\nu(A)(t-s)=\widetilde{N}(s,A), = N ( s , A ) + ν ( A ) ( t − s ) − ν ( A ) ( t − s ) = N ( s , A ) , where we have used E N ( t , A ) = t ν ( A ) \mathbb{E}N(t,A)=t\nu(A) E N ( t , A ) = t ν ( A ) . It follows that, for any fixed R > 0 R>0 R > 0 , the process M ( k ) = ( M t ( k ) ) t ≥ 0 M^{(k)}=(M_t^{(k)})_{t\geq0} M ( k ) = ( M t ( k ) ) t ≥ 0 , defined as
M t ( k ) : = ∫ 1 / k ≤ ∣ z ∣ < R z N ~ ( t , d z ) , k = 1 , 2 , … , M_t^{(k)}:=\int_{1/k\leq|z|<R}z\widetilde{N}(t,dz),\qquad k=1,2,\ldots, M t ( k ) := ∫ 1/ k ≤ ∣ z ∣ < R z N ( t , d z ) , k = 1 , 2 , … , is a (d d d -dimensional) martingale satisfying E ∣ M t ( k ) ∣ 2 < ∞ \mathbb{E}|M_t^{(k)}|^2<\infty E ∣ M t ( k ) ∣ 2 < ∞ . One can show that, as k → ∞ k\to\infty k → ∞ the sequence of processes ( M ( k ) ) k ∈ N (M^{(k)})_{k\in\mathbb{N}} ( M ( k ) ) k ∈ N converges in L 2 ( Ω , F , P ) L^2(\Omega,\mathcal{F},\mathbb{P}) L 2 ( Ω , F , P ) to a process M = ( M t ) t ≥ 0 M=(M_t)_{t\geq0} M = ( M t ) t ≥ 0 defined as
M t ≡ ∫ ∣ z ∣ < R z N ~ ( t , d z ) : = lim k → ∞ ∫ 1 / k ≤ ∣ z ∣ < R z N ~ ( t , d z ) , M_t\equiv\int_{|z|<R}z\widetilde{N}(t,dz):=\lim_{k\to\infty}\int_{1/k\leq|z|<R}z\widetilde{N}(t,dz), M t ≡ ∫ ∣ z ∣ < R z N ( t , d z ) := k → ∞ lim ∫ 1/ k ≤ ∣ z ∣ < R z N ( t , d z ) , which is also a martingale and satisfies E ∣ M t ∣ 2 < ∞ \mathbb{E}|M_t|^2<\infty E ∣ M t ∣ 2 < ∞ .
The following theorem gives the most general form of a Lévy process.
Theorem 10.1.12 (Itô-Lévy decomposition). Let η = ( η t ) t ≥ 0 \eta= (\eta_t)_{t\geq0} η = ( η t ) t ≥ 0 be a Lévy process in R d \mathbb{R}^d R d . Then η \eta η has the decomposition
η t = μ R t + σ W t + ∫ ∣ z ∣ < R z N ~ ( t , d z ) + ∫ ∣ z ∣ ≥ R z N ( t , d z ) , (160) \eta_t = \mu_R t + \sigma W_t + \int_{|z|<R} z\widetilde{N}(t,dz) + \int_{|z|\geq R} zN(t,dz), \tag*{(160)} η t = μ R t + σ W t + ∫ ∣ z ∣ < R z N ( t , d z ) + ∫ ∣ z ∣ ≥ R z N ( t , d z ) , (160) for some vector μ R ∈ R d \mu_R \in\mathbb{R}^d μ R ∈ R d , matrix σ ∈ R d × d \sigma\in\mathbb{R}^{d\times d} σ ∈ R d × d , constant R ∈ [ 0 , ∞ ] R \in[0,\infty] R ∈ [ 0 , ∞ ] , Poisson random measure N N N on R d \mathbb{R}^d R d , and d d d -dimensional Brownian motion W W W that is independent of N N N . The choice R = 0 R=0 R = 0 is possible if and only if ∫ ∣ z ∣ < 1 ∣ z ∣ ν ( d z ) < ∞ \int_{|z|<1}|z|\nu(dz)<\infty ∫ ∣ z ∣ < 1 ∣ z ∣ ν ( d z ) < ∞ , and the choice R = ∞ R=\infty R = ∞ is possible if and only if ∫ ∣ z ∣ ≥ 1 ∣ z ∣ ν ( d z ) < ∞ \int_{|z|\geq1}|z|\nu(dz)<\infty ∫ ∣ z ∣ ≥ 1 ∣ z ∣ ν ( d z ) < ∞ , where ν \nu ν is the Lévy measure associated with $N.
We can always choose R = 1 R=1 R = 1 in (160) . It is useful to recognize when we can choose R = 0 R=0 R = 0 and R = ∞ R=\infty R = ∞ as, in these cases, the right-hand side of (160) reduces from four to three terms.
Example 10.1.13. If the Lévy measure ν \nu ν satisfies
∫ ∣ z ∣ ≥ 1 ∣ z ∣ ν ( d z ) < ∞ , (161) \int_{|z|\geq1}|z|\nu(dz)<\infty, \tag*{(161)} ∫ ∣ z ∣ ≥ 1 ∣ z ∣ ν ( d z ) < ∞ , (161) then we can write (160) as
η t = ( μ 1 + ∫ ∣ z ∣ ≥ 1 z ν ( d z ) ) t + σ W t + ∫ ∣ z ∣ < 1 z N ~ ( t , d z ) + ∫ ∣ z ∣ ≥ 1 z ( N ( t , d z ) − ν ( d z ) t ) = μ ∞ t + σ W t + ∫ R d z N ~ ( t , d z ) , \begin{aligned}
\eta_t &= \left(\mu_1+\int_{|z|\geq1}z\nu(dz)\right)t+\sigma W_t+\int_{|z|<1}z\widetilde{N}(t,dz)+\int_{|z|\geq1}z\left(N(t,dz)-\nu(dz)t\right)\\
&=\mu_\infty t+\sigma W_t+\int_{\mathbb{R}^d}z\widetilde{N}(t,dz),
\end{aligned} η t = ( μ 1 + ∫ ∣ z ∣ ≥ 1 z ν ( d z ) ) t + σ W t + ∫ ∣ z ∣ < 1 z N ( t , d z ) + ∫ ∣ z ∣ ≥ 1 z ( N ( t , d z ) − ν ( d z ) t ) = μ ∞ t + σ W t + ∫ R d z N ( t , d z ) , where μ ∞ = μ 1 + ∫ ∣ z ∣ ≥ 1 z ν ( d z ) \mu_\infty=\mu_1+\int_{|z|\geq1}z\nu(dz) μ ∞ = μ 1 + ∫ ∣ z ∣ ≥ 1 z ν ( d z ) . We call μ ∞ \mu_\infty μ ∞ the center of η \eta η since E η t = μ ∞ t \mathbb{E}\eta_t=\mu_\infty t E η t = μ ∞ t . It is straightforward to show that if μ ∞ = 0 \mu_\infty=0 μ ∞ = 0 then η \eta η is a martingale.
Example 10.1.14. If the Lévy measure ν \nu ν satisfies
∫ ∣ z ∣ < 1 ∣ z ∣ ν ( d z ) < ∞ , (162) \int_{|z|<1}|z|\nu(dz)<\infty, \tag*{(162)} ∫ ∣ z ∣ < 1 ∣ z ∣ ν ( d z ) < ∞ , (162) then we can write (160) as
η t = μ 1 t + σ W t + ∫ ∣ z ∣ < 1 z N ( t , d z ) − ∫ ∣ z ∣ < 1 z ν ( d z ) t + ∫ ∣ z ∣ ≥ 1 z N ( t , d z ) = μ 0 t + σ W t + ∫ R d z N ( t , d z ) , (163) \begin{aligned}
\eta_t &= \mu_1t+\sigma W_t+\int_{|z|<1}zN(t,dz)-\int_{|z|<1}z\nu(dz)t+\int_{|z|\geq1}zN(t,dz)\\
&=\mu_0t+\sigma W_t+\int_{\mathbb{R}^d}zN(t,dz), \tag*{(163)}
\end{aligned} η t = μ 1 t + σ W t + ∫ ∣ z ∣ < 1 z N ( t , d z ) − ∫ ∣ z ∣ < 1 z ν ( d z ) t + ∫ ∣ z ∣ ≥ 1 z N ( t , d z ) = μ 0 t + σ W t + ∫ R d z N ( t , d z ) , (163) where μ 0 = μ 1 − ∫ ∣ z ∣ < 1 z ν ( d z ) \mu_0=\mu_1-\int_{|z|<1}z\nu(dz) μ 0 = μ 1 − ∫ ∣ z ∣ < 1 z ν ( d z ) . We call μ 0 \mu_0 μ 0 the drift of η \eta η .
Example 10.1.15. If the Lévy measure ν \nu ν satisfies
ν ( R d ) < ∞ , \nu(\mathbb{R}^{d}) < \infty, ν ( R d ) < ∞ , then (162) holds, and we can write (163) as a compound Poisson process
η t = μ 0 t + σ W t + ∑ n = 1 P t X n , \eta_t = \mu_0t + \sigma W_t + \sum_{n=1}^{P_t} X_n, η t = μ 0 t + σ W t + n = 1 ∑ P t X n , where P P P is a Poisson process with intensity ν ( R d ) \nu(\mathbb{R}^{d}) ν ( R d ) and the ( X n ) n ∈ N (X_n)_{n\in\mathbb{N}} ( X n ) n ∈ N are iid random vectors in R d \mathbb{R}^{d} R d with common distribution F X = ν / ν ( R d ) F_X = \nu/\nu(\mathbb{R}^{d}) F X = ν / ν ( R d ) .
We will typically drop the subscript from μ R \mu_R μ R , writing it instead simply as μ \mu μ .
Theorem 10.1.16. Lévy processes are Markov processes.
This should come as no surprise. We have already seen a Brownian motion and a Poisson process are Markov processes. As these processes serve as building blocks for more general Lévy processes, it follows that Lévy processes are Markov as well.
Theorem 10.1.17 (Lévy-Khintchine formula). Let η = ( η t ) t ≥ 0 \eta= (\eta_t)_{t\geq0} η = ( η t ) t ≥ 0 be the following d d d -dimensional Lévy process
η t = μ t + σ W t + ∫ ∣ z ∣ < R z N ~ ( t , d z ) + ∫ ∣ z ∣ ≥ R z N ( t , d z ) , \eta_t = \mu t + \sigma W_t + \int_{|z|<R} z\widetilde{N}(t,dz) + \int_{|z|\geq R} zN(t,dz), η t = μ t + σ W t + ∫ ∣ z ∣ < R z N ( t , d z ) + ∫ ∣ z ∣ ≥ R z N ( t , d z ) , where μ ∈ R d \mu\in\mathbb{R}^{d} μ ∈ R d , σ ∈ R d × d \sigma\in\mathbb{R}^{d\times d} σ ∈ R d × d , W W W is a d d d -dimensional Brownian motion and N N N is a Poisson random measure on R d \mathbb{R}^{d} R d . Then the Lévy measure ν \nu ν associated with N N N satisfies
∫ R d ( 1 ∧ ∣ z ∣ 2 ) ν ( d z ) < ∞ , (164) \int_{\mathbb{R}^{d}} (1 \wedge|z|^{2})\nu(dz) < \infty, \tag*{(164)} ∫ R d ( 1 ∧ ∣ z ∣ 2 ) ν ( d z ) < ∞ , (164) and the characteristic function φ η t : R d → C \varphi_{\eta_t} : \mathbb{R}^{d} \to\mathbb{C} φ η t : R d → C is given by
φ η t ( u ) : = E e i ⟨ u , η t ⟩ = e t ψ ( u ) , ⟨ a , b ⟩ : = ∑ i = 1 d a i b i . (165) \varphi_{\eta_t}(u) := \mathbb{E}e^{i\langle u,\eta_t\rangle} = e^{t\psi(u)}, \qquad\langle a,b\rangle:= \sum_{i=1}^{d} a_i b_i. \tag*{(165)} φ η t ( u ) := E e i ⟨ u , η t ⟩ = e t ψ ( u ) , ⟨ a , b ⟩ := i = 1 ∑ d a i b i . (165) where ψ \psi ψ , called the characteristic exponent of η \eta η , is given by
ψ ( u ) = i ⟨ μ , u ⟩ − 1 2 ⟨ u , σ σ T u ⟩ + ∫ ∣ z ∣ < R ( e i ⟨ u , z ⟩ − 1 − i ⟨ u , z ⟩ ) ν ( d z ) + ∫ ∣ z ∣ ≥ R ( e i ⟨ u , z ⟩ − 1 ) ν ( d z ) . (166) \psi(u) = i\langle\mu,u\rangle- \frac{1}{2}\langle u,\sigma\sigma^{T}u\rangle+ \int_{|z|<R}(e^{i\langle u,z\rangle}-1-i\langle u,z\rangle)\nu(dz) + \int_{|z|\geq R}(e^{i\langle u,z\rangle}-1)\nu(dz). \tag*{(166)} ψ ( u ) = i ⟨ μ , u ⟩ − 2 1 ⟨ u , σ σ T u ⟩ + ∫ ∣ z ∣ < R ( e i ⟨ u , z ⟩ − 1 − i ⟨ u , z ⟩) ν ( d z ) + ∫ ∣ z ∣ ≥ R ( e i ⟨ u , z ⟩ − 1 ) ν ( d z ) . (166) Conversely, given triplet ( μ , a , ν ) (\mu,a,\nu) ( μ , a , ν ) with a = σ σ T a=\sigma\sigma^{T} a = σ σ T and ν \nu ν satisfying (164) , there exists a Lévy process η \eta η satisfying (165) –(166) .
Proof. We will show why (165) –(166) holds for a scalar Lévy process of the compound Poisson type. In this case, η \eta η has the decomposition
η t = μ t + σ W t + ∑ n = 1 P t X n \eta_t=\mu t+\sigma W_t+\sum_{n=1}^{P_t}X_n η t = μ t + σ W t + n = 1 ∑ P t X n
where W W W is a scalar Brownian motion, P P P is a Poisson process with intensity λ \lambda λ and the ( X n ) n ∈ N (X_n)_{n\in\mathbb{N}} ( X n ) n ∈ N are iid random variables with common distribution F X F_X F X . We compute
E e i u η t = E e i u μ t + i u σ W t + i u ∑ n = 1 P t X n = e i u μ t E e i u σ W t E e i u ∑ n = 1 P t X n = e i u μ t − 1 2 σ 2 u 2 t E [ E [ e i u ∑ n = 1 P t X n ∣ P t ] ] = e i u μ t − 1 2 σ 2 u 2 t E [ ( E [ e i u X ] ) P t ] = e i u μ t − 1 2 σ 2 u 2 t E [ ( ∫ e i u z F X ( d z ) ) P t ] = e i u μ t − 1 2 σ 2 u 2 t exp ( λ t ( ∫ e i u z F X ( d z ) − 1 ) ) = e i u μ t − 1 2 σ 2 u 2 t exp ( λ t ∫ ( e i u z − 1 ) F X ( d z ) ) = e t ψ ( u ) , \begin{aligned}
\mathbb{E}e^{iu\eta_t}
&=\mathbb{E}e^{iu\mu t+iu\sigma W_t+iu\sum_{n=1}^{P_t}X_n}\\
&=e^{iu\mu t}\mathbb{E}e^{iu\sigma W_t}\mathbb{E}e^{iu\sum_{n=1}^{P_t}X_n}\\
&=e^{iu\mu t-\frac{1}{2}\sigma^2u^2t}\mathbb{E}\left[\mathbb{E}\left[e^{iu\sum_{n=1}^{P_t}X_n}\mid P_t\right]\right]\\
&=e^{iu\mu t-\frac{1}{2}\sigma^2u^2t}\mathbb{E}\left[\left(\mathbb{E}[e^{iuX}]\right)^{P_t}\right]\\
&=e^{iu\mu t-\frac{1}{2}\sigma^2u^2t}\mathbb{E}\left[\left(\int e^{iuz}F_X(dz)\right)^{P_t}\right]\\
&=e^{iu\mu t-\frac{1}{2}\sigma^2u^2t}\exp\left(\lambda t\left(\int e^{iuz}F_X(dz)-1\right)\right)\\
&=e^{iu\mu t-\frac{1}{2}\sigma^2u^2t}\exp\left(\lambda t\int(e^{iuz}-1)F_X(dz)\right)\\
&=e^{t\psi(u)},
\end{aligned} E e i u η t = E e i uμ t + i u σ W t + i u ∑ n = 1 P t X n = e i uμ t E e i u σ W t E e i u ∑ n = 1 P t X n = e i uμ t − 2 1 σ 2 u 2 t E [ E [ e i u ∑ n = 1 P t X n ∣ P t ] ] = e i uμ t − 2 1 σ 2 u 2 t E [ ( E [ e i u X ] ) P t ] = e i uμ t − 2 1 σ 2 u 2 t E [ ( ∫ e i u z F X ( d z ) ) P t ] = e i uμ t − 2 1 σ 2 u 2 t exp ( λ t ( ∫ e i u z F X ( d z ) − 1 ) ) = e i uμ t − 2 1 σ 2 u 2 t exp ( λ t ∫ ( e i u z − 1 ) F X ( d z ) ) = e t ψ ( u ) ,
where we have used E s P t = exp ( λ t ( s − 1 ) ) \mathbb{E}s^{P_t}=\exp(\lambda t(s-1)) E s P t = exp ( λ t ( s − 1 )) and ν = λ F X \nu=\lambda F_X ν = λ F X . The computation for a more general Lévy process in multiple dimensions is similar.
Let us provide a few examples of Lévy measures ν \nu ν and compute the integrals that appear in expression (166) for the characteristic exponent ψ \psi ψ .
Example 10.1.18 (Dirac comb). A Dirac comb Lévy measure on R \mathbb{R} R is a measure of the form
ν ( d z ) = ∑ i = 1 n λ i δ z i ( d z ) , \nu(dz)=\sum_{i=1}^{n}\lambda_i\delta_{z_i}(dz), ν ( d z ) = i = 1 ∑ n λ i δ z i ( d z ) , where λ i \lambda_i λ i is the intensity of a jump of size z i z_i z i . In this case, both (161) and (162) are satisfied, so we can choose either R = 0 R=0 R = 0 or R = ∞ R=\infty R = ∞ . Suppose we choose R = 0 R=0 R = 0 . Then the third term in (166) disappears and the last term becomes
∫ R ( e i u z − 1 ) ν ( d z ) = ∑ i = 1 n λ i ( e i u z i − 1 ) . \int_{\mathbb{R}}(e^{iuz}-1)\nu(dz)=\sum_{i=1}^{n}\lambda_i(e^{iuz_i}-1). ∫ R ( e i u z − 1 ) ν ( d z ) = i = 1 ∑ n λ i ( e i u z i − 1 ) .
Example 10.1.19 (Gaussian Jumps). A Gaussian Lévy measure on R \mathbb{R} R is a measure of the form
ν ( d z ) = λ 1 2 π s 2 exp ( − ( z − m ) 2 2 s 2 ) d z . \nu(\mathrm{d}z)=\lambda\frac{1}{\sqrt{2\pi s^2}}\exp\left(-\frac{(z-m)^2}{2s^2}\right)\mathrm{d}z. ν ( d z ) = λ 2 π s 2 1 exp ( − 2 s 2 ( z − m ) 2 ) d z . where m m m is the mean jump size, s 2 s^2 s 2 is the variance of the jumps, and the intensity of all jumps is ν ( R ) = λ \nu(\mathbb{R})=\lambda ν ( R ) = λ . In this case, both (161) and (162) are satisfied, so we can choose either R = 0 R=0 R = 0 or R = ∞ R=\infty R = ∞ . Suppose we choose R = 0 R=0 R = 0 . Then the third term in (166) disappears and the last term becomes
∫ R ( e i u z − 1 ) ν ( d z ) = λ ( e i u m − 1 2 s 2 u 2 − 1 ) . \int_{\mathbb{R}}(\mathrm{e}^{\mathrm{i}uz}-1)\nu(\mathrm{d}z)=\lambda\left(\mathrm{e}^{\mathrm{i}um-\frac{1}{2}s^2u^2}-1\right). ∫ R ( e i u z − 1 ) ν ( d z ) = λ ( e i u m − 2 1 s 2 u 2 − 1 ) .
Example 10.1.20 (Generalized tempered stable jumps). A Generalized tempered stable Lévy measure on R \mathbb{R} R is a measure of the form
ν ( d z ) = 1 { z < 0 } A − e − β − ∣ z ∣ ∣ z ∣ 1 + α − + 1 { z > 0 } A + e − β + z ∣ z ∣ 1 + α + . \nu(\mathrm{d}z)=\mathbf{1}_{\{z<0\}}\frac{A_{-}\mathrm{e}^{-\beta_{-}|z|}}{|z|^{1+\alpha_{-}}}+\mathbf{1}_{\{z>0\}}\frac{A_{+}\mathrm{e}^{-\beta_{+}z}}{|z|^{1+\alpha_{+}}}. ν ( d z ) = 1 { z < 0 } ∣ z ∣ 1 + α − A − e − β − ∣ z ∣ + 1 { z > 0 } ∣ z ∣ 1 + α + A + e − β + z . where the parameters ( A ± , α ± , β ± ) (A_{\pm},\alpha_{\pm},\beta_{\pm}) ( A ± , α ± , β ± ) are all positive. In this case (161) is satisfied by (162) is not. Thus, we must choose R = ∞ R=\infty R = ∞ . The fourth term in (166) then disappears and the third term becomes
∫ R ( e i u z − 1 − i u z ) ν ( d z ) = A − Γ ( − α − ) β − α − [ ( 1 + i u β − ) α − − 1 − i u α − β − ] + A + Γ ( − α + ) β + α + [ ( 1 − i u β + ) α + − 1 + i u α + β + ] . \begin{aligned}
\int_{\mathbb{R}}(\mathrm{e}^{\mathrm{i}uz}-1-\mathrm{i}uz)\nu(\mathrm{d}z)
={}&A_{-}\Gamma(-\alpha_{-})\beta_{-}^{\alpha_{-}}\left[\left(1+\frac{\mathrm{i}u}{\beta_{-}}\right)^{\alpha_{-}}-1-\frac{\mathrm{i}u\alpha_{-}}{\beta_{-}}\right]\\
&+A_{+}\Gamma(-\alpha_{+})\beta_{+}^{\alpha_{+}}\left[\left(1-\frac{\mathrm{i}u}{\beta_{+}}\right)^{\alpha_{+}}-1+\frac{\mathrm{i}u\alpha_{+}}{\beta_{+}}\right].
\end{aligned} ∫ R ( e i u z − 1 − i u z ) ν ( d z ) = A − Γ ( − α − ) β − α − [ ( 1 + β − i u ) α − − 1 − β − i u α − ] + A + Γ ( − α + ) β + α + [ ( 1 − β + i u ) α + − 1 + β + i u α + ] . assuming α ± ≠ 0 \alpha_{\pm}\ne0 α ± = 0 and α ± ≠ 1 \alpha_{\pm}\ne1 α ± = 1 . The above integral can also be computed explicitly for the cases α ± = 0 \alpha_{\pm}=0 α ± = 0 and α ± = 1 \alpha_{\pm}=1 α ± = 1 (see, e.g., (Cont and Tankov, 2004, Proposition 4.2)).
Assumption 10.1.21. From this point onward we assume E ∣ η t ∣ 2 < ∞ \mathbb{E}|\eta_t|^2<\infty E ∣ η t ∣ 2 < ∞ , and thus E ∣ η t ∣ < ∞ \mathbb{E}|\eta_t|<\infty E ∣ η t ∣ < ∞ . Thus, we can take R = ∞ R=\infty R = ∞ and express η \eta η as follows
η t = μ t + σ W t + ∫ R z N ~ ( t , d z ) . (167) \eta_t=\mu t+\sigma W_t+\int_{\mathbb{R}}z\widetilde{N}(t,\mathrm{d}z). \tag*{(167)} η t = μ t + σ W t + ∫ R z N ( t , d z ) . (167)
Lévy processes are semimartingales , which (roughly speaking) form the class of stochastic processes that can be used to construct an Itô integral. In § 19.3 we defined Itô integral of an ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\geq0} ( F t ) t ≥ 0 -adapted process Γ = ( Γ t ) t ≥ 0 \Gamma=(\Gamma_t)_{t\geq0} Γ = ( Γ t ) t ≥ 0 with respect to a Brownian motion W = ( W t ) t ≥ 0 W=(W_t)_{t\geq0} W = ( W t ) t ≥ 0 by introducing a sequence of simple processes Γ ( n ) = ( Γ t ( n ) ) t ≥ 0 \Gamma^{(n)}=(\Gamma_t^{(n)})_{t\geq0} Γ ( n ) = ( Γ t ( n ) ) t ≥ 0 that converged to Γ \Gamma Γ in the following sense
Γ t ( n ) : = ∑ j = 0 n − 1 Γ t j 1 { t j ≤ t < t j + 1 } , 0 ≤ t 0 < t 1 < ⋯ < t n = T , lim n → ∞ E ∫ 0 T ∣ Γ t − Γ t ( n ) ∣ 2 d t = 0. \Gamma_t^{(n)}:=\sum_{j=0}^{n-1}\Gamma_{t_j}\mathbf{1}_{\{t_j\leq t<t_{j+1}\}},\qquad0\leq t_0<t_1<\cdots<t_n=T,\qquad\lim_{n\to\infty}\mathbb{E}\int_0^T|\Gamma_t-\Gamma_t^{(n)}|^2\mathrm{d}t=0. Γ t ( n ) := j = 0 ∑ n − 1 Γ t j 1 { t j ≤ t < t j + 1 } , 0 ≤ t 0 < t 1 < ⋯ < t n = T , n → ∞ lim E ∫ 0 T ∣ Γ t − Γ t ( n ) ∣ 2 d t = 0.
We then by defined the Itô integral as the following limit
∫ 0 T Γ t d W t : = lim n → ∞ ∫ 0 T Γ t ( n ) d W t = lim n → ∞ ∑ j = 0 n − 1 Γ t j ( W t j + 1 − W t j ) . \int_0^T\Gamma_t\mathrm{d}W_t:=\lim_{n\to\infty}\int_0^T\Gamma_t^{(n)}\mathrm{d}W_t=\lim_{n\to\infty}\sum_{j=0}^{n-1}\Gamma_{t_j}(W_{t_{j+1}}-W_{t_j}). ∫ 0 T Γ t d W t := n → ∞ lim ∫ 0 T Γ t ( n ) d W t = n → ∞ lim j = 0 ∑ n − 1 Γ t j ( W t j + 1 − W t j ) .
We can define the Itô integral of Γ \Gamma Γ with respect to a Lévy process η \eta η in the same manner
∫ 0 T Γ t d η t : = lim n → ∞ ∫ 0 T Γ t ( n ) d η t = lim n → ∞ ∑ j = 0 n − 1 Γ t j ( η t j + 1 − η t j ) . \int_{0}^{T}\Gamma_t\,\mathrm{d}\eta_t := \lim_{n\to\infty}\int_{0}^{T}\Gamma_t^{(n)}\,\mathrm{d}\eta_t = \lim_{n\to\infty}\sum_{j=0}^{n-1}\Gamma_{t_j}(\eta_{t_{j+1}}-\eta_{t_j}). ∫ 0 T Γ t d η t := n → ∞ lim ∫ 0 T Γ t ( n ) d η t = n → ∞ lim j = 0 ∑ n − 1 Γ t j ( η t j + 1 − η t j ) .
Here, we must add a technical condition that the integrand Γ \Gamma Γ be a left-continuous process. If a process Γ \Gamma Γ is not left-continuous, then we can still define a stochastic integral as follows
I T = ∫ 0 T Γ t − d η t , I t − = lim s ↑ t Γ s . I_T=\int_{0}^{T}\Gamma_{t-}\,\mathrm{d}\eta_t,\qquad I_{t-}=\lim_{s\uparrow t}\Gamma_s. I T = ∫ 0 T Γ t − d η t , I t − = s ↑ t lim Γ s .
where ( Γ t − ) t ≥ 0 (\Gamma_{t-})_{t\geq0} ( Γ t − ) t ≥ 0 is the left-continuous version of ( Γ t ) t ≥ 0 (\Gamma_t)_{t\geq0} ( Γ t ) t ≥ 0 . Note that, if Γ \Gamma Γ is left-continuous then Γ t = Γ t − \Gamma_t=\Gamma_{t-} Γ t = Γ t − for all $t\geq 0. The process ( I t ) t ≥ 0 (I_t)_{t\geq0} ( I t ) t ≥ 0 will still be a right-continuous process as η \eta η is right-continuous. In view of (167) , for a left-continuous process Γ \Gamma Γ , we can separate an Itô integral into three terms
∫ 0 T Γ t d η t = ∫ 0 T Γ t μ d t + ∫ 0 T Γ t σ d W t + ∫ 0 T ∫ Γ t z N ~ ( d t , d z ) . (168) \int_{0}^{T}\Gamma_t\,\mathrm{d}\eta_t=\int_{0}^{T}\Gamma_t\mu\,\mathrm{d}t+\int_{0}^{T}\Gamma_t\sigma\,\mathrm{d}W_t+\int_{0}^{T}\int\Gamma_tz\widetilde{N}(\mathrm{d}t,\mathrm{d}z). \tag*{(168)} ∫ 0 T Γ t d η t = ∫ 0 T Γ t μ d t + ∫ 0 T Γ t σ d W t + ∫ 0 T ∫ Γ t z N ( d t , d z ) . (168)
Expression (168) suggests that we consider general processes of the form
d X t = μ t d t + σ t d W t + ∫ γ t ( z ) N ~ ( d t , d z ) , (169) \mathrm{d}X_t=\mu_t\mathrm{d}t+\sigma_t\mathrm{d}W_t+\int\gamma_t(z)\widetilde{N}(\mathrm{d}t,\mathrm{d}z), \tag*{(169)} d X t = μ t d t + σ t d W t + ∫ γ t ( z ) N ( d t , d z ) , (169)
where the processes ( μ t ) t ≥ 0 (\mu_t)_{t\geq0} ( μ t ) t ≥ 0 , ( σ t ) t ≥ 0 (\sigma_t)_{t\geq0} ( σ t ) t ≥ 0 and ( γ t − ( z ) ) t ≥ 0 (\gamma_{t-}(z))_{t\geq0} ( γ t − ( z ) ) t ≥ 0 , must be adapted to a filtration F = ( F t ) t ≥ 0 \mathbb{F}=(\mathcal{F}_t)_{t\geq0} F = ( F t ) t ≥ 0 obtained by observing the processes W W W and ∫ z N ( ⋅ , d z ) \int zN(\cdot,\mathrm{d}z) ∫ z N ( ⋅ , d z ) . Note that we have written things in differential form ; to obtain the integral form, simply integrate and add an initial condition. We have been somewhat vague about the dimension of the various objects appearing in (169) . In general, for X ∈ R d X\in\mathbb{R}^{d} X ∈ R d , we could have
μ : R + × Ω → R d , σ : R + × Ω → R d × n , γ : R + × R m × Ω → R d × k . \mu:\mathbb{R}_{+}\times\Omega\to\mathbb{R}^{d},\qquad\sigma:\mathbb{R}_{+}\times\Omega\to\mathbb{R}^{d\times n},\qquad\gamma:\mathbb{R}_{+}\times\mathbb{R}^{m}\times\Omega\to\mathbb{R}^{d\times k}. μ : R + × Ω → R d , σ : R + × Ω → R d × n , γ : R + × R m × Ω → R d × k .
where W W W is an n n n -dimensional Brownian motion (with W ( i ) ⊥ W ( j ) W^{(i)}\perp W^{(j)} W ( i ) ⊥ W ( j ) for i ≠ j i\ne j i = j ) and N ~ \widetilde{N} N is a k k k -dimensional Poisson random measure (with N ~ ( i ) ⊥ N ~ ( j ) \widetilde{N}^{(i)}\perp\widetilde{N}^{(j)} N ( i ) ⊥ N ( j ) for i ≠ j i\ne j i = j ) on R m \mathbb{R}^{m} R m . Component-wise, we have
d X t ( i ) = μ t ( i ) d t + ∑ j = 1 n σ t ( i , j ) d W t ( j ) + ∑ j = 1 k ∫ R m γ t − ( i , j ) ( z ) N ~ ( j ) ( d t , d z ) , i = 1 , 2 , … , d . \mathrm{d}X_t^{(i)}=\mu_t^{(i)}\mathrm{d}t+\sum_{j=1}^{n}\sigma_t^{(i,j)}\mathrm{d}W_t^{(j)}+\sum_{j=1}^{k}\int_{\mathbb{R}^{m}}\gamma_{t-}^{(i,j)}(z)\widetilde{N}^{(j)}(\mathrm{d}t,\mathrm{d}z),\qquad i=1,2,\ldots,d. d X t ( i ) = μ t ( i ) d t + j = 1 ∑ n σ t ( i , j ) d W t ( j ) + j = 1 ∑ k ∫ R m γ t − ( i , j ) ( z ) N ( j ) ( d t , d z ) , i = 1 , 2 , … , d .
We call any process of the form (169) a Lévy-Itô process .
We showed (for the scalar case) in Theorem 8.1.5 that if σ \sigma σ satisfies E ∫ 0 T ∣ σ t ∣ 2 d t < ∞ \mathbb{E}\int_{0}^{T}|\sigma_t|^2\mathrm{d}t<\infty E ∫ 0 T ∣ σ t ∣ 2 d t < ∞ then the Itô integral I t : = ∫ 0 t σ s d W s I_t:=\int_{0}^{t}\sigma_s\mathrm{d}W_s I t := ∫ 0 t σ s d W s is a martingale. Likewise, if
E ∫ 0 T ∫ R ∣ γ t ( z ) ∣ 2 ν ( d z ) d t < ∞ , \mathbb{E}\int_{0}^{T}\int_{\mathbb{R}}|\gamma_t(z)|^2\nu(\mathrm{d}z)\mathrm{d}t<\infty, E ∫ 0 T ∫ R ∣ γ t ( z ) ∣ 2 ν ( d z ) d t < ∞ ,
then the process M = ( M t ) 0 ≤ t ≤ T M=(M_t)_{0\leq t\leq T} M = ( M t ) 0 ≤ t ≤ T , defined by
M t : = ∫ 0 t ∫ R γ s ( z ) N ~ ( d s , d z ) , M_t:=\int_{0}^{t}\int_{\mathbb{R}}\gamma_s(z)\widetilde{N}(\mathrm{d}s,\mathrm{d}z), M t := ∫ 0 t ∫ R γ s ( z ) N ( d s , d z ) ,
is a martingale. It follows that if μ = 0 \mu=0 μ = 0 in (169) , and σ \sigma σ and γ ( z ) \gamma(z) γ ( z ) satisfy the above square integrability conditions, then the process X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 is a martingale.
The Itô formula for Lévy-Itô processes.
In this section, we suppose X is given by (169) and we show how to compute d f ( X t ) df(X_t) df ( X t ) . We simply state the result and then explain heuristically why it is true. To keep the statements of the results in this chapter readable, we make the following standing assumption.
Assumption 10.2.1 (Integrability). Whenever we write ∫ 0 T H t d W t \int_0^T H_t\,dW_t ∫ 0 T H t d W t or ∫ 0 T ∫ H t ( z ) N ~ ( d t , d z ) \int_0^T\int H_t(z)\widetilde{N}(dt,dz) ∫ 0 T ∫ H t ( z ) N ( d t , d z ) , we assume that
E ∫ 0 T H t 2 d t < ∞ , respectively, E ∫ 0 T ∫ H t 2 ( z ) ν ( d z ) d t < ∞ , \mathbb{E}\int_0^T H_t^2dt < \infty, \qquad\text{respectively,} \qquad\mathbb{E}\int_0^T\int H_t^2(z)\nu(dz)dt < \infty, E ∫ 0 T H t 2 d t < ∞ , respectively, E ∫ 0 T ∫ H t 2 ( z ) ν ( d z ) d t < ∞ , for every $T>0. Whenever we write ∫ 0 T H t d t \int_0^T H_tdt ∫ 0 T H t d t or ∫ 0 T ∫ H t ( z ) ν ( d z ) d t \int_0^T\int H_t(z)\nu(dz)dt ∫ 0 T ∫ H t ( z ) ν ( d z ) d t , we assume the integral is finite almost surely. In particular, under this assumption, all stochastic integrals with respect to W W W and N ~ \widetilde{N} N are martingales.
For example, in the Itô formula below, Assumption 10.2.1 requires that each of the integrands appearing in (171) (e.g., σ t f ′ ( X t ) \sigma_tf'(X_t) σ t f ′ ( X t ) and f ( X t − + γ t ( z ) ) − f ( X t − ) f(X_{t-}+\gamma_t(z))-f(X_{t-}) f ( X t − + γ t ( z )) − f ( X t − ) ) satisfy the relevant integrability condition.
Theorem 10.2.2 (One-dimensional Itô formula for a Lévy-Itô processes). Suppose Assumption 10.2.1 10.2.1 10.2.1 holds and X = ( X t ) t ≥ 0 X=(X_t)_{t\geq0} X = ( X t ) t ≥ 0 is a one-dimensional Lévy-Itô process of the form
d X t = μ t d t + σ t d W t + ∫ R γ t − ( z ) N ~ ( d t , d z ) . (170) dX_t=\mu_tdt+\sigma_tdW_t+\int_{\mathbb{R}}\gamma_{t-}(z)\widetilde{N}(dt,dz). \tag*{(170)} d X t = μ t d t + σ t d W t + ∫ R γ t − ( z ) N ( d t , d z ) . (170) where the drift μ \mu μ volatility σ \sigma σ and γ \gamma γ are maps
μ : R + × Ω → R , σ : R + × Ω → R + , γ : R + × Ω × R → R , \mu:\mathbb{R}_+\times\Omega\to\mathbb{R},\qquad\sigma:\mathbb{R}_+\times\Omega\to\mathbb{R}_+,\qquad\gamma:\mathbb{R}_+\times\Omega\times\mathbb{R}\to\mathbb{R}, μ : R + × Ω → R , σ : R + × Ω → R + , γ : R + × Ω × R → R , and N N N is a one-dimensional Poisson random measure on R \mathbb{R} R , which is independent of a one-dimensional Brownian motion W W W . Let f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R satisfy f ∈ C 2 ( R ) f\in C^2(\mathbb{R}) f ∈ C 2 ( R ) . Then
d f ( X t ) = ( μ t f ′ ( X t ) + 1 2 σ t 2 f ′ ′ ( X t ) ) d t + σ t f ′ ( X t ) d W t df(X_t)=\left(\mu_tf'(X_t)+\frac{1}{2}\sigma_t^2f''(X_t)\right)dt+\sigma_tf'(X_t)dW_t df ( X t ) = ( μ t f ′ ( X t ) + 2 1 σ t 2 f ′′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) N ~ ( d t , d z ) \qquad+\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})\right)\widetilde{N}(dt,dz) + ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) N ( d t , d z ) + ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) − γ t ( z ) f ′ ( X t − ) ) ν ( d z ) d t . (171) \qquad+\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})-\gamma_t(z)f'(X_{t-})\right)\nu(dz)dt. \tag*{(171)} + ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) − γ t ( z ) f ′ ( X t − ) ) ν ( d z ) d t . (171)
We are not going to prove Theorem 10.2.2 . However, we will attempt to understand why the formula is correct. Suppose for simplicity that ν ( R ) < ∞ \nu(\mathbb{R})<\infty ν ( R ) < ∞ . In this case we have a finite activity Lévy process and we can separate the compensated Poisson random measure N ~ \widetilde{N} N into two parts N ( d t , d z ) N(dt,dz) N ( d t , d z ) and ν ( d z ) d t \nu(dz)dt ν ( d z ) d t . In this case, we can write d X t dX_t d X t as
d X t = μ t d t + σ t d W t + ∫ R γ t ( z ) N ( d t , d z ) − ∫ R γ t ( z ) ν ( d z ) d t = ( μ t − ∫ R γ t ( z ) ν ( d z ) ) d t + σ t d W t + ∫ R γ t ( z ) N ( d t , d z ) = μ t ′ d t + σ t d W t + ∫ R γ t ( z ) N ( d t , d z ) , \begin{aligned}
dX_t=\mu_tdt+\sigma_tdW_t+\int_{\mathbb{R}}\gamma_t(z)N(dt,dz)-\int_{\mathbb{R}}\gamma_t(z)\nu(dz)dt \\
&= \left(\mu_t-\int_{\mathbb{R}}\gamma_t(z)\nu(dz)\right)dt+\sigma_t dW_t+\int_{\mathbb{R}}\gamma_t(z)N(dt,dz) \\
&=\mu'_t dt+\sigma_t dW_t+\int_{\mathbb{R}}\gamma_t(z)N(dt,dz),
\end{aligned} d X t = μ t d t + σ t d W t + ∫ R γ t ( z ) N ( d t , d z ) − ∫ R γ t ( z ) ν ( d z ) d t = ( μ t − ∫ R γ t ( z ) ν ( d z ) ) d t + σ t d W t + ∫ R γ t ( z ) N ( d t , d z ) = μ t ′ d t + σ t d W t + ∫ R γ t ( z ) N ( d t , d z ) ,
μ t ′ : = μ t − ∫ R γ t ( z ) ν ( d z ) , \mu'_t:=\mu_t-\int_{\mathbb{R}}\gamma_t(z)\nu(dz), μ t ′ := μ t − ∫ R γ t ( z ) ν ( d z ) ,
where we can now identify the drift of X X X as μ t ′ \mu'_t μ t ′ . Similarly, we can write d f ( X t ) df(X_t) df ( X t ) as
d f ( X t ) = ( μ t f ′ ( X t ) + 1 2 σ t 2 f ′ ′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) N ( d t , d z ) − ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) ν ( d z ) d t + ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) − γ t ( z ) f ′ ( X t ) ) ν ( d z ) d t = ( ( μ t − ∫ R γ t ( z ) ν ( d z ) ) f ′ ( X t ) + 1 2 σ t 2 f ′ ′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) N ( d t , d z ) = ( μ t ′ f ′ ( X t ) + 1 2 σ t 2 f ′ ′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) N ( d t , d z ) . (172) \begin{aligned}
df(X_t)={}&\left(\mu_t f'(X_t)+\frac{1}{2}\sigma_t^2f''(X_t)\right)dt+\sigma_t f'(X_t)dW_t\\
&+\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})\right)N(dt,dz)\\
&-\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})\right)\nu(dz)dt\\
&+\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})-\gamma_t(z)f'(X_t)\right)\nu(dz)dt\\
={}&\left(\left(\mu_t-\int_{\mathbb{R}}\gamma_t(z)\nu(dz)\right)f'(X_t)+\frac{1}{2}\sigma_t^2f''(X_t)\right)dt+\sigma_t f'(X_t)dW_t\\
&+\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})\right)N(dt,dz)\\
={}&\left(\mu'_t f'(X_t)+\frac{1}{2}\sigma_t^2f''(X_t)\right)dt+\sigma_t f'(X_t)dW_t\\
&+\int_{\mathbb{R}}\left(f(X_{t-}+\gamma_t(z))-f(X_{t-})\right)N(dt,dz). \tag*{(172)}
\end{aligned} df ( X t ) = = = ( μ t f ′ ( X t ) + 2 1 σ t 2 f ′′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) N ( d t , d z ) − ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) ν ( d z ) d t + ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) − γ t ( z ) f ′ ( X t ) ) ν ( d z ) d t ( ( μ t − ∫ R γ t ( z ) ν ( d z ) ) f ′ ( X t ) + 2 1 σ t 2 f ′′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) N ( d t , d z ) ( μ t ′ f ′ ( X t ) + 2 1 σ t 2 f ′′ ( X t ) ) d t + σ t f ′ ( X t ) d W t + ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) N ( d t , d z ) . (172)
Things are looking a bit more familiar now. The non-integral terms in (172) arise from the μ t ′ d t + σ t d W t \mu'_t dt+\sigma_t dW_t μ t ′ d t + σ t d W t part of d X t dX_t d X t . To understand the integral term in (172) , define
η t : = ∫ R z N ( t , d z ) . \eta_t:=\int_{\mathbb{R}}zN(t,dz). η t := ∫ R z N ( t , d z ) .
Observe that
Δ η t = ∫ R z Δ N ( t , d z ) , \Delta\eta_t=\int_{\mathbb{R}}z\Delta N(t,dz), Δ η t = ∫ R z Δ N ( t , d z ) ,
Hence, we have
Δ N ( t , d z ) = δ Δ η t ( z ) d z . \Delta N(t,dz)=\delta_{\Delta\eta_t}(z)dz. Δ N ( t , d z ) = δ Δ η t ( z ) d z .
It follows that
Δ X t : = X t − X t − = ∫ R γ t ( z ) Δ N ( t , d z ) = ∫ R γ t ( z ) δ Δ η t ( z ) d z = γ t ( Δ η t ) . \Delta X_t:=X_t-X_{t-}=\int_{\mathbb{R}}\gamma_t(z)\Delta N(t,dz)=\int_{\mathbb{R}}\gamma_t(z)\delta_{\Delta\eta_t}(z)dz=\gamma_t(\Delta\eta_t). Δ X t := X t − X t − = ∫ R γ t ( z ) Δ N ( t , d z ) = ∫ R γ t ( z ) δ Δ η t ( z ) d z = γ t ( Δ η t ) .
Finally, we have
Δ f ( X t ) = f ( X t ) − f ( X t − ) = f ( X t − + Δ X t ) − f ( X t − ) = f ( X t − + γ t ( Δ η t ) ) − f ( X t − ) = ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) δ Δ η t ( z ) d z = ∫ R ( f ( X t − + γ t ( z ) ) − f ( X t − ) ) Δ N ( t , d z ) . \begin{aligned}
\Delta f(X_t)=f(X_t)-f(X_{t-}) \\
&= f(X_{t-} + \Delta X_t) - f(X_{t-}) \\
= f(X_{t-} + \gamma_t(\Delta\eta_t)) - f(X_{t-}) \\
= \int_{\mathbb{R}} \left(f(X_{t-} + \gamma_t(z)) - f(X_{t-})\right)\delta_{\Delta\eta_t}(z)\,dz \\
= \int_{\mathbb{R}} \left(f(X_{t-} + \gamma_t(z)) - f(X_{t-})\right)\Delta N(t,dz).
\end{aligned} Δ f ( X t ) = f ( X t ) − f ( X t − ) = f ( X t − + γ t ( Δ η t )) − f ( X t − ) = ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) δ Δ η t ( z ) d z = ∫ R ( f ( X t − + γ t ( z )) − f ( X t − ) ) Δ N ( t , d z ) . = f ( X t − + Δ X t ) − f ( X t − )
This last expression agrees with the integral term in (172) .
Example 10.2.3. Consider the following Geometric Lévy process
d X t = b X t d t + a X t d W t + ∫ R ( e c ( z ) − 1 ) X t − N ~ ( d t , d z ) . (173) dX_t=bX_{t}dt+aX_tdW_t+\int_{\mathbb{R}}\left(e^{c(z)}-1\right)X_{t-}\widetilde{N}(dt,dz). \tag*{(173)} d X t = b X t d t + a X t d W t + ∫ R ( e c ( z ) − 1 ) X t − N ( d t , d z ) . (173)
Note, if X 0 > 0 X_0>0 X 0 > 0 then this process remains strictly positive since
X t − + ( e c ( z ) − 1 ) X t − = e c ( z ) X t − > 0. X_{t-}+\left(e^{c(z)}-1\right)X_{t-}=e^{c(z)}X_{t-}>0. X t − + ( e c ( z ) − 1 ) X t − = e c ( z ) X t − > 0.
Comparing (173) with (170) we identify
μ t = b X t , σ t = a X t , γ t ( z ) = ( e c ( z ) − 1 ) X t − . \mu_t=bX_t,\qquad\sigma_t=aX_t,\qquad\gamma_t(z)=\left(e^{c(z)}-1\right)X_{t-}. μ t = b X t , σ t = a X t , γ t ( z ) = ( e c ( z ) − 1 ) X t − .
We see an explicit expression for X t X_t X t . To this end, we define
f ( X t ) = log X t , f ′ ( X t ) = 1 X t , f ′ ′ ( X t ) = − 1 X t 2 . f(X_t)=\log X_t,\qquad f'(X_t)=\frac{1}{X_t},\qquad f''(X_t)=-\frac{1}{X_t^2}. f ( X t ) = log X t , f ′ ( X t ) = X t 1 , f ′′ ( X t ) = − X t 2 1 .
Plugging all of the above into (171) we obtain
d f ( X t ) = d log X t = ( b X t 1 X t − 1 2 a 2 X t 2 1 X t 2 ) d t + a X t 1 X t d W t + ∫ R ( log ( X t − + ( e c ( z ) − 1 ) X t − ) − log ( X t − ) ) N ~ ( d t , d z ) + ∫ R ( log ( X t − + ( e c ( z ) − 1 ) X t − ) − log ( X t − ) − ( e c ( z ) − 1 ) X t 1 X t ) ν ( d z ) d t = ( b − 1 2 a 2 ) d t + a d W t + ∫ R c ( z ) N ~ ( d t , d z ) + ∫ R ( c ( z ) − e c ( z ) + 1 ) ν ( d z ) d t = b ′ d t + a d W t + ∫ R c ( z ) N ~ ( d t , d z ) , \begin{aligned}
df(X_t) &= d\log X_t \\
&= \left(bX_t\frac{1}{X_t}-\frac{1}{2}a^2X_t^2\frac{1}{X_t^2}\right)dt+aX_t\frac{1}{X_t}dW_t \\
&\quad+\int_{\mathbb{R}}\left(\log\left(X_{t-}+\left(e^{c(z)}-1\right)X_{t-}\right)-\log(X_{t-})\right)\widetilde{N}(dt,dz) \\
&\quad+\int_{\mathbb{R}}\left(\log\left(X_{t-}+\left(e^{c(z)}-1\right)X_{t-}\right)-\log(X_{t-})-\left(e^{c(z)}-1\right)X_t\frac{1}{X_t}\right)\nu(dz)dt \\
&= \left(b-\frac{1}{2}a^2\right)dt+adW_t+\int_{\mathbb{R}}c(z)\widetilde{N}(dt,dz)+\int_{\mathbb{R}}\left(c(z)-e^{c(z)}+1\right)\nu(dz)dt \\
&= b'dt+adW_t+\int_{\mathbb{R}}c(z)\widetilde{N}(dt,dz),
\end{aligned} df ( X t ) = d log X t = ( b X t X t 1 − 2 1 a 2 X t 2 X t 2 1 ) d t + a X t X t 1 d W t + ∫ R ( log ( X t − + ( e c ( z ) − 1 ) X t − ) − log ( X t − ) ) N ( d t , d z ) + ∫ R ( log ( X t − + ( e c ( z ) − 1 ) X t − ) − log ( X t − ) − ( e c ( z ) − 1 ) X t X t 1 ) ν ( d z ) d t = ( b − 2 1 a 2 ) d t + a d W t + ∫ R c ( z ) N ( d t , d z ) + ∫ R ( c ( z ) − e c ( z ) + 1 ) ν ( d z ) d t = b ′ d t + a d W t + ∫ R c ( z ) N ( d t , d z ) ,
b ′ = b − 1 2 a 2 − ∫ R ( e c ( z ) − 1 − c ( z ) ) ν ( d z ) . b'=b-\frac{1}{2}a^2-\int_{\mathbb{R}}\left(e^{c(z)}-1-c(z)\right)\nu(dz). b ′ = b − 2 1 a 2 − ∫ R ( e c ( z ) − 1 − c ( z ) ) ν ( d z ) .
Integrating from 0 0 0 to t t t , we obtain
log X t − log X 0 = b ′ t + a W t + ∫ R c ( z ) N ~ ( t , d z ) . \log X_t-\log X_0=b't+aW_t+\int_{\mathbb{R}}c(z)\widetilde{N}(t,dz). log X t − log X 0 = b ′ t + a W t + ∫ R c ( z ) N ( t , d z ) .
Finally, adding log X 0 \log X_0 log X 0 to both sides and exponentiating, we have
X t = X 0 exp ( b ′ t + a W t + ∫ R c ( z ) N ~ ( t , d z ) ) . X_t=X_0\exp\left(b't+aW_t+\int_{\mathbb{R}}c(z)\widetilde{N}(t,dz)\right). X t = X 0 exp ( b ′ t + a W t + ∫ R c ( z ) N ( t , d z ) ) .