北京理工大学《概率论与数理统计》分布函数能够完整地描述随机变量的统计特性，但在某些实际问题中，不需要全面考查随机变量的变化，只需知道它的随机变量的某些数字特征也就够了.评定某企业的经营能力时，只要知道该企业例如：年平均赢利水平研究水稻品种优劣时，我们关心的是稻穗的平均粒数及平均重量考察一射手的水平，既要看他的平均环数是否高，还要看他弹着点的范围是否小，即数据的波动是否小.由上面的例子看到，平均盈利水平、平均粒数、平均环数、数据的波动大小等，都是与随机变量有关的某个数值，能清晰地描述随机变量在某些方面的重要特征，这些数字特征在理论和实践上都具有重要意义.另一方面，对于一些常用的重要分布，如二项分布、泊松分布、指数分布、正态分布等，其中的参数恰好就是某些数字特征，因此，只要知道了这些数字特征，就能完全确定其具体的分布.第四章随机变量的数字特征4.1随机变量的平均取值——数学期望4.2随机变量取值平均偏离平均值的情况——方差4.3 描述两个随机变量之间的某种关系的数——协方差与相关系数4.1 数学期望一离散型随机变量的数学期望二连续型随机变量的数学期望三常见分布的数学期望四随机变量函数的数学期望五数学期望的性质六、数学期望的应用一离散型随机变量的数学期望引例射击问题设某射击手在同样的条件下,瞄准靶子相继射击90次,(命中的环数是一个随机变量).射中次数记录如下命中环数Y0 1 2 3 4 5命中次数n k 2 13 15 10 20 30频率n k/n2/90 13/90 15/90 10/90 20/90 30/90试问:该射手每次射击平均命中靶多少环?解：平均命中环数这是以频率为权的加权平均命中环数Y0 1 2 3 4 5命中次数n k2 13 15 10 20 30频率n k /n 2/90 13/90 15/90 10/90 20/90 30/900211321531042053090×+×+×+×+×+×=21315102030012345909090909090=×+×+×+×+×+×50k k n k n =⋅∑ 3.37.==射中靶的总环数射击次数平均射中环数频率随机波动随机波动“平均射中环数”的稳定值?=由频率的稳定性知：当n 很大时：频率n k /n 稳定于概率p k 稳定于50k k n k n =⋅∑50k k k p =⋅∑50k k n k n =⋅∑“平均射中环数”等于射中环数的可能值与其概率之积的累加定义1 设X 是离散型随机变量，它的概率分布是:P {X =x k }=p k , k =1,2,…如果绝对收敛,则称它为X 的数学期望或均值.记为E (X ), 即如果发散，则称X 的数学期望不存在.1k k k x p ∞=∑1()k k k E X x p ∞==∑1||k k k x p∞=∑注意：随机变量的数学期望的本质就是加权平均数，它是一个数，不再是随机变量.注1：随机变量X 的数学期望完全是由它的概率分布确定的，而不应受X 的可能取值的排列次序的影响，因此要求绝对收敛1k k k xp ∞=<+∞∑11111(1)1ln 2234212n n−+−++−→− 1111111(2)1ln 22436852−−+−−+→注2.E (X )是一个实数，而非随机变量，它是一种以概率为权的加权平均，与一般的算术平均值不同，它从本质上体现了随机变量X 取可能值的真正的平均值，也称均值.当随机变量X 取各个可能值是等概率分布时，X 的期望值与算术平均值相等.假设X 1P80 85 90 1/4 1/4 1/21()800.25850.25+900.586.25E X =×+××=X 2P80 85 901/3 1/3 1/32()85.E X =注3.数学期望E(X)完全由随机变量X的概率分布确定，若X服从某一分布，也称E(X)是这一分布的数学期望.乙射手甲射手例1.甲、乙两个射击手，他们射击的分布律如下表所示，问：甲和乙谁的技术更好?击中环数8 9 10概率0.3 0.1 0.6击中环数8 9 10概率0.2 0.5 0.3单从分布列看不出好坏，解：设甲，乙两个射击手击中的环数分别为X 1，X 2E (X 1)=8×0.3+9×0.1+10×0.6=9.3（环）E (X 2)=8×0.2+9×0.5+10×0.3=9.1（环）例2.1654年职业赌徒德.梅尔向法国数学家帕斯卡提出一个使他苦恼很久的分赌本问题：甲、乙两赌徒赌技相同，各出赌注50法郎，每局中无平局.他们约定，谁先赢三局，则得到全部100法郎的赌本.当甲赢了2局，乙赢了1局时，因故要中止赌博.现问这100法郎如何分才算公平？解：假如比赛继续进行下去，直到结束为止. 则需要2局.这时，可能的结果为：甲甲，甲乙，乙甲，乙乙即：甲赢得赌局的概率为3/4，而乙赢的概率为1/4.设：X、Y分别表示甲和乙得到的赌金数. 则分布律分别为：X0 100 P1/4 3/4Y0 100 P3/4 1/4这时，可能的结果为：甲甲，甲乙，乙甲，乙乙即：甲赢得赌局的概率为3/4，而乙赢的概率为1/4.E(X)=0×1/4+100×3/4=75E(Y)=0×3/4+100×1/4=25即甲、乙应该按照3：1的比例分配全部的赌本.例3.确定投资决策方向?某人有10万元现金,想投资于某项目,预估成功的机会为30%,可得利润8万元,失败的机会为70%,将损失2万元.若存入银行,同期间的利率为5%,问是否做此项投资?解：设X 为此项投资的利润，则存入银行的利息：故应该选择该项投资.(注：投资有风险，投资须谨慎)X 8 −2P0.3 0.7此项投资的平均利润为：E (X )=8×0.3+(−2)×0.7=1(万元)10×0.05=0.5(万元)设X 是连续型随机变量，密度函数为f (x ).问题：如何寻找一个体现随机变量平均值的量.将X 离散化.二、连续型随机变量的数学期望在数轴上取等分点：…x −2<x −1<x 0<x 1<x 2<…x k +1−x k =∆x ，k =0,±1,….，并设x k 都是f (x )的连续点.则小区间[x i ,x i+1)阴影面积近似为f (x i )∆x i1()i x x f x dx+=∫()i f x x≈∆P {x i <X ≤x i +1}定义一个离散型随机变量X *如下：其数学期望存在，且绝对收敛时，P {X *=x i }=P {x i ≤X <x i +1} ≈f (x i )∆x对于X *，当当分点越来越密，即∆x →0时，可以认为X *=x i 当且仅当x i ≤X <x i +1(*)i i ix P X x =∑(*){*}i i iE X x P X x ==∑()i i ix f x x ≈∆∑0=lim ()i i x ix f x x ∆→∆∑则其分布律为E (X *) →E (X ) *0=lim x EX EX ∆→即有：+()xf x dx∞−∞=∫定义2：设X 是连续型随机变量，其密度函数为f (x )，如果绝对收敛，则称的值为X 的数学期望，如果积分发散，则称随机变量X 的数学期望不存在.+()xf x dx ∞−∞∫+||()x f x dx∞−∞∫即+()()E X xf x dx∞−∞=∫+()xf x dx ∞−∞∫记为E (X ).注意：随机变量的数学期望的本质就是加权平均数，它是一个数，不再是随机变量.三、常见分布的数学期望1.0−1分布设随机变量X服从参数为p的0−1分布，求EX.解：X的分布律为X0 1P1−p p则：E(X)=0×P{X=0}+1×P{X=1}=P{X=1}=p概率是数学期望的特例(第五章)2.二项分布X 的分布律为P {X =k }=C n k p k (1−p )n−k ,k =0,1,…,n .解：设随机变量X ~b (n ,p )，求EX .0{}nk EX kP X k ==∑0(1)n k k n k n k kC p p −=−∑1!(1)!()!n k n kk n k p p k n k −=−−∑1(1)(1)1(1)!(1)(1)!()!nk n k k n np p p k n k −−−−=−−−−∑11(1)1(1)n l k l ln ln l np Cp p −=−−−−=−∑1[(1)]n np p p −=+−np=抛掷一枚均匀硬币100次，能期望得到多少次正面3.泊松分布则解：X 的分布律为设随机变量X ~π(λ)，求EX .{},0,1,2,!kP X k e k k λλ−=== 00(){}!k k k e E X kP X k k k λλ−∞∞=====∑∑11(1)!k k ek λλλ−∞−==−∑1!ii k i e i λλλ∞=−−=∑=e e λλλλ−=1!k k e k k λλ−∞==∑泊松分布的参数是λ4.几何分布解：X 的分布律为P {X =k }=q k −1p ,k =1,2,….p+q =1设随机变量X 服从参数为p 的几何分布，求EX .111(){}k k k E X kP Xk k pq∞∞−=====⋅∑∑11k k p k q∞−=⋅∑1=()kk p q ∞=′∑1=()k k p q ∞=′∑()1q p q′=−211(1)p q p=−重复掷一颗骰子平均掷多少次才能第一次出现6点设X ~U (a , b )，求E (X ).解：X 的概率密度为：X 的数学期望为：数学期望位于区间(a ,b )的中点.5.均匀分布1()0a xb f x b a<<=− 其它()()2bax a b E X xf x dx dx b a +∞−∞+===−∫∫设X 服从指数分布，求E (X ).分部积分法6.指数分布当概率密度表示为：对应的数学期望为θ.,0()0,x e x f x x λλ− >=≤ 0xxedx λλ+∞−=∫()()E X xf x dx +∞−∞=∫1λ=1,0()0,0xe xf x x θθ− > = ≤解：X 的概率密度为：设X ~N (μ,σ2)，求E (X ).解：X 的概率密度为被积函数为奇函数，故此项积分为0.7.正态分布22()21()2x f x eµσπσ−−=()()E X xf x dx +∞−∞=∫22()212x xedxµσπσ−+∞−−∞=∫221()2x t t t edtµσσµπ−=+∞−−∞+∫ 2222122t t tedt edt σµππ+∞+∞−−−∞−∞+∫∫µ=N (0,1)的密度函数积分为1.注意：不是所有的随机变量都有数学期望例如：Cauchy 分布的密度函数为但发散故其数学期望不存在.21(),(1)f x x x π=−∞<<+∞+2||||()(1)x x f x dx dx x π+∞+∞−∞−∞=+∫∫四随机变量函数的数学期望设已知随机变量X的分布，我们需要计算的不是X的期望，而是X的某个函数的期望，比如说g(X)的期望. 那么应该如何计算呢？一种方法是，因为g(X)也是随机变量，故应有概率分布，它的分布可以由已知的X的分布求出来. 一旦我们知道了g(X)的分布，就可以按照期望的定义把E[g(X)]计算出来.例4.某商店对某种家用电器的销售采用先使用后付款的方式，记该种电器的使用寿命为X (以年计)，规定：X ≤1,一台付款1500元；1<X ≤2，一台付款2000元2<X ≤3，一台付款2500元；X >3，一台付款3000元设X 服从指数分布，且平均寿命为10年，求该商店一台电器的平均收费.解：设该商店一台电器的收费为Y .要求E (Y )X 的分布函数为：1101,()0,0x e x F x x − −>=≤设该商店一台电器的收费为YX ≤1，一台付款1500元1 <X ≤2，一台付款2000元2 <X ≤3，一台付款2500元X >3，一台付款3000元1101,0()0,0x ex F x x − −>=≤P {Y =1500}=P {X ≤1}=F (1)=1−e −0.1=0.0952P {Y =2000}=P {1<X ≤2}=F (2)−F (1)=0.0861P {Y =2500}=P {2<X ≤3}=F (3)−F (2)=0.0779P {Y =3000}=P {X >3}=1−F (3)=0.7408设X 服从指数分布，且平均寿命为10年.Y 的分布律为所以该商店一台电器的平均收费，即Y 的数学期望为Y 1500 2000 2500 3000P0.0952 0.0861 0.0779 0.7408()15000.095220000.086125000.0779 30000.74082732.15E Y =×+×+×+×=使用上述方法必须先求出g(X)的分布，有时这一步骤是比较复杂的.那么是否可以不先求g(X)的分布，而只根据X的分布求E[g(X)]呢？例5.设离散型随机变量X 的概率分布如下表所示,求:Z=X 2的期望.X−11P214141E (Z )= g (0)×0.5+g (-1)×0.25+g (1)×0.25解:=0.5注:这里的.)(2x x g =(1)当X 为离散型随机变量时，分布律为P {X = x k }=p k ，k =1,2,⋯(2)当X 为连续型随机变量时，概率密度函数为f (x ).定理：设Y 是随机变量X 的函数，Y =g (X )(g 是连续函数)若级数绝对收敛，则有若积分绝对收敛，则有1()[()]()kkk E Y E g X g x p∞===∑()[()]()()E Y E g X g x f x dx+∞==∫1()k k k g x p ∞=∑()()g x f x dx+∞−∞∫该公式的重要性在于：当求E [g (X )]时，不必知道g (X )的分布，而只需知道X 的分布就可以了，这给求随机变量函数的期望带来很大方便.k k k g x p X E Y E g X g x f x dx X 1(),()[()]()(),∞=+∞−∞== ∑∫离散型连续型例6.设随机变量X~b(n, p)，Y=e aX，求E(Y).解：因为X的分布律为所以有{}(1), 0,1,...,k k n knP X k C p p k n−==−= ()E Y=(1)nak k k n knke C p p−=−∑()(1)nk a k n knkC e p p−=−∑[(1)]a npe p=+−={}nakke P X k==∑例7.设X ~U [0,π]，Y=sinX ，求E (Y ).解：因为X 的概率密度为所以有1,0()0,x f x ππ≤≤ =其他()sin ()E Y xf x dx +∞−∞=∫01sin x dx ππ⋅∫2π=定理：设Z 是随机变量X 和Y 的函数，Z =g (X,Y )(g 是连续函数),Z 是一维随机变量(1)若(X,Y )是二维离散型随机变量，概率分布为(2)若(X,Y )是二维连续型随机变量，概率密度为f (x, y )，则有这里假定上两式右边的积分或级数都绝对收敛11()[(,)](,)ijijj i E Z E g X Y g x y p∞∞====∑∑()[(,)](,)(,)E Z E g X Y g x y f x y dxdy+∞+∞−∞−∞==∫∫{,},,1,2,i j ij P X x Y y p i j ====则有几个常用的公式()[(,)](,)(,)E Z E g X Y g x y f x y dxdy+∞+∞−∞−∞==∫∫(,)EX xf x y dxdy+∞+∞−∞−∞=∫∫(,)EY yf x y dxdy+∞+∞−∞−∞=∫∫22()(,)E Y y f x y dxdy+∞+∞−∞−∞=∫∫22()(,)E X x f x y dxdy+∞+∞−∞−∞=∫∫()(,)E XY xyf x y dxdy+∞+∞−∞−∞=∫∫例8.设二维随机变量(X ,Y )的密度函数为求E (X ),E (Y ),E (X +Y ),E (XY ).解：21(13),02,01,(,)40,x y x y f x y +<<<< =其它()(,)E X xf x y dxdy+∞+∞−∞−∞=∫∫212001(13)4x xdx y dy =⋅+∫∫43=()(,)E Y yf x y dxdy+∞+∞−∞−∞=∫∫212001(13)4xdx y y dy +∫∫58=数学期望的性质注意：X ,Y 相互独立()()(,)E X Y x y f x y dxdy+∞+∞−∞−∞+=+∫∫(,)(,)xf x y dxdy yf x y dxdy+∞+∞+∞+∞−∞−∞−∞−∞+∫∫∫∫()()E X E Y +45473824=+=()(,)E XY xyf x y dxdy +∞+∞−∞−∞=∫∫2120011(13)22x xdx y y dy=⋅⋅+∫∫455386=⋅=()()E X E Y ⋅设X =(X 1,…, X n )为离散型随机向量，概率分布为≥ 1nnj j j j n P X =x ,,x =p ,j ,,j .11{()}1Z = g (X 1,…, X n ),若级数绝对收敛，则.<∞∑ nnnj j j j j j g x ,,x p 111()=∑ nnnn j j j jj j E Z =E g X ,,X g x ,,x p 1111()(())()设X =(X 1,…, X n )为连续型随机向量，联合密度函数为 n f x x 1(,,)Z = g (X 1,…, X n ),若积分绝对收敛，则+∞+∞−∞−∞∫∫n n ng x x f x x x x 111(,,)(,,)d d n E Z E g X X 1()=((,,))+∞+∞−∞−∞=∫∫n n ng x x f x x x x 111(,,)(,,)d d五数学期望的性质1.设C 是常数，则E (C )=C 4.设X 、Y 相互独立，则E (XY )=E (X )E (Y );2.若k 是常数，则E (kX )=kE (X )3.E (X +Y )=E (X )+E (Y )注意:由E (XY )=E (X )E (Y )不一定能推出X ,Y 独立推广(诸X i 相互独立时)推广11[]()nni i i i i i E C X C E X ===∑∑11[]()n ni i i i E X E X ===∏∏性质4 的逆命题不成立，即若E (X Y ) = E (X )E (Y )，X ,Y 不一定相互独立.反例XY p ij -1 0 1-10181818181818181810p • j838382p i•838382X Y P-1 0 1828284EX EY ==0;E XY ()=0;=E XY EX EY ()但P X Y 1{=-1,=-1}=8≠=P X P Y 23{=-1}{=-1}8××=30+2103-3+5=92X XY Y X XY Y E(3+2-+5)=3E()+2E()-E()+E(5)性质2和3×××EX EY =310+2-3+5性质4例9.设X ～N (10,4)，Y ～U [1,5]，且X 与Y 相互独立，求E (3X ＋2XY －Y ＋5).解：由已知，有E (X )＝10, E (Y )＝3.例10: 设X 1 , X 2…,X n 相互独立且都服从B (1, p ),求Z = X 1 + X 2+…+X n 的数学期望E (Z ).解:注: 由二项分布的可加性易知Z = X 1 + X 2+…+X n ～B (n, p ).EZ = E (X 1 + X 2+…+X n )= E (X 1 ) +E ( X 2)+…+E (X n )= p +p +…+p =n p求二项分布的数学期望的又一种方法.例11.（超几何分布的数学期望）设一批同类型的产品共有N 件，其中次品有M 件.今从中任取n (假定n ≤N −M )件，记这n 件中所含的次品数为X ，求E (X ).则有所以解: 引入X ＝X 1+X 2+…+X n且易知抽签模型，概率与试验次数无关例10和例11：将X 分解成数个随机变量之和，然后利用随机变量和的期望等于期望的和这一性质，此方法具有一定的意义.1,,1,2,,0,i i X i n i ==第件是次品第件不是次品iMP X N{1}==1()ni i EX E X ==∑ni i P X 1{1}==∑1ni M N ==∑nM N =为普查某种疾病，N 个人需验血.有如下两种验血方案：（1）分别化验每个人的血，共需化验N 次；（2）分组化验.每k 个人分为1组，k 个人的血混在一起化验，若结果为阴性，则只需化验一次；若为阳性，则对k 个人的血逐个化验，找出有病者，此时k 个人的血需化验k+1次.设每个人血液化验呈阳性的概率为p ，且每个人化验结果是相互独立的.试说明选择哪一方案较经济.验血方案的选择例13.六、数学期望的应用解：只需计算方案(2)所需化验次数X 的期望.。
Limit Theorem极限定理Probability and Mathematical Statistics(概率与数理统计)Xi ZHANGIn many cases, we don’t need to calculate exactly the probability but roughly know it •Especially when the probability is very large or very small• e.g P{haze tomorrow in Lhasa}= ?•For example, suppose tossing a coin 1000 times, we would like to know if the probability of consecutive 17 appearance of heads•Let N be the number of occurrences of 17 consecutive heads in 1000 coin flips.N= I1+ … + I984E[I i ] = P(I i= 1) = 1/217E[N] = 984 ⋅1/217=0.007507Outlines•Chebyshev’s Inequality and the Weak Law of Large Numbers (切比雪夫不等式及弱大数定律）•The Central Limit Theorem(中心极限定理)•The Strong Law of Large Numbers（强大数定律）•SummaryMarkov’s Inequality （马尔可夫不等式）Proposition: Markov’s InequalityIf X is a random variable that takes only nonnegative values, then, for anyvalue a>0{}[] E XP X aa≥≤Hence, P[N ≥ 1] ≤E[N] / 1 ≤0.75%.E[X] = E[X | X ≥a]P(X≥ a) + E[X | X <a]P(X< a)≥ 0≥ a≥ 0E[X] ≥a P(X≥ a) + 0.Chebyshev’s Equality (切比雪夫不等式)Proposition: Chebyshev’s InequalityX is a random variable with finite mean μand variance σ2, then, for any value k >0,{}22P X k k σμ-≥≤P (|X –μ| ≥ k ) = P ((X –μ)2≥ k 2) ≤E [(X –μ)2] / k 2= σ2/ k 2Examples•Suppose that it is known that the number of items produced in a factory duringa week is a random variable with mean 50.(a) What can be said about the probability that this week’s production will exceed 75?(b) If the variance of a week’s production is known to equal 25, then what can be said about the probabilitythat this week’s production will be between 40 and 60?•Exercise•If X is uniformly distributed over the interval (0, 10),What can be said about the probability P X−5>4The Weak Law of Large Numbers （弱大数定理）Proposition:If Var(X)=0, then[]{}1P X E X ==Theorem: The weak law of large numbersLet X 1,X 2,...be a sequence of independent and identically distributed random variables, each having finite mean E[X i ]=μ.Then, for any ε>0,10 as n X X P n n με⎧++⎫-≥→→∞⎨⎬⎩⎭The Central Limit Theorem （中心极限定理）Let X 1,X 2,...be a sequence of independent and identically distributed random variables, each having mean μand variance σ2. Then the distribution oftends to the standard normal as n →∞. That is, for −∞<a <∞,1n X X n nμσ++-2/211 as 2a x n X X n P a e dx n n μσπ--∞++-⎧⎫≤→→∞⎨⎬⎩⎭⎰Example•An instructor has 50 exams that will be graded in sequence. The times required to grade the 50 exams are independent, with a common distribution that has mean 20 minutes and standard deviation 4 minutes. Approximate the probability that the instructor will grade at least 25 of the exams in the first 450 minutes of work.X = i=125X i E X = i=125E X i =25×20=500Var X = i=125Var X i =25×16=400P X ≤450=P X −500400≤450−500400≈P Z ≤−2.5=1−Φ2.5=0.006ExerciseIf 10 fair dice are rolled, find the approximate probability that the sum obtained is between 30 and 40, inclusive.The Strong Law of Large Numbers Theorem: The strong law of large numbersLet X 1,X 2,...be a sequence of independent and identically distributed ra ndom variables, each having a finite mean μ=E[X i ].Then, with proba bility 1,12 as n X X X n n μ+++→→∞Comparison between weak law of large numbers•Weak law of large numbers: For any specified large value n ∗, X 1+⋯+X n ∗n ∗is likely to be near μ, it does not say that (X 1+···+X n )/n is bound to stay near μfor all values of n larger than n ∗. Thus, it leaves open the possibility that large valu es of |(X 1+···+X n )/n −μ|can occur infinitely often (though at infrequen t intervals).•The strong law shows that this cannot occurSummary•Chebyshev’s Equality•Weak Law of Large Numbers •The Central Limit Theorem •The Strong Law of Large Numbers{}22P X kkσμ-≥≤10 asnX XP nnμε⎧++⎫-≥≤→∞⎨⎬⎩⎭2/211as2a xnX X nP a e dx nnμσπ--∞++-⎧⎫≤→→∞⎨⎬⎩⎭⎰12 asnX X Xnnμ+++→→∞•Homework 3 will be due on Nov. 6th•Midterm exam will be on Nov. 17th•Review lesson and sample questions will be given on Nov. 13th.Markov chains@stochastic processes •Stochastic processes•Many real-world systems contain uncertainty and evolve over time.•Stochastic processes (and Markov chains) are probability models for such systems.• A discrete-time stochastic process is a sequence of random variables:X0,X1,X2,. . . typically denoted by {X t}Time: t=0,1,2,...State: v-dimensional vector, s=(s1,s2,...,s v)In general, there are m states (a finite # of states): s1,s2,...,s mRandom walk problemA stochastic process whose state space is given by the integer i= 0,±1,±2,…is said to be a random walk if, for some number 0<p< 1,P i,i+1=p=1−P i,i−1,i=0,±1,±2,…A Markov ChainDefinition of the Markov chain•A stochastic process {X t}is called a Markov chain ifP X t+1=j X t=i,X t−1=i t−1,…,X1=i1,X0=i0=P X t+1=j X t=i=P ij←transition probabilities转移概率Discrete time means t T={0,1,2,...}The future behavior of the system depends only on the current state i and not on any of the previous states For all M states, ∑j p ij=1∀i and p ij≥0∀i,jP00⋯P0M⋮⋱⋮P M0⋯P MM ←transition matrix 转移概率矩阵X1X2X n-1X nP X t=i,X t−1=i t−1,…,X1=i1,X0=i0=P it−1,i t P X t−1=i t−1,…,X1=i1,X0=i0=P it−1,i t P it−2,i t−1…P i1,i2P i0,i1P{X0=i0}Matrix representationEach directed edge A B is associated with the positive transition probability from A to B 01000.800.200.300.50.200.0500.95A B B A C C DD0.20.30.50.050.950.20.81Example_Gambler’s RuinAt time zero the gambler has X 0=$2,and each day he makes a $1bet.He wins with probability p and lose with probability 1–p .He will quit if he ever obtains $4or if loses all my money.X t =amount of money he has after the bet on day t . State space is S ={0,1,2,3,4}3 with probability p 1with probability 1 –pif X t =4then X t+1= X t+2= • • • = 4, andif X t =0then X t+1= X t+2= • • • = 0.So , X 1 = {0123401000011-p0p 00201-p 0p 03001-p 0p400001n -step Transition Probabilities •Let p ij be probability of going from i to j in two transitions •In matrix form, P (2)= P ⨯P•For n = 3: P (3) = P (2) P = P 2 P = P 3•more generally, P (n ) = P (m )P (n -m )The ij th entry of this reduces top ij (n )= ∑p ik (m ) p kj (n -m )1 ≤m ≤n -1M k =0Chapman -Kolmogorov EquationsRevisit Gambler’s Ruin problem •Sometimes, the components P ij n in the transition matrix will converge•Limiting probability (极限概率) πj ,no matter what the initial state was.•Gambler ’s Ruin with p =0.75,t =300123401000010.325ε0ε0.67520.10ε00.930.025ε0ε0.975400001P (30)= 0123401000010.250 0.7500200.2500.75 03000.25 00.75400001Steady-State ProbabilitiesLet π= (π1, π2, . . . , πm) is the m-dimensional row vector of steady-state (unconditional) probabilities for the state space S= {1,…,m}. To find steady-state probabilities, solve linear system:π= πP, S j=1,mπj= 1, πj≥0, j= 1,…,mSteady-state probabilities might not exist unless the Markov chain is ergodic（遍历的）A Markov chain is ergodic if it is aperiodicand allows the attainment of any future state21from any initial state after one or moretransitions.3Game of Craps•The rules of craps is as follows:•The player rolls a pair of dice and sums the numbers showing.•A total of 7 or 11 on the first rolls wins for the player•Where a total of 2, 3, 12 loses•Any other number is called the point.•The player rolls the dice again.•If she rolls the point number, she wins•If she rolls number 7, she loses•Any other number requires another roll•The game continues until he/she wins or losesGame of Craps as a Markov ChainAll the possible statesStartLose WinP4P5P6P8P9P10ContinueGame of Craps NetworkProbability in crapsSum23456789101112Prob.0.0280.0560.0830.1110.1390.1670.1390.1110.0830.0560.028e.g.•Probability of win =P{7or11}=0.167+0.056=0.223•Probability of loss =P{2,3,12}=0.028+0.56+0.028=0.112Start Win Lose P4 P5 P6 P8 P9 P10Start 0 0.222 0.111 0.083 0.111 0.139 0.139 0.111 0.083Win 0 1 0 0 0 0 0 0 0Lose 0 0 1 0 0 0 0 0 0P4 0 0.083 0.167 0.75 0 0 0 0 0P =P5 0 0.111 0.167 0 0.722 0 0 0 0P6 0 0.139 0.167 0 0 0.694 0 0 0P8 0 0.139 0.167 0 0 0 0.694 0 0P9 0 0.111 0.167 0 0 0 0 0.722 0P10 0 0.083 0.167 0 0 0 0 0 0.75Transient Probabilities q(n)in CrapsThis is not an ergodic Markov chain so where you start is importantAbsorbing State Probabilities for CrapsInterpretation of Steady-State Conditions •Just because an ergodic system has steady-state probabilities doesnot mean that the system “settles down” into any one state• j is simply the likelihood of finding the system in state j after a large number of steps•The limiting probability πj that the process is in state j after a large number of steps is also equals the long-run proportion of time that the process will be in state j。
