September 23 :  Problems from the end of Chapter 4 Ross edition 12   Ch4-Exercises.pdf



Suggested problems, and some to be handed in and specified later;



1, 4, 5, 6 (more typically there is no simple algebraic formula for P^n)

8, 14, 16, 

17 (use the property that E(Y) = 2p-1 and the SLLN.  This does more formally what I suggested in one of the lectures).

20, 21, 29, 31, 35, 44 (are states 1 or m absorbing?)

50





Problems to hand in for Assignment 1

8 (a), (c) 

14

17

   problem 17 refers to Example 4.19 Ross 12.  This is the simple r.w. that we discussed in class, that is each increment is an independent r.v. Y that 

takes values +1 or -1 with probabilities p and  1-p respectively.

31

35

Plus a calculation related to the coupling argument from class 

X, Y are two independent MC with the transition matrix for problem 35.  

X ~ pi (stationary distribution), Y_0 = 1

Tau = inf( n >= 0 :  Y_n = X_n )

Find P(Tau = k), k = 0, 1



44

50 and 59 (additional problems, on limiting probabilities and mean time to end of game)



Oct 16, 2019 :  Solution to R12 Q59

Ross12-4-59.PDF



Problems for Assignment 2

(for now these are suggested problems and some will be required to hand in;  tentative due date will be the week after reading week)

Ross chapter 6 exercises Ch6-Exercises.pdf

chapter 6

1, 3, 8, 9 

(some additional problems will be added)

plus 

for the linear birth and death process finish deriving the DE for the second moment at time t.   Notice this is of the same form as the DE for the first moment, and uses the solution for the first moment as part of the forcing function (ie the additional function so the DE is not homogeneous).

For those students who are familiar with ODEs solve this and obtain the variance at time t.  Use initial condition X(0) = k and integer >= 1.

Notice there are two different expressions depending on mu = lambda or not equal.

Aside :  One could find higher moments in a similar fashion (do not cary out this higher moment DE part).

Problems for assignment 2 chapter 8 exercises Ross Ch8-Exercises.pdf