What This Document Is
This document contains complete solutions to Homework Two for Stochastic Processes I (MATH 4221) at Georgia Tech, Fall 2017. It addresses problems assigned after reading Chapter 1 of Lawler’s *Introduction to Stochastic Processes*. The homework focuses on determining if systems are Markov chains, calculating transition matrices, and applying concepts like stationary distributions and expected return times.
Why This Document Matters
This solutions guide is intended for students enrolled in MATH 4221 who have completed Homework Two and wish to check their work. It’s most valuable when used *after* an attempt to solve the problems independently, as a tool for understanding where errors were made and solidifying comprehension of the material. It serves as a key resource for self-assessment and reinforcing the core principles of stochastic processes.
Common Limitations or Challenges
This document provides *answers* to the homework problems, but it does not offer detailed explanations of the underlying concepts. It assumes familiarity with the material covered in Lawler’s Chapter 1. Users still need to understand the theory to fully benefit from the solutions. This is not a substitute for attending lectures, reading the textbook, or actively engaging with the course material.
What This Document Provides
The full document includes:
* Responses to preliminary questions regarding sources used, problem difficulty, and areas of interest.
* Detailed solutions for determining whether given examples represent Markov chains, including identifying state spaces and transition matrices where applicable.
* Calculations for finding stationary distributions using both matrix power approximation and direct eigenvector computation.
* Solutions for problems involving simple random walks on graphs, including calculating stationary distributions and expected return times.
* Specific answers to problems drawn from Lawler’s textbook (1.7 #1.1, 1.3, 1.4).
This preview does *not* include the student’s initial responses to the preliminary questions, nor does it provide step-by-step derivations of the solutions – only the final answers and some intermediate results are shown.
