Next: MC361 Generalized Linear Models
Up: Year 3
Previous: MC349 Complex Analysis
MC360 Stochastic Modelling
| Credits: 20 |
Convenor: Mr. B. English |
Semester: 1 |
| Prerequisites: |
essential: MC160, MC260 |
|
| Assessment: |
Coursework: 10% |
Three hour exam: 90% |
| Lectures: |
36 |
Classes: |
none |
| Tutorials: |
12 |
Private Study: |
102 |
| Labs: |
none |
Seminars: |
none |
| Project: |
none |
Other: |
none |
| Total: |
150 |
|
|
Explanation of Pre-requisites
The modules MC160 and MC260 provide the core probability and distribution theory
which form an essential prerequisite for this module. A number of disparate
mathematical tools are also required, which include the solution of simple
differential/difference equations, and results from linear algebra and analysis.
Courses which may cover these topics have not been included as prerequisites; a
brief and informal description of the necessary tools and results are
included when appropriate.
Course Description
In earlier courses we were mainly concerned with
sequences of independent observations. Thus in a sequence of Bernoulli trials,
the probabilities of success and failure remain constant, and the
outcomes of successive trials are independent. In this module, we study random
processes which somehow evolve in time. In general, the probability distribution
among outcomes at some time t, now depends on the outcomes of the process at
earlier times. For example, as a simple generalisation of Bernoulli trials, we
might allow the probabilities of success and failure at the nth
trial be governed by the outcome at the previous trial; this provides us with a
simple two-state Markov chain (named after the Russian mathematician
A. A. Markov who used this idea to model the alternation of vowels and
consonants in a poem by Pushkin). This process and its extension to many states
and more general time domains; the Markov process, provides a rich variety of
models which have been applied in diverse areas such as; the
biological sciences, including medicine; economic and financial modelling;
engineering and the physical sciences; the social sciences and queuing theory.
The emphasis of the course is on encouraging probabilistic intuition
and insight, and developing problem solving skills. A solid body of theory is
covered (together with formal proofs where appropriate), but in the main the
probabilistic content of results, and their application to problem solving, is
stressed over analytical detail and proof.
Aims
To provide a body of core knowledge for probability modelling and
basic stochastic processes. In particular, to provide a solid grounding
in Markov processes, including the random walk and simple branching process;
the Poisson and related processes, including the birth-death process and simple
models for queuing theory; an introduction to the renewal process. To enhance
the student's probabilistic insight and problem solving skills.
Objectives
On completion of this module, students should:
- be able to apply the Total Probability Theorem to establish valid
recurrence relationships for appropriate probability models;
- be able to solve or verify the solution to a recurrence relationship
either directly, via a suitable generating function, or by induction;
- know what is meant by a Markov process; the Chapman-Kolmogorov equation;
- know what is meant by a Markov chain, transition probability matrix,
stationary and equilibrium distributions, first step and last step analysis
(forward and backward equations), how to find the n-step
transition probability matrix;
- be able to define and classify an irreducible Markov chain
and define the terms transient, recurrent, null and positive recurrent,
periodic and aperiodic, and ergodic;
- know (for Markov chains) the basic limit theorems, and
where appropriate their justification,
the decomposition theorems, and be able to apply these;
- know the basic results for the simple branching process and their
justification;
- know and understand the basic characterisations of a Poisson process and to
be able to derive and apply the basic properties of this process;
- understand generalisations of the Poisson process to two dimensions, the
non-homogeneous Poisson process, models for general lifetimes, and the terms
hazard and reliability function;
- be able to establish the basic differential-difference equation for the
birth-death process, establish or validate solutions in simple tractable
cases, establish the equilibrium distribution when it exists, and be able to
apply these.
Transferable Skills
- A knowledge of some of the key areas of modern probabilistic modelling,
together with the ability to apply such knowledge.
Such knowledge has wide application in such diverse areas as economic and
financial modelling, in manufacturing industry, and most branches of pure
science.
- The ability to formalise and analyse a problem, and present a logically
argued solution.
Syllabus
Review of some aspects of probability theory; the Total Probability Theorem,
generating functions, random sums, limit theorems, the solution of difference
equations. Basic concepts of a
stochastic process; examples. Markov processes: Chapman-Kolmogorov equation.
Markov chains; transition probability matrices; n-step
transition probabilities; forward and backward equations; unconditional
probabilities. Two-state Markov chain, limiting distribution, mean recurrence
time. Classification of states; definition of terms irreducible, closed,
absorbing, ephemeral, transient, recurrent, positive and null recurrent, period
and ergodic. Basic Limit theorems; the decomposition theorems. Limiting
distributions and the classification of states; examples to include the simple
random walk with reflecting barrier. Absorption probabilities, mean time to
absorption. Periodic chains. The simple branching process; mean and variance of
the size of the nth generation, extinction probabilities.
The Poisson process; time to first and nth events; times between events, the
Markov property. Generalisations to two or more dimensions, the non-homogeneous
Poisson process. Exponential waiting times, and other characterisations of the
Poisson process. Modelling general lifetimes; hazard and reliability functions,
the reliability of complex systems. The birth-death process; alternative
specifications of particular forms, linear birth/death, immigration/emigration;
examples to include solution of the immigration-linear-death process. Limiting
distributions and the general birth-death process. A introduction to queuing
theory.
An introduction to the renewal process; some basic limit distributions. Stopping
times and Wald's Equation, a waiting time paradox. An application to queues.
Reading list
Background:
D. R. Cox and H. D. Miller,
The Theory of Stochastic Processes,
Chapman and Hall, 1977.
S. Ross,
Stochastic Processes,
J. Wiley, 1996.
H. M. Taylor and S. Karlin,
An Introduction to Stochastic Modelling,
Academic Press, 1984.
H. C. Tuckwell,
Elementary Applications of Probability Theory,
Chapman and Hall, 1988.
Next: MC361 Generalized Linear Models
Up: Year 3
Previous: MC349 Complex Analysis
Roy L. Crole
10/22/1998