Hazard Avoidance Alerting With Markov Decision Processes
Name
winder_icat_report.pdf
Size
1.24 MB
Format
Adobe PDF
Checksum (MD5)
d5fdfeddc58aed7c7fcbe90105c38137
Author(s)
Winder, Lee F.
Kuchar, James K.
Date Issued
August 2004
Series/Report no.
ICAT-2004-4
Abstract
This thesis describes an approach to designing hazard avoidance alerting systems based on a
Markov decision process (MDP) model of the alerting process, and shows its benefits over
standard design methods. One benefit of the MDP method is that it accounts for future decision
opportunities when choosing whether or not to alert, or in determining resolution guidance.
Another benefit is that it provides a means of modeling uncertain state information, such as
knowledge about unmeasurable mode variables, so that decisions are more informed.
A mode variable is an index for distinct types of behavior that a system exhibits at different
times. For example, in many situations normal system behavior is safe, but rare deviations from
the normal increase the likelihood of a harmful incident. Accurate modeling of mode
information is needed to minimize alerting system errors such as unnecessary or late alerts.
The benefits of the method are illustrated with two alerting scenarios where a pair of aircraft
must avoid collisions when passing one another. The first scenario has a fully observable state
and the second includes an uncertain mode describing whether an intruder aircraft levels off
safely above the evader or is in a hazardous blunder mode.
In MDP theory, outcome preferences are described in terms of utilities of different state
trajectories. In keeping with this, alerting system requirements are stated in the form of a reward
function. This is then used with probabilistic dynamic and sensor models to compute an alerting
logic (policy) that maximizes expected utility. Performance comparisons are made between the
MDP-based logics and alternate logics generated with current methods. It is found that in terms
of traditional performance measures (incident rate and unnecessary alert rate), the MDP-based
logic can meet or exceed that of alternate logics.
Subjects
hazard avoidance
alerting systems
Markov decision process
air transportation
Persistent DSpace Link
