H. YANG ,G.YIN,K.YIN and strongQstrong. ZHANG.pdf

ANZIAM J. 45(2003), 49–74 CONTROL OF SINGULARLY PERTURBED MARKOV CHAINS: A NUMERICAL STUDY 1 2 3 4 H. YANG , G. YIN , K. YIN and Q. ZHANG (Received 26 May, 2000; revised 13 March, 2001) Abstract This work is devoted to numerical studies of nearly optimal controls of systems driven by singularly perturbed Markov chains. Our approach is based on the ideas of hierarchical controls applicable to many large-scale systems. A discrete-time linear quadratic control problem is examined. Its corresponding limit system is derived. The associated asymp- totic properties and near optimality are demonstrated by numerical examples. Numerical experiments for a continuous-time hybrid linear quadratic regulator with Gaussian dis- turbances and a discrete-time Markov decision process are also presented. The numerical results have not only supported our theoretical findings but also provided insights for further applications. 1. Introduction This work is concerned with nearly optimal controls of systems driven by singularly perturbed Markov chains. We consider both discrete-time and continuous-time prob- lems. The main objectives are to study the issues related to reduction of complexity of the Markovian systems and to demonstrate the asymptotic properties numerically. Our numerical experiments have not only supported the theoretical findings, but also provided insights for further applications. In many real-world problems, a common practice in quantifying the dynamic rela- tionships of random events and uncertainties is to use stochastic processes in modelling and formulation. For systems having j