On reachability of Markov chains: A long-run average approach

Junca, Mauricio;Avila, Daniel
(2021) IEEE Transactions on Automatic Control — (2021)

Files

CORE_RP_3179.pdf
  • Open Access
  • Adobe PDF
  • 1.16 MB
Download

Details

Authors
  • Junca, Mauricio
    Author
  • Avila, DanielUCLouvain
    Author
Abstract
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we characterize the domain of attraction and escape set of the system, and a generalization called p-domain of attraction, using the aforementioned value function. Next, we solve the problem of maximizing the probability of reaching a set $A$ while avoiding a set B. Finally, we consider a constrained version of the previous problem where we ask for the probability of reaching the set $B$ to be bounded. In the finite case, we use linear programming formulations to solve these problem
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Junca, M., & Avila, D. (2021). On reachability of Markov chains: A long-run average approach. IEEE Transactions on Automatic Control. Accepted/in-press. https://doi.org/10.1109/tac.2021.3071334 (Original work published 2021)