Strategy complexity of finite-horizon Markov decision processes and simple stochastic games

Markov decision processes (MDPs) and simple stochastic games (SSGs) provide a rich mathematical framework to study many important problems related to probabilistic systems. MDPs and SSGs with finite-horizon objectives, where the goal is to maximize the probability to reach a target state in a given finite time, is a classical and well-studied problem. In this work we consider the strategy complexity of finite-horizon MDPs and SSGs. We show that for all $ε>0$, the natural class of counter-based strategies require at most $\log \log (\frac{1}ε) + n+1$ memory states, and memory of size $Ω(\log \log (\frac{1}ε) + n)$ is required. Thus our bounds are asymptotically optimal. We then study the periodic property of optimal strategies, and show a sub-exponential lower bound on the period for optimal strategies.

No reviews yet.

No comments yet.