On Skolemhardness and saturation points in Markov decision processes
Abstract
The Skolem problem and the related Positivity problem for linear recurrence sequences are outstanding numbertheoretic problems whose decidability has been open for many decades. In this paper, the inherent mathematical difficulty of a series of optimization problems on Markov decision processes (MDPs) is shown by a reduction from the Positivity problem to the associated decision problems which establishes that the problems are also at least as hard as the Skolem problem as an immediate consequence. The optimization problems under consideration are two nonclassical variants of the stochastic shortest path problem (SSPP) in terms of expected partial or conditional accumulated weights, the optimization of the conditional valueatrisk for accumulated weights, and two problems addressing the longrun satisfaction of path properties, namely the optimization of longrun probabilities of regular cosafety properties and the modelchecking problem of the logic frequencyLTL. To prove the Positivity and hence Skolemhardness for the latter two problems, a new auxiliary path measure, called weighted longrun frequency, is introduced and the Positivityhardness of the corresponding decision problem is shown as an intermediate step. For the partial and conditional SSPP on MDPs with nonnegative weights and for the optimization of longrun probabilities of constrained reachability properties (a U b), solutions are known that rely on the identification of a bound on the accumulated weight or the number of consecutive visits to certain sates, called a saturation point, from which on optimal schedulers behave memorylessly. In this paper, it is shown that also the optimization of the conditional valueatrisk for the classical SSPP and of weighted longrun frequencies on MDPs with nonnegative weights can be solved in pseudopolynomial time exploiting the existence of a saturation point.
 Publication:

arXiv eprints
 Pub Date:
 April 2020
 arXiv:
 arXiv:2004.11441
 Bibcode:
 2020arXiv200411441P
 Keywords:

 Computer Science  Logic in Computer Science
 EPrint:
 Conference version accepted for publication at ICALP'20