Polynomial VASS MDPs

Abstract

Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented during each state transition. VASS MDPs can serve as abstractions of probabilistic programs, offering many decidable properties. In this paper, we develop techniques for analyzing the polynomiality of asymptotic estimates for VASS MDPs. That is, for every initial configuration of size $n$, we consider the number of transitions needed to reach a configuration with some counter negative. We show that for any VASS MDP, either for all $ε>0$ and all sufficiently large $n$ the number of transitions is at least $2^{n^{\frac{1}{2}-ε}}$ with probability at least $1-ε$, or for any superpolynomial function $f$, all $ε>0$ and all sufficiently large $n$ the number of transitions is upper bounded by $f(n)$ with probability at least $1-ε$. Furthermore, if the VASS MDP is DAG-like then we can replace $f(n)$ with $n^{k+ε}$, where $k=2\cdot 3^{w\cdot d+|T|}$ and $w,d$ and $T$ are the number of MECs, counters and transitions of the VASS MDP, respectively. It is decidable in polynomial time which case holds.