Madani, O., Thorup, M., & Zwick, U. (2010). Discounted deterministic Markov decision processes and discounted all-pairs shortest paths. ACM Transactions on Algorithms (TALG), 6(2), 1-25.
We present algorithms for finding optimal strategies for discounted, infinite-horizon, Determinsitc Markov Decision Processes (DMDPs). Our fastest algorithm has a worst-case running time of O(mn), improving the recent bound of O(mn2) obtained by Andersson and Vorbyov . We also present a randomized O(m1/2n2)-time algorithm for finding Discounted All-Pairs Shortest Paths (DAPSP), improving an O(mn2)-time algorithm that can be obtained using ideas of Papadimitriou and Tsitsiklis .