Pure equilibrium strategies for stochastic games via potential functions

Strategic games with a potential function have quite often equilibria in pure strategies (Monderer and Shapley [4]). This is also true for stochastic games but the existence of a potential function is mostly hard to prove. For some classes of stochastic games with an additional structure, an equilibrium can be found by solving one or a finite number of finite strategic games.We call these games auxiliary games. In this paper, we investigate if we can derive the existence of equilibria in pure stationary strategies from the fact that the auxiliary games allow for a potential function. We will do this for zero-sum, two-person discounted stochastic games and non-zero-sum discounted stochastic games with additive reward functions and additive transitions (Raghavan et al. [8]) or with separable rewards and state independent transitions (Parthasarathy et al. [5]).