Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximation

arXiv:2404.12648v1 Announce Type: new
Abstract: We study infinite-horizon average-reward Markov decision processes (AMDPs) in the context of general function approximation. Specifically, we propose a novel algorithmic framework named Local-fitted Optimization with OPtimism (LOOP), which incorporates both model-based and value-based incarnations. In particular, LOOP features a novel construction of confidence sets and a low-switching policy updating scheme, which are tailored to the average-reward and function approximation setting. Moreover, for AMDPs, we propose a novel complexity measure -- average-reward generalized eluder coefficient …

abstract approximation arxiv confidence construction context cs.lg decision features framework function general horizon loop markov novel optimism optimization processes sample stat.ml study type value