Recommender systems (RSs) have become an inseparable part of our everyday lives. They help us find favorite items to purchase, friends on social networks, and movies watch. Traditionally, the recommendation problem was considered be a classification or prediction problem, but it is now widely agreed that formulating as sequential decision can better reflect user-system interaction. Therefore, f...