Elio Saade
Note

Symmetries and Model Minimization in Markov Decision Processes

The paper introduces the mathematical framework of reducing the size of a Markov Decision Process by identifying when different parts of the problem are essentially the same. They focus on MDPs with discrete states and actions but the ideas extend naturally to continuous.
The main idea is that many MDPs contain redundancy, and several state-action pairs look different on the surface but play the same role in decision making. The paper extends traditional model minimization methods by incorporating symmetry and allowing equivalence between state-action pairs when they have the same rewards and transition behavior.
The paper contains mostly mathematical definitions and theorems that formalize the problems, which I will not be repeating here.

Connections

Direct relationships to this note.