RL Benchmarking Environments

Topics: Reinforcement Learning

Benchmarking Environments

There are several environments used in RL research to benchmark environments in a standardized way:

• DeepMind Control Suite (1-2): continuous control tasks based on the MuJoCo physics engine. What makes DMC attractive for benchmarking is that all the rewards are scaled in the range $[0,1]$ with episodes of $1000$ steps, which makes the episode returns for any DMC environment limited in the range $[0,1000]$
• OpenAI Gym (3): offers a mix of discrete and continuous control tasks. The continuous control tasks are based on the MuJoCo physics engine. It is different from DMC in that the rewards are not scaled and are not limited in a given range. In other words, each environment has a different reward scale.
• PyBullet (4): another set of continuous control benchmarking tasks based on the MuJoCo physics engine. Not as popular as DMC and OpenAI Gym.
• TORCS (5): open-source 3D game simulator that offers an API that allows it to be used as a racing platform
• CARLA (6): open-source simulator built on Unreal Engine, designed for developing, training and testing autonomous driving systems. It offers realistic roads and traffic conditions. It it not very popular in the RL literature because it does not offer standardized benchmark tasks.
• Meta-World (6): benchmark for meta-RL and multi-task learning

References

1. https://github.com/google-deepmind/dm_control
2. S. Tunyasuvunakool et al., “dm_control: Software and tasks for continuous control,” Software Impacts, vol. 6, p. 100022, Nov. 2020, doi: 10.1016/j.simpa.2020.100022.
3. https://gymnasium.farama.org/index.html
4. https://pypi.org/project/pybullet/
5. https://sourceforge.net/projects/torcs/
6. https://meta-world.github.io/