Reinforcement Learning with Euclidean Data Augmentation for State-Based Continuous Control

The paper proposes a data augmentation method for state-based locomotion RL tasks. The idea is simple: if a robot is rotated around the gravity axis (yaw axis), the resulting transition should remain dynamically valid and should have the same reward as the original one.

Benchmarking simulators like DeepMind Control suite typically expose joint information as observations. The key contribution of the paper is showing that the data augmentation works well if the state representation is changed from the usual joint-based features to limb-based kinematic features such as limb positions, velocities, orientations, angular velocities, and joint axes. Unlike joint features, limb features transform naturally under Euclidean rotations, enabling meaningful augmented replay samples.

The method integrates with off-policy RL algorithms like DDPG as follows:

• sample transitions from the replay buffer
• rotate a subset of states and next states around gravity and keep actions and rewards unchanged
• train normally on the augmented mini-batch

The methods adds 1 extra hyperparameter $\rho_{aug}$ which is the percentage of the mini-batch samples that are augmented with Euclidean rotations during training; for example, $\rho_{aug}=75\%$.

Limitations

• The augmentation ratio $\rho_{aug}$ is task-specific and requires tuning.
• The method depends on knowing valid Euclidean symmetries of the environment.
• It is mainly demonstrated on robot locomotion; it may not apply to continuous-control domains without geometric physical symmetries.
• Symmetry assumptions can break under asymmetric bodies, uneven terrain, wind, obstacles, or other external forces.