Optimality Principles in Spacecraft Neural Guidance and Control

This paper reviews the use of Deep Neural Networks (DNNs) for end-to-end Guidance and Control in aerospace applications. This is known as Guidance & Control Networks, or G&CNets. The key points are summarized below.

G&C Introduction

• Current G&C approaches rely on computing the optimal trajectory before the launch of the mission, and the spacecraft follows the optimal trajectory and reduces deviations.
• If any event requires computing a new trajectory, then it is done at a ground station on Earth and transmitted to the spacecraft.
• This violates the minimum intervention principle and does not provide autonomy to the spacecraft. When the spacecraft deviates from the pre-computed trajectory or when uncertainty is high, there might be better actions required for optimality.
• This leads to the approach of continuously re-planning and computing the optimal guidance profile onboard via Model Predictive Control or global optimization for example
• Another approach is the use of DNNs to learn the optimal G&C function end-to-end. This has been studied in two main avenues:
  • Imitation Learning (behavioral cloning)
  • Deep Reinforcement Learning

Imitation Learning

Imitation learning or behavioral cloning consists of learning the parameters of the neural network via supervised learning given a dataset containing optimal state-action pairs of one or more tasks.
The key points to consider in imitation learning:

• the efficient creation of the dataset
• the need for the dataset to cover all the possible scenarios. In other words, if the spacecraft encounters a scenario that is not in its training set, it might extrapolate to an unusable or non-optimal control
• verification of the resulting onboard controller
• how many different tasks can be learned/embedded in a single DNN?

Deep Reinforcement Learning

In DRL, the agent learns a policy by interacting with the environment and receiving rewards. So, to be able to use DRL, the continuous control problem needs to be modeled as an MDP that includes all types of uncertainties, and a reward function must be designed. The main points to consider in DRL:

• the design of the reward function is crucial as optimality depends on it
• it requires intense computational requirements and a huge amount of data
• same issues of verification and number of tasks that can be effectively learned by one DNN

Future Work

• End-to-end training of large models, performed on-board, and subject to the availability of on-board accelerators and distributed systems
• Should G&CNets use imitation learning or reinforcement learning
• unresolved challenges in the complete certification and validation on DNN for space systems and mission-critical systems
• demonstrate robustness to stochastic parametric and dynamic uncertainties