Advantage Function

Topics: Reinforcement Learning

In addition to describing how good/bad an action is in an absolute sense, it is also useful to know how much better it is than other actions in average.
Hence, the advantage function corresponding to a policy $\pi$ describes how good it is to take a specific action from $\pi$ over randomly sampling actions from $\pi$:
$A^\pi (s,a) = Q^\pi(s,a) - V^\pi(s)$

References

• https://spinningup.openai.com/en/latest/spinningup/rl_intro.html