Application of reinforcement learning to balancing of acrobot

The acrobot is a two-link robot, actuated only at the joint between the two links. It is one of difficult tasks in reinforcement learning (RL) to control the acrobot because it has nonlinear dynamics and continuous state and action spaces. In this article, we discuss applying the RL to the task of balancing control of the acrobot. Our RL method has an architecture similar to the actor-critic. The actor and the critic are approximated by normalized Gaussian networks, which are trained by an on-line EM algorithm. We also introduce eligibility traces for our actor-critic architecture. Our computer simulation shows that our method is able to achieve fairly good control with a small number of trials.