Improving reinforcement learning for complex control: This research introduces a modular reinforcement learning architecture called Multiple Model-Based Reinforcement Learning (MMRL) designed for nonlinear, nonstationary control tasks. It decomposes complex tasks into multiple domains based on the predictability of environmental dynamics. The system comprises multiple modules, each with a state prediction model and a reinforcement learning controller. A “responsibility signal,” derived from the softmax function of prediction errors, weights module outputs and gates the learning of prediction models and controllers. The authors formulate MMRL for discrete-time, finite-state cases and continuous-time, continuous-state cases. MMRL's performance is demonstrated in a nonstationary hunting task in a grid world (discrete case) and in swinging up a pendulum with variable physical parameters (continuous case), highlighting its effectiveness for challenging control problems.
This Neural Computation publication aligns perfectly with the journal’s emphasis on theoretical and computational approaches to understanding neural and cognitive systems. By proposing a novel reinforcement learning architecture, the paper advances the field of machine learning and offers a new perspective on how complex control tasks can be tackled, which will engage a significant portion of the readership.