The mountain car problem. The upper left panel shows the landscape or potential energy function , with a minimum at position, (green dot) that exerts forces on the car. The car is shown at the target position at the top of the hill at (red dot). The equations of motion of the car are shown below. Crucially, at the force is unity and cannot be overcome by the agent, because a squashing function is applied to action. This means the agent can only access the target by starting on the left hill to gain enough moment to carry it up the other side. The right panels show the cost function and empirical priors (model of flow) that constitute the agent. Cost is a function of position and a hidden (e.g., physiological) state that plays a role of satiety . When satiety is high, cost is uniformly negative; . Conversely, when satiety is low cost becomes negative near, and only near, the target location; . The equations of motion on the lower right are constructed to ensure that fixed points are only stable in regions of negative cost or divergence: see main text.
