Value-based decision making

Imagine choosing between some money now and a larger amount later. The larger amount need not feel more worthwhile if the wait matters to you. How does the brain learn what an option is worth, compare it with alternatives and turn that comparison into a choice as circumstances change?[1] [2]

A choice can improve through repeating previously rewarded actions or through learning where different actions lead and planning ahead. How does the brain combine these routes and adjust them to changing circumstances? In Daw and colleagues’ task, a first choice usually led to one of two later choice situations, but sometimes led to the other; the later choices offered changing chances of a monetary reward. After a reward reached through an unusual route, simply repeating the first action differs from favoring the action that normally leads to that rewarding situation. This comparison helps distinguish stored action values from planning with a model of the task. The observed choices reflected both influences. Brain signals related to prediction errors—the difference between an expected and obtained outcome—also reflected their combination, rather than a clean separation into two independent systems.[1] [3]

Important signals are known. How the systems work together across changing situations remains unresolved.[1]

Known circuits and computational models explain parts of the process. A general account must distinguish alternative mechanisms and explain context-dependent behavior.[1] [3]

Choosing now changes which future outcomes remain possible. A choice also teaches something when the outcome differs from expectation. Explaining that loop connects learning from experience with planning, rather than treating each decision as an isolated expression of a fixed preference.[1] [3]

Planning lets an outcome change the appeal of an action that was not just taken, because the person knows where that action usually leads. Repeating rewarded actions supplies another way to learn without evaluating every future route. Understanding their interaction connects reward learning with flexibility, while leaving open how these laboratory distinctions apply to ordinary decisions involving uncertain consequences and changing goals.[1] [3]