From Occasional Choices to Inevitable Musts: A Computational Model of Nicotine Addiction

Figure 5

Error in expectation for each state of the system. In (a), (b), and (c), the δ error signal in expectation (blue lines) and smoking actions (red dots) are drawn. (a) Initially, the system randomly selects smoking or no smoking, but as time goes by, the system learns to select no smoking action. When no smoking behavior is firmly adopted, δ becomes 0 and no smoking action is always selected. (b) No particular behavior is adopted. The δ error signal oscillates between −1 and 1 throughout the simulation period, and the system randomly selects smoking or no smoking. The system does not learn any behaviors. (c) At first, the system randomly selects smoking or no smoking, but eventually, the system learns to select smoking action repeatedly. Even though no smoking is selected once in a while, the system quickly overcomes this decision and the learned behavior is adopted again. When addiction develops, δ becomes 0 and smoking action is always selected. (d) Bifurcation diagram of the action selection system (the branch colors are green, stable branch of nonaddictive behavior; blue, unstable branch of corresponding to exploration; red, stable branch of addiction).