An international team from the Champalimaud Centre for the Unknown (CCU), in Portugal, and the University College London (UCL), in the UK, has uncovered a previously unknown effect of serotonin on learning. Their results are published in the June 26 2018 edition of the journal Nature Communications.

Serotonin is one of the main chemicals that nerve cells use to communicate with each other, and its effects on behavior are still unclear. For a long time, neuroscientists have been set on constructing an integrated theory of what serotonin actually does in the normal brain. But it’s been challenging to pin down serotonin’s function, especially for learning. Using a new mathematical model, now the authors found out why.

“The study found that serotonin enhances the speed of learning”, says Zach Mainen, one of the study’s leaders. “When serotonin neurons were activated artificially, using light, it made mice quicker to adapt their behavior in a situation that required such flexibility. That is, they gave more weight to new information and therefore changed their minds more rapidly when these neurons were active.” Serotonin has previously been implicated in boosting brain plasticity, and this study adds weight to that idea, thus departing from the common conception of serotonin as a mood-enhancer.

The new finding may help to better explain a medical enigma: why so-called “selective serotonin reuptake inhibitors”, or SSRIs – a class of antidepressants that are thought to act by increasing brain levels of circulating serotonin –, are more effective in combination with behavioral therapies, based on the reinforced learning of behavioral strategies to stave off depressive symptoms.

Using mathematical tools developed at UCL by Peter Dayan – who led the study together with Mainen, of the CCU – Kiyohito Iigaya, also at UCL, worked in collaboration with CCU co-authors Madalena Fonseca and Masayoshi Murakami.

In the experiments, mice had to perform a learning task in which the goal was to find water. “Animals were placed in a chamber where they had to poke either a water-dispenser on their left side or one on their right – which, with a certain probability, would then dispense water, or not”, explains Fonseca.

When they analysed the data, the scientists found that the amount of time the mice waited between trials (attempts to find water) was variable: either they immediately tried again, poking on one of the water-dispensers, or they waited longer before making a new attempt. It was this variability that allowed the team to reveal the likely existence of a novel effect of serotonin on the animals’ decision-making.

The long waiting intervals were more frequent at the beginning and at the end of a day’s session (run of trials). This probably happens because initially the mice are more distracted and not very engaged in the task itself, “perhaps hoping to get out from the experimental chamber”, the authors write. At the end, having drunk enough water, they are likewise less motivated for seeking reward.

Whatever the case, the team found that, depending on the length of the interval between trials, the mice adopted one of two different decision-making strategies to maximize their chances of reward (obtaining water).

Specifically, when the interval between trials was short, the mathematical model that best predicted the animals’ next choice was based almost completely on the outcome (water or no water) of the immediately preceding trial (namely, they poked the same water-dispenser again; if that failed to provide water, they would next switch to the alternative water-dispenser, a strategy known as “win-stay-lose-switch”).

This, the authors write, suggests that when the interval between two trials was short, the animals were mostly relying on their “working memory” to make their next choice – that is, on the part of short-term memory concerned with immediate perceptions. It’s this kind of memory that allows us to memorize a telephone number for a short time – and then forget it if we do not repeat it to ourselves over and over again.

On the other hand, when the interval between two consecutive trials lasted more than seven seconds, the model that best predicted the mice’s next choice suggested that the mice were using the accumulation of several experiences of reward to guide their next move – in other words, their long-term memory “kicked in” (the one that allows us to store things we learn, like playing the piano).

The CCU group also stimulated the serotonin-producing neurons in the animals’ brain with laser light, through a technique called optogenetics, to look for the effects of higher levels of serotonin on their foraging behavior. They sought to determine whether and how an increase in serotonin levels would affect each of the two different decision-making strategies they had just uncovered.

Something surprising then occurred. When they pooled together all the trials in their calculations, without taking into account the duration of the preceding interval, the scientists found no significant effect of their serotonin manipulation on the behavior. It was only when they took into account the above mentioned different decision-making strategies that they were able to extract from the data an increase in the animals’ rates of learning. Stimulation of serotonin-producing neurons boosted the effectiveness of learning from the history of past rewards, but this only affected the choices made after long intervals.

“Serotonin is always enhancing learning from reward, but this effect is only apparent on a subset of the animals’ choices”, says Murakami.

“To our surprise, we found that animals’ choice behavior was generated from two distinctive decision systems”, summarizes Iigaya. “On most trials, choice was driven by a ‘fast system’, where the animals followed a win-stay-lose-switch strategy. But on a small number of the trials, we found that this simple strategy didn’t explain the animals’ choices at all. On these trials, we instead found that animals followed their ‘slow system’, in which it was the reward history over many trials, and not only the most recent trials, that affected their choices. Moreover, serotonin affected only these latter choices, in which the animal was following the slow system.”

As to the role of SSRIs in treating psychiatric disorders like depression, the authors conclude: “Our results suggest that serotonin boosts [brain] plasticity by influencing the rate of learning. This resonates, for instance, with the fact that treatment with an SSRI can be more effective when combined with so-called cognitive behavioral therapy, which encourages the breaking of habits in patients.”

Full bibliographic information

An effect of serotoninergic stimulation on learning rates for rewards apparent after long intertrial intervals
Nature communications