Figure 1
Narrative description of the social decision-making task. Over 64 days, the challenge is to maximise social encounters with Caroline. The agent has two moves (Caroline and Rudolph are both absorbing states, meaning that once the agent reaches them, it must stay there). For instance, on the first move, the agent can solicit information about Caroline’s availability by going on social media, and then, on the second move, decide where to go.
Table 1
Interventions.
| Baseline | The agent performs the social decision-making task in the absence of any adversity over a period of 64 days. |
| Severe depression | We induce social adversity on the 28th day by changing the uncertainty of social outcomes. The agent is now rejected by Rudolph (always) and by Caroline on a ‘bad day’. On a good day, the odds are inverted, such that Caroline is likely to afford a negative outcome. In other words, there is a flip in contingencies of the social environment. |
| Social support | We introduce social support on the 30th day, which reduces uncertainty about the outcomes of social encounters – and therefore resolves social adversity. This is modelled as an increase in Rudolph and Caroline’s reliability, which is increased when the agent forages for information on social media. Narratively, this could be interpreted as the agent signaling (implicitly or explicitly) to Caroline and Rudolph that they should be more consistent. Recovery thus depends on the sensitivity of the social environment and on how often the agent consults social media. |
| Pharmacotherapy | First-line pharmacotherapy typically employs either selective serotonin or norepinephrine reuptake inhibitors, and sometimes mixed serotonin or norepinephrine reuptake inhibitors (e.g., venlafaxine and duloxetine); the latter usually being used in patients who do not respond to serotonin reuptake inhibitors (Harmer et al., 2017). We simulate two types of synthetic pharmacotherapy: one motivated by serotonin and the other by norepinephrine. We assume, based on (Harmer et al., 2017), that serotonin upregulates prior expectations about initial states (i.e., increases the perceived probability of Caroline showing up), whereas noradrenaline introduces uncertainty about state transitions. Noradrenaline entails an overall loss of precise belief-updating during planning, a loss which underwrites the exploration of states that may lead to social reward. Condition 3 involves both noradrenaline and serotonin, condition 4 noradrenaline only, and condition 5 serotonin only. |
| Social support and pharmacotherapies | Condition 6 involves social support and both antidepressants; condition 7 involves support and noradrenaline only; and condition 8, support and serotonin only. |
Figure 2
Computational description of the decision-making task. The generative model and generative process of our decision-making task. Open circles represent random variables (hidden states and policies), filled circles represent the outcomes, squares represent model parameters (e.g., likelihood A, empirical priors B, D, G, and the evolutionary prior C). The generative model is shown in the upper part of the figure, while the process generating outcomes is shown in the lower part. The generative model and process are coupled through the same outcomes (o) and actions (u), where outcomes are used to infer hidden states and policies – and action is sampled from policies to change the states that are being inferred. States of the generative model are denoted by ‘s’ while states of the generative process are denoted as ‘s_bar’. The generative model is a joint probability distribution over outcomes and hidden states, which can be decomposed into factors. Factors are conditional densities (categorical: Cat; or Dirichlet: Dir) that make up the priors and likelihood of the generative model. Priors that depend on random variables, such as hidden states and policies, are empirical priors (e.g., priors that are learnt at a given hierarchical level or time scale). Priors that do not vary on this time scale are initialised as evolutionary priors (e.g., C). These are log preference vectors that rank the desirability of associated outcomes. Lower-case a and b correspond to matrices of concentration parameters for A and B respectively. The process whereby outcomes are generated decomposes into a series of belief updates: (i) Policy selection: the sequence of actions (i.e., plan or policy) is inferred under prior beliefs that the most likely policy minimises expected free energy (G); (ii) Inference about future states depends on state transitions encoded by the transition matrix (B) and the likelihood (A); (iii) Inference about outcome: the policy – with respect to the probability transitions – generates probabilistic outcomes at each time point. The likelihood of each outcome is encoded in the likelihood matrix (A), which attributes the probability of each possible outcome to each possible state; and (iv) Action: the agent selects the most likely action under posterior beliefs about policies. The green arrow highlights the circular causality that results when the generative model and process are coupled through outcomes and ensuing action. The process generating outcomes triggers the message-passing, under the generative model, which entails the evaluation of a policy, from which actions are selected. Actions change states in the generative process and a new outcome is generated. Thus, the cycle of perception and action continues. Learning corresponds to updating the concentration parameters that underwrite posterior beliefs about the likelihood of the sensory matrix (A). Each exchange with the environment is accumulated by concentration parameters. This accumulation encodes the probability of outcomes, given hidden states – enabling the agent to learn about environmental contingencies (and the social environment to change in response to the agent’s actions). The generative model and process can be defined for any scenario. The icons in the upper panel refer to changes in the generative model induced by (simulated) pharmacotherapy, or by changes in the generative process afforded by social adversity and support. These changes are described in the next figure. For a detailed description of the update equations and underlying theory, see (Friston, Parr, et al., 2017).
Table 2
Synthetic diagnostic criteria.
| Symptoms of normative depression | Anhedonia | Intensity | When the expected utility or reward is below the 95% confidence interval of the (healthy) control condition. Our subject experiences a lack of pleasure and disinterest in (prosocial) activities – of an intensity that a healthy phenotype experiences only about once every 20 days. |
| Duration | When the intensity criterion is met for multiple consecutive trials. Narratively, the subject experiences a lack of pleasure and disinterest in (social) activities – lasting many days. | ||
| Social withdrawal | Policies that do not lead to an encounter with social partners (see Figure 3) 1: Stay home, stay home (starting point, Figure 1) 4: Stay home, go to social media 7: Go to social media, go back home 10: Go to social media, stay on social media | ||
Figure 3
This figure details the likelihood and prior transition probabilities for our generative model of prosocial exchanges. The variables pertaining to the generative model are shown in light blue boxes, while the corresponding parameters of the generative process (i.e., the social world) are shown in light pink. The states and outcomes in this model are generated under two contexts pertaining to Caroline’s availability: available or not available. For ease of visualization, we have shown context-sensitive outcome likelihoods. In other words, there are six potential outcomes, but we have conditioned the epistemic (‘go’ and ‘no-go’) outcome on the context (to generate five outcomes). This simplifies the graphics and is licensed by the fact that only the epistemic outcome is context-sensitive. The top-left section corresponds to the contingencies during the initial exchanges (days 0–28) and corresponds with the narrative description in Figure 1. The adverse life event on the 28th day amounts to Rudolph and Caroline (on a good day) now yielding negative outcomes, and Caroline, even on a good day, affording negative outcomes. Adversity happens when the agent is sensitive to (i.e., prone to learn) the social environment. We implemented this by reinitialising the counts over the sensory prior beliefs of the agent (a). Social adversity and support are modelled by changing the precision or reliability of social outcomes in the generative process – in response to social signals. This is a subtle aspect of this model; namely, the generative process or social environment responds adaptively to the agent’s behaviour. As of the 30th day (for the conditions involving social support), we implement social support by adding counts (+10) to the likelihood of the environment counts (+10) for the cells corresponding to the mappings ‘Rudolph and positive outcomes’, ‘Caroline good day context and positive outcome’, and ‘Caroline busy day context and negative outcome’. The n_i corresponds to the number of times the agent visited the location a_i. The increase in counts has the ultimate consequence of driving the probability mapping in the (A) of the generative process towards and beyond their initial values more. A ‘+10’ is added to the cells every time the agent solicits the epistemic cue (i.e., social media). This implements the social signalling characteristic of adaptive low mood. Pharmacological interventions on the 35th day include the following: Serotonin provides an optimistic bias by changing prior beliefs about the initial states, in favour of the ‘Go’ context (from .5;.5 to .99;.01). Noradrenaline decreases the precision of the transition probability matrices B (i.e., it increases uncertainty about future states), which leads to a gradual accumulation of uncertainty about unvisited states. Through the expected ambiguity component of expected free energy G, it tends to motivate exploratory behaviours. The agent continues to learn the state transition after we administer noradrenaline.
Figure 4
Baseline. Top panel: The upper images show the posterior expectation of each of 10 policies (see method, Figure 2) as they evolve from day to day (64 in total). The small circles in the upper part of these panels indicate the observed outcomes (context in the first panel, and outcome in the second). The context changes every other day. The pragmatic value of these outcomes is shown as a (black) bar chart in the second panel. The lower panel describes the interventions that depend on the condition, and the symptoms, which are: (i) anhedonia when pragmatic value or reward (black bars) are below the pink bar over multiple days (duration, black shaded rounded rectangles), and (ii) social withdrawal, expressed by policies 1,4,7, and 10. The lower left panel provides a legend (upper) and a graphical description of the policies (lower). The intensity component of anhedonia corresponds inversely to the expected utility of a policy, or the extent to which it will yield preferred outcomes. Narratively speaking, this amounts to expecting socially rewarding outcomes when engaging a certain action. The intensity component of anhedonia is thus defined as low appetitive action. We assume that normal levels of appetitive action correspond to the expected utility experienced on most days, for a healthy (baseline) agent (pink line). The duration component of anhedonia corresponds to the number of consecutive days. A normative assessment of anhedonia thus would involve 14 consecutive days, as is the case in the condition of severe depression below.
Figure 5
Responses to intervention. This figure uses the same format as the upper panels of Figure 4. Interventions are indicated by the solid lines (red line: social adversity; blue line: social support; orange line: pharmacotherapy). The plots report the simulated responses to social adversity (red lines in all quadrants), and the remedial effects of social support (blue line in the second quadrant). Quadrants with orange lines show the corresponding effects of pharmacotherapy (serotonin or noradrenergic). The four treatment conditions show the same behavior over the first 28 days as the baseline scenario. This figure reports the results of conditions 1 to 4.
Figure 6
Responses to pharmacotherapy and social support. This figure uses the same format as Figure 4 and 5. Here, we report the responses to the final three conditions; namely, responses to serotonin and noradrenaline and combinations of drug treatment (yellow line), after social support (blue line).
Table 3
This table provides a summary of results in terms of the percentages of days post-adversity (out of 36) during which the synthetic subject met the subjective criteria for anhedonia in terms of intensity (expected utility below threshold) and duration (two or more consecutive days) and the behavioural criteria for social withdrawal.
| ANHEDONIA | SOCIAL WITHDRAWAL | |||
|---|---|---|---|---|
| INTENSITY CRITERION: EXPECTED UTILITY BELOW 95% CI (% OF 36 DAYS POST-ADVERSITY*) | DURATION CRITERION: 2 OR MORE CONSECUTIVE DAYS (% OF 36 DAYS POST-ADVERSITY*) | BEHAVIOR CRITERION: SELECTED POLICY 1,4,7, OR 1 (% OF 36 DAYS POST-ADVERSITY*) | ||
| CONDITION 1 Severe depression | 100% | 97% | 61% | |
| CONDITION 2 Adaptive mood (social support) | 6% | 3% | 0% | |
| CONDITION 3 Serotonin | 19% | 19% | 81% | |
| CONDITION 4 Noradrenaline | 53% | 53% | 75% | |
| CONDITION 5 Serotonin and noradrenaline combined | 61% | 61% | 50% | |
| CONDITION 6 Social support and serotonin combined | 6% | 0% | 0% | |
| CONDITION 7 Social support and noradrenaline combined | 44% | 31% | 22% | |
| CONDITION 8 Social support and serotonin and noradrenaline combined | 6% | 25% | 47% | |
[i] * Starting the first day after the adverse event (day 29th).