Table 1.
Variables used
| Variable | Description |
|---|---|
| Aa, Ap | Likelihood matrix (superscript denotes auditory or proprioceptive) |
| Ba, Bp | Transition matrix (superscript denotes auditory or proprioceptive) |
| oτa, oτp | Outcomes (agent observations; superscript denotes auditory or proprioceptive) |
| π | Policies |
| ζ | Likelihood precision |
| γ | Prior precision over policies |
| s | Hidden state (superscripts can be used to denote modality and subscripts to denote parameterization) |
| G | Expected free energy |
| F | Free energy |
| C | Prior preferences matrix |
| D | Beliefs about initial state |
Figure 1.
Effect of likelihood precision on state–outcome mapping. Top: A relatively precise likelihood matrix (high likelihood precision ζ) leads to a high-fidelity mapping between the state, s, and the outcome, o (in this case, if the state is equal to 1, then the probability of the outcome being equal to 1 is 0.9). Black = precise belief; gray = uncertainty regarding belief; white =4 very imprecise belief. Blue box highlights the probabilities of the outcomes associated with an arbitrary state. Bottom: In this case, the likelihood matrix has been made much less precise (all of its entries are now equal probabilities), corresponding to a low likelihood precision ζ. This leads to an uncertain state–outcome mapping.
Figure 2.
Neuronal message passing. This schematic illustrates the form of the (variational) message passing implied by active inference. Here sensory observations oτ inform beliefs about states under each policy sπτ (and this depends on the likelihood precision, ζ). These reciprocally influence beliefs about states in the past and future. Beliefs about states under each policy are used to compute the expected free energy for each policy. This informs the beliefs about policies, π, and is modulated by precision over policies, γ. Beliefs about policies are combined with beliefs about the states under each policy to compute the marginal beliefs about states (averaged under all policies), sτ. By manipulating γ, ζ, and π, we sought to induce changes in sτ. Bold terms represent vector quantities; italics are model parameters. A is the likelihood matrix; B is the state transition matrix; C is the prior preferences matrix. G is the expected free energy. H is the entropy of A. The filled circle containing oτ is sensory data (outcomes).
Figure 3.
Generative model. Here the generative model is presented more explicitly. In blue are the hidden states for listening or not listening and their associated outcomes—hearing or not hearing a voice. Mapping between states and outcomes occurs via the likelihood matrix A. This is either the mapping from Equation 2 or (when the hidden state for speaking has been inferred) a matrix with equal entries to simulate sensory attenuation (the reduction of auditory likelihood precision during speaking, to prevent inference that one is not speaking but listening). In pink are the hidden states for speaking or not speaking, mapped via identity matrices to the speaking or not speaking proprioceptive outcomes. Transitions between hidden states are accomplished via the transition matrices Ba and Bp.
Figure 4.
An imprecise likelihood matrix can cause false positive inference. The plots shown in this figure illustrate posterior beliefs about hidden states. True outcomes are noted above each trial. Darker colors mean greater probabilities of each state. Here we show that decreasing likelihood precision ζ (from left to right, ζ = 0.7; 0.525; 0.3) can lead to a false inference about the state of the world. Note that all other parameters were unchanged across these three simulations. Here we are looking at the beliefs of the agent about whether or not it is listening. In this example, it should listen (black box) and not listen (white box) in an alternating pattern to match the true outcomes of sound and no sound, respectively. Note that as there is an identity likelihood mapping between states and outcomes for speaking, the proprioceptive outcomes also indicate what the agent has inferred (i.e., whether it believes it is or is not speaking). In the leftmost figure, the agent infers the state of the world correctly, as reflected in its very certain beliefs (black squares) about states that correspond to external reality. In the middle figure, we have decreased the precision of the likelihood matrix, leading to uncertainty about whether or not the agent is in a listening state at the third and fifth time points; this decreased certainty is represented by the gray coloring over both possible states. In the third figure, a further decrease in ζ has led the agent to believe firmly that it is in the “listening to a voice” state (dark boxes at the third and fifth time points). These are the hallucinations (red circles). For Figures 4–6, the lower part of each panel, labeled “policies,” represents the inferences (posterior probabilities), over time, by the agent about which policy she is pursuing. Darker shading represents increasing probabilities. Each row represents an alternative policy. Plotted between the columns are the actions (listening or speaking) that would be selected during the transition between the preceding and the next time step, if the policy in that row were to be followed (i.e., in this figure, if Policy 1 is selected at the end of Time Step 3, the agent would choose to listen during Time Step 4). Note that the legend of icons is conserved for all figures.
Figure 5.
Increasing prior precision over policies given a noisy likelihood matrix can induce hallucinations. Darker boxes represent increasing levels of confidence. True outcomes are represented above each trial. On the left, an agent with a somewhat imprecise sensory mapping (likelihood precision ζ = 0.525) but a low prior precision over policies [γ = exp(64)] is uncertain about the state of the world and makes incorrect inferences (the inferred state does not concord with the outcome) at the second, third, and fourth time steps; at the third time step, the agent has hallucinated (infers it is listening to a voice when there is no voice present). Effects of low likelihood precision are denoted by the blue box. If γ is increased [γ = exp(−64)], the agent begins to hallucinate at the fifth time step as well (red rectangle). Note that this does not occur with baseline likelihood precision (γ = 0.7).
Figure 6.
Context sensitivity of synthetic hallucinations. At high likelihood precision, outcomes dominate state inference; at low likelihood precision, priors derived from policies dominate and hallucinations can occur in a policy-dependent manner. This figure demonstrates that only certain policy spaces can lead to hallucinations, and only in the presence of permissively low likelihood precision. When the agent has a relatively large policy space (top row) and likelihood precision is reduced (left to right likelihood precision ζ = 0.7, 0.525, 0.3), the agent correctly infers the presence of externally generated speech. When likelihood precision is high (left), it believes that it is able to listen (third time step) while generating speech. This is inconsistent with the policies and indicates that precise sensory evidence dominates the empirical priors derived from policy selection. At a sufficiently low likelihood precision (right), beliefs about the policy dominate sensory evidence about states, and this causes our agent to believe that she was not listening, having selected the speak policy. This is permitted by the sensory attenuation we have associated with the speaking state and the oppositional nature of our policies (which contributed to the attenuation). On the plots of posterior probabilities of policies over time, we see that the agent has inferred that she is pursuing the second policy by the fourth time step. As such, the agent has concluded that this policy is the most likely given the data observed and the actions taken. If this policy is removed (bottom row), this agent now no longer has access to Policy 2 and has selected alternative policies. This poses no problem when likelihood precision is high (left), as sensory evidence again dominates the inference. When precision is low, however, this policy does not explain the data well, leading to a hallucination at the third time step (highlighted in red).
Figure 7.
Putative mapping of Markov decision process (MDP) model to neurobiology. This figure shows the MDP model mapped onto putative neurobiology. This mapping should not be taken too seriously but serves as an illustration of how our model may relate to underlying functional anatomy. Here we have placed auditory outcomes, oa, in Wernicke’s area. Wernicke’s is connected via the arcuate fasciculus to the inferior frontal gyrus (IFG). The arcuate fasciculus represents the likelihood mapping between outcomes and auditory states, sa, which are located in IFG. Proprioceptive outcomes, op, are located in primary somatosensory cortex and map to proprioceptive states (representation of whether or not one is speaking), sp, in laryngeal motor cortex (LMC). States in IFG and LMC inform the selection of policies assigned to the striatum; the blue lines represent corticostriatal connections between IFG and striatum and LMC and striatum. The nucleus basalis represents cholinergic signaling, and the likelihood precision ζ modulates (blue arrow) the state–outcome mapping in the auditory modality. The ventral tegmental area/substantia nigra (VTA) represents dopaminergic signaling, encodes the prior precision over policies γ, and modulates the striatum (light blue arrow).