Bayesian inference describes how prior expectations and sensory evidence can be combined. Predictive coding proposes a possible neural architecture for implementing this process, in which the brain continually generates predictions and updates them in response to prediction errors.

In the previous step, we introduced Bayesian inference as a computational account of perception. We now turn to a related question: how might the brain implement this form of inference?

In 1999, neuroscientists Rajesh Rao and Dana Ballard proposed a computational model of visual processing called predictive coding, in which the visual cortex embodies the logic of Bayesian inference.

In the model, the architecture works like this. Higher-level brain areas continuously send predictions of lower-level neural activity downward: this is what I expect to see. Lower-level areas compare those predictions with the actual incoming activity. If the prediction is accurate, there is little or no mismatch. If it is inaccurate, a mismatch signal is propagated upward. That mismatch is the prediction error.

This is a surprisingly economical design. According to the predictive-coding model, the brain might not need to pass the full sensory signal unchanged through every level of the hierarchy. It passes only what the model did not already anticipate. Expected input is, in a sense, explained away before it gets any further.

Take a look at the figure below, which shows this architecture across three levels of the visual hierarchy.

Predictive coding architecture (Rao & Ballard, 1999)
Higher areas generate predictions of lower-level activity; lower areas return the mismatch, or prediction error.

Karl Friston extended this framework significantly in 2005. Whereas Rao and Ballard had focused on the visual cortex, Friston proposed that the same architecture could provide a more general theory of cortical responses, suggesting that hierarchical predictive coding could explain perceptual inference and learning across sensory systems.

In his formulation, the cortical hierarchy implements a generative model, with each level generating predictions of the activity at the level below. Perception, on this account, is not a single computation but an ongoing process. The hierarchy continually generates predictions, compares them against incoming signals, propagates errors upward, and revises its models. Prediction error is not a failure of the system. It is the engine that drives perceptual inference and learning.

A crucial refinement in Friston’s formulation is that not all prediction errors contribute equally to belief updating. In the model, prediction errors are weighted by their estimated precision, which reflects how reliable or trustworthy each signal is.

When sensory evidence is estimated to be more precise than the prior prediction, the resulting prediction error has greater influence on belief updating. When the sensory evidence is less precise relative to the prior, its prediction error is discounted. Precision weighting therefore acts as a gain control on prediction errors.

Precision weighting (Friston, 2005)
Each level generates predictions of the activity at the level below. Not all prediction errors are weighted equally: their influence depends on the estimated precision of the sensory evidence relative to the prior prediction.
Precision = inverse variance of the error signal. Friston links precision weighting to attentional gain: attending to a stimulus raises its estimated precision, amplifying its influence on belief updating.

Friston and colleagues have proposed that precision weighting captures an important aspect of attention. To attend to a stimulus is to increase the estimated precision of the sensory signal and hence the weight given to the prediction errors it generates, granting them more influence over perception. On this account, attention is not simply a spotlight that selects information, but also a mechanism for calibrating how much different signals are trusted.

Precision weighting also helps explain why strong expectations can sometimes override incoming evidence. When the brain’s prior is much more precise than the incoming sensory signal, the posterior is pulled strongly toward the prior. The sensory evidence is discounted: the brain explains away the mismatch rather than substantially updating its model.

A good demonstration is the Hollow Mask Illusion. When we view the concave interior of a rotating face mask, we perceive it as a normal convex face protruding toward us. In Bayesian terms, our prior expectation that faces are convex is so strong that it can dominate the sensory depth cues indicating concavity. The resulting percept is therefore determined primarily by the prior rather than by the sensory evidence.

The visualization below shows how this shift in perceptual inference can be described in Bayesian terms. When the prior and sensory evidence are similarly reliable, the posterior sits between them. When the prior is highly precise and the sensory signal is weak or ambiguous, the posterior is pulled strongly toward the prior, which may reflect how the brain resolves the ambiguity of the hollow mask.

Hollow Mask Illusion

When strong priors override sensory evidence
If the prior is highly precise relative to the sensory evidence, the posterior is pulled strongly toward it and the sensory evidence has less influence.

Authors: Fabian Mueller & William Palmer

References

Rao, R. P. N., & Ballard, D. H. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1), 79–87. https://doi.org/10.1038/4580

Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456), 815–836. https://doi.org/10.1098/rstb.2005.1622

Yildirim, I., Belledonne, M., Freiwald, W., & Tenenbaum, J. (2020). Efficient inverse graphics in biological face processing. Science Advances, 6(10), eaax5979. https://doi.org/10.1126/sciadv.aax5979