BLUEPRINTS OF THE MIND
4.12
One Mind, Many Models
The computational frameworks introduced in this chapter address different aspects of cognition. How do these approaches relate to one another, and what kinds of cognitive processes is each designed to explain?
We began with visual processing as a hierarchical system that transforms light into structured neural representations. From early feature detection to more complex combinations, the visual system extracts regularities from sensory input and transforms them into increasingly complex representations.
Pattern recognition offers an account of how these representations support object classification under variations in viewpoint, illumination, and context. Deep neural networks have become a powerful mathematical framework for modeling this capacity, classifying images by detecting and combining hierarchical features in ways that parallel certain properties of the ventral visual stream.
Yet not all aspects of perception are best explained by this approach. Because the same sensory input can arise from multiple external causes, perception also involves an inverse problem. Bayesian approaches describe its resolution as probabilistic inference, in which sensory evidence is integrated with prior expectations to estimate the most likely state of the world.
Analysis-by-synthesis and predictive coding extend this account by modeling perception as a continuous process of prediction and error correction: internal models generate expectations about sensory input, and mismatches between predicted and actual signals drive updating.
Discriminative and Generative Models
A fundamental distinction for understanding computational models of the brain is that between discriminative and generative approaches.
A discriminative model learns a direct mapping from input to output. Given a retinal image, it predicts the object category. It does not need to model how images are generated; it only needs to identify statistical regularities that distinguish one category from another. Deep neural networks trained for classification are a prototypical example.
A generative model, by contrast, represents a process through which observations could be produced. It captures aspects of the statistical structure of the environment and uses this model to infer the most likely causes of a given observation. Bayesian approaches to perception are often based on this logic.
The distinction matters because the two approaches carry different theoretical commitments. A generative model implies that perception involves inferring the hidden causes of sensory input. A discriminative model implies something closer to pattern classification: extracting task-relevant features without necessarily constructing a causal model of the world.
In biological vision, both modes may coexist. Rapid object recognition may rely substantially on feedforward discriminative processing. Slower recurrent processing may implement something closer to generative inference, as suggested by the effects of context, expectation, and ambiguity on perception. The question is therefore not which framework is universally correct, but which aspects of cognition each framework best explains.
The chapter then shifted from perception to learning. Reinforcement learning models describe how expectations are updated through prediction errors when outcomes differ from prior estimates. The Rescorla–Wagner model provided a formal account of this principle in Pavlovian conditioning, and later work on midbrain dopamine neurons suggested that the brain may implement functionally analogous reward-prediction-error signals.
The discussion then broadened to probabilistic generative models of intuitive physics, theory of mind, and self-perception. Across these domains, internal models were presented as supporting the simulation of physical events, the inference of other agents’ mental states, and the representation of the self as an acting agent within the world.
These frameworks differ in the processes they describe and in the theoretical claims they make. Some classify inputs, some infer hidden causes, some update expectations through experience, and others simulate possible states of the world. As the preceding step showed, they also leave important aspects of cognition unexplained.
This raises a final question: when a computational framework successfully predicts behavior or neural activity, what does that success allow us to conclude about how the mind actually works?
Authors: Fabian Mueller & William Palmer
References
Wichmann, F. A., & Geirhos, R. (2023). Are deep neural networks adequate behavioral models of human visual perception? Annual Review of Vision Science, 9, 501–524. https://doi.org/10.1146/annurev-vision-120522-031739
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23), 8619–8624.
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Computational Biology, 10(11), e1003915.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599.