How should we approach the challenge of understanding the human mind? Is it enough to look at just biology, or behavior, or its function? In this step, we explore a foundational perspective that has shaped cognitive science: the idea that understanding a complex system like the brain requires explanations at multiple levels.

When we ask, “How does the brain work?”, our first intuition might be to look at its physical structure – neurons, synapses, neurotransmitters, and electrical signals. One might think that understanding the brain simply means tracing these physical mechanisms, much like one explains a machine by analyzing its gears and circuits.

The answer must lie in how neurons fire, how they connect, and how their patterns of activity give rise to mental functions. Right?

Not quite. In his foundational work Vision (1982), British neuroscientist David Marr, one of the pioneers in the development of the field of Computational Neuroscience, argued that focusing on the neural level alone cannot explain perception.

“Trying to understand perception by studying only neurons is like trying to understand bird flight by studying only feathers. It just cannot be done. To understand bird flight, you need to understand aerodynamics. Only then can one make sense of the structure of feathers and the shape of wings. Similarly, you can’t reach an understanding of why neurons in the visual system behave the way they do just by studying their anatomy and physiology.”

David Marr, 1982

Marr challenged the view that biological mechanisms on their own are no complete explanation. Understanding cognition requires thinking across multiple levels: To truly understand a complex system like the brain, we must consider:

• What is the problem the system is solving, and why?
• How is the problem solved? What are the specific steps that the system uses to solve the problem?
• And lastly, where and how is this physically implemented in the brain?

These levels of inquiry form the basis of what has become one of the most influential frameworks in cognitive science: Marr’s Three Levels of Analysis. Below, we illustrate each level with two complementary examples: addition as a simple formal example, and color vision as a more applied example.

1. Computational (or functional) level

Question: “What is the goal of the computation, why is it appropriate, and what is the logic of the strategy by which it can be carried out?“ (David Marr, 1982)

Computational

This level is essentially defining what computation is being done or what problem the system is solving. Simply put, what is the computation for; and what is the goal (why is it useful)?

Addition: The goal is to determine the combined magnitude of two or more numbers. The inputs are numerical values, and the output is their sum.

Color Vision: From an evolutionary viewpoint, color supports efficient foraging by helping differentiate ripe from unripe or toxic fruit. The inputs are wavelengths of light striking the eyes, and the output is a color percept.

Example color vision

2. Algorithmic Level

Question: “How can this computational theory be implemented? In particular, what is the representation for the input and output, and what is the algorithm for the transformation?“ (David Marr, 1982)

Algorithmic & Representation

This level specifies how the computation is carried out: what representations are used, and what algorithm processes them. A representation is simply the form in which the actual inputs are used in the algorithm. Regarding the algorithm, if we think about computation like cooking, the algorithm would be the recipe outlining the explicit steps getting us from the inputs (i.e., ingredients) to the output (delicious meal/cognition).

Addition: Numbers can be represented in many formats (Arabic numerals, binary, Roman numerals). A simple algorithm is the familiar digit-wise summation taught in school, carrying over tens when needed.

Color Vision: Here we must define how patterns of incoming light map onto color categories (e.g., 530nm → “green” | 580 → “yellow” | 555 → “chartreuse”, etc.). Any algorithm solving color vision must take into account that the light hitting our eyes depends jointly on the reflectance and illumination of the observed object and therefore

1. separate the two and
2. normalize for lighting to extract the object’s color.

Because the sensory data conflate these two factors, this is an inherently ill-posed (undetermined) underdetermined problem, which we will revisit later.

3. Implementation Level

Question: “How can the representation and algorithm be realized physically?“ (David Marr, 1982)

Implementation

Addition: Calculators implement addition through circuits using logic gates. In other words, the algorithm for addition is physically realised in electronic components that encode and transform numerical inputs according to the rules of arithmetic. At the implementational level, the question is not what addition is for, or which procedure is used, but how that procedure is instantiated in a physical system.

Example Addition: 4-Bit Calculator Built Using Digital Logic Gates
© Cody Wabiszewski, Global Science Network (used with permission)

Color Vision: For color vision, this is how they are embedded into the brain: the neurons, circuits, and physiological mechanisms. Human color vision is implemented through three classes of cone photoreceptors, short (S), medium (M), and long (L) wavelength sensitive, whose relative activation supports trichromatic color perception. Neural circuits in the retina and visual cortex transform these signals into the colors we experience.

Schematic diagram of the biological vision system

The key idea here is that a proper theory of a cognitive process of the mind must be explained at all three of these levels which inherently constrain one another. A theory restricted to one level is incomplete: identifying neural circuits is insufficient without understanding the computational problem they solve, while specifying an algorithm is useless if it cannot be realized in biological tissue.

Once the computational goal is defined, it constrains the range of possible algorithms, which must in turn be feasible within the neural implementation. The figure below summarises these relationships using addition and color vision as examples.

Representations are especially important in theories of cognition because the mind does not merely respond to stimuli, but operates over internal states that stand in for relevant features of the world. To explain a cognitive process or capacity, we therefore often need to specify not only the sequence of operations involved, but also the form in which information is encoded.

Further Reading

If you are curious about whether Marr’s three levels are exhaustive, or how they relate to other multi-level frameworks, see Máté Lengyel (2024), “Marr’s three levels of analysis are useful as a framework for neuroscience.” Lengyel argues for a pluralistic approach that builds on Marr’s original insight while extending it to contemporary computational neuroscience.

Authors: Fabian Mueller & William Palmer

References

This step is broadly based on an open lecture by Nancy Kanwisher at the Massachusetts Institute of Technology (MIT), 2018: https://www.youtube.com/watch?v=Di_3pGAveGs

Colombo, M., & Piccinini, G. (2023). The Computational Theory of Mind. Cambridge University Press. https://doi.org/10.1017/9781009183734

Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (Chapter 1). MIT Press.

Gallistel, C. R., & King, A. P. (2009). Representations. In Memory and the computational brain: Why cognitive science will transform neuroscience (Chapter 4). Wiley-Blackwell.

Krakauer, D. C. (2024). The complex world: An introduction to the foundations of complexity science. SFI Press.

Pylyshyn, Z. (1984). Computation and Cognition: Toward a Foundation for Cognitive Science. MIT Press.

Lengyel M. Marr’s three levels of analysis are useful as a framework for neuroscience. J Physiol. 2024 May;602(9):1911-1914. doi: 10.1113/JP279549.

Willshaw DJ, Dayan P, Morris RG. Memory, modelling and Marr: a commentary on Marr (1971) ‘Simple memory: a theory of archicortex’. Philos Trans R Soc Lond B Biol Sci. 2015 Apr 19;370(1666)