In this step, we explore a foundational perspective that has shaped cognitive science: the idea that understanding a complex system like the brain requires explanations at multiple levels.

When we aim to understand how the brain works, our first intuition might be to look at its physical structure – neurons, synapses, neurotransmitters, and electrical signals. One might think that understanding the brain basically means tracing these physical mechanisms, much like one explains a machine by analyzing its gears and circuits.

The answer must lie in how neurons fire, how they connect, and how their patterns of activity give rise to mental functions. Right?

Not quite. In his foundational work Vision (1982), British neuroscientist David Marr, one of the pioneers in the development of the field of Computational Neuroscience, argued that focusing on the neural level alone cannot explain perception.

“Trying to understand perception by studying only neurons is like trying to understand bird flight by studying only feathers. It just cannot be done. To understand bird flight, you need to understand aerodynamics. Only then can one make sense of the structure of feathers and the shape of wings. Similarly, you can’t reach an understanding of why neurons in the visual system behave the way they do just by studying their anatomy and physiology.”

David Marr, 1982

Marr challenged the view that biological mechanisms on their own are a complete good explanation. To truly understand a complex system like the brain, we must consider:

• What is the problem the system is solving, and why?
• How is the problem solved? What are the specific steps that the system uses to solve the problem?
• And lastly, where and how is this physically implemented in the brain?

These levels of inquiry form the basis of what has become one of the most influential frameworks in cognitive science: Marr’s Three Levels of Analysis. Below, we illustrate each level with two complementary examples: addition as a simple formal example, and color vision as a more applied example.

1. Computational (or functional) level

Question: “What is the goal of the computation, why is it appropriate, and what is the logic of the strategy by which it can be carried out?“ (David Marr, 1982)

Computational

This level is essentially defining what computation is being done or what problem the system is solving. Simply put, what is the goal of the computation or what is the input and desired output and additionally why is this computation useful?

Addition: The goal is to determine the combined magnitude of two or more numbers. The inputs are numerical values, and the output is their sum.

Color Vision: From an evolutionary viewpoint, color supports efficient foraging by helping differentiate ripe from unripe or toxic fruit. The inputs are wavelengths of light striking the eyes, and the output is a color percept.

Example color vision

2. Algorithmic Level

Question: “How can this computational theory be implemented? In particular, what is the representation for the input and output, and what is the algorithm for the transformation?“ (David Marr, 1982)

Algorithmic & Representation

This level specifies how the computation is carried out: what representations are used, and what algorithm processes them. A representation is simply the form in which the actual inputs are used in the algorithm. Regarding the algorithm, if we think about computation like cooking, the algorithm would be the recipe outlining the explicit steps getting us from the inputs (i.e., ingredients) to the output (delicious meal).

Addition: Numbers can be represented in many formats (Arabic numerals, binary, Roman numerals). A simple algorithm is the familiar digit-wise summation taught in school, carrying over tens when needed.

Color Vision: Here we must define how patterns of incoming light map onto color categories.

For example, light around 450 nm generally appears blue, light around 530 nm green, and light around 650 nm red. Any algorithm solving color vision must take into account that the light hitting our eyes depends jointly on the reflectance and illumination of the observed object and therefore

1. separate the two and
2. normalize for lighting to extract the object’s color.

Because the sensory data conflate these two factors, this is an inherently underdetermined problem, which we will revisit later.

3. Implementation Level

Question: “How can the representation and algorithm be realized physically?“ (David Marr, 1982)

Implementation

The final level is how the representations and algorithm can implemented in the brain, that is, which neural circuits carry them out and how.

Addition: Calculators implement addition through circuits using logic gates. In other words, the algorithm for addition is physically realised in electronic components that encode and transform numerical inputs according to the rules of arithmetic. At the implementational level, the question is not what addition is for, or which procedure is used, but how that procedure is instantiated in a physical system.

Example Addition: 4-Bit Calculator Built Using Digital Logic Gates © Cody Wabiszewski, Global Science Network (used with permission)

Color Vision: For color vision, this level concerns how representations and algorithms are implemented in the brain through neurons, circuits, and physiological mechanisms. Human color vision is implemented through three classes of cone photoreceptors: short-wavelength (S; blue-sensitive), medium-wavelength (M; green-sensitive), and long-wavelength (L; red-sensitive) cones, whose relative activation supports trichromatic color perception. Neural circuits in the visual system across the retina, thalamus, and visual cortex transform these signals into the colors we experience.

Schematic diagram of the biological vision system

The key idea here is that a proper theory of a cognitive process must explain it at all three of these levels which inherently constrain one another. A theory restricted to one level is incomplete: identifying neural circuits is insufficient without understanding the computational problem they solve, while specifying an algorithm is insufficient as a biological explanation without showing how it could be realized in neural tissue.

The three levels inherently constrain one another. Once the computational goal is defined, it constrains the range of possible algorithms, which must in turn be feasible within the neural implementation. Addition and color vision illustrate these relationships across the three levels.

Representations are especially important in theories of cognition because the mind does not merely respond to stimuli, but operates over internal states that stand in for relevant features of the world. To explain a cognitive process or capacity, we therefore often need to specify not only the sequence of operations involved, but also the form in which information is encoded.

Authors: Fabian Mueller & William Palmer

---

Further Reading

If you are curious about whether Marr’s three levels are exhaustive, or how they relate to other multi-level frameworks, see Máté Lengyel (2024), “Marr’s three levels of analysis are useful as a framework for neuroscience.” Lengyel argues for a pluralistic approach that builds on Marr’s original insight while extending it to contemporary computational neuroscience.

References

This step is broadly based on an open lecture by Nancy Kanwisher at the Massachusetts Institute of Technology (MIT), 2018: https://www.youtube.com/watch?v=Di_3pGAveGs

Colombo, M., & Piccinini, G. (2023). The Computational Theory of Mind. Cambridge University Press. https://doi.org/10.1017/9781009183734

Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (Chapter 1). MIT Press.

Gallistel, C. R., & King, A. P. (2009). Representations. In Memory and the computational brain: Why cognitive science will transform neuroscience (Chapter 4). Wiley-Blackwell.

Krakauer, D. C. (2024). The complex world: An introduction to the foundations of complexity science. SFI Press.

Pylyshyn, Z. (1984). Computation and Cognition: Toward a Foundation for Cognitive Science. MIT Press.

Lengyel M. Marr’s three levels of analysis are useful as a framework for neuroscience. J Physiol. 2024 May;602(9):1911-1914. https://doi.org/10.1113/JP279549.

Willshaw DJ, Dayan P, Morris RG. Memory, modelling and Marr: a commentary on Marr (1971) ‘Simple memory: a theory of archicortex’. Philos Trans R Soc Lond B Biol Sci. 2015 Apr 19;370(1666)