Thirza Dado

PhD Candidate at the Neural Coding Lab who studies how the world is represented by the brain using synthesized reality.

Map is not the territory; limitations of decoding

2023, 12 July

The treachery of images The Treachery of Images, 1929, by Rene Magritte challenges us to recognize the limitations of representations. The painting urges us to understand the distinction between representation and reality - a key theme in the field of neural decoding.

Neural decoding involves the translation of neuronal activity into meaningful representations, such as images, language or sounds, so that we can interpret the information encoded in the brain. While we may be excited by the potential of this pioneering field, it’s also crucial to remain grounded. There are inherent limitations and potential biases that may arise within the decoding process that we must consider. As Alfred Korzybski famously said “the map is not the territory” we seek to explore.

Models are abstractions of reality

Light, upon entering the eyes, triggers a domino effect of electrical impulses in the visual stream of the brain. This process effectively encodes the visual stimuli from our environment into neural signals. The objective of decoding is then to model the reverse transformation from the neural responses back to the real-world concepts they represent. Let's acknowledge a fundamental truth: no matter how sophisticated or refined, models are simplifications of reality. These models, while offering significant insights, are essentially approximations that attempt to capture the complexities of a vast system using a set of assumptions and measured parameters.

Forgetting that the map is not the territory can lead us into the traps of reductionism and oversimplification.

Let’s consider an illustrative example:

Imagine researchers have developed a neural decoding model trained to reconstruct visual images from brain activity. This model is designed to “see” what a person sees by interpreting their neural signals. To ensure it works well across a variety of settings, it has been trained with a diverse array of images.

However, visual perception is not just about seeing — it’s deeply personal and influenced by emotions and memories. For instance, our emotional state can affect our attention, causing us to focus more on certain aspects of a visual scene or interpret colors and shapes in a way that aligns with our feelings. As a result, the model would likely reconstruct the painting in terms of its general outlines seen by the eyes, but it lacks the subjective depth of the actual experience that colors human perception. Thus, it does not fully represent the territory of the individual’s perceptual landscape.

Abstraction can be powerful

But there is another side to the coin. As Joan Robinson noted, “a model which took account of all the variegation of reality would be of no more use than a map at the scale of one to one”. Recognizing the limitations of models doesn’t imply we should aim for a perfect depiction of reality. Capturing every detail and variation within a model is not only impractical but would result in a tool too complex and cumbersome. It would lead to confusion rather than clarity. As such, generalization remains a powerful tool.

Models serve as a foundation for further exploration and discovery.

By simplifying the complexity of reality and identifying overarching principles, models provide a structured approach to grasp the world around us. They enable us to generate insights and make predictions about broader situations. Much like a map helps us navigate unfamiliar terrain, neural decoding models help us investigate the neural landscape — capturing its essential elements and illuminating critical relationships. While these models don’t encompass the full complexity of reality, they offer invaluable guidance. It’s crucial to strike a balance between representation provided by models and the actual reality they aim to simulate. Acknowledging the limitations and generalizations inherent in models is important, but we should also appreciate how they enhance our understanding and drive scientific inquiry forward. In essence, models are not just tools for representation; models serve as a foundation for further exploration and discovery. Through a continual cycle of hypothesis testing, experimentation, and revision, they enable us to ask precise questions and expand our knowledge.

Our mental model is also an abstraction of reality

The notion that representations are simplifications of reality goes even further. It follows the core of our own mental model and the way we perceive the world. Our neural representation of the world is, in fact, an abstraction of the real world. We can never have direct access to the full richness of reality but can only catch bits and pieces through our senses and cognitive processes. Our brains constantly construct and interpret an internal representation of the world based on this limited information colored by our subjective beliefs, expectations and biases. That said, it is not only important to be critical of the limitations of decoding models themselves but also of our own mental model that influences the interpretation of the results. This recognition invites us to humility and continuous reflection on the complex nature of our own thinking.

Conclusion

The promise of neural decoding is enticing. The prospect of unlocking thoughts and intentions purely by analyzing neural activity raises hope for the treatment of neurological disorders, the restoration of sensory and motor skills, and even the possibility of mind-reading. Yet, it’s essential to remember that this field is still in its infancy. Premature conclusions could lead to distorted results and misinterpretations. While we have made progress in decoding sensory inputs, fully understanding abstract thoughts, subjective experiences, and higher cognitive functions remains largely beyond our reach. The human mind is an elusive entity, and our current technologies still have a long way to go before they can truly grasp the full spectrum of our inner lives.

So let us, amidst the excitement about the possibilities of neural decoding, maintain humility. We must be mindful of the limitations and biases inherent in this field. In a world craving quick answers and immediate gratification, it is more important than ever to approach our research with patience and methodical diligence.

That’s all!

2023

Back to top ↑

2022

Back to top ↑