Multisensory Integration

In the struggle for survival in a complex and dynamic environment, animals have developed a multitude of sensory systems. These sensory systems form the link between events in the real world and the neuronal representation of the very same events in the brain of the animal. In order to realize an efficient view of the outside world, different senses--like vision, hearing, and touch, but also exotic senses such as the lateral line system of fish and amphibia or the infrared vision of snakes--have to be assembled to a coherent whole.

But how multisensory integration is achieved in the brain? First, it is necessary to understand how neuronal maps (topographic representations of the environment) are formed in the brain. Second, we need to understand how information from multiple sensory systems interacts in the brain to form an unambiguous representation of the environment.

The project currently focuses on the theoretical component of multisensory integration. We have concentrated on studying neuronal realizations of the general mechanisms that underlie sensory integration. That is, we study how a sensory modality dominates other sensory input and how such a dominance that has been suggested by several experimental studies can arise. This research is extended by the study of information processing in exotic sensory modalities. We currently put our knowledge to use in technical systems by an alliance with the TUM excellence cluster for cognition in technical systems, CoTeSys.

llustration of our model. On the left side, the mathematical model, where the input functions are taken as continuous and where the subsequent functions are mathematically calculated. On the right side, the neural model realized with spiking and noisy neural populations. In both cases, two Input Maps A and B are multiplicatively combined into the Integrated Multiplicative Map. The combined information can be used to intrinsically gauge the two input maps.