Project B4 – Multisensory Object Representation and Integration in Space-Time
Most ‘objects’ in the sensory world have a multimodal signature. We want to study how multi-sensory information in space-time leads to a unique neuronal representation of such objects. How does input from different senses interact in the central nervous system? How is object formation achieved, how are different object representations integrated, and how are these objects neuronally represented? And how is a consistent spatio-temporal representation of multimodal objects obtained in dynamic environments? These issues are to be studied in the avian midbrain, a model system where sensory circuits have been analyzed extensively [1] and new models of dynamic input computation as performed by neurons have been developed [2,3]. Preparatory work [4,5] has already focused on the mathematical analysis of sensory processing and map formation.
Objectives and description of the project
Combining experiments and theory, we aim at mathematically understanding audio-visual integration in the avian visual midbrain. We will experimentally determine the postsynaptic elements that constitute auditory and audio-visual maps in the optic tectum (OT), and work out the integration rules for bimodal input by systematically varying spatial and timing offsets (figure 1). We expect to find optimal integration windows and will test whether these can be shifted pharmacologically and by feedback. We will investigate the dynamics of multimodal object perception in the OT and the way in which it constrains the representation of spatio-temporally changing multimodal sensory signatures. We will pay special attention to feedback (via the isthmic nuclei), an obvious aspect of multisensory integration [3,6], and refine the theoretical framework within the context of STDP [4,7].
[1]: Luksch Rev Neurosci 2003. [2]: Luksch et al. Nat Neurosci 2004. [3]: Mahani et al. Biol Cybern 2006. [4]: van Hemmen ChemPhysChem 2001. [5]: van Hemmen & Schwartz Biol Cybern 2008. [6]: Meyer et al. Biol Cybern 2008. [7]: Friedel & van Hemmen Biol Cybern 2008.
