You are here: Home / Research / Bernstein I / Motion Vision: Neural Mechanisms and Technical Applications

Motion Vision: Neural Mechanisms and Technical Applications

Alexander Borst, Martin Buss, Kolja Kühnlenz

Project summary

We study the neural mechanisms underlying visual course control in flies and its application in autonomously navigating robots and vision-based robot control. In the fly, ample evidence exists for a specific mechanism of local motion detection, the so-called Reichardt detector. In the current project, we studied the integration of local motion signals by an array of large-field, motion sensitive cells, the so-called VS-cells. We found that, although VS-cells integrate with their dendrites motion information only within small adjacent stripes of the visual field; their receptive fields are broad and overlapping due to electrical coupling between them. We could show that such a circuit leads to a robust representation of important course parameters in the activity profile at the network output (Cuntz et al, PNAS, 2007). We have now started to investigate the transformation of optic flow information from the lobula plate cells onto descending neurons which control the motor output of the fly (Haag et al, J Neurosci, 2007; Wertz et al, J Neurosci, 2008). As a first step towards a technical implementation, a 2D-array of Reichardt detectors was simulated and fed with natural image sequences as filmed from within a driving car. We demonstrated that the output of such an array allows for simple video interpretation tasks such as segmentation of moving objects and motion estimation (Faille et al, Robotics and Autonomous Systems. submitted). In another part of our project, a hardware implementation of such a 2D-array of Reichardt detectors was realized on an FPGA platform, allowing for parallel, online computation of optic flow at high image framerates (Zhang et al., ICRA, 2008). Building upon this, we managed to control a simple one-axis manipulator visually in closed-loop at framerates up to 500Hz and validated the stabilizing effect of Reichardt-detectors in presence of large system latencies (Wu et al., ICRA 2009). Furthermore, we equipped a mobile outdoor robot (ACE) (Bauer et al., IJSR 2009) with means of optic flow-based 2D course estimation to be fused with wheel encoder data to cope with rough terrain where conventional odometry fails (Muehlbauer et al., ICRA 2009). We are now working towards closed-loop visual course control of a flying robot based on optic flow.

Project related journal publications

  • Kuehnlenz, K., Wu, H., Buss, M., Borst, A., Zhang, T. (2008): FPGA Design and Implementation of Insect-Inspired Reichardt Motion Detector and Receptive Fields. J Image and Graphics (Chin.), Chinese Academy of Sciences, in press.

  • Bauer, A., Klasing, K., Lidoris, G., Mühlbauer, Q., Rohrmüller, F., Sosnowski, S., Xu, T., Kühnlenz, K., Wollherr, D., Buss, M. (2008): The Autonomous City Explorer: Towards Natural Human-Robot Interaction in Urban Environments. Int J Social Robotics, Springer, in press.

  • Kuehnlenz, K., Buss, M. (2008): On sensor switching visual servoing. Int J Optomech, 2(3), Taylor & Francis.

  • Borst A (2009): Visual motion models. In: Encyclopedia of Neuroscience, vol. 10. Ed: LR Squire. Oxford Academic press, pp 297-305.

  • Haag, Borst A (2008): Electrical coupling of lobula plate tangential cells to a heterolateral motion-sensitive neuron in the fly. J Neurosci 28: 14435-14442.

  • Hines M, Eichner H, Schuermann F (2008): Neuron splitting in compute-bound parallel network simulations enables runtime scaling with twice as many processors. J Computat Neurosci 25: 203-210.

  • Joesch M, Plett J, Borst A, Reiff DF (2008): Response properties of motion-sensitive visual interneurons in the lobula plate of Drosophila melanogaster. Current Biology 18: 368-374.

  • Wertz A, Borst A, Haag J (2008): Nonlinear integration of binocular optic flow by DNOVS2, a descending neuron of the fly. J Neurosci 28: 3131-3140.

  • Weber F, Eichner H, Cuntz H, Borst A (2008): Eigenanalysis of a neural network for optic flow processing. New J Phys 10: 1-21.

  • Safran M, Flanagin V, Borst A, Sompolinsky H (2007): Adaptation and information transmission in fly motion detection. J Neurophysiol 98: 3309-3320.

  • Haag J, Borst A (2007): Reciprocal inhibitory connections within a neural network for rotational optic-flow processing. Frontiers in Neurosci 1: 111-121.

  • Borst A, Haag J (2007): Optic flow processing in the cockpit of the fly. In: Invertebrate Neurobiology. Eds: G North, RJ Greenspan. CSHL-Press, pp 101-122.

  • Cuntz H, Haag J, Foerstner F, Segev I, Borst A (2007): Robust coding of flow-field parameters by axo-axonal gap junctions between fly visual interneurons. PNAS 104: 10229-10233.

  • Haag J, Wertz A, Borst A (2007): Integration of lobula plate output signals by DNOVS1, an identified premotor descending neuron. J Neurosci 27: 1992-2000.

  • Borst A (2007): Correlation versus gradient type motion detectors - The pros and cons. Phil. Trans. Roy. Soc. B 362: 369-374.

Project related conference publications

  • Wu, H., Zhang, T., Borst, A,. Kuehnlenz, K., Buss, M. (2009): An Explorative Study of Visual Servo Control with insect-inspired Reichardt-Model, Proc IEEE Int Conf Rob Aut (ICRA), Kobe, Japan, in press.

  • Mühlbauer, Q., Sosnowski, S., Xu, T., Zhang, T., Kühnlenz, K.,  Buss, M. (2009) Navigation through Urban Environments by Visual Perception and Interaction. Proc IEEE Int Conf Rob Aut (ICRA), Kobe, Japan, in press.

  • Mühlbauer, Q., Sosnowski, S., Xu, T., Zhang, T., Kühnlenz, K., Buss, M. (2009) The Autonomous City Explorer: Towards Navigation by Interaction and Visual Perception. Proc 1st Int. Workshop on Cognition for Technical Systems, Munich, Germany, in press.

  • Wu, H., Zhang, T., Borst, A., Kuehnlenz, K., Buss, M. (2008): An Explorative Study of Applying Reichardt-Model in Visual Servo Control, Proc 1st Int. Workshop on Cognition for Technical Systems, Munich, Germany.

  • Zhang, T, Wu, H, Borst, A, Kühnlenz, K, Buss, M. (2008): An FPGA-Implementation of Insect-Inspired Motion Detector for High-Speed Vision Systems, Proc IEEE Int Conf Rob Aut (ICRA), Pasadena, USA.