Die aktuelle Vorhersage von Molly
Virtual fish at the Institute of Real-Time Learning Systems (EZLS)
Analysis-by-synthesis with virtual fishes as an experimental method for mate-choice studies
The interdisciplinary, DFG-funded project "Virtual Fish" of the Institute of Real Time Learning Systems (EZLS) and the Institute of Biology
consists of two parts. EZLS analyses the fish's movements and behaviour with the help of a computer-vision system. The gathered information is used in the sequel to create a photo-realistic simulation of fishes and their behaviour. The biologists will use the virtual fishes in combination with live test fish to conduct mate-choice experiments.
The used fish species is sailfin molly (Poecilia latipinna), which has a size of up to 10 cm and is able to move quite quickly and rather abruptly. Especially, its quick movements place special demands on the computer vision system. For the estimation of the fish's pose and position a new approach based on analysis-by-synthesis is used. To generate ground-truth data a very precisely calibrated computer-vision system with two orthogonally aligned cameras calculates positions and poses of the fishes (Figure 1 and 2). This allows later on to quantitatively evaluate the results stemming from the new analysis-by-synthesis approach.
Publications
M. Langer, L. Kuhnert, M. Ax, D.-V. Nguyen, and K.-D. Kuhnert. 3D object recognition and localization employing an analysis by synthesis system. In IADIS Int. Conf. on Applied Computing, pages 132-140, Rome, Italy, November 19-21, 2009.
S. Todt, M. Langer, C. Rezk-Salama, A. Kolb, and K.-D. Kuhnert. Spherical light field rendering in application for analysis by synthesis. In Int. J. on Intell. Systems and Techn. and Appl. (IJISTA), Issue on Dynamic 3D Imaging, volume 5:1, pages 304-314, 2008.
In the second phase a fish model is created with the help of the gathered shape, movement and behaviour information of the species. This steerable virtual fish can be used by the biologists to conduct behavioural experiments under strictly controlled conditions. Furthermore the model is utilised to build a tracking system based on analysis-by-synthesis.
This approach allows to track the fishes' 3D-position and 3D-deformation by using only one camera. Due to the model's information about fish's movements the algorithm renders the most probable image of the virtual fish and compares this to the captured image of the real fish (Figure 1). This step is repeated until the rendered image becomes undistinguishable from the captured image. As result the algorithm provides the best fit model parameters and consequently a complete fish description in 3D.
With the help of the tracking information of the live test fishes the virtual fishes which are projected onto an LCD screen can react to and also interact with the live fishes. This powerful tool will open up a new way to the biologists in conducting behavioural studies.
Figure 1. Screen-shot of the tracking-software. On the right side the current camera views are shown. The used method automatically cuts out the silhouettes of the fish (bottom left) and calculates the 3D-position and deformation of it (top left).
Figure 2. Experimental fish-tracking set-up. Besides a camera for the actual analysis-by-synthesis tracking method, a second camera is used to proof the calculated 3D-information. For camera-calibration red balls are mounted to the aquarium corners.
More information on the research
of the Institute of Biology.
