Neurobiologists from Belgium and Germany have successfully created realistic animated models of macaques. These avatars attracted the same intense attention from live macaques as real monkeys shown in videos.
Avatar Creation Methodology
During the research, the investigators avoided using physical markers or manually labeling every frame. Instead, they utilized data collected from eight cameras. Labeling was performed only on select frames, after which a neural network was trained to predict the location of key points on the avatar's body, allowing for the reconstruction of its movement trajectory.
The study was published in the scientific journal PLoS Biology.
The 'Uncanny Valley' Effect
It was found that as the avatar's realism decreased, the macaques' attention to it also diminished. This observation indicates the manifestation of the so-called 'uncanny valley' effect. Previously, scientists had created animated facial avatars to study perception in primates, which allowed them to overcome this effect: macaques paid almost as much attention to realistic facial avatars as to live conspecifics, whereas avatars without fur were ignored.
Challenges in Studying Movement
Studying the perception of moving bodies in primates is complicated by several factors. Motion capture technology using physical markers is inapplicable because macaques tend to dislodge them during grooming. Furthermore, manual frame-by-frame labeling would require a colossal amount of time, involving the placement of dozens of markers across hundreds of frames just to create a short animation.
Project Implementation Details
Martin Giese, the leader of the group from the University Hospital Tübingen and the Catholic University of Leuven, managed to bypass these obstacles by developing a moving avatar of the rhesus macaque (Macaca mulatta). For this, a system of eight cameras was set up around the macaque cage, recording 2.5 hours of video material. Short clips demonstrating the walking or turning of an individual, as well as a two-minute segment showing the grooming of two macaques, were selected from this material. Each camera operated at a frequency of 80 frames per second.
Instead of full manual labeling, a person marked the position of 42 joints, averaging two frames per second from two cameras. Then, the program projected these marked points onto the frames captured by the remaining six cameras. The Anipose algorithm used spatio-temporal triangulation to convert the resulting two-dimensional coordinates into three-dimensional ones. A neural network was responsible for predicting the position in frames that were not manually labeled. To ensure animation realism, the algorithm guaranteed the constancy of distances between joints and the smoothness of movements, preventing abrupt changes in coordinates between frames. This three-dimensional data was mapped onto a ready 3D model, resulting in the creation of a realistic animated avatar.
Avatar Testing Results
The correspondence between the avatar's movement trajectory and that of the real macaque from the video clips showed an error within 18–29 millimeters. Subsequently, the scientists demonstrated video recordings containing both real macaques and their avatars to eight male rhesus macaques. To assess the reaction specifically to the body, rather than the face, the faces in the videos and avatars were intentionally blurred. The macaques fixed their gaze on the silhouette of the real monkey and the avatar the same number of times if the avatar possessed realism, including fur and textures. However, attention was drawn less to the wire model of the monkey. Conversely, the macaques paid twice as little attention to the gray avatar, which lacked color, fur, and textures. The authors concluded that such an avatar causes discomfort in macaques due to the 'uncanny valley' effect.
Study Conclusions
The creation of the first animated macaque body avatar allowed the authors to demonstrate that the mechanisms for processing different social signals—both facial and bodily—in rhesus macaques likely operate on similar principles, as the 'uncanny valley' effect is observed in both cases. In the future, such a controlled avatar could become a tool for studying primate reactions to the moving bodies of their conspecifics. Nevertheless, the researchers noted that tracking the movements of two macaques simultaneously using eight cameras proved less accurate due to frequent point overlaps.