Poster abstracts:
Konstantin Steinmassl & Markus Paulus
Ludwig-Maximilians-Universität München (LMU)
Malleability of the sense of bodily self in early childhood: 5- and 6-year-old children children show the enfacement illusion
The mechanisms underlying the developing sense of bodily self are debated. Whereas some scholars stress the role of sensory factors, others propose the importance of contextual factors. By manipulating multisensory stimulation and social familiarity with the other person, we explored two factors that are proposed to relate to young children’s developing sense of bodily self. To this end, the study implemented an enfacement illusion with children (N = 64). Participants were exposed to one trial with synchronous interpersonal multisensory stimulation and one trial with asynchronous interpersonal multisensory stimulation—either with a stranger or with the mother as the other person. A self-recognition task using morph videos of self and other and an enfacement questionnaire were implemented as dependent measures. Results revealed evidence for the presence of the enfacement effect in children in both measures. The identity of the other person had a significant effect on the self-recognition task. Contrary to our hypothesis, the effect was significantly smaller in the caregiver condition. Our results demonstrate the role of both multisensory stimulation and contextual—here social familiarity—factors for the construction and development of a bodily self. The study provides developmental science with a novel approach to the bodily self by showing the validity of the self-recognition task in a child sample. Overall, the study supports proposals that the sense of bodily self is malleable early in development.
Konstantin Steinmassl & Markus Paulus
Ludwig-Maximilians-Universität München (LMU)
Are There Top-down Effects on the Developing Minimal Self? The Role of Attachment Representations
The development of the minimal self – assessed in terms of sense of agency, body ownership and interoception – has recently gained increased attention in developmental science. While research provided ample evidence for the presence of a minimal self in young children, the interrelations between these three facets as well as the cognitive mechanisms underlying their development are debated. Current theories propose that the minimal self is dynamic and plastic, and affected by top-down processes and cognitive representations. In order to advance developmental theorizing, the current study has four aims. It a) examines to which extent the three facets of the minimal self are related to each other. Moreover, it investigates the impact of one central cognitive process, the internal working model (IWM) of attachment, on young children’s b) explicit self and c), most central, minimal self. Furthermore, it d) examines whether the impact of the IWM is particularly strong in emotional contexts. To this end, 102 5-year-old children in an urban German area will be assessed. We will rely on established measures for all three facets of the minimal self, the explicit self-concept and attachment representations. The results will be discussed in terms of recent theorizing on top-down influences on the construction of the minimal self.
Carlotta Langer
Hamburg University of Technology.
Integrated Information in the Active Inference Framework
The active inference framework provides a principled approach to modeling sentient behavior. In this framework perception and action selection are treated in a unified way. The resulting agents form an internal generative model of the relevant dynamics of the world in order to infer their future observations, their internal states and to select actions. We combine this modeling framework with the Integrated Information Theory of consciousness and are therefore able to analyze the active inference agents from the perspective of integrated information. The Integrated Information Theory aims at quantifying the level of consciousness of a system by assessing its capability to integrate information. In this talk we define a measure of integrated information for the generative model by making an additional structural assumption. Experiments with simulated agents demonstrate the impact of modeling assumptions in the active inference framework on the integrated information value. Furthermore, we evaluate other information-theoretic measures within active inference agents, including one often referred to as Morphological Computation, which quantifies the interaction between an agent and its environment.
Filipe Gama
Automated head turn estimation from nose position in infant videos
Data from infant studies are often manually coded, requiring weeks of labor to analyze a relatively small window of time in the life of a few infants. In the past few years, automated Human Pose Estimation has been developed and used to study kinematics. However, it is primarily tuned for adult bodies, and its performance on infants has yet to be thoroughly evaluated, especially for motion. We compared the state-of-the-art 2D pose estimation method: ViTPose using 756 354 nose positions manually-coded at 15.15 Hz; and derived motion metrics (velocities and active bouts: defined as activity periods with velocities above 1.5cm/s). We used data from previous studies involving 5 infants recorded weekly for 5 minutes from 3 to 13 weeks old, in 5 conditions: baseline, mobile, parent talking, musical toys behind the head, and colorful toys in view. Automated pose estimation seems to generally provide accurate positions and velocity estimates, but its variability and failed estimates seem to make it unreliable in some cases. While it does not fully reproduce results from manually-coded data and needs manual verification to validate the output for further analyses, it helps reduce the overall amount of manual coding necessary to find the most notable results.
Jan Dohmen
Technische Universität Hamburg
Context inference for robust reinforcement learning
Reinforcement Learning (RL) has demonstrated the ability to solve complex problems. However, RL agents often lack robustness and struggle when faced with minor variations in the environment, such as changes in object mass or surface friction. An example is the sim-to-real gap, wherein an RL agent trained in a simulated environment is deployed in the real world. Slight differences in environment dynamics are expected due to the imperfections of the simulation. Our focus is on zero-shot generalization, wherein the agent is required to perform under novel, unseen dynamics without any prior adaptation phase. To model these problems, we utilize a contextual Markov Decision Process (CMDP) as the underlying formalism. We highlight scenarios where the contexts are unknown, i.e., the agent is context unaware, and propose a method to infer latent representations of these contexts using minimal experience in the environment.
Sofiya Karnovska
Ludwig-Maximilians-Universität München (LMU)
Charting the Emergence of Pointing: A Longitudinal Study on Social and Non-Social Index Finger Use
The pointing gesture plays an important role for early communicative development, yet little is known about how pointing emerges in early ontogeny. Relational developmental systems approaches to communicative development trace the origins of pointing back to early instances of non-social index finger use. So far, no longitudinal examination of this theoretical proposal exists. Moreover, early predictors of early non-social index finger use and pointing as a social gesture have not been studied in this context yet. To remedy this gap in the literature, this study reports on a longitudinal analysis of non-social index finger use and pointing from 6 to 18 months (n = 114) based on a parent questionnaire. In addition, it investigates maternal sensitivity, maternal non-intrusiveness, infant motor development and infant cognitive abilities at 6 months as early predictors of pointing. Our analyses revealed a gradual increase of both non-social index finger use and pointing from 6 to 18 months. We found that non-social index finger use at 12 months and 14 months, predicted pointing at 14 months 16 months respectively. Maternal sensitivity and non-intrusiveness (coded following Ainsworth) related to pointing at 8 months. Sensitivity had a positive effect, whereas the effect of non-intrusiveness was negative. Furthermore, cognitive development had a negative effect on index finger use at 14 months, and a positive effect on pointing at 16 months. This study illustrates maternal sensitivity and non-intrusiveness, as well as cognitive abilities are early predictors of non-social index finger use and pointing. It provides empirical support for the theoretical proposal that pointing emerges from non-social index finger use. Our longitudinal study underlines the importance of infants’ non-social index finger use for the development of pointing as a communicative gesture.
Anja Kaßecker1, Stephan Verschoor2, Bernhard Hommel3, Markus Paulus1
1Ludwig-Maximilians-Universität München, Munich, Germany
2 University of Bremen, Germany
3 Shandong Normal University, Jinan, China
Mimicry in Infancy: Applying a Dynamic Virtual Mirror to Infants
The minimal self describes the immediate, pre-reflective experience of selfhood derived from sensory information. Conceptually, it has been subdivided into the sense of agency and the sense of ownership (for an overview, see Gallagher, 2000). Influential developmental theories have proposed that infants develop precursors of body ownership and agency at the end of the first year of life (e.g., Butterworth, 1995, Verschoor & Hommel, 2017, Zmyj, et al., 2011). Yet it is not clear how exactly the ontogeny of agency and ownership occurs. Thus, we tested the developmental timeline of the self-by-doing theory (Verschoor & Hommel, 2017) claiming that precursors of agency develop before body ownership.
To accomplish this, we deployed a virtual mirror technology (Grewe et al., 2021), were infants (7- & 12-months old) interacted (head movements to the left and right) with an avatar to create an interactive enfacement paradigm designed specifically for infants. We measured infant’s facial expression (as a function of the me-other-overlap to the avatar) with cutting the edge facial encoding system OpenFace Technology (Baltrusaitis, et al., 2018). Data collection is still ongoing; thus, we will present first preliminary results.
Srihari Ramasubramony
ZIB Berlin
ActiveSelf: Cognitive Control of Social Resonance
This research explores the measurement, prediction, and influence of human mimicry during real-time human-avatar interactions, with a focus on cognitive science and social resonance. Mimicry, also known as behavioral matching or the chameleon effect, is crucial in social situations and impacts the outcome of social interactions. The study aims to understand the mechanisms underlying social resonance by integrating and evolving the Theory of Event Coding to better measure its influence.
An avatar was recreated using Unity 3D, a real time 3D development engine and parallelly a tool named Mediapipe was set to create facial landmarks using live webcam feed was developed, leveraging Action Units to measure participants’ mimicry and the avatars response to the facial expressions from the user. Through technical innovations, the study seeks to enhance how people naturally mimic and connect with lifelike avatars in virtual environments.
These avatars were designed to enhance social interactions and the research faced challenges in real-time interaction, including the integration of Unity 3D with Mediapipe, facial expression synchronization, and networking issues such as latency and data loss. The results of this study are expected to deepen our understanding of social resonance and its role in digital interactions, offering valuable insights for improving the realism and effectiveness of avatars.
By addressing challenges in synchronization and communication, the findings could significantly enhance the design of virtual environments, leading to more intuitive and authentic human-avatar interactions.
Suman Sardar
This research investigates the measurement, prediction, and influence of peoples’ mimicry in real-time during human-avatar interactions with a focus on cognitive science. It aims to determine the behavioural processes behind mimicry by integrating cognitive psychology principles such as Theory of Event Coding. Virtual avatars are developed in Unity and integrated with MediaPipe. This integration allows synchronization with participants’ head movements and facial expressions. A robust multimodal model is developed by collecting data through three distinct modalities- facial electromyography, webcam recordings, and infrared camera footage. The model specifically focuses on identifying the extent of participants’ mimicry, particularly in response to both doppelgänger avatar and other avatar for the facial expression smiling. The findings are expected to reveal critical insights into the degree and intensity of facial expression adoption which will enhance the understanding of human behaviour while interacting with virtual avatars. This study is crucial for advancing social robotics, virtual reality, and user experience design. It provides new insights into human-avatar interaction dynamics which could shape future technological innovations in these areas. The findings of this study may also help to improve the intuitiveness and alignment of virtual environment designs with human social resonance.
Shreejata Gupta
Aix-Marseille Université, Marseille, France
Offering and showing gestures in 12- to 15-month-old infants in natural contexts: A corpus-based study
Developmental theories propose that the ontogeny of sharing behavior builds on earlier emerging social routines of offering objects in caregiver-child-interactions. The current study explored in greater detail the occurrence of offering and showing behavior in naturalistic contexts, and how caregivers react to these gestures. To this end, we relied on video recordings of spontaneous infant-caregiver interactions from two CHILDES corpora, and analyzed data of six infants from 12 to 15 months. Across ages, infants showed more offering behavior than showing behavior. Caregivers responded vocally to both types of gestures and showed more object-directed action in the context of the offering gesture. Showing behavior was thus more likely to lead to an unimodal reaction, whereas offering behavior more often led to a bimodal reaction from caregivers. Overall, the study provides novel evidence for how infant gestures, and the early precursors of sharing behavior, develop in the context of caregiver-child-interactions.
Valentin Marcel
Czech Technical University in Prague
Extracting 3D motion of infants – video camera, depth camera, or marker-based motion capture?
Datasets of pictures of humans in everyday life are one of the biggest source of data in computer vision on which most computer vision algorithms are trained. However, mostly due to ethical issues: infants and especially newborns are rarely represented in the dataset, create a training domain gap in the algorithms extracting human poses.
In our lab, we are extracting motion data from recordings of infants, using standard and depth camera, and motion capture as ground truth measurement. Measuring few months old infants is tricky as they usually stays on their back and cannot pose. We talk about the measurement methods and present some applications of such data in experimental psychology, such as studying the development of self-touch, the ability to reach for the body or an external object with the aim of quantifying patterns of human development through automatic data analysis.
DFG SPP "The Active Self" 