Marco Pardini

I use Unreal Engine for high-fidelity simulation and biosignals to decode human emotional states. My aim is to embed this emotional awareness into cognitive architectures, so that people can interact with robots that genuinely understand and respond to the emotional state of their interlocutor.

My PhD sits at the interdisciplinary crossroads of Computer Engineering and Biomedical Engineering, jointly supervised by Prof. Mario Cimino and Prof. Alberto Greco. Prof. Cimino leads research in deep learning, explainable AI, computer vision, and cognitive architectures; Prof. Greco is a leading expert in electrodermal signal analysis and affective computing. Working at the intersection of these two domains, I apply deep learning and computer vision directly to robotic platforms — bridging cognitive AI and physiological affective response.

I work directly with hyper-realistic humanoid platforms, focusing on real-time control, collision avoidance, and facial retargeting to create believable interactions.

The first generation of Abel is a hyper-realistic humanoid conceived as a research platform for social interaction, emotion modeling, and embodied intelligence. Resembling an 11–12 year old boy, it is a collaboration between the University of Pisa and Gustav Hoegen (Biomimic Studio, London). It comprises a head and upper torso driven by 42 high-end servo motors (Futaba, MKS, Dynamixel); the head alone houses 21 motors dedicated to facial expressions, gaze, and speech simulation. It is equipped with a torso camera, binaural microphones, and an internal speaker (see Abel in action).

On the first generation I work on the control stack in ROS 2: the body and arms run through MoveIt for motion planning and collision avoidance, tested in Gazebo, while the face is driven by a middleware that blends expressions, visemes for lip-sync, and gaze.

The second generation of Abel, from the same collaboration, is built around 30 Dynamixel servos: a body-driven rotating neck, articulated arms and hands, and a face whose 13 dedicated motors drive the lips, jaw, eyes, eyelids, eyebrows, and mouth corners with precise control. It carries a chest-mounted Intel RealSense camera, a ReSpeaker microphone array, and an internal speaker.

With the second generation I'm currently exploring real-time facial retargeting — using deep learning to map a person's expressions from video onto its facial servos. This is ongoing, work-in-progress research.

In our work on continuous assessment of Social Anxiety Disorder (SAD) in Virtual Reality, we integrate continuous self-ratings with objective physiological data (ECG / EDA). Our Late-Fusion Transformer framework classified high- and low-anxiety groups with an F1-score of 0.853 and 82.5% accuracy.

We are now tackling the harder problem of multivariate continuous regression — mapping dynamic, in-the-moment physiological signals directly to the user's fluctuating emotional state.

Reviving historical figures begins with high-density point clouds, meticulously converted into workable static meshes inside Unreal Engine. Advanced grooming tools recreate fine details, followed by rigging the models to the MetaHuman framework.

These digital avatars can then be projected into physical Holoboxes. Displaying a MetaHuman holographically requires a multi-camera setup in Unreal Engine, rendering the same subject from several angles simultaneously to construct a seamless 3D illusion.