Human skill—from the fl uidity of a dancer’s movement to the precision of a surgeon’s hand, from the disciplined strokes of a calligrapher to the split-second coordination of athletes—represents one of the most sophisticated expressions of human achievement. And yet, despite their centrality to human capability, motor skills remain one of the least digitized and standardized forms of knowledge.

Traditional methods of learning rely on observation, imitation, and subjective feedback, leaving a vast gap between qualitative expertise and quantifi able data. ShuHaRi is a japanese concept describing three stages of learning and mastery: i) learn from master ii) understand and detach iii) transcend. This meeting convenes experts from fi elds analyzing human skilled performance to establish a common ground on methods to digitize skilled action, study its physiological and cognitive aspects, build computational models of skilled action, and design actuation technology aiming to scaffold skill transfer or to replicate it in artifi cial limbs or humanoids.

Recent advances in sensor technology, neuroimaging, computational modeling, offer an unprecedented opportunity to decode, digitize, and to study what leads to skilled performance. Furthermore, advances in neural representation learning, wearable actuation and robotics open new opportunities to model, transfer and/or replicate skilled performance.

Conventional methods for digitization rely on motion capture and pose estimation with electromagnetic, optical, or wearable sensors. Optical trackers are highly accurate, deliver high framerate and capture the outcome body posture and motion. Wearable sensors can be used to measure physiological parameters such as electrocardiogram (ECG), heart rate and blood oxygen saturation (pulse oximetry), body temperature, muscle tension (electro myography, EMG) [2], brain activity (EEG)[5], pupil and gaze deployment [1], or gait force distribution with sole pressure sensors[3]. Some modalities reveal information about the human state, while others reveal how the skill unfolds. When attempting multimodal recording, impending challenges such as a lack of guidelines for each sensing modality, cross-modal synchronization and sensor calibration make the process error prone, extensive and cumbersome.

When attempting a motor task, we obtain intrinsic feedback from multiple systems (auditory, visual, proprioceptive, somatosensory), which is continuously analyzed as part of the motor planning and execution process [11]. Skilled behavior potentially incurs high information load from sensation, skilled action preparation, execution, monitoring and updating. The human body continuously optimizes motor performance achieving improved performance outcome and the automation of the skill execution. Skill automation is fundamental in dealing with limited capacity. Successful performance under these constraints requires automated motor processes that operate independently of limited attentional resources, enabling parallel task execution without interference. Measurable task performance aspects help researchers differentiate aspects of a skilled performance from poor performance. When combined with physiological measurements one begins to understand which of those (perceptual, cognitive, motor coordination) define skilled performance.

With better knowledge of how human performance unfolds and performance data obtained from experts, how can we transfer the skill to a novice practitioner? Achieving such a challenge requires combining sensing technology, models of human performance [2] and actuation technology in a closely coordinated coaching system[4]. Models of human performance are used to identify the target skill to transfer. The type of regime may target three phases of motor learning. Pre-task serves as demonstration, during-task (real-time) aims to provide live correction of motor actions as they are performed, and post-task targets review and refl ection. Feedback is triggered by sensors, and may be enacted through diverse actuation methods, each affording diverse stimulation possibilities [6]. Major research challenges involve the models to match the skills, the sensing and actuating technology.

Reinforcement learning (RL) is the paradigm of choice when learning human level skills. RL trains an agent through its interaction with a simulated environment. Yet, some human performance tasks are better learned using human data, for instance autonomous racing [7]. This requires large scale human demonstration datasets [8].

Humanoid robots are poised to become general-purpose physical agents. Neural models trained through RL and Imitation Learning successfully capture the required combination of torque needed at each joint to achieve smooth nimble motion [10]. Learning humanoid motion from human demonstrations requires mapping human skeleton to the robot joints, and inverse kinematics, followed by model training [9,10]. In contrast, human expertise is naturally transferred through demonstration and verbal guidance by coaches or peers. An open question lies in studying the similarities, e.g., in attention, of learned representations compared with human performance.

ShuHaRi is conceived as a multiplier: by convening experts across these domains, we aim to: i) defi ne shared benchmarks and open datasets for skilled performance, ii) develop interdisciplinary methodologies for skill modeling and transfer, iii) foster collaborations that accelerate innovation in neuroengineering and embodied AI. This meeting is timely: the convergence of wearable sensing, AI, and robotics has created a unique window to establish skill science as a unified field.

We have invited a selected group of researchers and practitioners from: i) Neuroscience & Cognitive Science – Motor learning, neural plasticity, cognitive modeling, ii) Sports Science & Performing Arts – Skill acquisition, performance optimization, embodied knowledge, iii) Sensor and Technology & Wearables – EMG, EEG, motion capture, affective computing, haptics interfaces, iv) Computer Science & AI – Machine learning, reinforcement learning, human–robot interaction, v) Robotics – Anthropomorphic robots, skill replication, vi) Industry Partners – Applications in training, healthcare.

1. Lappi, Pekkanen, Krajnc, Iacono, Remonda, Veas. The racer’s gaze: Visual strategy in high-speed sports expertise. Journal of Vision, 25 (8), pp. 1-16. 2025.

2. Liu, Peng, Oku, Liao, Wu, Furuya, Koike.(2025) From Pose to Muscle: Multimodal Learning for Piano Hand Muscle Electromyography. Conference on Neural Information Processing Systems (NeurIPS 2025), 2025

3. Hirano, Tabei, Peng, Liao, Wu, and Koike. 2025. SkiTechCoach: A Multimodal Alpine Skiing Dataset with 3D Body Pose, Sole Pressure, and Expert Coaching. In Proceedings of the 8th International ACM Workshop on Multimedia Content Analysis in Sports (MMSports '25). Association for Computing Machinery, New York, NY, USA, 39–46.

4. Liao, Yu and Koike, "ShiftingGolf: Gross Motor Skill Correction Using Redirection in VR," in IEEE Transactions on Visualization and Computer Graphics, vol. 31, no. 5, pp. 3429-3439, May 2025, doi: 10.1109/TVCG.2025.3549170.

5. Wimmer, Weidinger, El Sayed, Müller-Putz, and Veas. Eeg-based error detection can challenge human reaction time in a vr navigation task. In 22nd IEEE International Symposium on Mixed and Augmented Reality: ISMAR 2023.

6. Eguchi, Kitagishi, Hiroi, & Hiraki (2025). Tactile Data Recording System for Clothing with Motion-Controlled Robotic Sliding. arXiv preprint arXiv:2511.11634.

7. Remonda, Veas, and Luzhnica, “Comparing driving behavior of humans and autonomous driving in a professional racing simulator,” PLoS one, vol. 16, no. 2, p. e0245320, 2021.
8. Remonda, Hansen, Raji, Musiu, Bertogna, Wang, Veas. A Simulation Benchmark for Autonomous Racing with Large-Scale Human Data. In Proc. of the International Conf. Neural Information Processing Systems NeurIPS2024, 2024.

9. Mao, J., Zhao, S., Song, S., Shi, T., Ye, J., Zhang, M., Geng, H., Malik, J., Guizilini, V., & Wang, Y. (2024). Learning from Massive Human Videos for Universal Humanoid Pose Control. http://arxiv.org/abs/2412.14172

10. Cheng, Li, Yang, Yang, & Wang (2024). Open-television: Teleoperation with immersive active visual feedback. arXiv preprint arXiv:2407.01512.

11. Lappi, & Dove. (2022). The Science of the Racer's Brain.