Modelling Two-person and Human-object Close Interactions


Introduction

Close interactions, not necessarily with any contacts, between different body parts of single or multiple characters or with the environment are common in computer animation and 3D computer games. Yoga, wrestling, dancing and moving through a constrained environment are some examples. In such motions, the spatial relationships between different body parts of characters are important in capturing the semantics of the scene. When an animator synthesizes or edits such movements, special care is needed to preserve these spatial relationships, for example, "arching back to avoid a punch", "hands extending around each other", "two bodies moving synchronously in close proximity" or "getting into a small car by bending down". However, traditionally, such spatial relationships exist only in the animator’s mind and are not digitally embedded into the data. Although humans use spatial relationships to recognize semantics of interactions, their usage has not been considered much in character animation.

We introduce a simple structure called an interaction mesh [Ho et al. SIGGRAPH2010, Ho et al. VRST2014] to represent such spatial relationships for editing and retargeting motions that involve close interactions between body parts of single or multiple articulated characters. Such an approach can also be used for controlling humanoid robots [Ho and Shum ICRA2013], synthesing virtual parner in VR [Ho et al. ACM TOMM2013]. The spatial relations between characters can also be used for facilitating the sampling process to generate a huge amount of pose-pairs [Yin et al. IEEE TVCG2019].

The spatial relations can also be used for analyzing interactions which can be used for motion retrieval [Shen et al. IEEE TVCG2020] and action recognition [Men et al. ICPR2020, Ho et al. VRST2015].

Readers are also referred to a closely related project on topology-based motion analysis and synthesis.
Publications

  1. Qianhui Men, Edmond S. L. Ho, Hubert P. H. Shum and Howard Leung, "Interaction Recognition with Pairwise Joint Distance and Fully Connected Graph Convolution LSTM"conference , Proceedings of the 2020 International Conference on Pattern Recognition (ICPR2020), accepted, Jan 2021. PDF
  2. Yijun Shen, Longzhi Yang, Edmond S. L. Ho and Hubert P. H. Shum, "Interaction-based Human Activity Comparison"journal , IEEE Transactions on Visualization and Computer Graphics, vol 26(8), pp. 2620-2633, Aug 2020. PDF video bibtex
  3. Kangxue Yin, Hui Huang, Edmond S. L. Ho, Hao Wang, Taku Komura, Daniel Cohen-Or and Hao Zhang, "A Sampling Approach to Generating Closely Interacting 3D Pose-pairs from 2D Annotations"journal , IEEE Transactions on Visualization and Computer Graphics, vol 25(6), pp. 2387-2396, June 2019. PDF video bibtex
  4. Edmond S. L. Ho, Jacky C. P. Chan, Yiu-ming Cheung, Pong C. Yuen, "Modeling Spatial Relations of Human Body Parts for Indexing and Retrieving Close Character Interactions"conference , Proceedings of the 2015 ACM Symposium on Virtual Reality Software and Technology (VRST 2015), Nov 2015. PDF bibtex
  5. Edmond S. L. Ho, He Wang, Taku Komura, "A Multi-resolution Approach for Adapting Close Character Interaction"conference , Proceedings of the 2014 ACM Symposium on Virtual Reality Software and Technology (VRST 2014), Nov 2014. PDF video bibtex
  6. Edmond S. L. Ho, Jacky C. P. Chan, Taku Komura, Howard Leung, "Interactive Partner Control in Close Interactions for Real-time Applications"journal , ACM Transactions on Multimedia Computing, Communications and Applications, vol 9(3), June 2013. PDF video bibtex
  7. Edmond S. L. Ho and Hubert P. H. Shum, "Motion Adaptation for Humanoid Robots in Constrained Environments"conference , Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA 2013), May 2013. PDF video bibtex
  8. Edmond S. L. Ho, Taku Komura, Chiew-Lan Tai, "Spatial Relationship Preserving Character Motion Adaptation"journal , ACM Transactions on Graphics (Proceedings of SIGGRAPH 2010), vol 29(4), July 2010. PDF video bibtex

The Team

Dr. Edmond S. L. Ho

Senior Lecturer, Northumbria University
e.ho@northumbria.ac.uk

Prof. Taku Komura

Professor, University of Hong Kong
taku@cs.hku.hk

Dr. Hubert P. H. Shum

Associate Professor, Durham University
hubert.shum@durham.ac.uk

Dr. He Wang

Associate Professor, University of Leeds
H.E.Wang@leeds.ac.uk

Dr. Jacky C. P. Chan

Lecturer, Hong Kong Baptist University
cpchan@comp.hkbu.edu.hk

Dr. Howard Leung

Associate Professor, City University of Hong Kong
howard@cityu.edu.hk