3D Motion Reconstruction from 2D Motion Data Using Multimodal Conditional Deep Belief Network

Heydari, Muhamad Javad and SHIRY GHIDARY, Saeed (2019) 3D Motion Reconstruction from 2D Motion Data Using Multimodal Conditional Deep Belief Network. IEEE ACCESS, 7. pp. 56389-56408. ISSN 2169-3536

Preview

Text 08707086.pdf - Publisher's typeset copy Available under License Creative Commons Attribution 4.0 International (CC BY 4.0) . Download (11MB) | Preview

Abstract or description

In this paper, we propose a deep generative model named Multimodal Conditional Deep Belief Network (MCDBN) for cross-modal learning of 3D motion data and their non-injective 2D projections on the image plane. This model has a three sectional structure, which learns conditional probability distribution of 3D motion data given 2D projections. Two distinct Conditional Deep Belief Networks (CDBNs), encode the real-valued spatiotemporal patterns of 2D and 3D motion time series captured from subjects’ movements into the compact representations. The third part includes a Multimodal Restricted Boltzmann Machines (MRBMs) which in the training process, learns the relationship between the compact representations of data modalities by variation information criteria. As a result, conditioned on a 2D motion data obtained from a video, MCDBN can regenerate 3D motion data in generation phase. We introduce Pearson correlation coefficient of ground truth and regenerated motion signals as a new evaluation metric in motion reconstruction problems. The model is trained with human motion capture data and the results show that the real and the regenerated signals are highly correlated which means the model can reproduce dynamical patterns of motion accurately.

Item Type:	Article
Faculty:	School of Computing and Digital Technologies > Computing
Depositing User:	Saeed SHIRY GHIDARY
Date Deposited:	05 Feb 2021 09:32
Last Modified:	24 Feb 2023 14:01
URI:	https://eprints.staffs.ac.uk/id/eprint/6787

Actions (login required)

View Item