DeepHandRobustHandPoseEstimationbyCompletingaMatrix.pdfVIP

DeepHand: Robust Hand Pose Estimation by Completing a Matrix Imputed with Deep Features Ayan Sinha? Chiho Choi? Karthik Ramani Purdue University West Lafayette, IN 47907, USA {sinha12, chihochoi, ramani}@purdue.edu Abstract We propose DeepHand to estimate the 3D pose of a hand using depth data from commercial 3D sensors. We discriminatively train convolutional neural networks to output a low dimensional activation feature given a depth map. This activation feature vector is representative of the global or local joint angle parameters of a hand pose. We ef?ciently identify ’spatial’ nearest neighbors to the activation feature, from a database of features corresponding to synthetic depth maps, and store some ’temporal’ neighbors from previous frames. Our matrix completion algorithm uses these ’spatio-temporal’ activation features and the corresponding known pose parameter values to estimate the unknown pose parameters of the input feature vector. Our database of activation features supplements large viewpoint coverage and our hierarchical estimation of pose parameters is robust to occlusions. We show that our approach compares favorably to state-of-the-art methods while achieving real time performance (≈ 32 FPS) on a standard computer. 1. Introduction Robust hand tracking is central to human-computer interaction interfaces and augmented reality applications. Although, there exists robust and accurate methods for full body tracking, hand tracking is far more challenging [10, 11, 26, 23, 29, 16, 14, 17, 18]. This is due to several reasons: (i) the hand pose exists in a high dimensional space because each ?nger and the palm is associated with several degrees of freedom, (ii) the ?ngers exhibit self similarity, are ?exible and often occlude each other, (iii) noise in acquired data coupled with fast ?nger articulations confounds continuous hand tracking. Multi camera setups or GPU acceleration eases some of these challenges, but limits deployment to the general public. We