TY - JOUR
T1 - Single Image Depth Map Estimation for Improving Posture Recognition
AU - Liu, Jiaqing
AU - Tsujinaga, Seiju
AU - Chai, Shurong
AU - Sun, Hao
AU - Tateyama, Tomoko
AU - Iwamoto, Yutaro
AU - Huang, Xinyin
AU - Lin, Lanfen
AU - Chen, Yen Wei
N1 - Publisher Copyright:
© 2001-2012 IEEE.
PY - 2021/12/1
Y1 - 2021/12/1
N2 - Image-based posture recognition is a very challenging problem since it is difficult to acquire rich 3D information from the posture in color image. To address this issue, we present a novel and unified framework for human posture recognition, applying single image depth map estimation from color images. The proposed method includes two stages. The first stage estimates the depth map from the single-color image by an improved Pix2Pix generation module. The generation module is equipped with a hybrid loss function that captures the high-level features and recovers the sharp depth discontinuities, thus improving the depth estimation results. The second stage (the recognition stage) improves the color image-based recognition performance by incorporating the estimated depth map. Thereby, a two-stream CNN architecture that separately processes the color image and its estimated depth image is developed for robust posture recognition. To verify its effectiveness, we first test the proposed method on a novel pose dataset, which contains 13800 samples of paired color-and-depth of 6 subjects with 15 poses. The dataset used in this work is been created and released, is available at http://media.ritsumei.ac.jp/iipl/database/pose/. Extensive experiments are also performed on the public OUHANDS hand gesture dataset. Experiments demonstrate that the proposed method achieves superior performance on both human pose and hand gesture recognition tasks.
AB - Image-based posture recognition is a very challenging problem since it is difficult to acquire rich 3D information from the posture in color image. To address this issue, we present a novel and unified framework for human posture recognition, applying single image depth map estimation from color images. The proposed method includes two stages. The first stage estimates the depth map from the single-color image by an improved Pix2Pix generation module. The generation module is equipped with a hybrid loss function that captures the high-level features and recovers the sharp depth discontinuities, thus improving the depth estimation results. The second stage (the recognition stage) improves the color image-based recognition performance by incorporating the estimated depth map. Thereby, a two-stream CNN architecture that separately processes the color image and its estimated depth image is developed for robust posture recognition. To verify its effectiveness, we first test the proposed method on a novel pose dataset, which contains 13800 samples of paired color-and-depth of 6 subjects with 15 poses. The dataset used in this work is been created and released, is available at http://media.ritsumei.ac.jp/iipl/database/pose/. Extensive experiments are also performed on the public OUHANDS hand gesture dataset. Experiments demonstrate that the proposed method achieves superior performance on both human pose and hand gesture recognition tasks.
UR - http://www.scopus.com/inward/record.url?scp=85118544005&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85118544005&partnerID=8YFLogxK
U2 - 10.1109/JSEN.2021.3122128
DO - 10.1109/JSEN.2021.3122128
M3 - Article
AN - SCOPUS:85118544005
SN - 1530-437X
VL - 21
SP - 26997
EP - 27004
JO - IEEE Sensors Journal
JF - IEEE Sensors Journal
IS - 23
ER -