Single Image Depth Map Estimation for Improving Posture Recognition

Jiaqing Liu, Seiju Tsujinaga, Shurong Chai, Hao Sun, Tomoko Tateyama, Yutaro Iwamoto, Xinyin Huang, Lanfen Lin, Yen Wei Chen

Research output: Contribution to journalArticlepeer-review

14 Citations (Scopus)

Abstract

Image-based posture recognition is a very challenging problem since it is difficult to acquire rich 3D information from the posture in color image. To address this issue, we present a novel and unified framework for human posture recognition, applying single image depth map estimation from color images. The proposed method includes two stages. The first stage estimates the depth map from the single-color image by an improved Pix2Pix generation module. The generation module is equipped with a hybrid loss function that captures the high-level features and recovers the sharp depth discontinuities, thus improving the depth estimation results. The second stage (the recognition stage) improves the color image-based recognition performance by incorporating the estimated depth map. Thereby, a two-stream CNN architecture that separately processes the color image and its estimated depth image is developed for robust posture recognition. To verify its effectiveness, we first test the proposed method on a novel pose dataset, which contains 13800 samples of paired color-and-depth of 6 subjects with 15 poses. The dataset used in this work is been created and released, is available at http://media.ritsumei.ac.jp/iipl/database/pose/. Extensive experiments are also performed on the public OUHANDS hand gesture dataset. Experiments demonstrate that the proposed method achieves superior performance on both human pose and hand gesture recognition tasks.

Original languageEnglish
Pages (from-to)26997-27004
Number of pages8
JournalIEEE Sensors Journal
Volume21
Issue number23
DOIs
Publication statusPublished - 01-12-2021
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Instrumentation
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Single Image Depth Map Estimation for Improving Posture Recognition'. Together they form a unique fingerprint.

Cite this