Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

Masaya Okano; Jia Qing Liu; Tomoko Tateyama; Yutaro Iwamoto; Yen Wei Chen

doi:10.1109/GCCE56475.2022.10014261

Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

Masaya Okano, Jia Qing Liu, Tomoko Tateyama, Yutaro Iwamoto, Yen Wei Chen

Intelligent Information Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.

Original language	English
Title of host publication	GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	645-648
Number of pages	4
ISBN (Electronic)	9781665492324
DOIs	https://doi.org/10.1109/GCCE56475.2022.10014261
Publication status	Published - 2022
Event	11th IEEE Global Conference on Consumer Electronics, GCCE 2022 - Osaka, Japan Duration: 18-10-2022 → 21-10-2022

Publication series

Name	GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics

Conference

Conference	11th IEEE Global Conference on Consumer Electronics, GCCE 2022
Country/Territory	Japan
City	Osaka
Period	18-10-22 → 21-10-22

All Science Journal Classification (ASJC) codes

Signal Processing
Information Systems and Management
Electrical and Electronic Engineering
Media Technology
Instrumentation
Social Psychology

Access to Document

10.1109/GCCE56475.2022.10014261

Cite this

Okano, M., Liu, J. Q., Tateyama, T., Iwamoto, Y., & Chen, Y. W. (2022). Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. In GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics (pp. 645-648). (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GCCE56475.2022.10014261

@inproceedings{6581d6917df242baa89a48d486717c18,

title = "Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion",

abstract = "Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.",

author = "Masaya Okano and Liu, {Jia Qing} and Tomoko Tateyama and Yutaro Iwamoto and Chen, {Yen Wei}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 11th IEEE Global Conference on Consumer Electronics, GCCE 2022 ; Conference date: 18-10-2022 Through 21-10-2022",

year = "2022",

doi = "10.1109/GCCE56475.2022.10014261",

language = "English",

series = "GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "645--648",

booktitle = "GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics",

address = "United States",

}

Okano, M, Liu, JQ, Tateyama, T, Iwamoto, Y & Chen, YW 2022, Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. in GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics, Institute of Electrical and Electronics Engineers Inc., pp. 645-648, 11th IEEE Global Conference on Consumer Electronics, GCCE 2022, Osaka, Japan, 18-10-22. https://doi.org/10.1109/GCCE56475.2022.10014261

Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. / Okano, Masaya; Liu, Jia Qing; Tateyama, Tomoko et al.
GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. Institute of Electrical and Electronics Engineers Inc., 2022. p. 645-648 (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

AU - Okano, Masaya

AU - Liu, Jia Qing

AU - Tateyama, Tomoko

AU - Iwamoto, Yutaro

AU - Chen, Yen Wei

PY - 2022

Y1 - 2022

N2 - Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.

AB - Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.

UR - http://www.scopus.com/inward/record.url?scp=85147256232&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85147256232&partnerID=8YFLogxK

U2 - 10.1109/GCCE56475.2022.10014261

DO - 10.1109/GCCE56475.2022.10014261

M3 - Conference contribution

AN - SCOPUS:85147256232

T3 - GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics

SP - 645

EP - 648

BT - GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 11th IEEE Global Conference on Consumer Electronics, GCCE 2022

Y2 - 18 October 2022 through 21 October 2022

ER -

Okano M, Liu JQ, Tateyama T, Iwamoto Y, Chen YW. Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. In GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. Institute of Electrical and Electronics Engineers Inc. 2022. p. 645-648. (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics). doi: 10.1109/GCCE56475.2022.10014261

Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this