Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

Masaya Okano; Jia Qing Liu; Tomoko Tateyama; Yutaro Iwamoto; Yen Wei Chen

doi:10.1109/GCCE56475.2022.10014261

Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

Masaya Okano, Jia Qing Liu, Tomoko Tateyama, Yutaro Iwamoto, Yen Wei Chen

研究推進ﾕﾆｯﾄ・知能情報工学分野

研究成果: 書籍/レポートタイプへの寄稿 › 会議への寄与

3 被引用数 (Scopus)

抄録

Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.

本文言語	英語
ホスト出版物のタイトル	GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics
出版社	Institute of Electrical and Electronics Engineers Inc.
ページ	645-648
ページ数	4
ISBN（電子版）	9781665492324
DOI	https://doi.org/10.1109/GCCE56475.2022.10014261
出版ステータス	出版済み - 2022
イベント	11th IEEE Global Conference on Consumer Electronics, GCCE 2022 - Osaka, 日本継続期間: 18-10-2022 → 21-10-2022

出版物シリーズ

名前	GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics

会議

会議	11th IEEE Global Conference on Consumer Electronics, GCCE 2022
国/地域	日本
City	Osaka
Period	18-10-22 → 21-10-22

All Science Journal Classification (ASJC) codes

信号処理
情報システムおよび情報管理
電子工学および電気工学
メディア記述
器械工学
社会心理学

文献へのアクセス

10.1109/GCCE56475.2022.10014261

他のファイルとリンク

引用スタイル

Okano, M., Liu, J. Q., Tateyama, T., Iwamoto, Y., & Chen, Y. W. (2022). Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. In GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics (pp. 645-648). (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GCCE56475.2022.10014261

Okano, Masaya ; Liu, Jia Qing ; Tateyama, Tomoko その他. / Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 645-648 (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics).

@inproceedings{6581d6917df242baa89a48d486717c18,

title = "Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion",

abstract = "Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.",

author = "Masaya Okano and Liu, {Jia Qing} and Tomoko Tateyama and Yutaro Iwamoto and Chen, {Yen Wei}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 11th IEEE Global Conference on Consumer Electronics, GCCE 2022 ; Conference date: 18-10-2022 Through 21-10-2022",

year = "2022",

doi = "10.1109/GCCE56475.2022.10014261",

language = "English",

series = "GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "645--648",

booktitle = "GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics",

address = "United States",

}

Okano, M, Liu, JQ, Tateyama, T, Iwamoto, Y & Chen, YW 2022, Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. in GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics, Institute of Electrical and Electronics Engineers Inc., pp. 645-648, 11th IEEE Global Conference on Consumer Electronics, GCCE 2022, Osaka, 日本, 18-10-22. https://doi.org/10.1109/GCCE56475.2022.10014261

Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. / Okano, Masaya; Liu, Jia Qing; Tateyama, Tomoko その他.
GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. Institute of Electrical and Electronics Engineers Inc., 2022. p. 645-648 (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics).

研究成果: 書籍/レポートタイプへの寄稿 › 会議への寄与

TY - GEN

T1 - Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

AU - Okano, Masaya

AU - Liu, Jia Qing

AU - Tateyama, Tomoko

AU - Iwamoto, Yutaro

AU - Chen, Yen Wei

PY - 2022

Y1 - 2022

N2 - Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.

AB - Gesture recognition is actively used, and has been applied in various fields, including games and medicine. For accurate gesture recognition, multi-modal information with color and depth images has recently been used. In multimodal gesture recognition, the fusion of color and depth images is crucial. To date, early and late fusion approaches have been widely used for the fusion of color and depth images. However, the enhancement of performance is limited due to the gap between the modalities. In this study, we proposed a modality-invariant fusion approach to overcome the modality gap issue. We applied the proposed approach to a public and our private data set and verified its effectiveness.

UR - http://www.scopus.com/inward/record.url?scp=85147256232&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85147256232&partnerID=8YFLogxK

U2 - 10.1109/GCCE56475.2022.10014261

DO - 10.1109/GCCE56475.2022.10014261

M3 - Conference contribution

AN - SCOPUS:85147256232

T3 - GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics

SP - 645

EP - 648

BT - GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 11th IEEE Global Conference on Consumer Electronics, GCCE 2022

Y2 - 18 October 2022 through 21 October 2022

ER -

Okano M, Liu JQ, Tateyama T, Iwamoto Y, Chen YW. Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion. In GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics. Institute of Electrical and Electronics Engineers Inc. 2022. p. 645-648. (GCCE 2022 - 2022 IEEE 11th Global Conference on Consumer Electronics). doi: 10.1109/GCCE56475.2022.10014261

Accurate Hand Gesture Recognition Using Color and Depth Images with Modality-invariant Fusion

抄録

出版物シリーズ

会議

All Science Journal Classification (ASJC) codes

文献へのアクセス

他のファイルとリンク

フィンガープリント

引用スタイル