TY - GEN
T1 - A Sentiment Pre-trained Text-Guided Multimodal Cross-Attention Transformer for Improved Depression Detection
AU - Teng, Shiyu
AU - Chai, Shurong
AU - Liu, Jiaqing
AU - Tateyama, Tomoko
AU - Lin, Lanfen
AU - Chen, Yen Wei
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Depression is a widespread mental health issue requiring efficient automated detection methods. Traditional single-modality approaches are less effective due to the disorder's complexity, leading to a focus on multimodal analysis. Recent advancements include transformer-based fusion methods, yet their application in depression detection is often limited by the dominant text modality. To address this, we propose the Text-Guided Multimodal Cross-Attention Transformer, enhancing cross-modal interactions between text, audio, and video for more effective depression detection. Our approach uniquely pre-trains encoders on a large sentiment dataset to better capture emotion-related features crucial for identifying depression-related sentiment changes. Our method demonstrates superior performance on the AVEC2019 benchmark, outperforming current state-of-the-art depression detection techniques.
AB - Depression is a widespread mental health issue requiring efficient automated detection methods. Traditional single-modality approaches are less effective due to the disorder's complexity, leading to a focus on multimodal analysis. Recent advancements include transformer-based fusion methods, yet their application in depression detection is often limited by the dominant text modality. To address this, we propose the Text-Guided Multimodal Cross-Attention Transformer, enhancing cross-modal interactions between text, audio, and video for more effective depression detection. Our approach uniquely pre-trains encoders on a large sentiment dataset to better capture emotion-related features crucial for identifying depression-related sentiment changes. Our method demonstrates superior performance on the AVEC2019 benchmark, outperforming current state-of-the-art depression detection techniques.
UR - http://www.scopus.com/inward/record.url?scp=85214970681&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85214970681&partnerID=8YFLogxK
U2 - 10.1109/EMBC53108.2024.10782904
DO - 10.1109/EMBC53108.2024.10782904
M3 - Conference contribution
AN - SCOPUS:85214970681
T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
BT - 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2024 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2024
Y2 - 15 July 2024 through 19 July 2024
ER -