Augmented Reality Visualization and Quantification of COVID-19 Infections in the Lungs

Jiaqing Liu, Liang Lyu, Shurong Chai, Huimin Huang, Fang Wang, Tomoko Tateyama, Lanfen Lin, Yenwei Chen

研究成果: ジャーナルへの寄稿学術論文査読

1 被引用数 (Scopus)

抄録

The ongoing COVID-19 pandemic has had a significant impact globally, and the understanding of the disease’s clinical features and impacts remains insufficient. An important metric to evaluate the severity of pneumonia in COVID-19 is the CT Involvement Score (CTIS), which is determined by assessing the proportion of infections in the lung field region using computed tomography (CT) images. Interactive augmented reality visualization and quantification of COVID-19 infection from CT allow us to augment the traditional diagnostic techniques and current COVID-19 treatment strategies. Thus, in this paper, we present a system that combines augmented reality (AR) hardware, specifically the Microsoft HoloLens, with deep learning algorithms in a user-oriented pipeline to provide medical staff with an intuitive 3D augmented reality visualization of COVID-19 infections in the lungs. The proposed system includes a graph-based pyramid global context reasoning module to segment COVID-19-infected lung regions, which can then be visualized using the HoloLens AR headset. Through segmentation, we can quantitatively evaluate and intuitively visualize which part of the lung is infected. In addition, by evaluating the infection status in each lobe quantitatively, it is possible to assess the infection severity. We also implemented Spectator View and Sharing a Scene functions into the proposed system, which enable medical staff to present the AR content to a wider audience, e.g., radiologists. By providing a 3D perception of the complexity of COVID-19, the augmented reality visualization generated by the proposed system offers an immersive experience in an interactive and cooperative 3D approach. We expect that this will facilitate a better understanding of CT-guided COVID-19 diagnosis and treatment, as well as improved patient outcomes.

本文言語英語
論文番号1158
ジャーナルElectronics (Switzerland)
13
6
DOI
出版ステータス出版済み - 03-2024

All Science Journal Classification (ASJC) codes

  • 制御およびシステム工学
  • 信号処理
  • ハードウェアとアーキテクチャ
  • コンピュータ ネットワークおよび通信
  • 電子工学および電気工学

フィンガープリント

「Augmented Reality Visualization and Quantification of COVID-19 Infections in the Lungs」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル