Augmented Reality-Based Lung Ultrasound Scanning Guidance
Published in MICCAI ASMUS, 2020
Recommended citation: Bimbraw, K., Ma, X., Zhang, Z., Zhang, H. (2020). "Augmented Reality-Based Lung Ultrasound Scanning Guidance". In: Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis. ASMUS 2020, PIPPI 2020. Lecture Notes in Computer Science, vol 12437. Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-030-60334-2_11
Lung ultrasound (LUS) is an established non-invasive imaging method for diagnosing respiratory illnesses. With the rise of SARS-CoV-2 (COVID-19) as a global pandemic, LUS has been used to detect pneumopathy for triaging and monitoring patients who are diagnosis or suspected with COVID-19 infection. While LUS offers a cost-effective, radiation-free, and higher portability compared with chest X-ray and CT, its accessibility is limited due to its user dependency and small number of physicians and sonographers who can perform appropriate scanning and diagnosis. In this paper, we propose a framework of guiding LUS scanning featuring augmented reality, in which the LUS procedure can be guided by projecting the scanning trajectory. To develop such a system, we implement a computer vision-based detection algorithm to classify different regions on human body. The DensePose algorithm is used to obtain a body mesh data for the upper body pictured with a mono-camera. Torso sub-mesh is used to extract and overlay the eight regions corresponding to anterior and lateral chests for LUS guidance. To minimize instability of the DensePose mesh coordinates based on different frontal angles of camera, a machine learning regression algorithm is applied to predict the angle-specific projection model with respect to the chest. ArUco markers are utilized for training the ground truth chest regions to be scanned and another single ArUco marker is used for detecting the center line of the body. The augmented scanning regions are highlighted one by one to guide the scanning path to execute the LUS procedure. We demonstrated the feasibility of guiding the LUS scanning procedure through the combination of augmented reality, computer vision, and machine learning.
Recommended citation: Bimbraw, K., Ma, X., Zhang, Z., Zhang, H. (2020). “Augmented Reality-Based Lung Ultrasound Scanning Guidance”. In: Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis. ASMUS 2020, PIPPI 2020. Lecture Notes in Computer Science, vol 12437. Springer, Cham.