| 제목 |
Automatic Detection of Penetration or Aspiration in Videofluoroscopy by Deep Learning Technology |
| 소속 |
Dankook University Hospital, Department of Rehabilitation Medicine1, Dankook University, Department of Nanobiomedical Science & BK21 PLUS NBM Research Center for Regenerative Medicine2, Dankook University, ) Institute of Tissue Regeneration Engineering (ITREN)3, Dankook University, Department of Applied Computer Engineering4, Dankook University, Department of Software Engineering5 |
| 저자 |
Hyun Il Kim5, Tae Uk Kim1, Seo Young Kim1, Jung Keun Hyun1,2, Sang Il Choi4, Seong Jae Lee1†, Joo Young Ko1* |
Objective
Analysis of Videofluoroscopic swallowing study (VFSS) can be time-consuming and tedious job and its reliability has been reported to be unsatisfactory. Recent advancement in image classification technology enables fully automatic detection of objects but its application but it has never been applied in VFSS image analysis. We aimed to develop an algorithm that can detect penetration or aspiration in VFSS images in a fully automated manner, using open source deep learning algorithms including convolutional neural network. This study may be the first step of development of fully automatic reading system of VFSS.
Methods
One hundred eighty VFSS files (108 normal and 72 with penetration or aspiration) from 71 subjects were collected for establishment of dataset. Each video files were split into individual frame images obtaining 138,890 normal and 5,682 dysphagic images with penetration or aspiration. Determination of normal and dysphagic images was performed by a physiatrist experienced in VFSS. Dataset was balanced by undersampling of normal images to 23,465 images and oversampling of dysphagic images to 24,672 images. Images were distributed to training, validation and test images at the ratio of 7:1:2. After equalizing by contrast limited adaptive histogram equalization (CLAHE) algorithm, automated detection of penetration or aspiration was trained by open source deep learning algorithms. Detection was accomplished in two steps. First, regions of interest (ROIs) were set around larynx with reference to cervical spinal column identified by cervical detector using Unet deep learning segmentation model that showed accuracy of 99.4% in previous study. Then penetration/aspiration detector identified presence of penetration or aspiration in those ROIs. Accuracy based on frame or video file was calculated and compared among the deep learning models.
Results
Accuracy of each deep learning model is presented in tables. Xception model showed the best result in frame based analysis but NasnetMobile did in file based analysis. Accuracy of detecting frame images that showed penetration or aspiration was 98.6%. When classifying files with penetration or aspiration, accuracy was 89.7%.
Conclusion
This study was first attempt to identify penetration or aspiration from VFSS images in a fully automated manner using deep learning image classification technology. The results show that deep learning algorithm can detect penetration or aspiration automatically from VFSS images with a significant accuracy. Further researches to improve accuracy and prospective clinical trials are required. |
|
Frame based accuracy of detection by deep learning model
File based accuracy of detection by deep learning model
|
