Proceedings of the Joint 3rd International Conference on Bioinformatics and Data Science (ICBDS 2022)

Evaluating Generalizability of Deep Learning Models Using Indian-COVID-19 CT Dataset

Authors
S. Suba1, *, Nita Parekh1, Ramesh Loganathan1, Vikram Pudi1, Chinnababu Sunkavalli2
1International Institute of Information Technology, Gachibowli, Hyderabad, 500032, India
2Grace Cancer Foundation, Hyderabad, India
*Corresponding author. Email: suba.s@research.iiit.ac.in
Corresponding Author
S. Suba
Available Online 5 June 2023.
DOI
10.2991/978-94-6463-164-7_3How to use a DOI?
Keywords
Convolutional Neural Network; Computed Tomography Scans; COVID-19; Deep Learning
Abstract

Computer tomography (CT) have been routinely used for the diagnosis of lung diseases and recently, during the pandemic, for detecting the infectivity and severity of COVID-19 disease. One of the major concerns in using machine learning (ML) approaches for automatic processing of CT scan images in clinical setting is that these methods are trained on limited and biased subsets of publicly available COVID-19 data. This has raised concerns regarding the generalizability of these models on external datasets, not seen by the model during training. To address some of these issues, in this work CT scan images from confirmed COVID-19 data obtained from one of the largest public repositories, COVIDx CT 2A were used for training and internal validation of machine learning models. For the external validation we generated Indian-COVID-19 CT dataset, an open-source repository containing 3D CT volumes and 12096 chest CT images from 288 COVID-19 patients from India. Comparative performance evaluation of four state-of-the-art machine learning models, viz., a lightweight convolutional neural network (CNN), and three other CNN based deep learning (DL) models such as VGG-16, ResNet-50 and Inception-v3 in classifying CT images into three classes, viz., normal, non-covid pneumonia, and COVID-19 is carried out on these two datasets. Our analysis showed that the performance of all the models is comparable on the hold-out COVIDx CT 2A test set with 90%–99% accuracies (96% for CNN), while on the external Indian-COVID-19 CT dataset a drop in the performance is observed for all the models (8%–19%). The traditional machine learning model, CNN performed the best on the external dataset (accuracy 88%) in comparison to the deep learning models, indicating that a lightweight CNN is better generalizable on unseen data. The data and code are made available at https://github.com/aleesuss/c19.

Copyright
© 2023 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

Volume Title
Proceedings of the Joint 3rd International Conference on Bioinformatics and Data Science (ICBDS 2022)
Series
Advances in Health Sciences Research
Publication Date
5 June 2023
ISBN
978-94-6463-164-7
ISSN
2468-5739
DOI
10.2991/978-94-6463-164-7_3How to use a DOI?
Copyright
© 2023 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Cite this article

TY  - CONF
AU  - S. Suba
AU  - Nita Parekh
AU  - Ramesh Loganathan
AU  - Vikram Pudi
AU  - Chinnababu Sunkavalli
PY  - 2023
DA  - 2023/06/05
TI  - Evaluating Generalizability of Deep Learning Models Using Indian-COVID-19 CT Dataset
BT  - Proceedings of the Joint 3rd International Conference on Bioinformatics and Data Science (ICBDS 2022)
PB  - Atlantis Press
SP  - 15
EP  - 28
SN  - 2468-5739
UR  - https://doi.org/10.2991/978-94-6463-164-7_3
DO  - 10.2991/978-94-6463-164-7_3
ID  - Suba2023
ER  -