Investigation of Parallel and Hyperparameters Strategy on Performance of Image Classification Training

DOI

10.2991/978-94-6463-300-9_105How to use a DOI?

Keywords

Convolutional Neural Networks; Image classification; GPU utilization; Data parallel

Abstract

Convolutional Neural Networks (CNNs) have witnessed widespread adoption in the domain of image classification, while deep neural networks have been developed to tackle more intricate tasks. In the experimental investigation, a remarkable downward trend in GPU utilization was observed as the batch size of the LeNet model was increased, regardless of the parallel or non-parallel mechanism employed. The research findings establish that this phenomenon can be ascribed to a constraint in data loading speed, which in turn diminishes the efficiency of training when dealing with larger batch sizes, ultimately resulting in reduced GPU utilization. To mitigate this issue, the data loading thread are enhanced by adjusting the "num_worker" parameter in the dataloader, thereby investigating its impact on GPU utilization. Moreover, a series of comprehensive experiments are conducted to ascertain the appropriate learning rates required for maintaining satisfactory classification accuracy when utilizing large batch sizes. This paper contributes to the field in two primary ways. Firstly, it identifies the cause of decreased GPU utilization when the batch size is increased and proposes a solution to enhance efficiency. Secondly, it verifies the adjustment of learning rates when adopting large batch sizes to achieve comparable loss curves and classification accuracies.

Copyright

© 2023 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Yannan Cao
AU  - Weiran Shen
PY  - 2023
DA  - 2023/11/27
TI  - Investigation of Parallel and Hyperparameters Strategy on Performance of Image Classification Training
BT  - Proceedings of the 2023 International Conference on Image, Algorithms and Artificial Intelligence (ICIAAI 2023)
PB  - Atlantis Press
SP  - 1046
EP  - 1061
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-300-9_105
DO  - 10.2991/978-94-6463-300-9_105
ID  - Cao2023
ER  -