Si-Ung Kim, Nammee Moon, Journal of Information Processing Systems Vol. 21, No. 5, pp. 484-493, Oct. 2025  

https://doi.org/10.3745/JIPS.02.0226
Keywords: Convolution Neural Network, Image Classification, Optimizer Fusion, Vision Transformer
Fulltext: PDF Full Text PubReader

Abstract

Training deep learning models involves the use of various optimization algorithms, each with its own advantages and disadvantages. Stochastic gradient descent (SGD) provides consistent performance and stable optimization but has the drawback of a slow convergence rate. On the other hand, Adam offers the advantage of fast convergence but can lead to overfitting. This study proposes a hybrid method that combines the stable convergence of SGD with the fast convergence of Adam, enabling the model to be optimized quickly and stably. This approach was applied to the EfficientNetV2 and Vision Transformer (ViT) architectures in image classification tasks. EfficientNetV2 used Adam up to the 6th block and then switched to SGD, achieving the best performance on the proposed dataset with an accuracy of 97.84%, a loss of 0.0990, and an F1-score of 98.04%. Similarly, ViT used Adam for the first 10 encoders and then switched to SGD for the remaining 10 encoders, showing optimal results on the same dataset with an accuracy of 98.54%, a loss of 0.1345, and an F1-score of 98.53%. This fusion optimizer approach effectively enhances training by using Adam for initial feature extraction and SGD for later stages.

Statistics
Cite this article