Toward Automatic Parallelization: Detecting Parallelizable Loops Using Large Language Modeling


Soratouch Pornmaneerattanatri, Keichi Takahashi, Yutaro Kashiwa, Kohei Ichikawa, Hajimu Iida, Journal of Information Processing Systems Vol. 21, No. 3, pp. 271-283, Jun. 2025  

https://doi.org/10.3745/JIPS.04.0349
Keywords: Automatic Parallelization, Deep Learning, Large Language Model, OpenMP, Parallel computing
Fulltext:

Abstract

To fully harness the capabilities of multi-core processors, parallel programming is indispensable, demanding a comprehensive understanding of both software and hardware. Although various tools have been developed to automate parallel programming by leveraging static analysis approaches, manually parallelized code continues to consistently outperform those that are automatically parallelized. Meanwhile, the emergence of transformer-based large language models has facilitated significant breakthroughs in understanding and generating programming languages. This study presents a model tailored to detecting parallelizable for-loops by fine-tuning the transformer-based model, CodeT5. The fine-tuned model assists programmers in identifying independent for-loops that have the potential for parallelization with libraries like OpenMP, leading to performance enhancements in software applications. The model was trained on 500,000 for-loops sourced from public GitHub repositories and demonstrated an F1-score of 0.860 in detecting parallelizable for-loops within a public GitHub dataset, along with an F1-score of 0.764 on the NAS Parallel Benchmark suite.


Statistics
Show / Hide Statistics

Statistics (Past 3 Years)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.




Cite this article
[APA Style]
Pornmaneerattanatri, S., Takahashi, K., Kashiwa, Y., Ichikawa, K., & Iida, H. (2025). Toward Automatic Parallelization: Detecting Parallelizable Loops Using Large Language Modeling. Journal of Information Processing Systems, 21(3), 271-283. DOI: 10.3745/JIPS.04.0349.

[IEEE Style]
S. Pornmaneerattanatri, K. Takahashi, Y. Kashiwa, K. Ichikawa, H. Iida, "Toward Automatic Parallelization: Detecting Parallelizable Loops Using Large Language Modeling," Journal of Information Processing Systems, vol. 21, no. 3, pp. 271-283, 2025. DOI: 10.3745/JIPS.04.0349.

[ACM Style]
Soratouch Pornmaneerattanatri, Keichi Takahashi, Yutaro Kashiwa, Kohei Ichikawa, and Hajimu Iida. 2025. Toward Automatic Parallelization: Detecting Parallelizable Loops Using Large Language Modeling. Journal of Information Processing Systems, 21, 3, (2025), 271-283. DOI: 10.3745/JIPS.04.0349.