Explorando Modelos GPT na Geração de Código Paralelo para Aplicações de Computação Stencil
Resumo
Este trabalho explora a aplicação dos modelos de linguagem GPT para a geração automática de código paralelo em aplicações de Computação Stencil, ressaltando a importância desta abordagem para reduzir o tempo de desenvolvimento e aumentar a eficiência computacional. Ao aplicar os modelos GPT-3.5 turbo e o GPT-4o em oito aplicações Stencil com estruturas de dados variadas e executadas em três arquiteturas multicore, mostramos que ambos os modelos geraram códigos paralelos que melhoraram o desempenho em relação às versões sequenciais. No entanto, o GPT-4o não demonstrou um desempenho superior ao GPT-3.5 turbo e ambos enfrentaram desafios na gestão de dependências de dados. De um modo geral, mostra-se também que, através do uso correto das diretivas e cláusulas paralelas do OpenMP para exploração do paralelismo, o GPT-3.5 turbo foi capaz de gerar códigos paralelos com desempenho 15% superior aos do GPT-4o.Referências
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
Buscemi, A. (2023). A comparative study of code generation using chatgpt 3.5 across 10 programming languages.
Cattaneo, R., Natale, G., Sicignano, C., Sciuto, D., and Santambrogio, M. D. (2015). On how to accelerate iterative stencil loops: a scalable streaming-based approach. ACM Transactions on Architecture and Code Optimization (TACO), 12(4):1–26.
Cesare, V., Colonnelli, I., and Aldinucci, M. (2020). Practical parallelization of scientific applications. In 2020 28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pages 376–384. IEEE.
Eijkhout, V. (2010). Introduction to high performance scientific computing. Lulu. com.
Godoy, W., Valero-Lara, P., Teranishi, K., Balaprakash, P., and Vetter, J. (2023). Evaluation of openai codex for hpc parallel programming models kernel generation. In Proceedings of the 52nd International Conference on Parallel Processing Workshops, ICPP-W 2023. ACM.
Kalender, M. E., Mergenci, C., and Ozturk, O. (2014). Autopar: An automatic parallelization tool for recursive calls. In 2014 43rd International Conference on Parallel Processing Workshops, pages 159–165. IEEE.
Lorenzon, A. F. and Beck Filho, A. C. S. (2019). Parallel computing hits the power wall: principles, challenges, and a survey of solutions. Springer Nature.
Marques, S. M. V., Serpa, M. S., Muñoz, A. N., Rossi, F. D., Luizelli, M. C., Navaux, P. O., Beck, A. C. S., and Lorenzon, A. F. (2022). Optimizing the edp of openmp applications via concurrency throttling and frequency boosting. Journal of Systems Architecture, 123:102379.
Navaux, P. O. A., Lorenzon, A. F., and da Silva Serpa, M. (2023). Challenges in high-performance computing. Journal of the Brazilian Computer Society, 29(1):51–62.
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al. (2018). Improving language understanding by generative pre-training.
Schmidt, B., Gonzalez-Martinez, J., Hundt, C., and Schlarb, M. (2017). Parallel programming: concepts and practice. Morgan Kaufmann.
Sterling, T., Brodowicz, M., and Anderson, M. (2017). High performance computing: modern systems and practices. Morgan Kaufmann.
Valero-Lara, P., Huante, A., Lail, M. A., Godoy, W. F., Teranishi, K., Balaprakash, P., and Vetter, J. S. (2023). Comparing llama-2 and gpt-3 llms for hpc kernels generation.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2023). Attention is all you need.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
Buscemi, A. (2023). A comparative study of code generation using chatgpt 3.5 across 10 programming languages.
Cattaneo, R., Natale, G., Sicignano, C., Sciuto, D., and Santambrogio, M. D. (2015). On how to accelerate iterative stencil loops: a scalable streaming-based approach. ACM Transactions on Architecture and Code Optimization (TACO), 12(4):1–26.
Cesare, V., Colonnelli, I., and Aldinucci, M. (2020). Practical parallelization of scientific applications. In 2020 28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pages 376–384. IEEE.
Eijkhout, V. (2010). Introduction to high performance scientific computing. Lulu. com.
Godoy, W., Valero-Lara, P., Teranishi, K., Balaprakash, P., and Vetter, J. (2023). Evaluation of openai codex for hpc parallel programming models kernel generation. In Proceedings of the 52nd International Conference on Parallel Processing Workshops, ICPP-W 2023. ACM.
Kalender, M. E., Mergenci, C., and Ozturk, O. (2014). Autopar: An automatic parallelization tool for recursive calls. In 2014 43rd International Conference on Parallel Processing Workshops, pages 159–165. IEEE.
Lorenzon, A. F. and Beck Filho, A. C. S. (2019). Parallel computing hits the power wall: principles, challenges, and a survey of solutions. Springer Nature.
Marques, S. M. V., Serpa, M. S., Muñoz, A. N., Rossi, F. D., Luizelli, M. C., Navaux, P. O., Beck, A. C. S., and Lorenzon, A. F. (2022). Optimizing the edp of openmp applications via concurrency throttling and frequency boosting. Journal of Systems Architecture, 123:102379.
Navaux, P. O. A., Lorenzon, A. F., and da Silva Serpa, M. (2023). Challenges in high-performance computing. Journal of the Brazilian Computer Society, 29(1):51–62.
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al. (2018). Improving language understanding by generative pre-training.
Schmidt, B., Gonzalez-Martinez, J., Hundt, C., and Schlarb, M. (2017). Parallel programming: concepts and practice. Morgan Kaufmann.
Sterling, T., Brodowicz, M., and Anderson, M. (2017). High performance computing: modern systems and practices. Morgan Kaufmann.
Valero-Lara, P., Huante, A., Lail, M. A., Godoy, W. F., Teranishi, K., Balaprakash, P., and Vetter, J. S. (2023). Comparing llama-2 and gpt-3 llms for hpc kernels generation.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2023). Attention is all you need.
Publicado
23/10/2024
Como Citar
DIAS, João Vitor M.; BECK, Antonio Carlos S.; LORENZON, Arthur F..
Explorando Modelos GPT na Geração de Código Paralelo para Aplicações de Computação Stencil. In: SIMPÓSIO EM SISTEMAS COMPUTACIONAIS DE ALTO DESEMPENHO (SSCAD), 25. , 2024, São Carlos/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 348-359.
DOI: https://doi.org/10.5753/sscad.2024.244736.
