LINS DE ALBUQUERQUE, Beatriz Ventorini; SOUZA DA CUNHA, Antonio Fernando; SOUZA, Leonardo; SIQUEIRA, Sean Wolfgand Matsui; SANTOS, Rodrigo Pereira. Generating and Reviewing Programming Codes with Large Language Models: A Systematic Mapping Study. In: XX Brazilian Symposium on Information Systems (SBSI), 20. , 2024, Juiz de Fora/MG. Proceedings […]. New York: Association for Computing Machinery, 2024 . Article No. 70, p. 1-10. DOI: https://doi.org/10.1145/3658271.3658342.
Generating and Reviewing Programming Codes with Large Language Models: A Systematic Mapping Study
Authors
Beatriz Ventorini Lins de Albuquerque (UNIRIO e Petrobras) Antonio Fernando Souza da Cunha (UNIRIO e Petrobras) Leonardo Souza (UNIRIO) Sean Wolfgand Matsui Siqueira (UNIRIO) Rodrigo Pereira dos Santos (UNIRIO)
Abstract
Context: The proliferation of technologies based on Large Language Models (LLM) is reshaping various domains, also impacting on programming code creation and review. Problem: The decision-making process in adopting LLM in software development demands an understanding of associated challenges and diverse application possibilities. Solution: This study addresses the identified challenges linked to LLM utilization in programming code processes. It explores models, utilization strategies, challenges, and coping mechanisms, focusing on the perspectives of researchers in software development. IS Theory: Drawing on Task-Technology Fit (TTF) theory, the research examines the alignment between task characteristics in code generation and review, and LLM technology attributes to discern performance impacts and utilization patterns. Method: Employing the Systematic Mapping of the Literature method, the research analyzes 19 selected studies from digital databases—IEEE Digital Library, Compendex Engineering Village, and Scopus—out of 1,257 retrieved results. Summary of Results: The research reveals 23 models, 13 utilization strategies, 15 challenges, and 14 coping mechanisms associated with LLM in programming code processes, offering a comprehensive understanding of the application landscape. Contributions to IS: Contributing to the Information Systems (IS) field, This study provides valuable insights into the utilization of LLM in programming code generation and review. The identified models, strategies, challenges, and coping mechanisms offer practical guidance for decision-making processes related to LLM technology adoption. The research aims to support the IS community in effectively navigating the complexities of integrating large language models into the dynamic software development lifecycle.
Keywords:
Code Generation; LLM; automatic refactoring; code auto-suggestion; code completion; natural language models; neural networks; systematic mapping study; transformer architecture. doi: 10.1145/3658271.3658342
