How Good Is AI at Coding, Really?
Study finds that while AI can be great, it also struggles due to training limitations
In collaboration with IEEE Xplore, this piece is a part of
our unique IEEE Journal Watch series.
It's a full circle moment: after decades of programming code
for AI models, programmers are now using AI to write code. But what is the
difference between a human coder and an AI code generator?
A research assessing the functionality, complexity, and
security of the code generated by OpenAI's ChatGPT was published in the IEEE
Transactions on Software Engineering in June.
According to the results, ChatGPT's success rate in
producing functional code varies greatly, ranging from as low as 0.66 percent
to as high as 89 percent, depending on the task's complexity, the programming
language, and a variety of other factors.
The investigation also reveals significant security issues
with AI-generated code, even though in some circumstances the AI generator
could produce better code than humans.
Yutian Tang, a University of Glasgow lecturer, participated
in the research. He points out that there may be some benefits to using
AI-based code generation in terms of increasing productivity and automating
software development processes, but it's critical to comprehend the advantages
and disadvantages of these models.
“By undertaking a complete analysis, we will discover
cappotential troubles and obstacles that get up withinside the
ChatGPT-primarily based totally code era... [and] enhance era techniques,” Tang
explains.
To discover those obstacles in extra detail, his crew sought
to check GPT-3.5`s cappotential to deal with 728 coding troubles from the
LeetCode checking out platform in 5 programming languages: C, C++, Java,
JavaScript, and Python.
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)