A “new milestone in competitive programming” was trumpeted by Google’s UK AI subsidiary DeepMind earlier this month when it unveiled AlphaCode, a system it claims can write programs full-fledged computers that compare favorably to the work of humans. Software development has long been identified as an area where AI can have a significant impact, and with advances in AlphaCode and other systems, is the prospect of machines replacing human coders realistic?
DeepMind says AlphaCode is able to understand a problem and then write a program that solves that problem. He says he tested the system against people who entered coding contests and found that his results rank among the top 54% of human participants.
“Solving competitive programming problems is a very difficult thing to do, requiring both good coding skills and creative problem-solving in humans,” said Petr Mitrichev, software engineer at Google, who is involved in coding contests. “I was very impressed that AlphaCode could make progress in this area and excited to see how the model uses its understanding of instructions to produce code and guides its random exploration to create solutions.”
With AlphaCode showing such impressive progress, Mitrichev and his colleagues could be forgiven for looking over their shoulders.
Why is DeepMind AlphaCode important?
DeepMind used a combination of unsupervised and supervised machine learning to train the AI models that underpin AlphaCode. The system first absorbed 715 gigabytes of data from the GitHub coding repository, before the DeepMind team tweaked it using a database they had created from problem statements, of test cases and submissions – correct and incorrect – from coding contests.
When faced with a new problem, AlphaCode analyzes the information it has, comes up with thousands or, in some cases, millions of solutions, then filters them down to the top ten that it submits to the contest. If one of them turns out to be correct, the team considers it a success.
This process sets AlphaCode apart from other AI coding tools, such as the OpenAI Codex, able to generate specific code snippets on command. “This is a breakthrough in the sense that they solve much more complex programming problems than previous systems,” says Professor Ernest Davis of New York University’s computer science department.
But, according to Professor Davis, it is not without its limits. Since his method involves finding so many potential solutions to a problem (before filtering them to find the best ones), the amount of training data needed to write more complex programs is likely to be enormous. “It is reasonable to expect that the number of samples required will increase exponentially with program duration,” says Professor Davis. It also depends on the relevance of its training data to come up with accurate solutions, which a human programmer could circumvent even if he had no knowledge of the subject.
AlphaCode is a useful contribution to AI research, but coding AI at the level of a trained human is still light years away.
Dzmitry Bahdanau, ServiceNow
And compared to DeepMind’s other AI achievements, AlphaCode is a relatively minor success, says Dzmitry Bahdanau, a research scientist at workflow software provider ServiceNow, who has studied how AlphaCode works. “Right now, AlphaCode is a useful contribution to AI research, but coding AI at the level of a trained human is still light years away,” he says. “It’s not Alpha Go in terms of beating humans, not Alpha folding in terms of revolutionizing an entire field of science.
Will AI replace human programmers?
While an AI such as AlphaCode could overcome these limitations, the conditions under which it has been tested are very different from the world of professional software development, says Dr Peter Schrammel, senior lecturer in computer science at the University of Sussex. and CTO and co-founder of software company Diffblue. “The coding challenges are relatively specific because the assignments need to be as unambiguous as possible in order to be able to compare the solutions fairly,” he explains. “The description is fairly well structured and the examples are given coherently.”
Real-world development is “a different beast,” he argues. “The requirements are vague and determined by incremental iterations,” he says. “Software developers negotiate requirements in conversations with stakeholders, implement a first release, then demo it and discuss with stakeholders what to improve for the next iteration.” Whereas in a coding challenge “one knows perfectly well what the requirements are,” says Dr. Schrammel, “in real-world software development, extracting the requirements from the heads of the stakeholders is the main challenge.”
Bahdanau agrees and says he doesn’t believe many people in the industry “seriously take [AI replacing human programmers] as a goal”. “Humans understand the larger context, the true real-world purpose for which code should be written,” he says. “AlphaCode, Codex and others are just very advanced statistical methods that contain billions of gigabytes of distilled code. That said, AI has the potential to help human software developers get their jobs done faster. »
How will AI help software development?
But with the continued shortage of tech skills threatening to slow the pace of digital transformation, AI has an important role to play in software development, helping increasingly stretched IT teams. Technical monitor reported on the growing popularity of low-code and no-code systems, which allow non-technical personnel to create digital functions to facilitate their work, with no-code having to be taken into account 65% app development by 2024, according to Gartner.
For professionals, tools such as GitHub CoPilot, which suggests lines of code or functions to aid development, are likely to be the most productive use of AI in the short to medium term, says Dr. Schrammel . “Software development itself has benefited greatly from automation since its inception,” he says. “Nobody creates machine code manually anymore. Automated tools such as compilers and code generators turn the high-level formal specifications of an application into machine-executable code.”
Software development itself has benefited greatly from automation since its inception.
Dr Peter Schrammel, University of Sussex
The rise of these compilers has helped automate “tedious tasks”, but has not led to a decrease in the number of human developers, adds Dr. Schrammel. “One reason could be that many software development activities such as requirements engineering rely on a wide range of domain knowledge and have to deal with large degrees of ambiguity,” he says. “These activities are much more difficult to automate than more regular activities, such as coding from well-defined specifications and tests.”
Indeed, further automation of test systems is likely to be the next frontier of AI in programming, says Dr. Schrammel. As automation increases, Bahdanau believes an increasingly important role for human developers will be to ensure that how AI generates code is understood. “Professional developers will need to understand and verify the proposed AI algorithm,” he says. “For citizen developers using low-code or no-code programming interfaces, the reliability bar for AI will be even higher, and they will need developer experience that explains the logic offered by AI. .”
Matthew Gooding is editor for Technical monitor.