As the world of software development continues to evolve, the role of AI has become increasingly important. With the benefits like the ability to reduce the time and effort required for coding and more, it's no wonder AI in software development is a hot topic nowadays.
Although AI hasn't reached a stage where it can independently create, test, and launch software products without human input, compared to the past, it has greatly speeded up the overall turnaround time.
Many AI-driven tools are available that can assist in developing custom applications by generating and autocompleting code, as well as detecting and fixing bugs. The ELEKS Research & Development team thoroughly assessed one such tool - GitHub Copilot - to evaluate its influence on developers' tasks, completion duration, and quality standards for recommendations provided. Let's take a closer look.
GitHub Copilot is a tool for code autocompletion and generation. It is built based on OpenAI Codex and trained on public GitHub repositories. As a result, it suggests code for any programming language. However, the quality of proposals depends on the number of public repositories that are based on a specific programming language and framework.
GitHub Copilot works with program code using Integrated Development Environment (IDE) plugins. Only the following IDEs are currently supported:
This investigation aimed to explore the impact of using Copilot on both development speed and quality.
To conduct this study, a small pet project was initiated as a testing ground for various assumptions. With a team comprising four skilled middle developers armed with expertise in React, Redux, TypeScript, Jest, Vite, PHP, Symfony, and Codeception, we started the investigation and defined our goals and approaches as follows.
|Evaluate the dependency between the developer’s competency and performance when using Copilot
|We selected the Symfony framework for back-end development. The back-end team had no substantial experience with this framework.
To assess the quality of the back-end solution, a Symfony expert was involved in a code review. The expert also developed a basic project structure that went beyond the typical one and was used in a large commercial project.
|Investigate the impact on typical developer tasks
|Before the project started, we created a WBS that covered the typical work of a developer on a commercial project. Also, we conducted a task estimation session.
|Investigate the impact on programming languages and frameworks by popularity
|We developed a web application with test coverage.
|Evaluate the impact on task completion time
|We made a preliminary estimation of individual tasks on the project.
|Enhance the research effectiveness
|We conducted knowledge-sharing sessions within the team during daily syncs.
|Explore the quality of suggestions
|We tested how Copilot deals with a non-classical and more complex project structure.
ELEKS' investigation of GitHub Copilot yielded several key findings that shed light on the tool's capabilities and benefits. Here is a more detailed description:
Please note that the investigation outcomes are based on the team's subjective feedback, observations of their work, and code reviews of their solutions.
On a side note, during the project, the team encountered several bugs in GitHub Copilot. For example, Copilot proposed code suggestions based on the code from another project open in another IDE window. It offered syntactically incorrect code. Copilot conflicts with the features of some plugins (built-in or added by third-party) that also have a code auto-completion functionality.
GitHub Copilot is a powerful tool that positively impacts development speed and developer job satisfaction. Like any other development tool, it requires time to master it.
At this point, an individual monthly license costs only $10, while the business plan is priced at $19 per month. GitHub Copilot will undoubtedly be financially beneficial for use on the projects, especially for the specialists of middle competency level and higher.
The team predicts that with the growth of experience using GitHub Copilot, the development speed can increase by 5–25%, depending on the technology's popularity and the developer's competency level. GitHub Copilot is guaranteed not to slow down development because developers will gradually gain the expertise and understanding of when it is beneficial to use the tool and when not.
In conclusion, the impact of AI in software development, exemplified by tools like GitHub Copilot, is undeniable. Overall, such tools as GitHub Copilot cannot replace a developer but serve as valuable tool to enhance developers' comfort and productivity, contributing to the evolution of software development in an increasingly AI-driven world.
The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.
Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.
ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.