Contact Us
ELEKS' GitHub Copilot Investigation – Exploring the Potential of AI in Software Development

ELEKS’ GitHub Copilot Investigation – Exploring the Potential of AI in Software Development

AI is revolutionising software development, making it more efficient, productive, and innovative. In this article, we explore the impact of AI in software development, delving into the insights gained from our investigation of GitHub Copilot performed by the ELEKS R&D team.

As the world of software development continues to evolve, the role of AI has become increasingly important. With the benefits like the ability to reduce the time and effort required for coding and more, it's no wonder AI in software development is a hot topic nowadays.

of developers based in the U.S. are already using AI coding tools both at work and outside of it.
of developers believe that AI coding tools will give them a distinct edge in their professional endeavours, including improved code quality, reduced completion time, and enhanced incident resolution capabilities.
4 out of 5
developers expect AI coding tools to make their team more collaborative.

Although AI hasn't reached a stage where it can independently create, test, and launch software products without human input, compared to the past, it has greatly speeded up the overall turnaround time.

Many AI-driven tools are available that can assist in developing custom applications by generating and autocompleting code, as well as detecting and fixing bugs. The ELEKS Research & Development team thoroughly assessed one such tool - GitHub Copilot - to evaluate its influence on developers' tasks, completion duration, and quality standards for recommendations provided. Let's take a closer look.

What is GitHub Copilot?

GitHub Copilot is a tool for code autocompletion and generation. It is built based on OpenAI Codex and trained on public GitHub repositories. As a result, it suggests code for any programming language. However, the quality of proposals depends on the number of public repositories that are based on a specific programming language and framework.

GitHub Copilot works with program code using Integrated Development Environment (IDE) plugins. Only the following IDEs are currently supported:

  • Visual Studio Code
  • Visual Studio
  • Neovim
  • all IDEs from JetBrains

Overview of ELEKS' GitHub Copilot investigation

This investigation aimed to explore the impact of using Copilot on both development speed and quality.

To conduct this study, a small pet project was initiated as a testing ground for various assumptions. With a team comprising four skilled middle developers armed with expertise in React, Redux, TypeScript, Jest, Vite, PHP, Symfony, and Codeception, we started the investigation and defined our goals and approaches as follows.

Goal Approach
Evaluate the dependency between the developer’s competency and performance when using Copilot We selected the Symfony framework for back-end development. The back-end team had no substantial experience with this framework.
To assess the quality of the back-end solution, a Symfony expert was involved in a code review. The expert also developed a basic project structure that went beyond the typical one and was used in a large commercial project.
Investigate the impact on typical developer tasks Before the project started, we created a WBS that covered the typical work of a developer on a commercial project. Also, we conducted a task estimation session.
Investigate the impact on programming languages and frameworks by popularity We developed a web application with test coverage.
Evaluate the impact on task completion time We made a preliminary estimation of individual tasks on the project.
Enhance the research effectiveness We conducted knowledge-sharing sessions within the team during daily syncs.
Explore the quality of suggestions We tested how Copilot deals with a non-classical and more complex project structure.

Key findings from ELEKS' GitHub Copilot investigation

ELEKS' investigation of GitHub Copilot yielded several key findings that shed light on the tool's capabilities and benefits. Here is a more detailed description:

Please note that the investigation outcomes are based on the team's subjective feedback, observations of their work, and code reviews of their solutions.

The GitHub Copilot has two modes of operation:

  1. Code auto-completion mode – almost always positively affects performance.
  2. Code generation mode based on a comment with a prompt – works well only for typical code structures. It can be counterproductive when generating business logic or non-typical solutions because analysing the proposed code variants takes a lot of time.

The quality of GitHub Copilot code suggestions depends on the following:

  • Popularity of the selected tech stack in public repositories. The more popular the technology or language, the higher the quality of GitHub Copilot suggestions.
  • Code structure and application architecture. The more typical your solution is, the more effective suggestions GitHub Copilot provides. This is especially noticeable at the start of the project development.

Other advantages of GitHub Copilot:

  • Over time, the quality of code suggestions increases. GitHub Copilot learns and adapts to the project structure, code style, and typical approaches to technical solutions on the project.
  • Positively influences developer job satisfaction. GitHub Copilot takes on a portion of routine, typical, and boring tasks. The ratio between developing interesting and routine tasks changes, which increases developers’ comfort.

General team findings:

  • The tool showed no impact on the code quality—neither positive, nor negative. We think that for now, it cannot replace the code review process, manual code testing done by the developer, running autotests, security testing, and so on.
  • The effectiveness depends on the developer's competence in the given tech stack. The better the developer's skills, the more effective this tool proves to be.
  • The value of GitHub Copilot for learning new technologies is very doubtful.
  • The team formed an impression that code suggestions become better if all code-related files are open in the IDE.
  • Like any other tool, GitHub Copilot requires certain skills. The more the team practiced, the more positive feedback was received.
  • The feedback regarding increase in performance varies depending on the tech stack, specialist’s competency level, and subjective reasons.
the average increase in productivity of team members.
ELEKS R&D team

On a side note, during the project, the team encountered several bugs in GitHub Copilot. For example, Copilot proposed code suggestions based on the code from another project open in another IDE window. It offered syntactically incorrect code. Copilot conflicts with the features of some plugins (built-in or added by third-party) that also have a code auto-completion functionality.


GitHub Copilot is a powerful tool that positively impacts development speed and developer job satisfaction. Like any other development tool, it requires time to master it.

At this point, an individual monthly license costs only $10, while the business plan is priced at $19 per month. GitHub Copilot will undoubtedly be financially beneficial for use on the projects, especially for the specialists of middle competency level and higher.

The team predicts that with the growth of experience using GitHub Copilot, the development speed can increase by 5–25%, depending on the technology's popularity and the developer's competency level. GitHub Copilot is guaranteed not to slow down development because developers will gradually gain the expertise and understanding of when it is beneficial to use the tool and when not.

In conclusion, the impact of AI in software development, exemplified by tools like GitHub Copilot, is undeniable. Overall, such tools as GitHub Copilot cannot replace a developer but serve as valuable tool to enhance developers' comfort and productivity, contributing to the evolution of software development in an increasingly AI-driven world.

Unlock the path to an efficient software development process and successful outcomes with ELEKS!
Contact an expert
Application development
We’ll help you bring your most complex software vision to life with our leading full-cycle custom application development service. So you can focus on delivering an incredible user experience that sets you apart from the competition.
View service
Data science
Deep-dive into your data and boost business performance by understanding what your users really want.
View expertise
Have a question?
Speak to an expert
app development
Explore our application development services
Contact Us
  • We need your name to know how to address you
  • We need your phone number to reach you with response to your request
  • We need your country of business to know from what office to contact you
  • We need your company name to know your background and how we can use our experience to help you
  • Accepted file types: jpg, gif, png, pdf, doc, docx, xls, xlsx, ppt, pptx, Max. file size: 10 MB.
(jpg, gif, png, pdf, doc, docx, xls, xlsx, ppt, pptx, PNG)

We will add your info to our CRM for contacting you regarding your request. For more info please consult our privacy policy
  • This field is for validation purposes and should be left unchanged.

The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.

sam fleming
Sam Fleming
President, Fleming-AOD

Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.

Caroline Aumeran
Caroline Aumeran
Head of Product Development, appygas

ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.

Samer Awajan
Samer Awajan
CTO, Aramex