Article

GitHub Copilot: A 55% Speed Boost in Development – Myth or Reality?

GitHub Copilot has become a topic of significant interest in the developer community due to its potential to accelerate development processes. This article presents a GitHub Copilot review and assesses its impact on enhancing development speed in real-life scenarios.

Since its launch, Copilot has sparked various discussions, ranging from enthusiasm and optimism to scepticism and caution about its potential impact on programming and the future role of developers. Its influence on different aspects of software development - from speed to code quality and learning - continues to be a subject of analysis and discussion.

What is GitHub Copilot?

GitHub Copilot is an AI-powered code completion tool developed by GitHub and OpenAI. It functions as your programming assistant, using machine learning to suggest code snippets and entire functions based on your data, the context of your codebase and natural language comments.

Copilot provides real-time coding recommendations by analysing code patterns, comments, and context, much like an intelligent autocomplete. It can help with simple code completions and generate complex functions, test cases, and documentation.

GitHub Copilot offers support for numerous programming languages, including:

Python, JavaScript, and TypeScript
Java, Go, and Ruby
C++, C#, and C
PHP, Rust, and Scala
Swift and Kotlin
SQL and Shell scripts
HTML/CSS and Vue
React and Angular frameworks

Copilot integrates with several development environments:

Visual Studio Code (most popular integration);
Visual Studio IDE;
JetBrains IDEs (including IntelliJ IDEA, PyCharm, WebStorm);
Neovim;
GitHub.com's web editor.

Moreover, GitHub Copilot keeps track of context throughout your project, including the libraries you install and any external dependencies. As a result, Copilot becomes progressively tailored as it learns your coding preferences and style. It can also understand code comments written in natural language, allowing you to describe what you want to achieve, and it will suggest appropriate implementations.

How to start with GitHub Copilot

To install and begin using Copilot, one needs a GitHub account and an active GitHub Copilot subscription. GitHub offers different subscription plans tailored for individuals, businesses, and enterprises, each with varying features.

Exploring GitHub Copilot in action

GitHub Copilot users demonstrated an acceleration in task completion, achieving a 55% faster rate than developers who did not use the tool.

GitHub

Intrigued by the bold claims regarding the speed boost attributed to GitHub Copilot, we embarked on a journey to verify its effectiveness, particularly in the realm of AI in software development. In our pursuit of truth, we conducted our own testing of Copilot's use on real projects. To ensure optimal results, we took the following approach:

Varied project selection: We deliberately chose several projects with different tech stacks and architectural approaches, aiming to cover a wide range of use cases.
Diverse developer expertise: We enlisted developers with different levels of experience and competence to test the tool.

The key objective for the team was to conduct a GitHub Copilot review and assess its impact on coding productivity, identify its key influences, and find the most effective ways of using it. The testing period lasted three months to mitigate the potential bias influenced by a learning curve. Let’s dive into the outcomes.

GitHub Copilot review: How it impacts development speed

ELEKS team has conducted an in-depth investigation into GitHub Copilot, aiming to assess its impact on developers' tasks, completion duration, and the quality standards of the recommendations and suggestions it provides. The findings of this investigation can be reviewed here: ELEKS’ GitHub Copilot Investigation – Exploring the Potential of AI in Software Development.

This study's key focus was to explore how the use of Copilot affects different types of projects. We tested and analysed the effectiveness of GitHub Copilot in monolithic applications and microservices in both backend and frontend applications to understand where this tool is most effective.

In broad terms, we can assert that the impact of Copilot on development speed is highly variable and depends on many factors. The following are key dependencies that emerge regarding the effective utilisation of Copilot:

1. Size of the existing codebase

Depending on the type of project and code structure, Copilot's impact on development speed varies: in frontend monolithic applications, we got approximately 20-25% development speed improvement; in backend monolithic applications - about 10-15% improvement, and in backend microservices - an average of 5-7%.

Ihor Mysak

Tech Lead at ELEKS

The verdict? GutHub Copilot thrives in projects with a large codebase, where it can support developers with existing templates and solutions. However, its prowess dwindles in the microservice realm, characterised by a small codebase. It indicates Copilot's ineffectiveness in projects that are just starting and do not yet contain enough developed solutions.

2. Technological stack

Testing Copilot on projects with different tech stacks showed a significant dependence on the quality of Copilot's suggestions based on the popularity of the technology.

React applications reveal a significant productivity surge, overshadowing the now outdated and less popular Zend framework.
.Net projects find themselves in the middle ground; performance was observed to be intermediate, better than with Zend but not as high as with React, suggesting a correlation with the relative popularity and volume of .Net materials available.

We believe this is because Copilot was trained on public GitHub repositories and had more training material for technologies that were more popular among developers.

3. Quality of code in the existing codebase

Copilot tends to generate higher-quality suggestions with proper and logical naming of variables and methods. This leads us to believe that quality naming helps GitHub Copilot better understand the context of the code and provides more accurate and useful suggestions.

Meanwhile, when the naming of variables and methods is unclear or ambiguous, Copilot has less information to analyse, which decreases the quality of its contribution to the development process. Thus, high-quality naming in code not only simplifies the work of programmers but also enhances the effectiveness of artificial intelligence tools.

4. Type of tasks performed by the developer

Despite its effectiveness in certain aspects of development, we also found that Copilot has limitations, especially when writing complex code that implements new business logic.

Copilot writes only the code according to the prompt, not complete solutions. Copilot is most effectively used for clear and template tasks. The time spent on a detailed description of business logic can outweigh the time needed to implement this business logic without using Copilot.

Ihor Mysak

Tech Lead at ELEKS

While Copilot gives better suggestions for the templated tasks, it struggles with the intricacies of new ideas or creative programming. The message is clear: GitHub Copilot is a developer's trusted companion in routine, but the realm of innovation demands the touch of human creativity.

Key tips regarding the effective use of Copilot:

Precision in the key: The more precise and detailed the prompt, the higher the likelihood of receiving a quality proposal from Copilot.
Context is everything: Avoid confusion by closing unrelated projects while using Copilot. If multiple projects are open in the IDE, Copilot can confuse contexts and generate suggestions for Project A based on the code of Project B.
Comments matter: Adding comments before creating a class or method enhances autocompletion quality.
File focus: Copilot is sensitive to the open tabs with project files, so one can artificially create a more targeted context for it.

Unleashing GitHub Copilot's potential: adaptability and indirect impact

Adaptability to project-specific environments

One of the most interesting characteristics of GitHub Copilot is its ability to adapt to the specifics of a particular project. Over time, Copilot "learns" the coding style and specific features of the project, leading to an improvement in the quality of the code generated by its suggestions.

Initially, Copilot may provide generic or less precise solutions. However, as the tool accumulates more exposure and interaction within the project, the accuracy and relevance of its suggestions significantly improve. This is especially noticeable in projects with an established coding style and a large amount of existing code for GitHub Copilot to "train" on. This adaptability makes Copilot not only a tool for increasing efficiency but also a powerful aid in maintaining code consistency within a project.

Developers have also highlighted Copilot's positive influence on code complexity, noting a shift towards more readable and maintainable solutions, especially among those accustomed to crafting convoluted and intricate code structures.

Elevating automated testing

GitHub Copilot doesn't stop at coding; it has also mastered the art of automated testing. The tool offers templates and recommendations for potential test scenarios, allowing developers to save time and resources.

Copilot's ability to generate unique test cases that may not be obvious to developers is particularly valuable. It expands the testing coverage, improving the software product's examination depth.

Interestingly, the quality of tests created with GitHub Copilot is directly related to the quality and structure of the tested code. Our developers noted that the clarity of variable names, methods, and the overall structure of the code significantly affect the quality and accuracy of Copilot's test generation. Therefore, the effectiveness of utilising Copilot for writing unit tests depends on the tool itself and the quality of the tested code.

Overall, Copilot has proven to be a useful tool in the process of writing Unit Tests, enhancing not only the speed but also the quality of the final product.

Indirect impact

GitHub Copilot increases the coding speed and improves the overall nature of a developer's work. According to developers' feedback, Copilot allows them to shift focus from routine, time-consuming work to more creative and challenging tasks.

Additionally, Copilot can be an effective alternative to searching the Internet or documentation, reducing the time spent switching between different windows and allowing developers to concentrate on their current tasks. This feature is handy when needing to quickly find answers to questions without being distracted from the main work.

Copilot positively impacts the comfort and satisfaction of a developer. It streamlines getting answers to different questions and helps when there is no opportunity to turn to senior colleagues or find a solution on the Internet.

Olena Hladych

QA Lead at ELEKS

Interestingly, we found a correlation between the soft skills of developers and their satisfaction with using Copilot: developers with less developed communication skills often are less satisfied with its performance, possibly due to difficulties in precisely formulating prompts.

Conclusions

GitHub Copilot is a powerful tool that substantially enhances development productivity in specific scenarios, particularly during unit test composition and when navigating extensive codebases built on popular technologies. However, its efficacy faces constraints in tasks demanding innovative approaches or the creation of novel concepts.

Contrary to the claim suggesting a 55% boost in productivity, the actual outcome fell short. On average, teams experienced a moderate 10-15% improvement in productivity related to generating new code. However, it's essential to highlight various advantages attributed to Copilot utilisation. Overall, developers appraise Copilot as an invaluable tool that contributes significantly to development speed and fostering satisfaction among developers.

We recommend that teams and developers consider GitHub Copilot and approach it with an understanding of its potential limitations. The key to effectively using Copilot lies in understanding that it is an auxiliary tool, not a replacement for human intellect and creativity. It can enhance productivity and job satisfaction, reduce the time spent on routine aspects of development, and allow developers to focus on more complex and creative tasks.

Deliver your software project with top development experts at ELEKS.

Contact an expert

AI development

Partner with ELEKS to implement AI-powered strategies that drive breakthrough performance.

View service

Data science

Deep-dive into your data and boost business performance by understanding what your users really want.

View expertise

FAQs

What are the disadvantages of GitHub Copilot?

GitHub Copilot has limitations in generating innovative ideas and creatively approaching programming tasks. Its effectiveness diminishes when dealing with smaller codebases, which highlights Copilot's inefficiency in projects at their initial stages, lacking a sufficient number of established solutions.

Additionally, the tool may not consistently deliver accurate results, particularly when employed with less commonly used programming languages.

Can GitHub Copilot review code?

Is ChatGPT better than GitHub Copilot?

Will GitHub Copilot replace coders?

Can GitHub Copilot write unit tests?

Does GitHub Copilot work offline?

Have a question?

Speak to an expert

Explore AI development services

Full name*
We need your name to know how to address you

Email*
We need your email to respond to your request

Phone number*
We need your phone number to reach you with response to your request

Country*
We need your country of business to know from what office to contact you

Company*
We need your company name to know your background and how we can use our experience to help you

Message*

Attach file
Accepted file types: jpg, gif, png, pdf, doc, docx, xls, xlsx, ppt, pptx, Max. file size: 10 MB.

Add an attachment

(jpg, gif, png, pdf, doc, docx, xls, xlsx, ppt, pptx, PNG)

- I want to receive news and updates once in a while

We will add your info to our CRM for contacting you regarding your request. For more info please consult our privacy policy

Email
This field is for validation purposes and should be left unchanged.

What our customers say

The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.

Sam Fleming

President, Fleming-AOD

Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.

Caroline Aumeran

Head of Product Development, appygas

ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.

Samer Awajan

CTO, Aramex