One such enhancement is Graph RAG, which structures information into a knowledge graph to capture relationships between entities better. Graph RAG excels at handling more complex queries involving multiple documents, making it more powerful, computationally demanding, and expensive.
This article provides a detailed comparison of Classical RAG and Graph RAG, explaining how they work, their implementations, testing results, and cost considerations.
What is RAG?
Retrieval-augmented generation, or RAG, is a process that makes large language models LLMs' outputs more relevant in specific contexts. RAG improves LLMs by allowing them to refer to trusted external knowledge sources before generating a response. While LLMs are trained on massive datasets and use billions of parameters to perform tasks, their training data has limits.
RAG helps overcome those limits by connecting LLMs to specific domains or an organisation's internal knowledge base—without needing to retrain the model. This makes it a cost-effective solution that keeps responses accurate, relevant, and applicable across different use cases, particularly useful for knowledge intensive tasks in data engineering.
How RAG works
Traditional Retrieval-Augmented Generation (RAG) uses a four-step process called retrieve-then-generate approach to answer queries by combining retrieval with language generation.
- Document chunking: As part of the information retrieval component, large documents are split into smaller, overlapping text chunks—typically 1,200 tokens with a 100-token overlap. This keeps context intact across sections and improves retrieval accuracy.
- Embedding and indexing: Each chunk is transformed into a numerical vector using an embedding model. These vectors are stored in a vector database such as FAISS, allowing efficient similarity search.
- Retrieval: When a user submits a query, the system embeds it into the same vector space. It then retrieves the top-matching document chunks based on similarity to the query vector.
- Generation: The selected chunks are passed to a language model (e.g., GPT-4), which generates a natural language response using retrieved relevant information. Advanced natural language processing algorithms ensure the retrieval-augmented generation work is accurate and coherent, effectively retrieving relevant information.
This approach quickly and effectively retrieves direct answers from single documents or well-contained topics. However, it struggles when the response requires reasoning across multiple documents or understanding relationships between separate pieces of information.
What is Graph RAG?
Graph RAG builds on the standard RAG framework by introducing a knowledge graph to structure and connect information across documents. This enables better multi-document reasoning and deeper understanding. By organising data into a graph format, it becomes easier to draw connections, make inferences, and synthesise knowledge from diverse sources. This approach allows language models to handle more complex queries and provide more accurate responses.
How Graph RAG enhances RAG
Graph RAG enhances the traditional RAG approach with several key improvements, including:
- Entity and relationship extraction: Using tailored prompts with GPT-4, the system identifies key entities (e.g., people, events, terms) and the relationships between them. This step transforms flat text into structured knowledge.
- Graph construction: The extracted entities become nodes, and their relationships become edges in a graph. The result is a network of connected information, where ideas from different documents can logically link.
- Graph-based retrieval that occurs in two ways:
- Local search: Targets nodes directly related to the query, giving focused and accurate results.
- Global search: Pulls in connected nodes from different parts of the graph, enabling broader reasoning and synthesis across documents.
This graph structure allows Graph RAG to move beyond isolated chunks. It can combine relevant documents and facts from various sources, follow relationships, and produce responses that reflect a deeper understanding of the underlying knowledge.
Experiment at ELEKS: implementing Classical vs. Graph RAG
Both retrieval approaches—Classical RAG and Graph RAG—were implemented using GPT-4 as the final language model to ensure a consistent and meaningful comparison. The configuration was standardised across both methods, including the chunking strategy and dataset, to isolate the impact of the retrieval technique itself, demonstrating expertise in machine learning and data science.
- Classical RAG: It was implemented using the LangChain framework, with FAISS employed for embedding-based document retrieval. In this setup, user queries are matched against vectorised document chunks based on semantic similarity. Relevant sections are then retrieved and passed to GPT-4, generating responses grounded in the source material.
- Graph RAG: This approach followed Microsoft’s GraphRAG framework. It extends the classical method by extracting entities and their relationships from documents using GPT-4. These are structured into a knowledge graph, enabling the system to retrieve contextually connected information rather than isolated chunks—making it particularly effective for handling more complex or nuanced queries.
- Chunking strategy: Both methods used the same chunking setup, dividing the documents into 1200-token segments with a 100-token overlap.
- Dataset: The evaluation was conducted using publicly available information about the company from eleks.com.
Testing results: Graph RAG vs. Classical RAG performance
Our evaluation compared Graph RAG and Classical RAG across several question types to uncover where each approach excels — and where it doesn’t. To reflect realistic use cases, we tested the systems using targeted questions, list-based queries and multi-document questions. Check it out below!
1. Targeted questions specific to a particular section
- Performance: Both Classical RAG and Graph RAG (local search) provided correct answers.
- Classical RAG: Delivered concise responses containing the key information.
- Graph RAG (local search): Included additional context, often retrieving surrounding information that wasn’t strictly necessary but provided a broader understanding.
- Example: When asked about a particular clause, and using publicly available information about the company from eleks.com, Classical RAG returned the exact sentence, while Graph RAG provided the entire paragraph with related details.
Question: What can you tell me about the company ELEKS?
Graph Rag (global search) answer:
Overview of ELEKS
ELEKS is a multifaceted technology service provider that has carved a niche for itself in the realms of artificial intelligence (AI), generative AI software development, custom software development, product design, quality assurance (QA), and consultancy services. The company is renowned for its extensive AI consulting services, which include strategy development, implementation guidance, and post-deployment support, specifically catering to businesses that lack in-house AI expertise. ELEKS stands out for its ability to optimize complex creative processes through advanced AI techniques, with its generative AI software development services finding applications across various sectors such as e-commerce and media [Data: Entities (0)].
ELEKS' Core Services and Expertise
ELEKS offers a broad spectrum of services designed to meet the evolving needs of enterprises, SMEs, and technology challengers. The company's offerings span from AI consulting and generative AI software development to custom software development and digital transformations. This broad service range ensures that clients remain competitive in a rapidly changing digital landscape. ELEKS' commitment to solving complex challenges with software expertise is evident in its robust portfolio, which includes over 120 active client accounts, more than 2000 specialists, and the completion of over 1000 end-to-end projects [Data: Reports (0)].
Client Satisfaction and Innovation
The company boasts an impressive client satisfaction rate of over 97%, with 96% of clients ready to recommend ELEKS. This high level of client satisfaction underscores the effectiveness and impact of ELEKS' services, contributing to its reputation as a trusted partner in the technology services sector [Data: Entities (6)]. ELEKS specializes in the development and deployment of generative AI solutions, including conversational AI systems, natural language generation, speech synthesis, and text-to-image generative AI solutions. These innovative services find applications across various sectors, enhancing business operations, customer engagement, and creative processes [Data: Entities (10, 11, 12, 13)].
Sector-Specific Solutions and AI Consulting
ELEKS leverages its generative AI and machine learning expertise to offer customized solutions for the retail and finance sectors, focusing on dynamic pricing optimization, inventory management, personalized customer engagement, automated financial services, credit scoring, and fraud detection. These sector-specific solutions demonstrate ELEKS' ability to apply its technological capabilities to address the unique challenges and opportunities within different industries [Data: Entities (54, 55)]. Furthermore, ELEKS offers AI consulting services that include strategic guidance on implementing AI solutions for significant business benefits. The company's AI consulting process is a comprehensive three-stage approach that includes awareness, opportunity evaluation, and AI strategy development tailored to an organization's needs [Data: Entities (56, 59, 61, 63)].
Conclusion
In summary, ELEKS is a technology service provider that excels in delivering a wide range of services, with a strong focus on artificial intelligence and digital transformation solutions. The company's dedication to quality, innovation, and client satisfaction has established it as a key player in the technology services sector, capable of addressing the complex challenges and needs of its diverse clientele.
Classical RAG answer:
ELEKS is a technology partner for many of the world's leading enterprises, SMEs, and technology challengers. The company specializes in providing custom software development, product design, QA, and consultancy services. They help businesses elevate their value through these services.
The company has more than 2,000 specialists and over 120 active client accounts. It has completed over 1,000 end-to-end projects. ELEKS offers assistance in building perfect development teams, adopting digital transformations, and unifying an enterprise's digital ecosystem.
ELEKS is known for its passion for engineering, performance focus, and dedication to success. The company prides itself on its ability to explore new technologies to design best-fit solutions, which has led 96% of their clients to be ready to recommend them. They have a client satisfaction rate of over 97%.
They also offer nearshore software development teams, providing their clients access to a talent pool of multiskilled specialists. The company has 20 offices distributed in key cities over three continents.
In addition, the company offers innovative AI solutions, including conversational AI systems, Natural Language Generation (NLG) tools, and speech synthesis and signal processing.
2. List-based queries spread across the document
- Graph RAG (global search): Successfully retrieved all items from a list distributed throughout multiple sections.
- Classical RAG: Often returned only a portion of the list because it relied on retrieving the most relevant chunks individually, rather than synthesising content from multiple locations.
- Example: When asked to extract all the responsibilities listed on eleks.com, Graph RAG (global search) returned the full set, whereas Classical RAG retrieved only the most prominent ones from the most relevant chunk.
Question: Across all sectors, what types of generative AI models and techniques does ELEKS use in its solutions?
Graph Rag (global search) answer:
ELEKS, a prominent player in the field of generative AI, employs a diverse range of models and techniques to cater to various sectors, enhancing business operations, customer engagement, and creative processes. Their expertise spans the development and deployment of cutting-edge generative AI solutions, including conversational AI systems, natural language generation, speech synthesis, and text-to-image generative AI solutions. These technologies are not only pivotal in transforming how businesses interact with their customers but also play a crucial role in streamlining creative endeavors [Data: Reports (0)].
In addition to these innovative services, ELEKS has tailored its generative AI and machine learning capabilities to offer specialized solutions for the retail and finance sectors. For retail, the focus is on dynamic pricing optimization and inventory management, which are essential for maintaining competitive advantage and operational efficiency. Personalized customer engagement strategies are also developed using these technologies, ensuring that businesses can connect with their customers in more meaningful and impactful ways. In the finance sector, ELEKS leverages its expertise to automate financial services, enhance credit scoring systems, and bolster fraud detection mechanisms. These applications not only improve the efficiency and reliability of financial operations but also significantly reduce the risk of fraudulent activities, safeguarding both the businesses and their clients [Data: Reports (0)].
Furthermore, ELEKS's generative AI capabilities are integrated into a broader spectrum of services that include AI consulting and strategy development. This comprehensive approach ensures that enterprises, SMEs, and technology challengers can leverage AI and digital transformation solutions that are aligned with their evolving needs. By offering a wide range of AI solutions, ELEKS positions itself as a versatile partner capable of addressing the diverse challenges faced by businesses in today's rapidly changing digital landscape [Data: Reports (0)].
In summary, ELEKS's deployment of generative AI models and techniques across various sectors underscores its commitment to innovation and excellence. By harnessing the power of conversational AI, natural language generation, speech synthesis, and text-to-image generative AI, along with sector-specific solutions for retail and finance, ELEKS is at the forefront of driving digital transformation and enhancing business efficiency and customer engagement across the board.
Classical RAG answer:
The text does not provide specific information about the types of generative AI models and techniques ELEKS uses in its solutions across all sectors.
3. Multi-document queries requiring knowledge synthesis across documents
- Graph RAG (global search): Effectively combined insights from multiple documents, providing a cohesive answer.
- Classical RAG: Struggled when the required information was spread across different chunks in separate documents. If the information fit within the retrieved chunks, it worked well, but otherwise, it lacked the ability to synthesise knowledge across multiple sources.
- Example: When asked how different topics interacted with each other, Graph RAG (global search) connected related sections from different documents, while Classical RAG either provided an incomplete answer or required multiple separate queries.
Question: What are the main benefits of using generative AI in both the finance and manufacturing sectors, and how does ELEKS help organizations implement these benefits?
Graph RAG (global search) answer:
Generative AI in Finance and Manufacturing
Generative AI is revolutionizing various sectors, notably finance and manufacturing, by offering innovative solutions that enhance efficiency, security, and product development.
Finance Sector Benefits
In the finance sector, generative AI plays a pivotal role in automating financial services. This includes advanced credit scoring systems and fraud detection mechanisms. By leveraging generative AI, financial institutions can significantly enhance their operational efficiency and security measures. This technological advancement allows for more accurate risk assessments and quicker identification of fraudulent activities, ensuring a safer financial environment for both the institutions and their clients [Data: Reports (0)].
Manufacturing Sector Benefits
The manufacturing sector benefits from generative AI through its application in generative design and process automation. Generative design utilizes AI to explore all possible permutations of a solution, efficiently generating design alternatives and optimizing products for performance, cost, and material usage. This capability not only accelerates the product development process but also introduces a level of innovation previously unattainable. Additionally, process automation facilitated by generative AI streamlines production processes, reducing manual labor, minimizing errors, and increasing overall productivity [Data: Reports (2)].
ELEKS' Role in Implementing Generative AI
ELEKS, a provider of AI consulting services, plays a crucial role in helping organizations harness the benefits of generative AI. The company offers strategic guidance on implementing AI solutions, ensuring that organizations can effectively integrate AI technology to maximize value across their operations. ELEKS specializes in the development and deployment of generative AI solutions, demonstrating a commitment to leveraging cutting-edge technology. This approach provides tailored solutions that address the specific needs of its clients, ensuring they can capitalize on the advantages of generative AI [Data: Reports (0)].
Moreover, ELEKS' comprehensive AI and digital services, including custom software development and digital transformations, are designed to meet the evolving needs of enterprises. This ensures that organizations remain competitive in a rapidly changing digital landscape. By offering a suite of services that span from strategic planning to implementation and beyond, ELEKS positions itself as a key partner for businesses looking to innovate and improve their operations through generative AI [Data: Reports (0)].
In summary, the integration of generative AI into the finance and manufacturing sectors offers substantial benefits, including enhanced efficiency, security, and innovation. ELEKS facilitates this integration by providing expert consulting, development, and deployment services, ensuring organizations can fully leverage the potential of generative AI to stay ahead in their respective industries.
Classical RAG answer:
The text doesn't provide specific information on the benefits of using generative AI in both the finance and manufacturing sectors, or how ELEKS helps organizations in these specific sectors implement these benefits.
Although it is possible to run Classical RAG using a local, offline language model, the same cannot be said for Graph RAG—at least not effectively. Several key challenges in the tests conducted by ELEKS made it difficult or even impossible to apply Graph RAG locally using current open-source models.
One of the main steps in Graph RAG is turning unstructured generated text into structured knowledge without relying on costly fine tuning of local models. This means identifying key entities (like names, places, or terms) and the relationships between them. Most local models struggled with this task. Even when using advanced prompting techniques, the results were poor. For example, Gemma 2 could create a graph but still performed poorly when answering follow-up questions, showing that accurate understanding was missing.
Creating a helpful knowledge graph depends on the model's ability to correctly recognise and link related information. Unfortunately, local models often don't have the level of understanding needed to build these relationships properly. As a result, the graphs they produced were either incomplete or empty, making them unusable for deeper reasoning or search.
Running Graph RAG also places much heavier demands on computing resources. It requires a high-quality language model with extended context support to process multiple documents simultaneously. Most local models currently available don't have the memory or processing capabilities to handle this, especially without access to powerful hardware.
Even if a local model manages to build a graph, its ability to search it and reason over the results is still far behind that of advanced, cloud-based models like GPT-4. This leads to weaker retrieval accuracy and poorer answers, limiting the practical use of Graph RAG in an offline environment.
Conclusion
The experiment run by ELEKS shows that both Classical RAG and Graph RAG have their own strengths, and each works best for different types of tasks. Classical RAG is a great option when you need quick, direct answers from a document. It's faster, cheaper, and works well when the information you seek is in a specific place. Because it uses fewer resources, it’s also a better fit for projects where cost matters.
Graph RAG, on the other hand, is more powerful when you’re dealing with complex questions. It can connect information across many documents and give deeper, more complete answers. However, this extra capability comes with a higher cost, as it needs more processing power.
The right choice depends on what your project needs. If you want simple answers pulled from a single source, Classical RAG is usually enough. But if your questions require combining ideas from different places, Graph RAG is the better choice—even if it’s more expensive. In cases where you need to work offline or have limited computing resources, Classical RAG is usually the only practical option, especially when combined with a carefully selected local language model. For teams and businesses working with AI development, it’s crucial to weigh the trade-offs between cost, accuracy, and depth of understanding.
Related Insights
