Abstract
Retrieval-Augmented Generation (RAG) has become the basis in modern question answering (QA) systems, combining Large Language Models (LLMs) with external document retrieval. However, traditional RAG architectures often struggle with retrieving semantically structured or context-rich content, which can impact accuracy and relevance. This paper presents a comparative evaluation of a standard RAG model and a GraphRAG model. The GraphRAG approach combines graph-based document representation within the retrieval pipeline to assess their efficiency on a structured question answering task. This research leverages two modern Ollama-hosted models, Phi-4 and LLaMA 3.2 3B, as the base language models for both retrieval pipelines. Using a custom dataset derived from German coalition policy documents, this research evaluates performance through both lexical and semantic metrics. The results demonstrate that GraphRAG consistently outperforms traditional RAG in semantic alignment and contextual accuracy, particularly when paired with Phi-4. These findings aim to contribute to the growing body of work on hybrid retrieval strategies and support the case for graph-enhanced architectures in long-form QA systems, which are central to advancing structured and knowledge-aware retrieval methods in complex information domains.
| Original language | English |
|---|---|
| Pages | 73-80 |
| Number of pages | 8 |
| Publication status | Published online - 19 Apr 2026 |
| Event | The Eighteenth International Conference on Advanced Cognitive Technologies and Applications - Portugal, Lisbon, Portugal Duration: 19 Apr 2026 → 23 Apr 2026 Conference number: 18 https://www.iaria.org/conferences2026/COGNITIVE26.html |
Conference
| Conference | The Eighteenth International Conference on Advanced Cognitive Technologies and Applications |
|---|---|
| Abbreviated title | COGNITIVE 2026 |
| Country/Territory | Portugal |
| City | Lisbon |
| Period | 19/04/26 → 23/04/26 |
| Internet address |
Funding
This research is funded by UKRI (UK Research and Innovation) under the Hartree National Centre for Digital Innovation Programme (Hartree Centre Northern Ireland Hub )
| Funders |
|---|
| UKRI/ STFC Hartree Centre |
Keywords
- Evaluation
- GraphRAG
- Generative AI
- Large language models (LLMs)
Fingerprint
Dive into the research topics of 'Comparative Evaluation of RAG and GraphRAG for Open-Ended Question Answering'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver