Core Concepts What is KG RAG?

What is KG RAG?

Knowledge Graph RAG (Knowledge Graph Retrieval-Augmented Generation) is an advanced AI system that allows you to build intelligent document repositories by converting documents into searchable knowledge graphs. Unlike traditional RAG systems that rely solely on vector similarity, KG RAG creates semantic connections between document chunks, enabling more accurate and contextually aware retrieval of information.

KG RAG uses webAI's Navigator to process documents, extract text and images, build knowledge graphs that represent relationships between concepts, and retrieve relevant information for answering questions. This system is perfect for creating secure, private AI assistants that can answer questions based on your internal documents without relying on cloud services.

Overview

KG RAG consists of three specialized elements that work together to transform your documents into an intelligent knowledge system:

  • Document Ingestion - Processes documents and extracts text, images, and metadata
  • Knowledge Graph Builder - Constructs knowledge graphs from processed documents and creates vector embeddings
  • Knowledge Graph Retrieval - Retrieves relevant context from the knowledge graph based on user queries

The system operates in two main phases: a document ingestion pipeline that processes and indexes your documents, and an inference pipeline that allows you to query your documents using a Large Language Model (LLM). The knowledge graph structure enables the system to understand relationships between different pieces of information, leading to more accurate and contextually relevant answers.

System Requirements

KG RAG has specific system requirements to ensure optimal performance:

Production Requirements

  • Operating System: macOS Sequoia (15.xx)
  • Python: 3.10
  • Hardware: Apple M3 Ultra with 256 GB RAM (recommended for full production use)
  • Active RAM Usage: Up to 150 GB (to run Llama 4 Scout 17B 16E Instruct 8bit with Navigator and other OS-related processes)
  • Disk Space: Up to 150 GB (to store Llama 4 Scout 17B 16E Instruct 8bit with Navigator and other OS-related processes)

Local Testing Requirements

For local testing of ingestion and retrieval, you can use smaller models on less powerful hardware:

  • Model: Gemma 3 4B IT BF16 (requires about 10 GB RAM and disk space)
  • Hardware: MacBook Pro with at least 16 GB RAM (to leave room for OS and other processes)

While smaller models can be used for testing, accuracy of numerical values and cited information will not be as good as with the recommended production model.

The Elements

Document Ingestion

The Document Ingestion element is the starting point of the KG RAG pipeline. It ingests documents, extracts text and metadata, performs chunking, and organizes content for further processing. This element processes documents to extract their content in a structured manner.

Key capabilities include:

  • Extracting text and images from documents
  • Organizing content into manageable chunks for processing
  • Supporting multiprocessing for faster document ingestion on large document sets
  • Configurable image resolution settings for balancing quality and performance

The output from Document Ingestion serves as the input for the Knowledge Graph Builder, creating a seamless pipeline for document processing.

Knowledge Graph Builder

The Knowledge Graph Builder element builds, saves, and visualizes knowledge graphs from document chunks and embeddings using high-performance core algorithms. This element takes the processed content from the Document Ingestion element and constructs a knowledge graph that represents the relationships between different entities and concepts found in the documents.

Key capabilities include:

  • Creating vector embeddings from document chunks
  • Building knowledge graphs that capture semantic relationships between concepts
  • Storing knowledge graph data in a vector database for efficient retrieval
  • Providing interactive visualization tools to explore document relationships
  • Supporting custom vector database configurations for advanced use cases

The knowledge graph structure enables the system to understand not just what information exists in your documents, but how different pieces of information relate to each other, leading to more intelligent retrieval.

Knowledge Graph Retrieval

The Knowledge Graph Retrieval element retrieves context and metadata from a knowledge graph based on similarity and graph connectivity. This element allows for querying the knowledge graph to retrieve relevant information, which is a key component for the RAG system as it provides the context needed to answer questions accurately.

Key capabilities include:

  • Searching the knowledge graph for relevant document chunks based on query similarity
  • Filtering results by similarity threshold to ensure high-quality context
  • Providing citations to specific pages where information was found
  • Supporting custom instructions for guiding LLM responses
  • Preloading embedding models for faster first query response
  • Supporting custom search strategies for advanced retrieval scenarios

The Knowledge Graph Retrieval element works seamlessly with LLM elements to provide accurate, contextually relevant answers with proper citations to source documents.

How It Works

KG RAG operates through two main pipelines that work together to create an intelligent document querying system:

Document Ingestion Pipeline

The ingestion pipeline processes your documents and prepares them for retrieval:

  1. Document Ingestion processes files, extracting text, images, and metadata, and organizing content into chunks
  2. Knowledge Graph Builder takes the processed chunks, creates vector embeddings, and builds a knowledge graph that captures relationships between concepts and entities
  3. The knowledge graph and vector embeddings are stored in a vector database for efficient retrieval

Inference Pipeline

The inference pipeline allows you to query your documents and get intelligent answers:

  1. A user submits a query through an API element
  2. Knowledge Graph Retrieval searches the knowledge graph for relevant document chunks based on similarity and graph connectivity
  3. The retrieved context (including images and text) is passed to an LLM element along with the user's query
  4. The LLM generates an answer based on the retrieved context, and the response is returned through the API with citations to source pages

The knowledge graph structure enables the system to retrieve not just similar content, but also related information that may be contextually relevant, leading to more comprehensive and accurate answers.

Key Features

KG RAG includes several advanced features that enhance performance and usability:

  • Multiprocessing Support: Document ingestion uses page-level parallelization for faster processing on multi-document workloads, delivering up to 2x performance improvements
  • Real-time Progress Tracking: UI-readable progress updates during document ingestion and knowledge graph building for monitoring processing status
  • Interactive Visualization: Visualize knowledge graphs to explore relationships between document concepts
  • Citation Support: Automatic citations to specific page numbers where information was retrieved
  • Custom Instructions: Provide custom instructions to guide how the LLM processes retrieved context and answers queries
  • Multimodal Support: Works with both text and images, allowing the system to understand visual content in documents

Use Cases

KG RAG is ideal for a wide range of document-based AI applications:

  • Enterprise Knowledge Base → Company documentation and policy Q&A
  • Customer Support → Automated responses from support materials
  • Legal Document Analysis → Contract and compliance document search
  • Technical Documentation → API docs, manuals, and troubleshooting guides
  • Research Assistant → Academic paper and research document querying
  • Medical Information → Healthcare documentation and procedure guides
  • Educational Content → Textbook and curriculum material assistance
  • Compliance Systems → Regulatory and standards documentation

Known Limitations

Please be aware of these current limitations when working with KG RAG:

  • Supported Document Types: The Document Ingestion element can ingest PDF, XML, JSON, and HTML in text-only mode.
  • User Query Images: You can include up to 2 images along with your text query when asking questions. If you upload more than 2 images, only the first 2 will be processed.
  • Context Images: The Knowledge Graph Retrieval element will pass up to 5 images to the LLM. Using Max Results = 4 means you get 4 context images plus room for 1 user-uploaded image (total of 4 images to the LLM).
  • Total Image Limit: Most multimodal LLMs have a limit of 5 total images per query (user images + context images combined).

Next Steps

Now that you understand what KG RAG is and how it works, you can get started building your own knowledge graph-based RAG system: