Language Models and Chatbots Knowledge Graph RAG Inference Pipeline

Knowledge Graph RAG Inference Pipeline

Knowledge Graph RAG (Knowledge Graph Retrieval-Augmented Generation) inference enables you to query multiple documents with an LLM and visualize semantic links between text segments. This inference pipeline is the second step in the KG RAG workflow, allowing you to chat with your processed documents and receive answers with citations to specific pages.

This step-by-step guide walks you through setting up the knowledge graph retrieval pipeline using the API, LLM, and Knowledge Graph Retrieval elements. Once complete, you'll have a working chat interface that can answer questions about your ingested documents while providing source citations.

To use the a Knowledge Graph RAG Inference Pipeline, you must first complete a successful Knowledge Graph RAG Document Ingestion Pipeline run. For detailed steps, please refer to the Knowledge Graph RAG Document Ingestion Pipeline article first. 

See our What is KG RAG? article for more information on system requirements, element details, key features, and known limitations. 

Knowledge Graph Retrieval

If you already completed a KG RAG Document Ingestion flow and used the preconfigured settings to create an inference canvas, you can skip to step 6 below to select an LLM. 

  1. Open Navigator and on the Projects tab create a new KG RAG Inference Pipeline template. Create → Templates →  KG RAG Inference Pipeline
  2. We will see three Elements for this inference process.
    • API Element
    • LLM Element
    • Knowledge Graph Retrieval
  3. If you selected the template, these should be preconfigured, however if you needed to manually apply these connectors here are the inputs and outputs for each element.
    • The API Output should connect to the Knowledge Graph Retrieval Input
    • The Knowledge Graph Retrieval Output should connect to the LLM Input

    • The LLM Output should connect to the API Input

      The Canvas should now look like the image below

  4. Click on the Knowledge Graph Retrieval element → View Settings or click the three dots on this element ... → Settings to open the Element Settings panel.

  5. Select the Collection Name that you created for the vector database during the Document Ingestion process in the Knowledge Graph RAG Document Ingestion Pipeline guide from the Collection Name dropdown.

    If this inference canvas was preconfigured from the completion of the ingestion process, this Collection Name should already be auto-selected. 

  6. Click on the LLM element or click the three dots on this element ... → Settings to open the Element Settings panel.
  7. From the Large Language Model dropdown you can select the model you would like to use.
    For more information on Large Language Models see our Support LLM Base Models guide.

  8. You can now click Run to get this retrieval process started!

    If you want more information on what these specific elements do and their settings, see our Element Registry.

    Once this completes you should see a chat preview window appear for you to begin asking questions about your specific documents. 

Knowledge Graph Retrieval takes less than 1 second per query on a MacBook Pro and Mac Studio. The first query will take about 3-5 seconds for cache warmup.

You have now created a KG RAG (Knowledge Graph Retrieval-Augmented Generation) inference pipeline within Navigator!

Next Steps