Document QnA

Want to use LLMs to query or summarize documents or other information privately and securely? The Document QnA is the fastest way to get started. Follow the steps below to upload your documents and get going. Want an easy one out of the box? Create your own Support Chat app here.

1. Open the Document QnA Element settings and make the following adjustments:
   1. Trained Artifact: If you have previously trained a custom LLM you can select it here. It will override the Base Model Architecture and Quantization Rank settings.
   2. Base Model Architecture: Select any of the supported models, or leave this as the default.
   3. References Folder Path: Upload a folder containing the documents you want to interact with. It can be any mix of Word Docs, PDFs, or .txt files.
      

      Even if you have only 1 document it needs to be in a folder.

   4. Base Model Storage Path: Select where you want the base LLM model to be stored on your machine.
   5. Model System Prompt: This default prompts your model to act as an assistant. You can leave this setting alone, or play with it to change how the model responds.

      For example: ask the model to talk like a Pirate or Rock Star, speak in a casual tone, or speak like a poet.

2. Leave all other settings as default.
3. Press Run and wait for the dependencies to install.
4. Start chatting with your newly created Document QnA Assistant.