LLM API

The LLM API allows external applications to interact with your LLM workflows created in Navigator. This integration enables you to build applications that leverage your custom LLM flows.

Getting Started

Setting Up the API Element

Add the API element to your LLM flow and connect it properly:

Required Elements

• LLM Element Core language model component
• API Element Enables external API access

Required Connections

• Input Connect LLM element input to the API element output
• Output Connect LLM element output back to your API input
• Input/Output API Element This creates the response loop for continuous interaction

Here is an example of a Canvas that could be utilized by the LLM API:

See our Getting Started with the LLM Chatbot Template article to create this flow.

Configuring API Element Settings

In the API element settings panel:

• API Key Create a custom alphanumeric key for authentication
• Port Address 10105

Port Addresses are automatically assigned after an API element is ran or deployed inside of Navigator.

Using the API

We are now compatible with OpenAI Standard. See the OpenAI API Reference documentation for more information.

Endpoint Details

• URL http://<host>:<port> Example: http://localhost:10105
• Methods POST
  GET
• POST Routes
  • /v1/chat/completions - Chat-based completions (OpenAI format)
  • /v1/completions - Legacy text completions (OpenAI format)
  • /v1/responses - OpenAI Responses API
• GET Routes
  • /v1/models - Model listing and retrieval
• Authentication API keys should be provided via HTTP Bearer authentication.
  Authorization: Bearer <WEBAI_API_KEY>
• Response Format Streamed as server-sent events (SSE)

Examples

The canvas inside of Navigator must to be running or deployed for the API Element to be discoverable.

POST /v1/chat/completions 

More Documentation Here

Text Requests

Parameters

• model string - required
• messages array - required

Request body example

{
    "stream": false,
    "model": "webai-llm",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Write three taglines for a pizza shop?"
                }
            ]
        }
    ]
}

The stream parameter controls how the model returns data. With stream=false, the model waits to finish, then sends the full response at once. With stream=true, the model sends data as it generates it, using server-sent events. See the Streaming documentation and the streaming responses guide for implementation details.

Response

{
    "choices": [
        {
            "citations": [],
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "message": {
                "content": "Here are three potential taglines for a pizza shop:\n\n1. \"Heating up your appetite, one pie at a time.\"\n2. \"A slice above the rest.\"\n3. \"Fresh dough, fiery passion.\"",
                "role": "assistant"
            }
        }
    ],
    "citations": [],
    "created": 1677664795,
    "id": "webaichat-6ee7cdbd9fb0",
    "metrics": {
        "tps": 43.0,
        "ttft": 0.204036
    },
    "model": "nav",
    "object": "chat.completion",
    "usage": {}
}

Image Requests

Parameters

• model string - required
• messages array - required

Request body example

{
    "stream": false,
    "model": "webai-llm",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What is the location of this picture?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url":  "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
                    }
                }
            ]
        }
    ]
}

Response

{
    "choices": [
        {
            "citations": [],
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "text": "This picture was taken at the University of Wisconsin-Madison Arboretum, specifically on the nature boardwalk trail. The boardwalk appears to be located in a wetland or marshy area within the arboretum, which is a nature preserve and research facility."
        }
    ],
    "citations": [],
    "created": 1677664795,
    "id": "webaichat-6ee7cdbd9fb0",
    "metrics": {
        "tps": 59.0,
        "ttft": 0.204737
    },
    "model": "nav",
    "object": "chat.completion",
    "usage": {}
}

Is the response stating that it cannot see the image you provided? If so, may need to change the model you've selected within the LLM Element Settings. The image icon next to the listed models will indicate that a model supports multimodal inputs. Also checkout the Configuring LLM Settings video to learn more on the LLM Element. 

POST /v1/completions 

More Documentation Here

Parameters

• model string - required
• prompt string or array - required

Request body example

{
    "model": "webai-llm",
    "prompt": "write a tagline for a taco shop.",
    "max_tokens": 150
}

Response

{
    "choices": [
        {
            "citations": [],
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "text": "Here are a few tagline options for a taco shop:\n\n1. \"Wrap yourself in flavor\"\n2. \"Tacos that taste like home\"\n3. \"Fold, fill, repeat: the taco life\"\n4. \"Taste the culture, one bite at a time\"\n5. \"Tacos made with love, served with pride\"\n6. \"The perfect fold, every time\"\n7. \"Your taco, your way\"\n8. \"Flavors that fold you over\"\n9. \"Taco 'bout great food\"\n10. \"Get wrapped up in the flavor\""
        }
    ],
    "citations": [],
    "created": 1677664795,
    "id": "webaichat-6ee7cdbd9fb0",
    "metrics": {
        "tps": 59.0,
        "ttft": 0.204737
    },
    "model": "nav",
    "object": "text_completion",
    "usage": {}
}

POST /v1/responses 

More Documentation Here

Parameters

• model string - required
• input string or array - required

Request body example

{
    "model": "webai-llm",
    "input": "tell me a short story in three sentences about a wizard.",
    "max_tokens": 150
}

Response

{
    "choices": [
        {
            "citations": [],
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "text": "In the mystical realm of Aethoria, a powerful wizard named Eryndor Thorne was known for his extraordinary ability to communicate with the ancient trees, which granted him wisdom and knowledge beyond the understanding of mortal men. Eryndor spent most of his days within the heart of the enchanted forest, studying the ancient lore and mastering the mystical arts that allowed him to wield the elements themselves. As the night of the summer solstice approached, Eryndor prepared to face the great challenge that lay ahead: a ritual to bring balance and harmony to the land, threatened by a growing darkness."
        }
    ],
    "citations": [],
    "created": 1677664795,
    "id": "webaichat-6ee7cdbd9fb0",
    "metrics": {
        "tps": 60.0,
        "ttft": 0.205079
    },
    "model": "nav",
    "object": "response",
    "usage": {}
}

GET /v1/models 

More Documentation Here

Response

{
    "data": [
        {
            "created": 1747937728,
            "id": "webai/llm",
            "object": "model",
            "owned_by": "webai"
        }
    ],
    "object": "list"
}