RAG use case

The rag use case uses candidate documents that are inserted into a LLM’s context to ground the generated response to those documents instead of generating an answer from details stored in the LLM’s trained weights. This type of search adds guardrails so the LLM can search private data collections.

The RAG search can perform queries against external documents passed in as part of the request.

POST /ai/prediction/rag/{MODEL_ID}

RAG use case

Python

import requests

url = "https://{APPLICATION_ID}.applications.lucidworks.com/ai/prediction/rag/{MODEL_ID}"

payload = { "batch": [
        {
            "text": "Why did I go to Germany?",
            "documents": [
                {
                    "body": "I'm off to Germany to go to the Oktoberfest!",
                    "source": "http://example.com/112",
                    "title": "Off to Germany!",
                    "date": "2022-01-31T19:31:34Z"
                }
            ]
        }
    ] }
headers = {
    "Authorization": "<authorization>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

200

{
  "predictions": [
    {
      "tokensUsed": {
        "promptTokens": 606,
        "completionTokens": 23,
        "totalTokens": 629
      },
      "answer": "The reason for going to Germany was to attend Oktoberfest.",
      "answerFound": true,
      "sources": [
        "http://example.com/112"
      ],
      "memoryUuid": "53417d2f-6b0e-47e4-8610-e6842b84a87b",
      "response": "SOURCES:\n- 0\nANSWER: The reason for going to Germany was to attend Oktoberfest."
    }
  ]
}

Headers

Authorization · string · required

Bearer token used for authentication. Format: Authorization: Bearer ACCESS_TOKEN.

Content-Type · string

application/json

Example:

"application/json"

Path Parameters

MODEL_ID · string · required

Unique identifier for the model.

Body

application/json

batch · BatchRag · object[]

Show child attributes

useCaseConfig · UseCaseConfigRagExtDoc · object

Show child attributes

modelConfig · ModelConfig · object

Provides fields and values that specify ranges for tokens. Fields used for specific use cases and models are specified. The default values are used if other values are not specified.

Show child attributes

Response

200 - application/json

OK

predictions · object[]

Show child attributes