> ## Documentation Index > Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt > Use this file to discover all available pages before exploring further. # LWAI Prediction > Index pipeline stage configuration specifications export const schema = { "type": "object", "title": "LWAI Prediction", "description": "Integrates Fusion index pipelines with Lucidworks AI to generate model predictions during document indexing. Runs asynchronously by default, allowing the pipeline to continue while waiting for the AI response. Combine with a Solr Asynchronous Query stage to handle incremental AI responses when using asynchronous mode.", "required": ["accountName", "useCase", "modelName", "inputContextVariable"], "properties": { "skip": { "type": "boolean", "title": "Skip This Stage", "description": "When `true`, this stage is bypassed and documents pass to the next stage unchanged. Use this to disable a stage temporarily without removing it from the pipeline. A skipped stage isn't initialized, so configuration errors in it aren't reported until you clear this option.", "default": false, "hints": ["advanced"] }, "label": { "type": "string", "title": "Label", "description": "A descriptive name for this stage. The label appears in the pipeline editor, log messages, and stage metrics, so a clear label makes it easier to identify a stage when troubleshooting. For example, `Enrich product data`.", "hints": ["advanced"], "maxLength": 255 }, "condition": { "type": "string", "title": "Condition", "description": "JavaScript expression that determines whether this stage runs for each document. The stage runs when the expression returns `true` and is skipped when it returns `false`. The expression can use the `doc` (current document) and `ctx` (pipeline context) variables. For example, `doc.getFirstFieldValue('type') == 'premium'` runs this stage only for documents whose `type` field is `premium`. The expression must return a boolean. If it returns another value or throws an error, processing fails for that document.", "hints": ["code", "code/javascript", "advanced"] }, "accountName": { "type": "string", "title": "Account Name", "description": "Specifies the Lucidworks AI account name registered in the Lucidworks AI Gateway Service. This value must match an existing account. An invalid name causes all prediction calls to fail. Use the account selector to choose from configured accounts.", "hints": ["enumUrl:/api/query-stages/lwai-accounts"] }, "useCase": { "type": "string", "title": "Use Case", "description": "Specifies the Lucidworks AI use case that determines the type of prediction to perform, such as embedding or summarization. Available use cases are filtered to those compatible with the selected account and `index` caller context. Refer to the Lucidworks AI documentation for supported identifiers.", "hints": ["enumUrl:/api/query-stages/lwai-usecase?account=${accountName}&caller=index"] }, "modelName": { "type": "string", "title": "Model", "description": "Specifies the Lucidworks AI model to invoke for generating predictions. The model must be registered under the selected account and compatible with the chosen use case. Selecting an incompatible model causes a processing error.", "hints": ["enumUrl:/api/query-stages/lwai-model?account=${accountName}&useCase=${useCase}"] }, "inputContextVariable": { "type": "string", "title": "Input context variable", "description": "Specifies the name of the pipeline context variable whose value is sent to the Lucidworks AI model as input. Supports template expressions such as `` for dynamic resolution. The variable must be populated by an upstream stage before this stage executes." }, "outputContextVariable": { "type": "string", "title": "Destination Field Name & Context Output", "description": "Specifies the destination used to store the prediction result as both a document field and a pipeline context variable under the same name. The name must be compatible with the Solr schema field type expected for the prediction output. Both the document field and the context variable are populated simultaneously." }, "useCaseConfig": { "type": "array", "title": "Use Case Configuration", "description": "Sends additional key-value parameters to the Lucidworks AI use case alongside the standard request. Use this to pass use-case-specific options not covered by the primary configuration. Each entry requires a `key` and an optional `value`.", "minItems": 0, "items": { "type": "object", "required": ["key"], "properties": { "key": { "type": "string", "title": "Parameter Name" }, "value": { "type": "string", "title": "Parameter Value" } } } }, "modelConfig": { "type": "array", "title": "Model Configuration", "description": "Sends additional key-value configuration parameters to the Lucidworks AI model alongside the prediction request. Use this to supply model-specific options not covered by the standard configuration. Each entry requires a `key` and an optional `value`.", "minItems": 0, "items": { "type": "object", "required": ["key"], "properties": { "key": { "type": "string", "title": "Parameter Name" }, "value": { "type": "string", "title": "Parameter Value" } } } }, "callAsynchronously": { "type": "boolean", "title": "Call Asynchronously?", "description": "Controls whether the Lucidworks AI API call blocks the pipeline thread until a response is received. When `true`, the call is non-blocking and the pipeline continues immediately. When `false`, the stage waits synchronously for the response. Asynchronous mode improves throughput but requires additional stages to handle the deferred response.", "default": true }, "maxTries": { "type": "integer", "title": "Maximum Asynchronous Call Tries", "description": "Sets the maximum number of attempts for each asynchronous Lucidworks AI API call. Increase this value to add retry resilience for transient network or service errors. Each retry is attempted immediately after a failure with no delay.", "default": 1, "minimum": 1, "exclusiveMinimum": false }, "failOnError": { "type": "boolean", "title": "Fail on Error", "description": "Controls whether a prediction error causes the entire pipeline to fail. When `true`, any model error throws an exception and stops the document. When `false`, errors are logged and the document continues. Set to `true` in production to surface integration issues immediately.", "default": false }, "apiKey": { "type": "string", "title": "API Key", "description": "Specifies the API secret key used to authenticate with the AI model service, such as an OpenAI key starting with `sk-`. Store this value as a Fusion secret rather than entering it as plain text. An invalid or missing key causes authentication failures for all prediction calls.", "hints": ["secret"] } }, "category": "AI", "categoryPriority": 10, "unsafe": false }; export const SchemaParamFields = ({schema}) => { const sanitize = str => { if (typeof str !== "string") return str; return str.replace(/^"(.*)"$/s, "$1").replace(/\\/g, "").replace(/"/g, "'"); }; const renderMd = str => { const s = sanitize(str); const text = (/[.!?]\)*$/).test(s) ? s : `${s}.`; return text.split(/(\*\*[^*]+\*\*|_[^_]+_|`[^`]+`)/g).map((part, i) => { if (part.startsWith("**")) return {part.slice(2, -2)}; if (part.startsWith("_")) return {part.slice(1, -1)}; if (part.startsWith("`")) return {part.slice(1, -1)}; return part; }); }; const {description, properties = {}, required: requiredProps = []} = schema; const visibleProps = useMemo(() => Object.entries(properties).filter(([, prop]) => !prop.hints?.includes("hidden")), [properties]); const renderProp = ([name, prop]) => { const isRequired = requiredProps.includes(name); const hasDefault = prop.default !== undefined; const rawDefault = prop.default; const hints = prop.hints || []; const isComplexDefault = hasDefault && (typeof rawDefault === "object" || typeof rawDefault === "string" && (rawDefault.length > 20 || rawDefault.includes('"'))); const postBadges = []; if (prop.title) { postBadges.push(<>API property: {name}); } const constraints = []; if (prop.minimum !== undefined && prop.maximum !== undefined) { constraints.push(`Range: ${prop.minimum} – ${prop.maximum}`); } else if (prop.minimum !== undefined) { constraints.push(`Min: ${prop.minimum}`); } else if (prop.maximum !== undefined) { constraints.push(`Max: ${prop.maximum}`); } if (prop.minLength !== undefined && prop.maxLength !== undefined) { constraints.push(`Length: ${prop.minLength} – ${prop.maxLength}`); } else if (prop.minLength !== undefined) { constraints.push(`Min length: ${prop.minLength}`); } else if (prop.maxLength !== undefined) { constraints.push(`Max length: ${prop.maxLength}`); } const fieldProps = { key: name, body: prop.title || name, type: prop.type, ...postBadges.length > 0 && ({ post: postBadges }), ...isRequired && ({ required: true }), ...!isComplexDefault && hasDefault ? { default: sanitize(String(rawDefault)) } : {} }; const isObject = prop.type === "object" && prop.properties; const isArrayOfObjects = prop.type === "array" && prop.items?.type === "object" && prop.items.properties; return {prop.description &&

{renderMd(prop.description)}

} {prop.enum &&

Allowed values: {prop.enum.map((v, i) => <>{i > 0 && ", "}{String(v)})}

} {constraints.length > 0 &&

{constraints.join(" · ")}

} {isComplexDefault &&

Default: 

              
                {JSON.stringify(rawDefault, null, 2)}
} {isArrayOfObjects && } {isObject && } ; }; return
{description &&

{renderMd(description)}

} {visibleProps.map(renderProp)}
; }; export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => { const [isLoaded, setIsLoaded] = useState(false); useEffect(() => { const timer = setTimeout(() => { setIsLoaded(true); }, 500); return () => clearTimeout(timer); }, []); return
{isLoaded && ` }} />} Powered by Lucidworks Agent Studio
; }; [localhost link]: http://localhost:3000/docs/lucidworks-search/09-developer-documentation/config-specs/index-pipeline-stages/lucidworks-ai-prediction-index-stage [mintlify link]: https://doc.lucidworks.com/docs/lucidworks-search/09-developer-documentation/config-specs/index-pipeline-stages/lucidworks-ai-prediction-index-stage [old doc.lw link]: https://doc.lucidworks.com/fusion/5.9/bzik28 The LWAI Prediction index stage is an integration between Fusion and Lucidworks AI to enrich your index with [Generative AI](/docs/lw-platform/lw-ai/lw-ai-generative-ai) predictions. Generative AI predictions can enrich product descriptions and other metadata that improves search relevance and recommendations. In addition, this stage can return summaries of long documents or clarify descriptions and other data. For example, if you are a B2B organization that provides industrial technology components and information, this stage can enhance product part descriptions, and category classification and tags that improve search relevance. For technical specifications and detailed instructional guides, the stage can summarize that information and enhance discoverability. In a similar fashion, the stage can enrich catalog items and information to improve relevance and customer search experiences for B2C organizations. If you enable the **Call Asynchronously?** field, the stage processes predictions without blocking the indexing pipeline. Asynchronous predictions are particularly useful for large documents or generative AI models where response times may be longer. Fusion submits the selected document content to Lucidworks AI and continues processing the pipeline while the Lucidworks AI model generates the response. This behavior follows a processing pattern similar to [asynchronous Tika parsing](/docs/managed-fusion/04-move-data-in/parsers/asynchronous-tika-parsing), where documents are indexed first and enrichment results are applied at a different time when system performance allows. Running predictions asynchronously helps maintain indexing throughput when generating AI responses that may take longer to complete. An example of a simplified asynchronous prediction flow includes the following processes: * A document enters the Fusion index pipeline. * Fusion processes the document and prepares it for indexing or update. * The LWAI Prediction index stage sends the configured input field to Lucidworks AI. * Lucidworks AI processes the request and generates the prediction asynchronously. * Fusion receives the prediction and writes the result to the configured destination field in the document. ## Prerequisites ### Lucidworks AI Gateway Make sure you have configured a [Lucidworks AI Gateway](/docs/lw-platform/lw-ai/lw-ai-gateway) integration before you begin. Lucidworks AI Gateway provides a secure, authenticated connection between Fusion and your hosted models. ### Permissions To use this stage, non-admin Fusion users must be granted the `PUT,POST,GET:/LWAI-ACCOUNT-NAME/**` permission in Fusion, which is the Lucidworks AI API Account Name defined in [Lucidworks AI Gateway](/docs/lw-platform/lw-ai/lw-ai-gateway) when this stage is configured. ## Configuration This section describes how to configure the stage when your organization generates and manages the authentication and API keys. If your organization has opted to contract with Lucidworks to provide the API keys to execute generative AI (Gen-AI) models, see [Generative AI using Lucidworks-managed keys](/docs/lw-platform/lw-ai/lw-ai-generative-ai-using-lw-keys). 1. Sign in to Fusion and click **Indexing > Index Pipelines**. 2. Click **Add+** to add a new pipeline. 3. Enter the name in **Pipeline ID**. 4. Click **Add a new pipeline stage**. 5. In the AI section, click **LWAI Prediction**. 6. In the **Label** field, enter a unique identifier for this stage. 7. In the **Condition** field, enter a script that results in true or false, which determines if the stage should process. 8. In the **Account Name** field, select the Lucidworks AI API account name defined in [Lucidworks AI Gateway](/docs/lw-platform/lw-ai/lw-ai-gateway). 9. In the **Use Case** field, select the Lucidworks AI use case to associate with this stage. * To generate a list of the use cases for your organization, see [Use Case API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-use-case-api). * If the **Call Asynchronously?** check box is selected, the index pipeline continues to process documents while the prediction is handled separately. For more information, see available use cases described in [Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview). * If the **Call Asynchronously?** check box is *not* selected, the index pipeline waits until the prediction response is returned before continuing. For more information, see available use cases described in [Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-prediction-api/overview). 10. In the **Model** field, select the Lucidworks AI model to associate with this stage. If you do not see any model names and you are a non-admin Fusion user, verify with a Fusion administrator that your user account has these permissions: `PUT,POST,GET:/LWAI-ACCOUNT-NAME/**` Your Fusion account name must match the name of the account that you selected in the **Account Name** dropdown. For more information about models, see: * [Pre-trained embedding models](/docs/lw-platform/lw-ai/lw-ai-pre-trained-embedding-models) * [Custom embedding model training](/docs/lw-platform/lw-ai/lw-ai-custom-embedding-model-training/overview) * [Generative AI models](/docs/lw-platform/lw-ai/lw-ai-generative-ai) 11. In the **Input context variable** variable field, enter the name of the variable in context to be used as input. Template expressions are supported. 12. In the **Destination field name and context output** field, enter the name that will be used as both the field name in the document where the prediction is written and the context variable that contains the prediction. * If the **Call Asynchronously?** check box is selected and a value is entered in this field: * `{destination name}_t` is the full response. * In the document: * `_lw_ai_properties_ss` contains the Lucidworks account, boolean setting for async, use case, input for the call, and the collection. * `_lw_ai_request_count` is the number of GET requests by `predictionId` and `_lw_ai_success_count` is the number of responses without errors. Because asynchronous predictions may be retried multiple times (based on the **Maximum Asynchronous Call Tries** setting), the request count may be greater than one. These fields are only used for debugging. When troubleshooting, the most useful measure is the ratio of `_lw_ai_success_count` divided by `_lw_ai_request_count`. A value close to **1.0** indicates that prediction requests are completing successfully. Lower ratios may indicate retries, API errors, or connectivity issues with the Lucidworks AI service. If `_lw_ai_success_count` remains **0** while `_lw_ai_request_count` increases, verify the following configuration is set correctly: * Lucidworks AI Gateway configuration * API authentication settings * Network connectivity between Fusion and the AI service * Selected use case and model are valid for asynchronous predictions * `enriched_ss` contains the use case. This can be used as a boolean value to verify if the use case indexed successfully. * If the **Call Asynchronously?** check box is *not* selected and a value is entered in this field: * `{destination name}_t` is the full response. * If *no* value is entered in this field (regardless of the **Call Asynchronously?** check box setting): * `_lw_ai_{use case}_t` is the `response.response` object, which is the raw model output. * `_lw_ai_{use case}_response_s` is the full response. 13. In the **Use Case Configuration** section, click the **+** sign to enter the parameter name and value to send to Lucidworks AI. The `useCaseConfig` parameter is only applicable to certain use cases. * If the **Call Asynchronously?** check box is selected, `useCaseConfig` information for each applicable use case is described in [Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview). * If the **Call Asynchronously?** check box is *not* selected, `useCaseConfig` information for each applicable use case is described in [Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-prediction-api/overview). 14. In the **Model Configuration** section, click the **+** sign to enter the parameter name and value to send to Lucidworks AI. Several `modelConfig` parameters are common to generative AI use cases. * If the **Call Asynchronously?** check box is selected, `modelConfig` information is described in [Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview). * If the **Call Asynchronously?** check box is *not* selected, `modelConfig` information is described in [Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-prediction-api/overview). 15. In the **API Key** field, enter the secret value specified in the external model. For: * OpenAI models, `"apiKey"` is the value in the model’s `"[OPENAI_API_KEY]"` field. For more information, see [Authentication API keys](https://platform.openai.com/docs/api-reference/authentication). * Azure OpenAI models, `"apiKey"` is the value generated by Azure in either the model’s `"[KEY1 or KEY2]"` field. For requirements to use Azure models, see [Generative AI models](/docs/lw-platform/lw-ai/lw-ai-generative-ai). * Google VertexAI models, `"apiKey"` is the value in the model’s `"[BASE64_ENCODED_GOOGLE_SERVICE_ACCOUNT_KEY]"` field. For more information, see [Create and delete Google service account keys](https://cloud.google.com/iam/docs/keys-create-delete). 16. To run the API call asynchronously, select the **Call Asynchronously?** check box to specify the stage is to use the [Lucidworks AI Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview) endpoints. If this is selected, the API call does not block the pipeline while waiting for a response from Lucidworks AI. If the check box is *not* selected, the API call uses the [Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-prediction-api/overview), which uses the pipeline until a response is received from Lucidworks AI. Performance of other API calls can be impacted. 17. In the **Maximum Asynchronous Call Tries** field, enter the maximum number of times to send an asynchronous API call before the system generates a failure error. 18. Select the **Fail on Error** checkbox to generate an exception if an error occurs while generating a prediction for a document. 19. Click **Save**. ## Additional requirements Asynchronous enrichment stages update documents after the initial indexing operation. For this reason, the indexing pipeline must support partial document updates. Include the following additional requirements: * Use a **V2 connector**. Only V2 connectors support this workflow. * Remove the `Apache Tika` parser stage if asynchronous parsing is enabled, as it may conflict with asynchronous processing. If the parser stage is not removed, the datasource fails with the following error: "The following components failed: \[class com.lucidworks.connectors.service.components.job.processor.DefaultDataProcessor : Only Tika Container parser can support Async Parsing.]" * Replace the `Solr Indexer` stage with the `Solr Partial Update Indexer` stage with the following settings: * `Enable Concurrency Control` set to **off** * `Reject Update if Solr Document is not Present` set to **off** * `Process All Pipeline Doc Fields` set to **on** * `Allow reserved fields` set to **on** * Configure the required partial update parameters By default, the Fusion `Call Asynchronously?` field is selected, which specifies this stage uses the [Lucidworks AI Async Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-async-prediction-api/overview) endpoints. If the Fusion `Call Asynchronously?` field is *not* selected, this stage uses the [LWAI Prediction API](/docs/lw-platform/lw-ai/lw-ai-apis/lw-ai-prediction-api/overview) endpoints. When entering configuration values in the UI, use *unescaped* characters, such as `\t` for the tab character. When entering configuration values in the API, use *escaped* characters, such as `\\t` for the tab character. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.