Query Pipeline Stages

A query pipeline is made up of a series of query stages that process incoming search queries.

A pipeline stage definition associates a unique ID with a set of properties. These definitions are registered with the Fusion API service and stored in ZooKeeper for re-use across pipelines and search applications.

Fusion includes a number of specialized query stages as well as a JavaScript stage that allows advanced processing via a JavaScript program.

Configuring query pipeline stages

• In the Fusion UI, the Query Workbench provides an environment for configuring the stages in a query pipeline.
• The Query Stages API is used to create, list, update, or delete query stages using JSON. See also the Query Pipelines API.

Asynchronous query pipeline processing

Query pipeline processing performance can be improved by enabling asynchronous processing for certain stages that make requests to secondary collections, external databases, and so on. The following stages support asynchronous processing:

• Active Directory Security Trimming
• Apply Rules
• Boost with Signals
• JDBC Lookup
• Security Trimming
• Solr Subquery

This feature uses the fork-and-join model, where any of the stages above can create a fork on the pipeline. The parallel processes are joined again using the Merge Async Results stage at a later point in the pipeline.

Conditional query processing

Query Pipeline stages are used to modify Request objects and Response objects. Each stage can include a conditional JavaScript expression (the condition property in its configuration) that can access these objects.

For example, this condition first checks that the property fusion-user-name is specified in the Request object, then checks for a particular value:

request.hasParam("fusion-user-name") && request.getFirstParam("fusion-user-name").equals("SuperUser");

Reference topics

See these reference topics for complete details about each query pipeline stage:

Setup

• Active Directory Security Trimming
• Field Facet
• Query Fields
• Security Trimming

Relevancy

• Block Documents
• Landing Pages
• Parameterized Boosting
• Recommend More Like This
• Recommend Items for User
• Recommend Items for Item

Query rewriting

• Text Tagger
• Apply Rules

Response rewriting

• Response Document Exclusion Stage
• Response Document Field Redaction Stage
• Response Pairwise Swap
• Response Shuffle Stage
• Modify Response with Rules

Fetch data

• JDBC Lookup
• REST Query Stage
• Solr Query
• Solr Subquery

Troubleshooting

• Logging
• Send PagerDuty Message
• Send Slack Message
• Send SMTP Email
• Write Log Message

Advanced

• Additional Query Parameters
• JavaScript Query Stage
• Managed Javascript
• Retrieve Stored Parameters

Other

• Analytics Catalog Query
• Call Pipeline
• Experiment Query
• Parameterized Faceting
• Return Query Parameters
• Rollup Aggregation