> ## Documentation Index > Fetch the complete documentation index at: https://doc.lucidworks.com/llms.txt > Use this file to discover all available pages before exploring further. # Migration guide export const LwTemplate = ({title = "Key questions to get you started", icon = "sparkles", cta = "Powered by Agent Studio", linkHref = "https://lucidworks.com/demo/?utm_source=docs&utm_medium=referral&utm_campaign=docs_cta_ai"}) => { const [isLoaded, setIsLoaded] = useState(false); useEffect(() => { const timer = setTimeout(() => { setIsLoaded(true); }, 500); return () => clearTimeout(timer); }, []); return
{isLoaded && ` }} />} Powered by Lucidworks Agent Studio
; }; This guide explains the client tasks required to migrate from Predictive Merchandiser to Commerce Studio in Lucidworks Platform. See [Commerce Studio versus Predictive Merchandiser](/docs/lw-platform/lw-cs/cstudio-versus-pm) for a feature comparison. While Lucidworks will complete the majority of the migration, you will need to perform certain pre-migration and post-migration tasks. The migration is one-way. The migration can be reverted back to Predictive Merchandiser, but any changes made in Commerce Studio are not reverted back into Predictive Merchandiser. ## Prepare for the migration Complete these client tasks before Lucidworks begins the migration. ### Determine which rules and rewrites will be migrated * **Unpublished** rules or rewrites have never been published. * **Staged** rules or rewrites have been published at least once, but then were changed and those changes are not yet published. It also includes rules and rewrites that were deleted, but the deletion has not yet been published. #### Unpublished rules If you want unpublished rules migrated, then inform Lucidworks to set `excludeUnpublishedRules` to `false` during the migration. This is the default option. If you do not want unpublished rules to be migrated, tell Lucidworks to set `excludeUnpublishedRules` to `true`. #### Unpublished rewrites If you want unpublished rewrites migrated, then inform Lucidworks to set `excludeUnpublishedRewrites` to `false` during the migration. This is the default option. If you do not want unpublished rules to be migrated, tell Lucidworks to set `excludeUnpublishedRewrites` to `true`. #### Publish only approved content By default, migrated rules and rewrites are published in Commerce Studio if they are published in Predictive Merchandiser. If you want only approved rewrites and rules to be migrated as published, then inform Lucidworks to set `publishApprovedOnly` to `true` during the migration. #### Skip rules or rewrites If you want to skip all rules, then inform Lucidworks to set `skipRules` or `skipRewrites` to `true` during the migration. #### Staged rules and rewrites Staged rules and rewrites are skipped in the migration, so you must publish or revert all staged changes to rules and rewrites. You can only view staged changes in Predictive Merchandiser that belong to your user. To check for staged changes that belong to other users in your organization, do the following: 1. Copy the `precheck.sh` script below to check for staged (unpublished) rules and rewrites: ```bash expandable theme={"dark"} #!/bin/bash set -e # Function to perform curl requests with error handling and retries perform_curl_request() { local url=$1 local max_retries=3 local retry_count=0 local wait_time=5 local curl_opts="-s -w '\n%{http_code}' -H 'Accept: application/json' -u '$FUSION_USERNAME:$FUSION_PASSWORD'" while [ $retry_count -le $max_retries ]; do # Execute curl and capture response with status code local response=$(eval "curl $curl_opts '$url'") local status_code=$(echo "$response" | tail -n1) local body=$(echo "$response" | sed '$d') # Check if status code is in 2xx range if [[ $status_code =~ ^2[0-9][0-9]$ ]]; then echo "$body" return 0 else # Report the error echo "Error: HTTP request failed with status code $status_code" >&2 echo "URL: $url" >&2 # If it's a 4xx error, don't retry if [[ $status_code =~ ^4[0-9][0-9]$ ]]; then echo "Client error. Not retrying." >&2 exit 1 fi # For other errors (5xx), retry if we haven't reached max retries if [ $retry_count -lt $max_retries ]; then retry_count=$((retry_count + 1)) echo "Retrying in $wait_time seconds... (Attempt $retry_count of $max_retries)" >&2 sleep $wait_time else echo "Maximum retries reached. Giving up." >&2 exit 1 fi fi done exit 1 } # Function to validate JSON blob rules validate_json_blob_rule() { local doc="$1" local doc_id="$2" local json_blob_s=$(echo "$doc" | jq -r '.json_blob_s // empty') # Guard: Exit early if no json_blob_s field [ -z "$json_blob_s" ] && return 0 # Decode base64 and validate JSON structure local decoded_json=$(echo "$json_blob_s" | base64 -d 2>/dev/null || echo "") # Guard: Exit early if decoding failed [ -z "$decoded_json" ] && return 0 # Check if the decoded JSON is valid if ! echo "$decoded_json" | jq empty 2>/dev/null; then # Invalid JSON in blob bad_json_blob_rule_ids+=("$doc_id") echo "found JSON blob rule with invalid JSON: $doc_id" # Add this entry to our JSON structure for bad JSON blobs local content_entry=$(jq -n --arg id "$doc_id" \ --argjson content "$doc" \ --arg decoded_content "$decoded_json" \ '{ "id": $id, "ruleContent": $content, "decodedJsonBlob": $decoded_content, "error": "Invalid JSON syntax" }') # Add this entry to the second problem (index 1) in our JSON structure json_output=$(echo "$json_output" | jq --argjson entry "$content_entry" '.problems[1].content += [$entry]') return 0 fi # Valid JSON - now check if it's multiple objects # A simple heuristic: if we can't parse it as a single JSON value, it's likely multiple objects if echo "$decoded_json" | jq -e 'type' >/dev/null 2>&1; then # Guard: Exit early if it's a valid single JSON object return 0 fi # This means jq couldn't parse it as a single value, likely multiple objects bad_json_blob_rule_ids+=("$doc_id") echo "found JSON blob rule with multiple objects: $doc_id" # Add this entry to our JSON structure for bad JSON blobs local content_entry=$(jq -n --arg id "$doc_id" \ --argjson content "$doc" \ --arg decoded_content "$decoded_json" \ '{ "id": $id, "ruleContent": $content, "decodedJsonBlob": $decoded_content }') # Add this entry to the second problem (index 1) in our JSON structure json_output=$(echo "$json_output" | jq --argjson entry "$content_entry" '.problems[1].content += [$entry]') } # Function to check for staged published rules check_staged_published_rule() { local doc="$1" local doc_id="$2" local deployed="$3" # Guard: Exit early if rule is deployed (not staged) [ "$deployed" != "false" ] && return 0 # Check if the doc exists in the live rewrites collection local live_url="$FUSION_BASE_URL/api/apps/$FUSION_APP_ID/query-rewrite/search?q=id:$doc_id&isLive=true&doc_type=rule&fl&rows=1&start=0" echo "fetching live rule via $live_url" local live_response=$(perform_curl_request "$live_url") # Check if we have any rules in the live response array local live_docs_count=$(echo "$live_response" | jq -r '.response.docs | length') # Guard: Exit early if doc doesn't exist in live collection [ "$live_docs_count" -eq 0 ] && return 0 # Found a published rule with staged changes staged_published_rule_ids+=("$doc_id") echo "found published rule with staged changes: $doc_id" # Extract the live doc from the response local live_doc=$(echo "$live_response" | jq '.response.docs[0]') # Create a new entry for this rule with both staging and live content local content_entry=$(jq -n --arg id "$doc_id" \ --argjson staging "$doc" \ --argjson live "$live_doc" \ '{ "id": $id, "stagingContent": $staging, "liveContent": $live }') # Add this entry to our JSON structure json_output=$(echo "$json_output" | jq --argjson entry "$content_entry" '.problems[0].content += [$entry]') } # Function to display help message show_help() { echo "Usage: $0 [output_file]" echo "Example: $0 admin password https://fusion.example.com MyFusionApp precheck_results.json" echo "This script performs pre-checks for Fusion content before migration." echo "Arguments:" echo " fusion_username Fusion username" echo " fusion_password Fusion password" echo " fusion_base_url Base URL of the Fusion instance" echo " fusion_app_id Application ID of the Fusion app to check" echo " output_file (Optional) Output file to save results (default: precheck_results.json)" } # Check if required arguments are provided if [ "$#" -lt 4 ]; then show_help exit 1 fi FUSION_USERNAME=$1 FUSION_PASSWORD=$2 FUSION_BASE_URL=$3 FUSION_APP_ID=$4 OUTPUT_FILE=${5:-precheck_results.json} # Input validation if [ -z "$FUSION_USERNAME" ]; then echo "Error: fusion_username is required" show_help exit 1 fi if [ -z "$FUSION_PASSWORD" ]; then echo "Error: fusion_password is required" show_help exit 1 fi if [ -z "$FUSION_BASE_URL" ]; then echo "Error: fusion_base_url is required" show_help exit 1 fi if [ -z "$FUSION_APP_ID" ]; then echo "Error: fusion_app_id ID is required" show_help exit 1 fi # Check if jq is installed if ! command -v jq &> /dev/null; then echo "Error: jq command line tool is required but not installed. Please install jq and try again: https://jqlang.org/" exit 1 fi # Check if output file exists if [ -f "$OUTPUT_FILE" ]; then read -p "File $OUTPUT_FILE already exists. Overwrite? (y/n): " -n 1 -r echo if [[ ! $REPLY =~ ^[Yy]$ ]]; then echo "Operation cancelled." exit 1 fi fi # Ensure the output file is writeable if ! touch "$OUTPUT_FILE" &> /dev/null; then echo "Error: Output file $OUTPUT_FILE is not writeable. Check output path / permissions and try again." exit 1 fi # Initialize JSON structure json_output='{ "problems": [ { "problem": "Published rule with staged changes will be skipped during migration", "fix": "Publish or revert the changes, or manually re-create the skipped content after migrating", "content": [] }, { "problem": "JSON blob rule contains multiple JSON objects instead of a single object", "fix": "Update the JSON blob rule to contain only a single JSON object at the top level", "content": [] } ] }' # Initialize array to store staged published rule IDs declare -a staged_published_rule_ids # Initialize array to store bad JSON blob rule IDs declare -a bad_json_blob_rule_ids # Fetch query rewrite instances with pagination start=0 rows=10 processed_docs=0 while true; do page_url="$FUSION_BASE_URL/api/apps/$FUSION_APP_ID/query-rewrite/instances?sort=id+asc&doc_type=rule&rows=$rows&editSessionId=*&start=$start" echo "fetching next page of rules via $page_url" response=$(perform_curl_request "$page_url") # Check if we got an empty array, which means we're done paginating array_length=$(echo "$response" | jq -r '. | length') if [ "$array_length" -eq 0 ]; then break fi echo "Retrieved $array_length rules at offset $start" # Process each doc in the array while read -r doc; do doc_id=$(echo "$doc" | jq -r '.id') deployed=$(echo "$doc" | jq -r '.deployed') doc_type=$(echo "$doc" | jq -r '.type // empty') # Check for bad JSON blob rules if [ "$doc_type" = "json_blob" ]; then validate_json_blob_rule "$doc" "$doc_id" fi # Check for staged published rules check_staged_published_rule "$doc" "$doc_id" "$deployed" processed_docs=$((processed_docs + 1)) done < <(echo "$response" | jq -c '.[]') # Move to the next page start=$((start + rows)) done # Write the final JSON output to file echo "$json_output" | jq '.' > "$OUTPUT_FILE" # Output results total_staged_issues=0 total_json_blob_issues=0 if [ -n "${staged_published_rule_ids[*]:-}" ]; then total_staged_issues=${#staged_published_rule_ids[@]} fi if [ -n "${bad_json_blob_rule_ids[*]:-}" ]; then total_json_blob_issues=${#bad_json_blob_rule_ids[@]} fi total_issues=$((total_staged_issues + total_json_blob_issues)) # Guard: Exit early if no issues found if [ $total_issues -eq 0 ]; then echo "All pre-checks passed. No issues found." echo "Processed $processed_docs rules." exit 0 fi # Report issues found echo "The following pre-checks failed:" # Report staged published rule issues if [ $total_staged_issues -gt 0 ]; then echo " Found published rules with staged changes. Publish or revert the changes to fix this issue." # Display up to 10 IDs display_count=$([ $total_staged_issues -gt 10 ] && echo 10 || echo $total_staged_issues) for ((i=0; i | Placeholder | Description | | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `FUSION_USERNAME` | Your Fusion username. | | `FUSION_PASSWORD` | The password for your Fusion user. | | `FUSION_BASE_URL` | The URL of your Fusion instance including `https://`. For example, `https://FUSION_HOST.com`. | | `FUSION_APP_ID` | The application ID of the Fusion app that contains your Predictive Merchandiser. This is the value of the `id` field, and may or may not match the `name` of the Fusion app. To obtain the ID, sign in to your Fusion instance. For example, `FUSION_INSTANCE.com`. Then also open a different browser window and enter `https://FUSION_INSTANCE.com/api/apps`. The `id` and `name` display for each valid Fusion app. The value in the `id` field is the value that needs to be used in this field. | If you receive an error stating `bash: ./precheck.sh: Permission denied`: 1. Make the script executable: `chmod +x precheck.sh`. 2. Run the script again. 3. Check the output to see if any issues were detected. If there are no issues, the output returns `All pre-checks passed. No issues found.` Continue to [retrieve list of objects in Predictive Merchandiser](#retrieve-lists-of-objects-in-predictive-merchandiser). If there are issues, the output returns a list of issues like the following: ``` The following pre-checks failed: Found published rules with staged changes. Publish or revert the changes to fix this issue. - 2fRtM1GEsU - E1YkUTX9r6 - Eb2A6WARag - FNS7Whciwx - LlIsMfK4kq - OidpIUrVv3 - P39TDQ2X1K - RYbY2XPMOI - ZHvW2LiQpe - aCseNzc4Xc ... and 6 more. See precheck_results.json for the full list of invalid content ``` 4. Review `precheck_results.json` and inspect each reported rule. If you created the rules, sign into Predictive Merchandiser and publish or revert the staged changes. If the unpublished rules were created by a different user, you cannot publish or revert the rules in the Predictive Merchandiser UI. There are a few options for resolving unpublished rules owned by another user: * After you complete the pre-migration tasks, tell Lucidworks to proceed with the migration. These rules are skipped and you can re-create them in Commerce Studio. * Use the Fusion [Query Rewrite](/api-reference/query-rewrite-api/get-the-query-rewrite-status) and [Custom Rule](/api-reference/custom-rule-type-api/get-a-custom-rule-type) APIs to manually publish or revert the unpublished rules. * Use the Fusion [Query Rewrite](/api-reference/query-rewrite-api/get-the-query-rewrite-status) and [Custom Rule](/api-reference/custom-rule-type-api/get-a-custom-rule-type) APIs, or Solr, to manually change the edit session ID of the content in Solr to your current user. Then, manually fix the content in the Predictive Merchandiser UI. * Have the user that owns the content in Predictive Merchandiser sign in and either publish or revert the changes. 5. After resolving issues, re-run the script. If there are no issues, continue to [retrieve list of objects in Predictive Merchandiser](#retrieve-lists-of-objects-in-predictive-merchandiser). You can run the precheck.sh script at any time, even after a migration if desired. It makes no changes to contents and works even if Commerce Studio is enabled. After preparing Predictive Merchandiser for migration, Lucidworks recommends that you export lists of rules, rewrites, templates, and zones from your Predictive Merchandiser instance to compare against Commerce Studio later. This will help serve as a reference when performing your post-migration validation checks. 1. Export a list of your rules and rewrites: ```bash theme={"dark"} curl --request GET \ --u USER:PASSWORD \ --url 'https://FUSION_HOST/api/apps/APP_NAME/query-rewrite/instances' \ --header 'accept: application/json' \ > exported-rules.json ``` 2. Create an inventory of your rules, noting how many of each type and tag appear in the list. 3. Export a list of your zones: ```bash theme={"dark"} curl --request GET \ -u USER:PASSWORD \ --url 'https://FUSION_HOST/api/templating/zones?context=app:APP_NAME' \ --header 'Accept: application/json' \ > zones.json ``` 4. Export a list of your templates: ```bash theme={"dark"} curl --request GET \ -u USER:PASSWORD \ --url 'https://FUSION_HOST/api/templating/templates?context=app:APP_NAME' \ --header 'Accept: application/json' \ > templates.json ``` Notify Lucidworks to begin the migration, but keep in mind that once the migration begins, you will no longer be able to access Predictive Merchandiser. Lucidworks will set up the Commerce Studio integration and instance, and migrate your data from Predictive Merchandiser to Commerce Studio. Do not proceed with subsequent procedures until Lucidworks confirms the migration is successful. ## Post-migration tasks Complete these client tasks when Lucidworks notifies you the migration is complete. Lucidworks will run the validation checks in this section, but you must also validate that all Predictive Merchandiser objects appear in Commerce Studio. Because you can no longer access Predictive Merchandiser, you must consult the [exported JSON](#retrieve-lists-of-objects-in-predictive-merchandiser) you generated in **Retrieve lists of objects in Predictive Merchandiser** for validation checks. 1. Sign in to [Lucidworks Platform](https://platform.lucidworks.com) as a workspace owner and navigate to **Commerce Studio**. 2. Navigate to **Rules**, then compare your rule count in Commerce Studio against the JSON you [exported from Predictive Merchandiser](#retrieve-lists-of-objects-in-predictive-merchandiser). 3. Navigate to **Rewrite**, then validate that your query rewrites in Commerce Studio match the exported JSON. 4. Navigate to **Settings**, then validate your templates have been correctly migrated. 5. Open each page, then validate your Predictive Merchandiser zones have become sections in Commerce Studio, and they are associated with the correct page. 6. Navigate to the **Editor** and check that rules and rewrites are applied as expected. Some rules and rewrites only fire under specific conditions: 1. For rules that fire under specific templates, change your page to match the required template. 2. For rules that fire only when specific tags are applied, verify the tags are displayed on the Rules page for the rule. 7. Perform additional rules and rewrites validations as needed, using the exported rules JSON as a reference. For example, check that the direction of synonym rewrites matches the expected direction. If you encounter any issues, contact your Lucidworks representative. Web traffic continues to use rules and rewrites from Predictive Merchandiser. Changes made in Commerce Studio do not affect live traffic. Contact Lucidworks when you complete the validation tasks. ## Revert to Predictive Merchandiser If you want to revert back to using Predictive Merchandiser, inform Lucidworks. Then Lucidworks must delete the Commerce Studio instance. If the Commerce Studio instance is deleted, changes you made to Commerce Studio after the initial migration from Predictive Merchandiser are *not* populated back into Predictive Merchandiser. When the Commerce Studio instance is deleted, the system should automatically revert the Fusion app back to Predictive Merchandiser. This restores access to Predictive Merchandiser and routes live traffic back to its rules and rewrites, if traffic had previously been switched to Commerce Studio. ## Frequently asked questions **Can I revert to Predictive Merchandiser after switching to Commerce Studio?** Yes. However, any changes made in Commerce Studio after switching will *not* be reverted back into Predictive Merchandiser. **Does switching to Commerce Studio immediately affect live web traffic?** No. After a successful migration, web traffic continues to use rules and rewrites from Predictive Merchandiser. **What happens if I forget to publish changes in Predictive Merchandiser before migrating?** If you fail to publish pending rules and rewrites before starting the migration, the migration may fail or result in incomplete or inaccurate data. All changes must be published before proceeding. ## Troubleshooting This section provides troubleshooting steps for resolving issues with the Commerce Studio [Editor](/docs/lw-platform/lw-cs/cstudio-editor) when integrated with Fusion. Use this guidance to debug unexpected behavior that may be caused by Fusion query pipeline or query profile configuration. This content is intended for users with experience in Fusion and search engineering. ### Symptoms You may be experiencing a configuration issue if you observe any of the following: * No results or facets returned in the Editor * Error messages when performing searches * Rules not firing or not appearing after creation * Rewrites not triggering or not affecting live traffic * Mismatch between facets and results list * “Targeted documents” column in the rules table is empty or incorrect * No field value suggestions when configuring rule conditions * Missing stage data in the Ranking Factors UI ### General troubleshooting steps 1. Open your browser’s developer tools and go to the Network tab. 1. Retry the failing action. 2. Look for requests with a non-200 status or check `render` endpoint responses for `errorMessage` fields. 3. Use the `fusionRequestUrl` to replicate the request in Fusion Query Workbench. 2. Verify that the query pipeline behaves as expected in Fusion Query Workbench. 1. Confirm there are no errors. 2. Verify expected results are returned. 3. In the **JSON view**, check for `fusion.applicable_rules` or `fusion.tagger` values. 3. Confirm that rules and rewrites are saved to the correct Solr collections. 1. In Solr Admin, check `APP_NAME_query_rewrite_staging_em` and `APP_NAME_query_rewrite_em`. 2. Use the rule ID to query and confirm the entry exists. 4. Ensure the staging collection has only one replica. 1. If multiple replicas exist, set `shards.preference=replica.leader` in the **Apply Rules** and **Text Tagger** stage parameters. 5. Verify that the pipeline supports the `fl` parameter. 1. Do not overwrite `fl`. Add required fields only when needed. 2. Do not use `fl=*`. ### Required pipeline configuration Your Fusion query pipeline must include the following stages in this order: 1. **Text Tagger stage** with a blank value in the **Tagger Collection** field 2. **Apply Rules stage** with a blank value in the **Collection** field 3. **Solr Query stage** 4. **Modify Response with Rules stage** 5. **Response Diagnostics stage** In addition, the pipeline must: * Use the `edismax` query parser (`defType=edismax`) * Preserve the following query parameters: `tags`, `lw.rules.simulate`, `lw.app.emStatus`, `context`, `lw.tagger.debug`, `lw.rules.debug`, `lw.em.staging` * Preserve required response fields: `response.docs`, `response.numFound`, `fusion.tagger`, `fusion.applicable_rules`, `fusion.applicable_stages`, `response.facet_counts`, `responseHeader.params` ### Feature-specific checks #### Targeted document lookup * The Fusion app and query profile must have the same name. * The query profile must support `q=**:**` and standard `fq` parameters. * Avoid using grouping, collapse/expand, or custom logic that may interfere with standard filter behavior. #### Field value suggestions * The Solr collection must match the Fusion app name. * The collection must support the `/terms` request handler with standard parameters. * Suggestions will fail if `/terms` is missing or misconfigured. #### Rewrites and rules interaction * If the **Text Tagger** stage appears before the **Apply Rules** stage, rewrites apply first. * If the **Apply Rules** stage appears before the **Text Tagger** stage, only the original terms are used for rule evaluation. * Rules must target the rewritten query term, not the original term. #### Grouping via collapse/expand * The query pipeline must set an `fq` with the collapse field specified, indicating what field identifies members of a group. For example, set `fq={!collapse field=product_id_s}` when `expand=true` is passed as a query param. * Set the following query parameters: ```bash theme={"dark"} expand=true expand.rows=5 ``` * Relevant response fields include the following: ```bash theme={"dark"} expanded response.docs ``` #### Query Workbench debugging tips * Use the correct query profile and associated pipeline. * Set the following query parameters: ```bash theme={"dark"} lw.rules.debug=1 lw.tagger.debug=1 lw.em.staging=1=_query_rewrite_staging_em lw.app.emStatus=CONNECTED queryProfileID= tags= lw.rules.target_segment= ``` * Use **View As: JSON** to verify `fusion.applicable_rules` and the final query string. * Use custom JavaScript stages to inspect `ctx` or log intermediate values. ### Known configuration issues * Hardcoded `tags` in pipelines may prevent rules from matching if users forget to apply corresponding tags in the Editor. * Query pipelines that use grouping may break document targeting. Only collapse/expand-based grouping is supported. Grouping via Solr `group` params or blockjoin query parser is not supported. * If a boost stage and a rule both adjust relevancy, only one may take effect, depending on rule priority and creation date.