Moving Analytics indexes and structured analytics sets

Moving Analytics indexes and structured analytics sets

The steps below assist with Analytics index and structured analytics set moves, where the Analytics installation remains on its original server. These instructions differ depending on whether the Analytics server is using Single Data Directory mode.

Note: Please note that there can only be one index and structured analytics set location per Analytics server. Indexes and structured analytics sets must use the same location. Moving indexes requires downtime of all Analytics functionality in Relativity, such as clustering, categorization, index creation, etc. This doesn't impact any functionality in Relativity other than Analytics functions. For this reason, we recommend performing the move during off-hours.

Step 1 - Identifying the directory mode

Single Data Directory mode is only available on new Analytics installations; you can't enable it on upgrades. To determine whether or not the server is using Single Data Directory, take the following steps:

  1. Open a browser on the Analytics server and navigate to the Analytics REST URL:

    https://<servername.FQDN>:8443/nexus/r1/system/devices

  2. Log in with the Analytics REST credentials.
  3. Verify the configured device and the device's path.

    If the server is in Single Data Directory mode, the device appears with a special name of single-data-directory and the file path.

Step 2a - Moving the data in non-Single Data Directory Mode

  1. After verifying that the server is not in Single Data Directory Mode, stop the Content Analyst (CAAT)/ Relativity Analytics Engine Windows service on the Analytics server.
  2. Copy the indexes and sets from the original location to the new location. You must copy all subfolders, even if they are empty. The CAAT® software needs this folder structure in order to access the index or set. Be sure to copy all indexes and structured analytics sets to the same folder location:
    1. Move the contents of the C:\CAAT\data-default folder to the new location. This folder contains a .ufx folder and a @README text file.
    2. Move the contents of the C:\CAAT\indexes folder to the new location.
    3. If applicable, move the contents of the C:\CAAT\staging folder the new location.
  3. Verify that the new location contains all indexes and structured analytics sets with no separation using subfolders, as shown below.

  1. Navigate to the C:\CAAT\device-links folder. Delete the caat symbolic link folder.

    1. Open a Windows command prompt.
    2. Run the following command: cd C:\CAAT\device-links
    3. Run the following command: mklink /d caat F:\NewDataLocation
      (In the previous command, "F:\NewDataLocation" represents the file path where the data was migrated from the data-default folder.)
    4. Double-click the new caat symbolic link.
      This displays your new folder location. In Windows Explorer, it will look like you are in C:\CAAT\device-links\caat. However, this is not really a folder (it is a link pointing to the data location).

      You should see a folder structure similar to the following in the caat directory:

      5. There should be a .ufx folder as well as any Analytics indexes listed outside of the .ufx folder.

  2. Open the .ufx folder and verify that it contains both caat and db folders.

  3. Navigate to the C:\CAAT folder, and then delete the C:\CAAT\data-default folder.
  4. Navigate to the C:\CAAT\etc directory.
  5. Open the itemstore.properties file in a text editor. Change the “device-caat” line to point to your new index location. The file path should have a forward slashes ( / ).

  1. Save the itemstore.properties file.
  2. Start the Content Analyst (CAAT)/Relativity Analytics Engine Windows service.
  3. Verify the new location.

    To verify that the new directory is working properly, perform the following steps:

    1. Open a browser on the Analytics server, and then navigate to the Analytics REST URL:

      https://<servername.FQDN>:8443/nexus/r1/

    2. Verify that you can see and access all expected Analytics indexes and structured analytics sets.
    3. Log in to Relativity, and then navigate to the Servers tab.
    4. Activate the Analytics server.
    5. Verify that you're able to run concept searches against the pre-existing indexes.
    6. Create a small Analytics index in one of the demo workspaces, such as Salt vs. Pepper.
    7. Once complete, confirm the new index resides in the new location.
    8. Create a small Language ID structured analytics set in one of the demo workspaces, such as Salt vs. Pepper.
    9. Once complete, confirm the new structured analytics set resides in the new location in the .ufx folder.

Step 2b - Moving the data in Single Data Directory Mode

  1. After verifying that the server is in Single Data Directory Mode, stop the Content Analyst (CAAT)/Relativity Analytics Engine Windows service on the Analytics server.
  2. Copy the indexes and sets from the original location to the new location.

    Note: You must copy all subfolders, even if they are empty. The Analytics software needs this folder structure in order to access the index or set. Be sure to copy all indexes and structured analytics sets to the same folder location.

  3. Move the index directory to the new location. This will include all indexes, older structured analytics sets, a .ufx folder, and an .initialized file.
  4. Verify that the new location contains all indexes and structured analytics sets with no separation using subfolders, as shown below.

  1. Navigate to the C:\CAAT\etc directory.
  2. Open the single-data-dir.properties file in a text editor. Change the “caat.single.data.directory” line to point to your new index location. The file path should have a forward slashes ( / ).

  1. Save the single-data-dir.properties file.
  2. Start the Content Analyst (CAAT)/Relativity Analytics Engine Windows service.
  3. Verify the new location.

    To verify that the new directory is working properly, perform the following steps:

    1. Open a browser on the Analytics server, and then navigate to the Analytics REST URL:

      https://<servername.FQDN>:8443/nexus/r1/

    2. Verify that you can see and access all expected Analytics indexes and structured analytics sets.
    3. Log in to Relativity, and then navigate to the Servers tab.
    4. Activate the Analytics server.
    5. Verify that you're able to run concept searches against the pre-existing indexes.
    6. Create a small Analytics index in one of the demo workspaces, such as Salt vs. Pepper.
    7. Once complete, confirm the new index resides in the new location.
    8. Create a small Language ID structured analytics set in one of the demo workspaces, such as Salt vs. Pepper.
    9. Once complete, confirm the new structured analytics set resides in the new location in the .ufx folder.

Migrating the Analytics server

If you need to migrate the Analytics server to a new server, use these instructions. For this procedure, you need to move the Analytics installation as well as the indexes and structured analytics sets.

Note: This requires downtime of all Analytics functionality in Relativity, such as clustering, categorization, index creation, etc. This doesn't impact any functionality in Relativity other than Analytics functions. For this reason, we recommend performing the move during off-hours.

Step 1 - Copying the Analytics indexes

You must copy all indexes from the old Analytics server to the new Analytics server. If the index directory is stored in another path, you will copy this directory to the desired location on the new server in Step 4 - Restoring the Analytics indexes.

  1. Navigate to the Analytics index location.
  2. Stop the Content Analyst (CAAT) service, which has a hold of all of the indexes.

    Note: This causes an outage of all Analytics functionality on this server.

  3. Copy the existing folders with the following naming convention:

    AnalyticsEngineIndexID

  4. Be sure to copy all subfolders, including empty ones. Take care when using the command line copy to verify that you copy all folders.

    Note: Do not rename any of these folders.

  5. Start the Content Analyst (CAAT) Windows service after completion.

Step 2 - Copying over the structured analytics sets

You must migrate all structured analytics sets from the old Analytics server to the new Analytics server. Contact Support for the Analytics Backup/Restore script and follow the below instructions:

Verifying the structured analytics sets for backup:
  1. From the Analytics server, run the Analytics backup script as an admin.
  2. Input option 1 to verify the structured analytics sets that need to be backed up on the Analytics server.
  3. Input the location of where the structured analytics sets are stored on the server, and then press Enter.
    A notepad file opens indicating which sets are available for backup.
Backing up the structured analytics sets:
  1. Input option 3 to begin the process of backing up the structured analytics sets.
    The system prompts you to input the location of where the structured analytics sets reside. In this will be the folder location you inputted during the verification process.
  2. Input the location of the staging-backup.cmd file. This file is stored in the C:\CAAT\bin location. Depending on your folder structure, you may need to rename the CAAT portion to point to the bin folder correctly.
  3. Input the location of where the backup folder is created. This folder will be created automatically by the script.
  4. Press Enter to begin the backup process.
    To verify that the backup is running, navigate to the location of the backup folder. You should see staging area files being written to the backup folder specified in step 3 and the size of the files growing. If you run into any issues during backup, please contact Support.
  5. Once the backup process completes, copy over the backup folder to the new Analytics server.

Step 3 - Installing Analytics on the new server

To install Analytics on the new server:

  1. Run through the pre-installation steps.
  2. Run the Analytics installer as a clean install on the new server. Be sure to use the correct Analytics server installer for the Relativity environment.
  3. Stop the Content Analyst (CAAT) Windows service after the installer completes.

Step 4 - Restoring the Analytics indexes

To restore your Analytics indexes on the new server:

  1. Navigate to the Analytics index location.
  2. Using the Analytics indexes you copied in Step 1 - Copying the Analytics indexes, paste in the backed up indexes, including all empty folders that may reside in the parent index folder.
  3. Restart the Content Analyst (CAAT) Windows service after all the files are copied over.

Step 5 - Restoring the structured analytics sets

After installation, restore the structured analytics sets from the old Analytics server to the new Analytics server. Contact Support for the Analytics Backup/Restore script and follow these instructions:

Verifying the structured analytics sets for restoring:
  1. Ensure that the backup folder is present on the new Structured Analytics server.
  2. From the Analytics server, run the Analytics Backup/Restore script as an admin.
  3. Input option 2, which provides you a list of structured analytics sets that are available for restore.
Restoring structured analytics sets:
  1. Input option 4, to restore the structured analytics sets.
  2. Input the location of the staging-restore.cmd file. This file is stored in the C:\CAAT\bin location. Depending on your folder structure, you may need to rename the CAAT portion to point to the bin folder correctly.
  3. Input the location of the backup folder where the structured analytics sets are backed up.
  4. Press Enterto kick off the restore.
    To verify that the process is running, navigate to the location where your current structured analytics sets are stored. The sets are commonly restored to the C:\CAAT\.ufx\work location. Depending on your configuration, the sets may be restored to a different location.

Step 6 - Adding the Analytics server to Relativity

Analytics is now configured on the new server. You now need to add this server to Relativity.

To add the Analytics server to Relativity, use the following steps:

  1. Verify that you have a valid URL value entered for the RestUriForCAAT instance setting. This is the fully qualified domain name (FQDN) URL to the web server hosting your Kepler services (e.g., https://client.domain.name/Relativity.REST/API).
  2. Add a new Analytics server from the Servers tab in Relativity. See Adding an Analytics server for more information.
    When entering the URL:
    1. Use this format: https://<servername.FQDN>:8443/.
    2. Duplicate all other settings from the original Analytics server.
  3. Add the new Analytics server to all of the same Resource Pools as the original server.
  4. Add the Analytics Move script to the Relativity Script Library and run the script.
    1. Navigate to the Relativity Script Library tab.
    2. Click New Relativity Script.
    3. Select and copy the contents of the Analytics Move script file. Paste the script text into the Script Body field, overwriting the default script body text.
    4. Click Save.
  5. Test functionality by creating a small structured analytics set or index.
  6. Run the Analytics Move script to swap all references from the original server to the new server just created.
  7. Delete the old Analytics server from the Servers tab in Relativity.

Step 7 - Verifying the migration

To verify that the new installation is working properly, follow these steps:

  1. Log in to Relativity to verify you are able to query against the current indexes.
  2. Create a small Analytics index in one of the demo workspaces, such as Salt vs. Pepper.
  3. Once complete, confirm the new index resides in the new location.
  4. Create a small Language ID structured analytics set in one of the demo workspaces, such as Salt vs. Pepper.
  5. Once complete, confirm the new structured analytics set resides in the new location in the .ufx folder.

Step 8 - Uninstalling the old Analytics installation

Now that Analytics is installed on the new server, perform the following steps:

  1. Delete the old Analytics server from the Servers tab.
  2. Uninstall the Analytics installation from the old Analytics server. For steps on how to do this, see Uninstalling the Relativity Analytics server.
  3. Delete the Analytics installation files and indexes from the old server.

Backing up or archiving Analytics data

Use these instructions for backing up or archiving Analytics data.

Backing up or archiving an Analytics index

Copy all needed indexes from the Analytics server using these instructions:

  1. Navigate to the Analytics index location.
  2. Stop the Content Analyst (CAAT) Windows service, which has ahold of all of the indexes.

    Note: This causes an outage of all Analytics functionality on this server.

  3. Copy the folders with the following naming convention:

    AnalyticsEngineIndexID

  4. Be sure to copy out all subfolders, including empty ones. Take care when using the command line copy to verify that you copy all folders.

    Note: Do not rename any of these folders.

  5. Start the Content Analyst (CAAT) Windows service after completion.

Backing up an individual structured analytics set

To back up only select structured analytics sets, use these instructions:

  1. Open a browser on the Analytics server and navigate to the Analytics REST URL: https://<servername.FQDN>:8443/nexus/r1/staging/
  2. Log in with the Analytics REST credentials. This will show the list of all structured analytics sets.
  3. Take note of the ones you would like to back up. The naming convention is as follows: ArtifactIDofWorkspace_ArtifactIDofStructuredAnalyticsSet
  4. Open a Windows command prompt, and then change to the following directory: 
    cd C:\CAAT\bin
  5. Run the next command to back up the structured analytics set, using the name of the structured analytics set that was recorded.
    In the following example command, 1016636_1111253 is the name of the structured analytics set, and C:\migration\1016636_1111253backup is the location where the backup file will be written.
    staging-backup 1016636_1111253 C:\migration\1016636_1111253backup
  6. Copy the resulting file.
  7. Repeat these steps for all needed structured analytics sets.

Backing up all structured analytics sets

If you would like to back up all structured analytics sets on the server, use the Analytics Backup/Restore script from Relativity Support. This script sequentially backs up every structured analytics set.

Note: This script differs based on the directory mode that the server is using. If Single Data Directory is enabled, the structured analytics sets are commonly stored in C:\CAAT\.ufx\work. If the server is not on Single Data Directory, the structured analytics sets are commonly stored in C:\CAAT\device-links\caat\.ufx\caat. See Identifying the directory mode above.

Verifying the structured analytics sets for backup:
  1. From the Analytics server, run the Analytics Backup/Restore script as an Administrator.
  2. Input option 1 to verify the structured analytics sets that need to be backed up on the Analytics server.
  3. Input the location of where the structured analytics sets are stored on the server and then press Enter. A notepad file will open indicating which sets are available for backup.
Backing up the structured analytics sets:
  1. Input option 3 to begin the process of backing up the structured analytics sets.
  2. You will be prompted to input the location of where the structured analytics sets reside in this will be the folder location you inputted during the verification process.
  3. Input the location of the staging-backup .cmd file.

    This file is stored in the C:\CAAT\bin location. Depending on your folder structure, you may need to rename the CAAT portion to point to the bin folder correctly.

  4. Input the location of where the backup folder is created. This folder will be created automatically by the script.
  5. Press Enter and then the backup process will begin.

    Note: To verify that the backup is running, navigate to the location of the backup folder. You should see staging area files being written to the backup folder specified in step 4 and the size of the files growing. If you run into any issues during backup, please contact Relativity Support.

  6. Once the backup process completes, copy over the backup folder to a safe location.

Restoring Analytics data

Use these instructions for restoring Analytics data.

Restoring necessary Analytics indexes

Restore the needed indexes to the Analytics server using these instructions:

  1. Navigate to the Analytics index location.
  2. Paste in the backed up indexes, including all empty folders that may reside in the parent index folder.
  3. Restart the Content Analyst (CAAT) Windows service after the copy is complete.

Restoring an individual structured analytics set

Perform the following steps to restore a structured analytics set.

  1. Make sure that the backed up structured analytics sets are on a lettered drive on the Analytics server, but not located in the Analytics indexes directory.
  2. From the Analytics server, open a Windows command prompt and change to the following directory: 
    cd C:\CAAT\bin
  3. Run the staging-restore.cmd script as follows: 
    staging-restore <new structured analytics set name> <path to backup file>

    For example, to restore a structured analytics set called 1015041_1111253 from a backup file called 1015041_1111253backup in the Backups directory on the C: drive, the following statement would be used: 

    staging-restore 1015041_1111253 C:\Backups\1015041_1111253backup
  4. If you receive the message in parentheses after the restore ("The staging area was successfully restored, but CAAT had some issue in recognizing this new staging area"), then navigate to the following URL on the Analytics server to allow the structured analytics set to be recognized:

    http://localhost:8443/nexus/r1/staging/1015041_1111253

  5. Repeat these steps for all structured analytics sets in the workspace.

Restoring all structured analytics sets

If you would like to restore all structured analytics sets to the server, use the Analytics Backup/Restore script from Relativity Support. This script sequentially restores every structured analytics set.

Note: This script differs based on the Directory mode that the server is using. If Single Data Directory is enabled, the structured analytics sets are commonly stored in C:\CAAT\.ufx\work. If the server is not on Singe Data Directory, the structured analytics sets are commonly stored in C:\CAAT\device-links\caat\.ufx\caat. See Identifying the directory mode above.

Verifying the structured analytics sets for restoring:
  1. Ensure that the backup folder is present on the new Structured Analytics server.
  2. From the Analytics server, run the Analytics Backup/Restore script as an admin.
  3. Input option 2. This will provide you a list of structured analytics sets that are available for restore.
Restoring structured analytics sets:
  1. Input option 4, to restore the structured analytics sets.
  2. Input the location of the staging-restore .cmd file. This file is stored in the C:\CAAT\bin location. Depending on your folder structure, you may need to rename the CAAT portion to point to the bin folder correctly.
  3. Input the location of the backup folder where the structured analytics sets are backed up.
  4. Press Enter to kick off the restore.

    Note: To verify that the process is running, navigate to the location where your current structured analytics sets are stored. The sets are commonly restored to the C:\CAAT\.ufx\work location. Depending on your configuration, you may be able to restore the sets to a different location.