Upgrading Elasticsearch

If you are upgrading to Relativity Server 2022 and your environment uses a version of Elasticsearch below 6.x to store audits, you have the option to upgrade to Elasticsearch 7.x. Relativity versions below Server 2021 are limited to Elasticsearch 6.x, per the compatibility matrix below. We recommend first upgrading Relativity and then upgrading Elasticsearch using one of the workflows on this topic.

The following table breaks down Elasticsearch compatibility per Relativity version:

Relativity version	Elastic version
10.1 and lower	2.3.3
10.2	2.3.3 and 6.x
10.3	2.3.3 and 6.x
Server 2021	2.3.3 and 6.x and 7.12.1
Server 2022	2.3.3 and 6.x and 7.12.1

Pre-upgrade steps

Download the IncrementalUpgrade7x.zip from the Relativity Community in the Misc. Customer Support Files library under the Files tab. This package has all the scripts required for the upgrade. We recommend starting off with a new cluster with a number of nodes equal to the number of primary and replica shards in your cluster. For example, if your current cluster has 1 primary and 1 replica shard, your cluster requires 2 nodes. These nodes should be the same size as the nodes on the existing cluster, ensuring the indexes from the node have enough room to be allocated onto the new destination cluster.
Note: You can find the number_of_shards and number_of_replicas count using the Elastic API. Send a GET request to the following URL: (https://<host>:<port>/_template/audit).
Whitelist the source cluster so that it can be re-indexed into the destination cluster:
1. Open the elasticsearch.yml (<elastic_install_directory_path>\RelativityDataGrid\elasticsearch-main\config) file from a node where you are running the re-index script.
2. Update the reindex.remote.whitelist parameter to allow list the endpoint of the source cluster. For example:
```
reindex.remote.whitelist: emttest:9200
```
  You can set this to a comma delimited list of allowed remote host and port combinations (e.g. otherhost:9200, another:9200, 127.0.10.*:9200, localhost:*). You must configure the whitelist on all nodes.
3. Re-start the cluster. You only need to restart the cluster the first time you complete this step.
If you do not already have it installed and configured, install the latest version of Python - 3.7.4 and select Add Python 3.7 to PATH during the installation.
1. Run Powershell as administrator, and then navigate to the <python_install_directory_path>\Python37\Scripts folder where Python is installed.
2. Run the following to install the package
```
.\pip3 install elasticsearch
```
Ensure you have SQL access at the EDDS level to run SQL scripts.

Upgrading from Elasticsearch 2.3.3 to 7.x

Upgrading to Elasticsearch 7.x consists of two steps:

Re-indexing into the new cluster.
Running the incremental upgrade script.

Re-indexing into the new cluster

In order to upgrade from Elasticsearch 2.3.3 to 7.x, you must incrementally move all the workspace indexes on the existing 2.3.3 cluster to the newly provisioned 7.x cluster. You can do this in one of three ways.

To move workspace indexes off of a specific node identified from the source cluster, complete the following:

From the destination cluster, run the identify_shards_to_move_off_node.py script to move workspace indexes off the node you identified from the source 2.3.3 cluster. You can find the name of the node using the Elastic API. Send a GET request to the following URL: https://<host>:<port>/_cat/nodes?v

This script identifies indexes to re-index, taking into account the space available on the destination 7.x cluster.

To run the script, complete the following:

Run Powershell as an administrator.
Navigate to the directory of the script:
```
 IncrementalUpgrade\python
```

Execute the following to run the script:

Note: Enter the username and password for the source cluster in --source_user and --source_password. Enter the username and password for the destination cluster in --destination_user and --destination_password.

py identify_shards_to_move_off_node.py --source_endpoint "https://2xendpoint:9200" --source_user "test" --source_password "test123" --destination_endpoint "https://6xendpoint:9202" --destination_user "test" --destination_password "test123" --node_name "P-DV-VM-CUB8DAL"

The script outputs a list of workspaces identified to be re-indexed. This list is located in IncrementalUpgrade\python\workspaces_to_reindex.txt.

Before running the reindex.py script, you must disable audit migration for the workspace(s) you are re-indexing so that audits don't write to the 2.3.3 cluster during the re-index process. To disable audits, complete the following:

Open the DisableDeletionAndMigration.sql script (IncrementalUpgrade\sql) in a text editor.
Edit the @workspaceIds SQL variable with the Artifact IDs of the workspaces you want to re-index.
Execute the script.

Once you complete the above steps, you can run the re-index.py script from the destination cluster to re-index the workspace indexes. To run the script:

Notes:

Enter the username and password for the source cluster in --source_user and --source_password. Enter the username and password for the destination cluster in --destination_user and --destination_password.
The parameter --thread_count is set to 10 by default. You can increase or decrease this value.
If you set --delete_source_indices to "True", set --exclude_node_name to the node from which you moved the indexes. That node will then be excluded from the cluster.

py reindex.py --source_endpoint "https://localhost:9200" --source_user "test" --source_password "test123" --destination_endpoint "https://localhost:9202" --destination_user "test" --destination_password "test123" --delete_source_indices "True" --thread_count "10" --exclude_node_name "P-DV-VM-CUB8DAL"

To move specific workspace indexes, complete the following:

Open the DisableDeletionAndMigration.sql script (IncrementalUpgrade\sql) in a text editor.
Edit the @workspaceIds SQL variable with the Artifact IDs of the workspaces you want to re-index.
Execute the script.

Once you complete the above steps, you can run the re-index.py script from the destination cluster to re-index the workspace indexes. To run the script:

Notes:

Enter the username and password for the source cluster in --source_user and --source_password. Enter the username and password for the destination cluster in --destination_user and --destination_password.
Set the --delete_source_indices "True" parameter if you want to automatically delete the indexes on the source cluster after the re-indexing to free up space. If you don't set this parameter, you must manually delete the indexes on the source cluster using the Elastic API. For more information, see Deleting the source index.
The parameter --thread_count is set to 10 by default. You can increase or decrease this value.

reindex.py --workspaces "1015024,1021384,1062116,1021304,1014823,1062116,1018427,1021385,-1" --source_endpoint "https://2xendpoint:9200" --source_user "test" --source_password "test123" --destination_endpoint "https://6xendpoint:9200" --destination_user "test" --destination_password "test123" --delete_source_indices "True" --thread_count "10"

Deleting the source index

If you didn't set the --delete_source_indices "True" parameter to automatically delete the indexes on the source cluster, you must manually delete the indexes using the Elastic API.

Running the incremental upgrade script

To complete the upgrade, run the incremental_upgrade_setup.sql script (\IncrementalUpgrade\sql) from a SQL editor at the EDDS level. This script sets up the Audit application to write and read from two endpoints during the upgrade.

Edit the following variables in the script:

@primaryShards - enter the number of primary shards. The default value is 1.
@replicaShards - enter the number of primary shards. The default value is 1.
@previousAuditDataGridEndpoint - enter the previous Audit endpoint. For example, https://2xendpoint:9200..
@previousElasticSearchMajorVersion - enter the previous Elasticsearch major version. For example, if you are on Elasticsearch 2.3.3, enter 2.
@currentAuditDataGridEndpoint - enter the current Audit endpoint. For example, https://6xendpoint:9200.
@currentElasticSearchMajorVersion - enter the current Elasticsearch major version. In this case, enter 6.
@workspaceIds - enter a comma separated list of the Artifact IDs of the workspaces identified for upgrade.

Rolling upgrades

You have the option of performing a rolling upgrade of Elastic, specifically from versions 6.6.0 and 6.8.13 directly to 6.8.22 or 7.12.1, by accessing the rolling upgrade .exe files from the Relativity Community.

Those files can be found here:

ElasticRollingUpgrade_7.16.3

Updating the instance setting

Once you complete the upgrade, you must also update the DataGridEndpointinstance setting with the endpoint for the Elasticsearch cluster. For example, https://6xendpoint:9200. This value is required to activate Data Grid operation for the Relativity instance.