Elasticsearch Upgrade and Migration Guide

Table of Contents

• Purpose
• Upgrade vs Migration
• Upgrade & Migration Strategies
• Recommended Migration/Upgrade Strategies & Version Matrix
• Index Compatibility for Snapshot Restore
• Elasticsearch Upgrade Approaches
  • Upgrade Using Snapshot & Restore
  • Upgrade Using Reindex from Remote
• Post-Upgrade Checklist
• Troubleshooting

Purpose

This document provides high-level, generic guidance for upgrading or migrating Elasticsearch clusters between major and minor versions. It is intended to assist with planning and executing safe, reliable upgrades or migrations, and references official Elastic documentation for detailed, version-specific steps.

Refer to the official Elastic documentation for the most current and detailed procedures:

• Upgrade a deployment or cluster: https://www.elastic.co/docs/deploy-manage/upgrade/deployment-or-cluster/elasticsearch
• Snapshot & Restore: https://www.elastic.co/docs/deploy-manage/tools/snapshot-and-restore
• Prepare to upgrade & Upgrade Assistant: https://www.elastic.co/docs/deploy-manage/upgrade/prepare-to-upgrade

Upgrade vs Migration

Upgrade & Migration Strategies

1. Rolling Upgrade (In-Place)

• Upgrade nodes one at a time within the same major version.
• Supported only within minor or patch versions, or as dictated by Elastic.
• Not suitable for major version jumps.

Applicable when:

• Upgrading within the same major version (e.g., 8.17 → 8.19).

2. Snapshot & Restore

• Back up data from the source cluster to a snapshot repository.
• Restore the snapshot into the target cluster.
• Works only if snapshots and created indices are compatible with the target version.
• See full details here: https://www.elastic.co/docs/deploy-manage/tools/snapshot-and-restore

Applicable when:

• A snapshot infrastructure is available.
• Snapshot and index versions are compatible with the target Elasticsearch version.

3. Reindex from Remote

• Read data from the source cluster and write into a new index on the target cluster using the _reindex API.
• Useful when snapshot restore of older indices is not compatible.

Applicable when:

• Direct snapshot restore is not possible due to version compatibility.

Recommended Migration/Upgrade Strategies & Version Matrix

When planning an Elasticsearch upgrade or migration, choose the strategy based on the source and target versions. The following matrix summarizes recommended approaches:

Legend:

• ✅ = Supported
• ❌ = Not supported

Recommendations:

• Always upgrade to the latest patch of the current major version before attempting a major migration.
• Run the Upgrade Assistant to identify and resolve issues before migration.
• Prefer snapshot & restore for supported direct migrations; otherwise, reindex from remote.
• Rolling upgrades are only supported within the same major version (minor/patch upgrades).
• Test all migrations in a non-production environment before production deployment.

Index Compatibility for Snapshot Restore

Any index restored from a snapshot must also be compatible with the current cluster’s version. Attempting to restore an index created in an incompatible version will cause the restore attempt to fail.

¹ These indices must be reindexed to upgrade them to 7.x or later.
² These indices must be reindexed to upgrade them to 8.x or later.

For the most up-to-date compatibility matrix, see Elastic's official docs: https://www.elastic.co/docs/deploy-manage/tools/snapshot-and-restore#index-compatibility

Elasticsearch Upgrade Approaches

Upgrade Using Snapshot & Restore

When to Use

• Major version upgrades (e.g., 6.x → 8.x / 9.x)
• Cluster replacement or hardware refresh
• When downtime is acceptable

1. Pre-Upgrade Checks

Verify that:

• Cluster health status is green.

  Copy

  curl -k -u <USER>:<PASSWORD> -X GET https://<ES_HOST>:9200/_cluster/health
  

  Expected responses (examples):

  Copy

  {
    "cluster_name": "es-cluster",
    "status": "green",
    "number_of_nodes": 3,
    "active_shards": 120
  }
  

  Warning
  In development environments, cluster health may show yellow due to single-node setup or unassigned replicas. This is expected.
  For production upgrades, ensure the cluster health status is green before proceeding.

• No snapshot or reindex tasks are currently running.

  Copy

  # Filter reindex tasks
  curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_tasks?detailed=true&actions=*reindex"
  
  
  # Filter snapshot tasks
  curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_tasks?detailed=true&actions=*snapshot*"
  

  Expected responses (examples):

  Copy

  {
    "nodes": {}
  }
  

2. Configure Snapshot Repository

elasticsearch.yml

path.repo: ["\\\\sharedPath\\es-backups"]

Restart all nodes after updating the configuration.

Create repository:

curl -k -u <USER>:<PASSWORD> -X PUT "https://<ES_HOST>:9200/_snapshot/es_backup_repo" -H 'Content-Type: application/json' -d '
{
  "type": "fs",
  "settings": {
    "location": "\\\\sharedPath\\es-backups",
    "compress": true
  }
}'

Expected response:

{ "acknowledged": true }

Verify repository:

curl -k -u <USER>:<PASSWORD> -X POST "https://<ES_HOST>:9200/_snapshot/es_backup_repo/_verify"

Expected response (example):

{ "nodes": { /* ... node details ... */ }  }

3. Take Snapshot

curl -k -u <USER>:<PASSWORD> -X PUT "https://<ES_HOST>:9200/_snapshot/es_backup_repo/pre_upgrade_snapshot" -H 'Content-Type: application/json' -d '
{
  "indices": "app-index-*,logs-*,metrics-*",
  "ignore_unavailable": true,
  "include_global_state": false
}'

Expected response (example):

{ "accepted": true }

Check snapshot status:

curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_snapshot/es_backup_repo/pre_upgrade_snapshot"

Expected response (example):

{
  "snapshots": [
    { "snapshot": "pre_upgrade_snapshot", "state": "SUCCESS", "indices": ["my-index"] }
  ]
}

4. Install New Elasticsearch Version

• Stop Elasticsearch Service on All Nodes

  Copy

  net stop <service-name>
  

  Or using PowerShell:

  Copy

  Stop-Service <service-name>
  

• Uninstall Elasticsearch on All Nodes Run from Elasticsearch install directory

  Copy

  bin\elasticsearch-service.bat remove <service-name>
  

• Install new Elasticsearch version
  Refer to the Installation Guide

5. Configure Snapshot Repository on New Version

elasticsearch.yml

path.repo: ["\\\\sharedPath\\es-backups"]

Restart all nodes after updating the configuration.

Register repository again:

curl -k -u <USER>:<PASSWORD> -X PUT "https://<ES_HOST>:9200/_snapshot/es_backup_repo" -H 'Content-Type: application/json' -d '
{
  "type": "fs",
  "settings": {
    "location": "\\\\sharedPath\\es-backups"
  }
}'

Expected response:

{ "acknowledged": true }

Verify repository:

curl -k -u <USER>:<PASSWORD> -X POST "https://<ES_HOST>:9200/_snapshot/es_backup_repo/_verify"

Expected response (example):

{ "nodes": { /* ... node details ... */ }  }

6. Restore Snapshot on New Cluster

Restore snapshot:

curl -k -u <USER>:<PASSWORD> -X POST "https://<ES_HOST>:9200/_snapshot/es_backup_repo/pre_upgrade_snapshot/_restore" \
-H "Content-Type: application/json" -d '
{
  "indices": "app-index-*,logs-*,metrics-*",
  "ignore_unavailable": true,
  "include_global_state": false
}'

Expected response (example):

{ "accepted": true }

Monitor restore:

curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_cat/recovery?v"

Expected response (example):

index recovery_source shard time
my-index snapshot        0    30s

7. Post-Restore Validation

Verify that:

• Cluster health status is green.

  Copy

  curl -k -u <USER>:<PASSWORD> -X GET https://<ES_HOST>:9200/_cluster/health
  

  Expected responses (examples):

  Copy

  {
    "cluster_name": "es-cluster",
    "status": "green",
    "number_of_nodes": 3,
    "active_shards": 120
  }
  

  Warning
  In development environments, cluster health may show yellow due to single-node setup or unassigned replicas. This is expected.
  For production upgrades, ensure the cluster health status is green before proceeding.

• Verify indices.

  Copy

  curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_cat/indices?v"
  

  Expected Responses - Indices List (examples) :

  Copy

  #/cat/indices sample output
  health status index        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
  green  open   my-index     someUuid               5   1       1000            0     10mb          5mb
  

Upgrade Using Reindex from Remote

When to Use

• Snapshot restore is not supported between versions
• Need selective index migration
• Zero or minimal downtime required

1. Prepare New Elasticsearch Cluster

• Install new Elasticsearch version
  Refer to the Installation Guide
• Verify that cluster health status is green

curl -k -u <USER>:<PASSWORD> -X GET https://<ES_HOST>:9200/_cluster/health

Expected response (example):

  {
    "cluster_name": "es-cluster",
    "status": "green",
    "number_of_nodes": 3,
    "active_shards": 120
  }

2. Whitelist Old Cluster for Reindex

On the new cluster, update elasticsearch.yml:

reindex.remote.whitelist: ["old-es-host:9200"]

Restart Elasticsearch.

3. Reindex Individual Index

curl -k -u <USER>:<PASSWORD> -X POST "https://<ES_HOST>:9200/_reindex" \
-H "Content-Type: application/json" \
-d '{
  "source": {
    "remote": {
      "host": "https://<OLD_ES_HOST>:9200",
      "username": "<OLD_USER>",
      "password": "<OLD_PASSWORD>"
    },
    "index": "source_index"
  },
  "dest": {
    "index": "target_index"
  }
}'

4. Reindex All Indexes (One by One)

Example script logic:

curl -k -u <USER>:<PASSWORD> -X GET "https://<OLD_ES_HOST>:9200/_cat/indices?h=index"

Expected response (example):

my-index
another-index

Reindex each index individually by running the _reindex API for each index.

5. Monitor Reindex Progress

curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_tasks?detailed=true&actions=*reindex"

Expected response (example):

{ "nodes": { /* task details */ } }

6. Post-Reindex Validation

Verify that:

• Cluster health status is green.

  Copy

  curl -k -u <USER>:<PASSWORD> -X GET https://<ES_HOST>:9200/_cluster/health
  

  Expected responses (examples):

  Copy

  {
    "cluster_name": "es-cluster",
    "status": "green",
    "number_of_nodes": 3,
    "active_shards": 120
  }
  

  Warning
  In development environments, cluster health may show yellow due to single-node setup or unassigned replicas. This is expected.
  For production upgrades, ensure the cluster health status is green before proceeding.

• Verify indices.

  Copy

  curl -k -u <USER>:<PASSWORD> -X GET "https://<ES_HOST>:9200/_cat/indices?v"
  

  Expected Responses - Indices List (examples) :

  Copy

  #/cat/indices sample output
  health status index        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
  green  open   my-index     someUuid               5   1       1000            0     10mb          5mb
  

Compare document counts between old and new clusters.

Post-Upgrade Checklist

• Cluster health is green
• All required indices restored/reindexed
• Applications connected successfully
• Old cluster decommissioned (after validation)

Troubleshooting

• Mapping conflicts: inspect index mappings with GET <index>/_mapping and resolve conflicts before reindexing.
• Snapshot failures: inspect repository verification and Elasticsearch logs; use GET _snapshot/<repo>/_status.
• Reindex slow or stuck: inspect _tasks and consider slicing, throttling, or increasing resources.
• Plugin incompatibility: verify that required plugins are installed and compatible on the target cluster.

Rollback

• Keep snapshots until validation is complete.
• To rollback, restore the snapshot to the original cluster or redeploy the previous cluster using saved configuration and snapshots.
• Verify application functionality after rollback.