Short message search index (Advanced Access)

Short message search index enables you to perform more complex searches on RSMF documents processed in Relativity. It provides enhanced accuracy by storing and searching on the message-level and event-level metadata by using Elasticsearch. For more information on searching short message metadata after building a search index, see Short message search (Advanced Access).

See related pages:

While Elasticsearch is similar to dtSearch and filtering, there are differences:

	Description	Example	Search precision
Elasticsearch	Searches on message-level and event-level metadata.	If you enter a keyword without writing a field-specific query, Relativity searches all message-level fields, including the message body. Searching for "John" returns results that match that term everywhere Relativity finds it. Such as in the Sender Display Name and Message Body fields.	High
dtSearch	Searches cannot search specific aspects.	If you searched "John Smith," the results would show that the phrase "John Smith" appears in the document, but not if John specifically sent a message, reacted, left the chat, and so forth.	Medium
Filter	Searches on document-level fields only.	If you want to find documents that have a message sent by John Smith, the closest you can get is by checking the Participants field. The Participants field tells you if John Smith was in the DM or channel at any point in time.	Low

Installing the application

To start searching on short message metadata, you must first install the Search AI app to your workspace.

Navigate to the workspace where you want to install the application.
Navigate to the Relativity Applications tab.
Click New Relativity Application to display an application form.
Click the Select from Application Library radio button in the Application Type section.
Click the ellipses icon next to the Choose from Application Library field.
Select Search AI in the Select Library Application list. This list only displays applications added to the Application Library. If Search AI is not included in the list, see Installing applications.
Click Ok to display the application in the Choose from Application Library field. The application form also displays the following fields:
- Schema Version—displays the version of the application that you are installing.
- User-friendly URL—displays a user-friendly version of the application's URL. This field may be blank.
- Application Artifacts—displays object types and other application components by clicking the plus icon.
- Map Fields—there are no fields available in Search AI for mapping.
Click Import to install the Search AI app into the workspace.
Review the import status of the application. Verify that the installation was successful or resolve errors. See Viewing import status and Troubleshooting application installation errors.

The Search AI app is now in your workspace. You can begin building an index.

Short message search index workflow

The short message search index workflow is similar to other search indexes. You must first build an index before being able to search it. For more information on searching short messages, see Short message search (Advanced Access).

Although Elasticsearch works with messages and events, it returns results at the document level.

Documents returned may contain one or more hits at the message or event level.

Building the full index

To build the full index, perform the following steps:

All RSMF documents processed in Relativity before to May 8, 2025 must be republished before building the full index.

Navigate to the Search Indexes tab.
Select All Search Indexes from the View drop-down next to the New Search Index button.
Select the Elasticsearch entry.
Click Build Full Index located in the Build and Manage Index console.

The Status field shows Completed when the build finishes successfully. The build process includes all RSMF documents in the workspace, so you do not have to select a saved search or specific indexed fields.

Below are the possible statuses when building a short message index:

In progress—states that the index is currently building.
Completed—the build finished successfully.
Failed—indicates 50% or more of the documents failed. Review the Retry Errors report for details. See Retry errors for more information.
Completed, Eventing active with errors—indicates an error on a message-level coding decision. Resolve the error before continuing. See Retry errors for more information.

After the full index build is complete, you can run incremental index builds as needed instead of performing full builds. For more information, see Running incremental index builds.

Refer to Creating a short message search to start searching on the index results from the Documents list.

Running incremental index builds

After running a successful full index build, you can run incremental builds. Incremental builds only run on new RSMF documents processed into the workspace and will add them to the index. As a result, incremental builds run faster than a full build.

Consider the following items when running an incremental build:

Incremental builds are only possible when the index status is Active. For more information, see Index Status.
Incremental builds do not index any changes made to existing document data since the last full build.
Canceling an incremental build before it finishes deletes the existing index data. As a result, you must run a full index before rebuilding.

To start an incremental build:

Click Build Incremental Index located in the Build and Manage Index console.
In the Incremental build confirmation window, select the checkbox if you want to activate the index once it's complete. Otherwise, leave it blank.
Click Yes to begin the incremental build.

The Index Manager Agent will start indexing the job as soon as it becomes available.

Upon successful completion, the newly indexed files merge with the existing search index. If the job is unsuccessful, click Retry Errors located in the Errors and Details console. See Retry errors.

Automating incremental index builds using Automated Workflows

Automated workflows can trigger an incremental index build when the system adds new documents into a workspace. For more information, see Automated Workflows.

To set up the Automated Workflow:

Navigate to the Automated Workflows tab.
Click New Automated Workflow.
Select New Documents Added as the trigger.
Click Add Action and select Build Elasticsearch Index.
Select the Elasticsearch index from the drop-down menu.
Add a Name and Email for notification on the left panel.

After creating the automated workflow, the building of Elasticsearch indexes is now automated.

Retry errors

To view and retry index build errors, use the options in the Errors and Details console:

Retry Errors—click to retry any documents that failed to index during the earlier build.
Show Document Errors—click to see individual files and their errors on the Errored Documents modal. Click the Document Artifact ID link to display the document in the Viewer. Click the Export link to download a .csv file with the error details so that you can fix them and rebuild the index. See Retry errors workflow for more information.
View Documentation—click to navigate to the RelativityOne documentation site.

Retry errors workflow

When errors occur, you can first try to rebuild the index with the errored documents using the Retry Errors option in the Errors and Details console. This may resolve any system issues that may have occurred, allowing the errored documents to index successfully.

If there are still errors, click the Show document errors link to review each document and error. Error categories include:

Indexing Error—a problem occurred while indexing documents. Click Retry Errors to re-index the failed documents.
Coding Error—a problem occurred when updating a message-level coding field. Click Retry Errors to re-index the failed documents.
Metadata Storage Errors—a problem occurred when trying to index the metadata. Click Retry Errors to re-index the failed documents.
Empty File—the file has no data in it. The document file did not process correctly during Processing or an error occurred during data ingestion through Integration Points. Reprocess the document in Processing or Integration Points.

You can export the document information to a .csv file using the Export link. Then, either create a saved search to exclude the documents or proceed with reprocessing them.

Eventing

When you change the fields below using message-level coding in the Viewer, the system triggers an event. Their data is automatically refreshed in the index.

Responsive—this field reflects any message-level coding decisions made on the Responsive field using the coding layout.
Privileged—this field reflects any message-level coding decisions made on the Privileged field using the coding layout.
Notes—this field reflects any message-level coding notes made in the Notes field using the coding layout.

Events are searchable shortly after the "Last updated on" time, so there is no need to build an index.

If an event fails to index, the index status becomes Completed, Eventing active with errors. Click the Show Document Errors link to review the documents in the Errored Documents modal. Then, click Retry Errors to start fixing the issue. For more information, see Retry errors.

Also, if one or more message-level coding decisions did not index, a banner displays on the Documents list page. Navigate to the index details page to retry the documents.