Running text migration

To run a text migration job, complete the following steps: 

  1. Identify workspaces to migrate
  2. Identify long text fields to migrate
  3. Create a text migration job
  4. Run the breakage report
  5. Create and/or activate a dtSearch index
  6. Resolve saved searches and views
  7. Run text migration
  8. Review and resolve migration errors

Identifying workspaces to migrate

Identify the workspace(s) you want to migrate. Be sure to review the special considerations for text migration as well as the list of supported and unsupported functionality for Data Grid. For more information, see Considerations.

Identifying long text fields to migrate

Identify which long text field(s) you want to migrate. You can migrate up to three fields per migration job.

Creating a text migration job

To create a text migration job, complete the following:

  1. Navigate to the Text Migration Jobs tab underneath the Data Grid tab.
  2. Click New Text Migration Job.
  3. Complete the following fields:

    • Name - enter a name for the migration job.
    • Workspaces - click , and then select the from the list on the left the workspaces you want to migrate. Use the arrows to move your selection(s) to the list on the right. Once you are finished, click Save.
    • Fields - click , and then select the from the list on the left the fields you want to migrate. This list contains all long text fields that aren't Data Grid enabled. Use the arrows to move your selection(s) to the list on the right. Once you are finished, click Save.

      Note: You can migrate up to three fields per job.

    • Email notification Recipients - enter the email address(es) of the recipient(s) you want to send a notification to when your migration job finishes running. Separate entries with a semicolon.
  4. Click Save. When you first save the job, it has a default status of Pending.
  5. Once you are ready to start the job, click Start Job on the console.

Running the Breakage Report

To run the Breakage Report, click Breakage Report from the text migration console. The Breakage Report provides a list of all views, saved searches, indexes, and custom objects that will no longer function once you migrate SQL text to Data Grid. We recommend resolving these issues before running a migration job.

One of the primary functions of the Breakage Report is to identify areas where any of the objects above are tied to a keyword search or using keyword search specific functions like Contains or Does Not Contain. Most of the issues identified can be resolved by modifying the object to use an active dtSearch index.

The Breakage Report contains the following columns:

  • Workspace ID - the Artifact ID of the workspace.
  • Workspace Name - the name of the workspace.
  • Name - the name of the view, saved search, index, or custom object that will break upon migration.
  • Object Type - search, view, index, or custom object.
  • Owner - the owner of the view, saved search, index, or custom object that will break upon migration.
  • Field - the name of the field causing the object to break.
  • Operator – the search operator (is, is not, etc.) incompatible with Data Grid.
  • Term – the text string used in the broken search.

Creating and/or activating a dtSearch index

Most of the issues identified by the breakage report can be resolved by modifying the object to use an active dtSearch index. To create or activate a dtSearch index, see Creating a dtSearch index.

Note: Ensure all fields you want to search on are included in the searchable set for the dtSearch index. You can include a combination of Data Grid and SQL fields in your saved search.

The Data Grid Text Migration application includes logical to automatically fix active dtSearch indexes using the <all documents in workspace> searchable set. This logic only applies if the fields being migrated have the Include in Text Index field set to Yes.

The Data Grid Text Migration application creates a saved search called SS_TextMigration_[JobName] which contains all fields where Include in Text Index is set to Yes. It also creates a new dtSearch index called Migration_[existing dtSearch index name] which uses this new saved search. Once text migration completes, navigate to the old dtSearch index and click Swap Index. Select Migration_[existing dtSearch index name] as the replacement index.

Resolving saved searches and views

Use the results of the breakage report to help you resolve index or field-level search issues that affect saved searches and views

Note: We recommend copying saved searches prior to making edits.

Resolving index searches

  1. Navigate to the saved search.
  2. Right-click on the name, and then click Edit.
  3. Click the (Index Search) condition.
  4. Copy the terms in the Search Terms field.
  5. Change the Index field to an active dtSearch index.
  6. Paste the search terms if needed.
  7. Click Apply.
  8. Click Save & Search.

Resolving field-level searches

  1. Navigate to the saved search.
  2. Right-click on the name, and then click Edit.
  3. Click the condition using the field being migrated (for example, extracted text).
  4. Copy the text query, and note the operator (is, is like, contains).
  5. Click Add Condition, and then select (Index Search).
  6. Next to Index, select an active dtSearch index.
  7. Paste the query text you copied in step 4. Depending on the operator that was used, you may need to convert your search. For more information, see Search operator conversion.
  8. Click Apply.
  9. Remove the field condition.
  10. Click Save & Search.

Search operator conversion

Data Grid only supports the IS SET and IS NOT SET operators. If your field-level search uses another operator, you need to convert the search as follows:

Operator Example search Conversion
is Jane has a broken search "Jane has a broken search"
is not Jane has a broken search NOT "Jane has a broken search"
is set Supported operator No modification required
is not set Supported operator No modification required
is less than N/A N/A
is greater than N/A N/A
is less than or equal to N/A N/A
is greater than or equal to N/A N/A
is like Jane has a broken search *Jane has a broken search*
is not like Jane has a broken search NOT *Jane has a broken search*
begins with Jane has a broken search Paul*
does not begin with Jane has a broken search NOT Paul*
ends with Jane has a broken search *broken search
does not end with Jane has a broken search NOT *broken search
contains Jane has a broken search *Jane has a broken search*
does not contain Jane has a broken search NOT *Jane has a broken search*

Running a text migration job

Once you've resolved or taken note of any items in the breakage report, click Start Job to start the text migration job. If another text migration job is already in progress, the new job is added to the queue and begins as soon as any running or pending jobs complete

The job displays one of the following statuses which you can use to monitor the state of your migration:

  • Pending
  • In Progress
  • Complete
  • Completed with Errors

The Migration Job Details table shows the progress of each workspace in the migration job. This table refreshes every two seconds. Once the job completes or completes with errors, you can filter the table.

The table contains the following columns:

  • Workspace Name - the name of the workspace being migrated.
  • Fields - the name of the field being migrated
  • Migration Status - the status of the field being migrated. The following lists the field migration statuses:
    • Pending
    • In Progress
    • Not In Workspace
    • Already Migrated
    • Completed
    • Completed with Errors
  • Documents Migrated - the count of documents migrated
  • Total Documents - the total number of documents migrated.
  • Errors - the count of documents with errors.

After a successful migration, the application runs a verification step to ensure all documents migrated from SQL to Data Grid. Then, the application drops the SQL column, permanently deleting the migrated text from SQL.

    Notes:
  • If all documents are migrated, but a workspace and field are still marked as In Progress, this mean they are currently being verified.
  • Although the application automatically deletes the column in SQL, you are still required to reclaim the space that was taken up by your long text document field in SQL.

Viewing migration errors

Run the Data Grid Migration Error Report from the Text Migration Reports tab to view migration errors. To run this report, click Run. Click the drop-down to toggle between Field Level Errors and Document Level Errors. Once you've resolved any errors, return to the migration job and click Retry. The job status returns to In Progress and any errors on the fields marked with errors are reset.

Field Level Errors

The following results populate the bottom of the window:

  • Job Name - the name of the migration job where the errors occurred.
  • Workspace Id - the artifact ID of the workspace.
  • Workspace Name - the name of the workspace.
  • Field Id - the Artifact ID of the field with errors.
  • Field Name - the name of the field with errors.
  • Error Message - the text of the error.

Document Level Errors

The following results populate the bottom of the window:

  • Job Name - the name of the migration job where the errors occurred.
  • Workspace Id - the artifact ID of the workspace.
  • Workspace Name - the name of the workspace.
  • Document ID - the artifact ID of the document.
  • Field Name - the name of the field with errors.
  • Error Message - the text of the error.