Propagate coding post-import

Note: Some of the recipes listed below now exist as knowledge base articles on the Relativity Community. When you click those links, you must enter valid Community credentials to access those articles. The remaining recipes will soon be integrated into their corresponding feature documentation so that they'll show up either as new topics or new headings within existing topics. Once this relocation is complete, we will be deprecating this home page, and all of the content below will be accessible via search on our sites.

You must have valid Relativity Community credentials in order to download any Community file linked to the documentation site. You'll need to enter those credentials on the Community login screen if you're not already logged in. If you're already logged in to the Community at the time you click a link, the file is automatically downloaded in the bottom left corner of your screen. If you get an error message stating "URL No Longer Exists" after clicking a Community link, it may be due to a single sign-on error related to the SAML Assertion Validator, and you should contact your IT department.

The Propagate Coding Post-import solutions passes along, or spreads, the value of one or more fields to documents that are part of the same relational group. For more information, see Relational groups. The solution also reports any coding variances within those relational groups. For example, the relational group "EA53" includes one document with a "Responsive" designation. When the solution runs, the other documents with the "EA53" are given the "Responsive" designation . For more information, see Example of propagating.

This page contains the following sections:

Before you begin

The Propagate Coding Post-import solution performs several tasks to help you assign field values from one document to others that are part of the same relational group and returned by a saved search that you specify. You can customize the solution by specifying which field defines relational groups, which fields to propagate values to, and whether and how to address any coding variances within relational groups. The solution also reports coding variances, which can help you work with reviewers to address the variances. This solution provides two primary ways to propagate values. The first is a set of scripts, agents, and other components that propagate values to multiple fields when you run the solution manually or automatically according to a schedule that you define. The second is a script that propagates values to only one field when you run the script manually, "on demand." To learn more about the script, see Propagating to one field on demand.

Supported versions

This solution is supported in Relativity 9.0 – Server 2021 and RelativityOne.

Click on any of these links to download the appropriate version from the Relativity Community:

Solution version Supported Relativity version
15.1.1.0 9.5-Server 2021, RelativityOne
14.3 9.5.292.12, 9.5.411.4
14.1 9.5.162.111 - 9.5.259.2
13.5 9.0 - 9.4.343.65

Components

This solution consists of the following components:

  • Event handlers
  • Custom agents
  • Custom pages
  • Relativity script that runs at the environment level

Configuration

To use this solution, the Relativity Services API must be configured on your web server. To ensure that the Relativity Services API is configured correctly, please contact Relativity Client Services at support@relativity.com.

Considerations

Before you deploy and run the solution, it's important to keep the following in mind:

  • Audit records are created for each document that is updated by the propagate coding worker agent. The audit records have an action of "Update", not "Update - Propagation", and the associated user is the Relativity Service Account.
  • You can install multiple propagate coding worker agents in one Relativity instance. Each agent picks up the next available job and runs it to completion.
  • This solution changes field values and those changes cannot be undone. We recommend that you back up the database before you run the solution.
  • Solution performance depends primarily on the number of documents in the workspace and the number of documents that are updated. To optimize performance, we recommend that you run propagation jobs based on saved searches that return no more than 50,000 documents and return unique sets of documents. For example, don't reference the same saved search in more than one propagation job.
  • This solution doesn't support the client domains (multi-tenancy) and Data Grid features of Relativity.

Deploying and configuring the solution

To deploy and configure the solution in a Relativity instance, you perform three basic tasks:

  1. Install the Relativity application.
  2. Create and enable agents that monitor and perform propagation tasks.
  3. Configure email notifications.

Adding the application to the Application Library

To add the application to the Application Library:

  1. On the Applications & Scripts tab, click the Application Library tab.
  2. Click Upload Application.
  3. Click Browse, navigate to and select the RA_Propagate_Coding_Post-Import.rap file, and then click Open.
  4. Click Save.
  5. Click Install in the Workspaces Installed section to install the application on workspaces.
  6. Click in the Workspaces field to display the Select Workspaces dialog.
  7. Select the workspaces where you want to install the application, and then click Ok.
  8. Click Clear to remove a workspace from the list.
  9. Click Save to install the application to the selected workspaces. These workspaces now contain the application. Relativity lists the workspaces in the Workspaces Installed section on the detail view of the application.

Creating and enabling the propagation agents

After you add the solution to the application and script libraries, you're ready to create and enable a Propagate Coding Manager agent, which monitors propagation tasks, and one or more Propagate Coding Worker agents, which perform propagation tasks. Note that only one Propagate Coding Manager agent can be enabled at a time. To optimize solution performance however, you can create and enable multiple Propagate Coding Worker agents.

To create and enable the Propagate Coding Manager agent:

  1. Click the Server & Agent Management tab, and then click the Agents tab.
  2. Click New Agent.
  3. Complete the following fields:
    • Agent Type - click Ellipsis (...) button, select the Propagate Coding Manager - KCD_1040835 radio button, and then click OK.
    • Number of Agents - enter 1.
    • Agent Server - click Ellipsis (...) button, select the radio button for the server that you want the agent to run on, and then click OK.
    • Run interval - enter 5.
    • Logging level - select the type of information that you want to log for the agent.
    • Status - click Yes to enable the agent.
  4. Click Save and Back.

The "Propagate Coding Manager - KCD_1040835" agent now appears in the list of agents.

To create and enable a Propagate Coding Worker agent:

  1. On the Agents tab, click New Agent.
  2. Complete the following fields:
    • Agent Type - click Ellipsis (...) button, select the Propagate Coding Worker - KCD_1040835 radio button, and then click OK.
    • Number of Agents - enter the number of instances of the propagation worker agent you want to run. The default value is "1".
    • Agent Server - click Ellipsis (...) button, select the radio button for the server that you want the agent to run on, and then click OK.
    • Run interval - enter 5.
    • Logging level - select the type of information that you want to log for the agent.
    • Status - click Yes to enable the agent.
  3. Click Save and Back.

The "Propagate Coding Worker - KCD_1040835" agent now appears in the list of agents.

Setting up email configuration

You can configure the email address you want to notify users from. Update the ‘Email From’ column in the Configuration table for Relativity 9.2 and below, or the Instance Details table for Relativity 9.3 and above with a desired and valid email address to set this up. This configuration is part of the kCura.Notification section in the table.

If the value is not set or the configuration is missing, the agent sends a notification via the Message column that there is a missing configuration. If no messages appear in the message column, navigate to the event viewer of your agent server and look for errors. The following screen shot shows the error that appears in the event viewer if the Email From field is misconfigured.

Preparing the workspace

After you deploy the solution to the Relativity instance, you're ready to install and configure the solution in a workspace by performing a few basic tasks:

  1. Create and configure a propagation processing job.
  2. Optionally define a schedule for running the propagation processing job.

Creating and configuring the propagation job

To create and configure a propagation processing job:

  1. In the workspace, click the Propagate Coding Post-Import tab.
  2. Click New Job.
  3. Complete the following fields:
    • Name - enter a descriptive name for the job. Job names must be unique.
    • Priority - enter a number that indicates the priority of the job in the processing queue. Jobs are sent to the propagate coding worker agent first by priority and then by creation date and time. A lower priority number results in a higher priority for running the job. To learn how to view all jobs and their priority, see Viewing job status and history below.
    • Email notification recipients - enter the email address of each user whom you want to receive an email notification when the job is complete. The notification message provides details about the job, including status and any errors that occurred. To notify multiple users, separate each user's email address with a semi-colon (;).
      An example of the Job page with propagation settings.
  4. Click Save.
  5. In the console, click Configure Job.
  6. Complete the following fields:
    • Relational Field - select the name of the relational field that defines groups of related documents to propagate values to.
    • Saved Search - select the saved search that contains the documents you want to propagate values to. Ensure that the saved search returns all related documents; including the documents that values will be propagated to as well as documents that values will be propagated from. Values are propagated only to documents that are returned by the specified saved search.
    • Normalize Multi-Choice Fields - when selected, multiple choice fields are coded as a combination of values in the relational group instead of only propagating the value of a parent document.
    • Select Fields To Propagate – select the checkbox next to each field to propagate values to.
  7. Click Save.

After you create and configure the propagation job, you can run it manually or optionally define a schedule to run it automatically at specific times.

Defining a schedule for running the propagation job

If you prefer to run the propagation job automatically at specific times, you can define a schedule for it. Note that this is optional. You always have the option of running the job manually by clicking Start Job in the job console. If you choose to define a schedule, you can also override the schedule and run the job manually at any time by clicking Start Job in the console.

To define a schedule for running the job:

  1. In the console on the job page, click Set Schedule.
  2. Under Frequency, complete the following fields:
    • Occurs - the base time interval for running the job. Currently, Daily is the only available option.
    • Recurs Every - specify how often you want to run the job based on the selected time interval. For example, type 1 to run the job every day, type 2 to run the job every other day, and so on.
  3. Under Daily Frequency, complete the following fields to specify how often and when you want to run the job during the specified time interval:
    • Occurs Once – select this option to run the job only once a day.
    • Occurs Every – select this option to run the job multiple times a day according to a specific interval, and then select the interval.
    • Between – in the first field, enter the earliest time of day to run the job. In the second field, optionally enter the latest time of day to run the job. The first Between field is required if the Occurs Every option is selected and optional if the Occurs Once option is selected.

      Note: If a job is running when the end time occurs, the job finishes processing the current set of documents, sets the job status to "Complete", and resumes processing when the next available time frame begins. If you don't enter an end time, the job runs until it finishes processing all documents.

  4. Under Duration, complete the following fields to specify a date range for running the job:
    • Start Date - indicates the first date to run the job. This field is required.
    • End Date or No End Date- indicates the date to end the job recurrence. To stop running the job on a specific date, click End Date, and then select the date. To continue running the job at the specified frequency and times indefinitely, click No End Date.
    An example of the Set Schedule page with schedule settings for the propagation job.
  5. Click Save.

In the Schedule section of the job page, the Description field displays a summary of the schedule that you defined and the Next Run field indicates the next time when the job is scheduled to run.

Running the solution

This solution runs primarily through the propagation job that you created and configured when you prepared the workspace. This means that you have two options for running the solution, depending on whether you defined a schedule to run the job automatically. You can run the job manually or wait for the next scheduled run to finish. To check the status of a job, you can run a script that's included in this solution to generate a report with the history and details of propagation jobs.

Running the job manually

To run the job manually:

  1. In the workspace, click the Propagate Coding Post-Import tab.
  2. Click the name of the propagation job.
  3. In the console, click Start Job.

The job is added to the queue with a high priority and is run when a propagation worker agent becomes available.

Viewing job status and history

When jobs are sent to the processing queue, they are ordered first by priority and then by creation date and time across the Relativity instance. This solution includes a script that you can run to view a report of all the propagation jobs for the Relativity instance as well as details about each one, including status, priority, and workspace.

To view a report of propagation jobs:

  1. Click the Applications & Scripts tab, and then click the Relativity Script Library tab.
  2. Click the name of the Propagate Coding Post-Import: View Jobs script.
  3. In the console, click Run Script, and then click Run in the pop-up window that appears. The page displays the report.
    An example of a job report that lists details about all the jobs that exist in the Relativity instance.

The report lists all the jobs that exist in the Relativity instance and are associated with a Propagate Coding Post-Import application, and the order in which they'll be executed.

Viewing the results

To view the results of running the solution, navigate to any view or layout that contains the fields that you set to propagate when you configured the propagation job. For each field that's set to propagate, the results are one of the following:

  • Documents updated - If all the documents in a relational group had the same value for a field, the value was propagated to all other documents in the group where the field was not set.
  • Documents not updated - If documents in a relational group didn't have the same value for a field, no values were propagated to any documents in the relational group.

To view a detailed report of the results, you can run the "Propagate Coding Post-Import Last Run Results" script that's included in this solution. The script generates a report indicating the number of documents that were updated for each field and other information about those updates.

To run the script:

  1. In the workspace, click the Case Admin tab, and then click the Scripts tab.
  2. Click the name of the Propagate Coding Post-Import Last Run Results script.
  3. In the console, click Run Script, and then click Run in the pop-up window that appears. The page displays the report.
    An example of a report that provides information about the changes that were made.

In addition to viewing the results directly in Relativity or by running the results script, you might also receive an email notification each time a propagation job finishes running. When you created the propagation job, you had the option of entering the email address of each person whom you want to receive an email notification when the job finishes running. The notification message provides details about the job, including any errors that occurred. The following is an example of a notification message.

An example of a notification message that contains details about a propagation job.

Viewing coding variances

To help you identify and research instances where documents weren't updated due to coding variances, this solution creates a Multiple Object field (on the Document object) named “Variance Indicator – job name”. You can use the field in a saved search condition to view coding variances.

To view coding variances:

  1. In the workspace, create a new saved search.
  2. Under Conditions, click Variance Indicator - job name in the Field drop-down list, and then click these conditions in the Operator drop-down list.
  3. Under Value, click Ellipsis (...) button, and then create a value with the following criterion:
    • Field – Variance Indicator - job name
    • Operator – any of these
    • Value – Each field that is set to propagate

    • An image of the criterion for the condition value of the saved search.
  4. Click OK.
  5. Specify the fields and sort order for the resulting view.
  6. Click Save and Search. The result set appears and, depending on the fields that you added to it, displays coding variances.
    An example of saved search results that lists coding variances across documents.

You can then use the results of the saved search to review and resolve any coding variances before the propagation job runs again.

Deleting job results

After you view the results of one or more propagation jobs, you can optionally delete details about those jobs to ensure that they don't appear in future reports. To do so, run the "Propagate Coding Post-Import Last Run Results Clean Up" script, which is included in the solution.

To delete job results:

  1. In the workspace, click the Case Admin tab, and then click the Scripts tab.
  2. Click the name of the Propagate Coding Post-Import Last Run Results Clean Up script.
  3. In the console, click Run Script.
  4. In the Delete Results Before field, click Ellipsis (...) button, and then click the latest date of the data that you want to delete. Results for all dates before the date that you choose are deleted when the script runs.
  5. Click Run.

Propagating to one field on demand

If you want to quickly propagate values to a single field on demand or you see performance issues when you run the propagation jobs and agents included in the complete Propagate Coding Post-Import solution, you can use the Propagate Coding Post-Import script that's also included in the solution. The script runs at the workspace level and does the following for each relational group of documents that are returned by a specified saved search:

  1. Identifies the parent document or document with the lowest artifact ID and finds the value in a specified field.
  2. Depending on how you configure the Overwrite Child Values setting for the script, propagates the value of the field to all other documents in the same relational group.

If you set Overwrite Child Values to "No", the following occurs:

  • If all the documents contain the same value for the field to propagate, the value is propagated to any documents that are in the relational group and where the field is not set.
  • If the documents contain different values for the field to propagate, no values are propagated to documents in the relational group and coding variances are displayed in a report.

If you set Overwrite Child Values to "Yes", the following occurs:

  • The value of the field for the parent document is propagated to all child documents that are in the relational group and where the field is not set.
  • If the value of the field for a child document is different from the parent document, the value for the child document is overwritten with the value from the parent document.
  • If value of the field is not set for the parent document, the value for all child documents is changed to "not set" to match the parent document.

For a helpful example of how values are propagated in specific scenarios, see the example below.

Running the script

For optimal performance, we recommend running the script with a saved search that returns no more than 100,000 documents. In addition, script performance depends on the type of field that you're propagating data to and, for Single Choice or Multiple Choice fields, the number of choices for the field.

Note: Propagating field values to the same set of documents by using both the script and the propagation agents can produce unexpected results. In addition, you cannot run more than one instance of the script at the same time. In other words, if you start running the script on one computer, you cannot start running the script on another computer until the first instance finishes running.

To run the script:

  1. In the workspace, navigate to the Scripts tab.
  2. Click the name of the Propagate Coding Post-Import script.
  3. In the console, click Run Script.
  4. Complete the following fields:
    • Relational Identifier - select the relational field that defines groups of related documents to propagate values to. The following field types are available: fixed length, long text, date, user, Yes/No, whole number, decimal, currency, single choice, multiple choice, and multiple object.
    • Field to Propagate – Click the name of the field that you want to propagate values to. The field must be one of the following field types: fixed length, long text, date, user, Yes/No, whole number, decimal, currency, single choice, multiple choice, and multiple object.
    • Saved Search - select the saved search that contains the documents you want to propagate values to. Ensure that the saved search returns all related documents; including the documents that values will be propagated to as well as documents that values will be propagated from. Values are propagated only to documents that are returned by the specified saved search.
    • Normalize Multiple Choice Fields - enables overwrite to child values. If you select Yes, multiple choice fields are coded as a combination of values in the relational group. If you select No, multiple choice fields are only propagated if documents in the relational group share the same value.
    • Audit Propagation - select Yes to create an audit record for each change that's made to documents by the script. The resulting audit records have an "Update-Propagation" action and are logged under the user who runs the script. select No if you don't want to create any audit records for changes that are made by the script.
    • Overwrite Child Values – select Yes to propagate the same value to all documents in a relational group, regardless of any variances for the field. Click No to propagate a value to all documents in a relational group only if there are no variances for the field within the relational group.
  5. Click Run.

Viewing the results

After the solution runs, the results appear as a report on the script page.

If Overwrite Child Values is set to Yes:

  • The parent document’s field value is propagated to all child documents in the relational group which do not already contain the same field value.
  • If a child document contains a different field value than the parent document, the child document’s field is overwritten with the value from the parent document.
  • If the parent document field is not set, any child documents which have the field value set are unset to match the parent document. This does not apply to multiple choice fields when the normalization option is enabled.

If Overwrite Child Values is set to No:

  • If there are no coding variances between documents, the value is propagated to the documents in the relational group where the field to propagate is not set.
  • If there are coding variances between documents, the relational group is not propagated and the coding variance appears in a report.

If there weren't any coding variances or the Overwrite Child Values setting was set to "Yes", the report displays the number of documents that were updated. If there were coding variances and the Overwrite Child Values setting was set to "No", the report provides details about those variances.

An example of a script report that provides details about coding variances that were found.

The following table lists and describes the columns in the report.

Column Description
Document with Conflict The identifier of the document.
Relational Identifier The identifier of the relational group.
Field Name The name of the field that contains the coding variance.
Field Value The value of the field for that document.

You can use the report to review and resolve the coding variances.

Example

This example walks through a specific scenario that demonstrates how the "Propagate Coding Post-Import" script propagates values based on various conditions. In the example, the script is run with the following settings:

  • Saved Search - All Documents
  • Relational Identifier - MD5 Hash
  • Field to Propagate - Designation
  • Overwrite Child Values - No

The original coding values for the documents were:

An image of the original coding values for the propagation example.

When the script runs, it does the following:

  • The relational group "EA53B1B19A449C0539B324522D7A5CD8" propagates the "Responsive" choice to all related documents where the Designation field was not set because all related documents that were already set for the field had the value "Responsive".
  • The relational group "11A3AA1E226C273C8D6174E59F49903D" propagates the "Non-Responsive" choice to all related documents where the Designation field was not set because all related documents that were already set for the field had the value "Non-Responsive".
  • The relational group "A95C2F62DE9F4B342ADA216156649027" doesn't propagate any values for the Designation field because there were coding variances within the related documents. For this group, the script generates a report with details about the variances.
    An example a script report that identifies coding variances for a field.

The overall results of running the script are as follows.

An image of the new coding values for the propagation example.

Although relational group "A95C2F62DE9F4B342ADA216156649027" contains coding variances, you might want to overwrite those variances and set the value of the Designation field to "Responsive" for all the child documents because the parent document is coded as "Responsive". You can do this by changing the Overwrite Child Values setting to "Yes" and running the script again. If you do this, no coding variances are reported and the parent document's "Responsive" value is propagated to all child documents, as shown below.

An image of the final coding values for the propagation example, after running the script again with a different Overwrite Child Values setting.

Upgrading this application

Propagate coding post-import can be upgraded using the Application Library.