Promoting data with Integration Points

Once you've tagged documents for inclusion in and/or exclusion from the data you want to promote for review, you can access Integration Points to start the job or jobs that will send those documents to the review workspace.

This topic provides details on Relativity Integration Points as it is used in an ECA and Investigation workflow only. For information, see Relativity Integration Points.

This page contains the following information:

Note: For workspace-to-workspace integration points, Relativity supports transfers across the Document object only and not other RDO's.

Special considerations - job batches for large sync workflows

We recommend configuring integration point jobs into batches for large sync workflows. The following are recommendations when creating job batches:

  • Document count should be no more than 500k documents in a single batch.
  • When including extracted text as a mapped field in the integration point job, the sum of the extracted text in the batch should not exceed 25 GBs.
  • The recommended number of fields to be mapped should be no greater than 100, but it is best to map as few long text fields as possible.
  • When using the append/overlay or overlay only configuration, the batch size recommendations should be half the amount described above, specifically:
    • 250k document count
    • 12.5 GBs extracted text sum

Working with promoted documents

To view the documents you promoted to the review workspace, perform the following steps:

  1. Navigate to the destination workspace.
  2. Select the Saved Search browser and select the search you created to bring back documents that were promoted from ECA.
  3. Note the following ECA-relevant fields on the promoted-from ECA saved search view:
    • Relativity Source Case - the name of the workspace in which you tagged documents for inclusion and exclusion and from which you promoted your tagged documents to the review workspace.
    • Relativity Source Job - the name of the Integration Point that you used to promote tagged documents to the review workspace.

You can now review these documents and apply coding decisions for responsiveness and/or issues designation.

Reusing coding decisions

You can re-use the coding decisions you made on reviewed documents and promote them back into the ECA workspace through another Integration Point. For example, you could run another promote job to conduct a privilege overlay on documents in the source workspace.

To do this, perform the following steps:

  1. Select the saved search you created to promote documents back to the ECA (source) workspace.
  2. Navigate to the Integration Points tab.
  3. Create a new integration point that specifies the following values, which differ from those you entered for the promote job you ran previously:
    1. Destination Workspace - select the original source workspace, specifically the workspace from which you previously promoted documents to the review workspace.
    2. Saved Search - select the saved search you created to promote documents back to the ECA (source) workspace.
    3. Field Mappings - map only Control Number (Object Identifier) and Privilege Designation.
    4. Overwrite - select Overlay Only.
  4. Click Run.
  5. Navigate to the ECA Dashboard and select the promoted-from-review saved search.
  6. In the widgets drop-down list, select Coding Values. Note that the widgets and document list in this dashboard reflect the coding decisions that were applied to each document in the review workspace.

Performance baselines

The following table presents performance baselines for using the Relativity provider in Integration Points to promote documents and metadata between workspaces.

The jobs which yielded the following metrics were performed on a basic test environment of Relativity. Due to differences in data, infrastructure, and configuration, these should not be used as a benchmark of what you expect to see in a production environment. The results may not scale linearly.

Document Count

Job Size (GB)

Copy Native Setting

Job duration (HH:MM)

10,000 documents

0.85

Copy

00:04

10,000 documents

0.85

Links Only

00:04

100,000 documents

9.42

Copy

00:42

100,000 documents

9.42

Links Only

00:27

100,000 documents

0

No

(Metadata only)

00:16

100,000 documents (337,000 images)

22.13

Copy Images

00:57

100,000 documents (337,000 images)

22.13

Link Images

00:43

500,000 documents

47.5

Copy

02:38

500,000 documents

47.5

Links Only

02:03

Field information:

Total field count Fixed length Long text Multi choice Single choice Yes/No Date Single object Multi object Decimal Whole number
100 25 39 3 2 13 4 1 8 1 1

Handling errors

Errors can occur at both the item and the job-level when you run your integration point.

The following are examples of potential error causes:

  • Relativity is unable to retrieve a workspace.
  • The user who last created or edited the integration point is deleted before a scheduled job runs.
  • You promote documents with the same control numbers from two different source workspaces to the same destination workspace, which causes control number collisions.

Note: Any item-level errors that you encounter in your integration point are generated by the Import API.

To view these errors, click the View Errors link on the integration point console.

(Click to expand)

Clicking the View Errors link takes you to the Job History Errors tab. This link is unavailable if the job has a status of Stopped.

Job history errors view

In the Job History Errors tab, you can use the condition lists at the top of the view to find errors based on any error-related metadata fields, such as Artifact ID, Error, Error Status, Error Type, JobHistory, Name, Source Unique ID, Stack Trace, System Created By, System Created On, System Last Modified On, and Timestamp (UTC).

Once you specify your conditions, you can search for the targeted errors by clicking Search on the right side of the condition lists.

You can then open any of the individuals errors returned in the list by clicking the Name value.

Single RIP error layout

The Job History Error Layout provides the following fields:

  • Name - the system-generated name of the error.
  • Source Unique ID - the identifier of the item in which the error occurred.
  • Job History - the name of the integration point containing the file in which the error occurred.
  • Error - the error message.
  • Error Status - the current state of the error. The possible statuses are:
    • New - the error is new and no action has been taken.
    • Expired - the state assigned to an item level error in either of the following scenarios:
      • You received an item-level error on a job, you didn't retry the error, and the job ran on a schedule, meaning the Enable Scheduler field is set to Yes on the integration point.
      • You received an item-level or a job-level error, you didn't retry the error, and you click Run on the integration point, or it is a scheduled job.
    • In Progress - the error is currently in the process of being retried.
    • Retried - the item level error in a job has been retried, meaning you clicked Retry Errors on the integration point and the retry job is complete.
  • Error Type - an indicator of whether the error is item- or job-level.
  • Full Error- the error message with details, including a stack trace of the error, when available.
  • System Created On - the date and time when the agent created the error during the integration point job.
  • System Created By - the agent that created the error during the integration point job.
  • System Last Modified By - the agent that last updated the status value of the error.

With the information provided in the Job History Errors Layout, you can identify those files on which errors occurred. You can then access those files to manually address the causes of those errors. From there, you can return to the integration points console and retry the errors.

Item-level versus job-level errors

Note the following differences in the way Relativity handles item-level and job-level errors.

Relativity handles a mix of item-level and job-level errors in the following way:

  • When you click Run on an integration point that contains item- and job-level errors, the entire job is re-run and all errors are marked as Expired.
  • When you click Retry Errors on an integration point that contains item- and job-level errors, the entire job is rerun and Relativity assigns a value of Expired to the item-level errors while job-level errors are marked as In Progress and then Retried.

Relativity handles item-level errors in the following way:

  • When you click Run , the entire job is re-run, and Relativity assigns a value of Expired to the item-level errors.
  • When you click Retry Errors, Relativity assigns a value of Retried to the item-level errors and starts a retry job containing only those error documents.

Note: When you retry errors for a job that contains only item-level errors, Relativity creates a temporary saved search that contains only those item-level error documents, which the Export Service Manager and Export API can refer to. This search is called "Temporary Search" and is public and visible to users in the saved search browser for the duration of the retry errors job. Once Relativity completes the retry errors job, it deletes this temporary search.

Relativity handles job-level errors in the following way:

  • It registers only one job-level error for the integration point.
  • When you click Run , the entire job is re-run and Relativity assigns a value of Expired to the job-level error.
  • The Retry Errors and Run buttons are both available.
  • The Retry Error button re-runs the whole job, but the job-level error is marked as In progress and then retried.
  • The Run button re-runs the whole job, but the job-level error is marked as Expired.

Scheduled job is not inserted and an error is thrown that shows up in the errors tab in Admin mode that a scheduled job did not run for this integration because the job was already running.

Troubleshooting job history errors

Relativity provides informative and actionable error messages when you attempt to save and run an integration point in any of the following situations:

(Click to expand)

  • The destination workspace is missing.
  • The destination field is missing.
  • You don't have permissions to a saved search, production, or destination workspace.
  • The destination field has missing child object data.
  • The production is missing from the destination workspace.

The following table provides some of the errors you could encounter when attempting to run an Integration Points job, along with information on how to resolve them:

Error Likely resolution
Destination field(s) mapped has different type. Review the mapping for the following field(s): field_1, field_2 Review the fields listed in the error message, and check if you have the same type as expected. If the reason for changing the type was to delete the file and re-create it, then it will be un-mapped. Correct the mapping saved in the Integration Point job, and run the job again.
Destination field(s) mapped may no longer be available or has been renamed. Review the mapping for the following field(s) [field name]. Review if the failing field(s) still exist in the destination workspace, correct the mapping saved in the Integration Point job, and run the job again.
Destination workspace is not available. Verify if destination workspace exists.
Failed to copy source field into destination field due to missing child object. Review the following destination field(s): [field names] Review the list of failing fields, create the child/parent relation in the destination workspace as it is in the source workspace and try again. If that doesn't work, try to use the RDC export and import functionality to move those missing fields only.
Saved search is not available or has been secured from this user. Contact your system administrator. Verify that you have the required permissions for the saved search, or check to see if the saved search has been deleted. If needed, contact your system administrator to verify the saved search configuration.
User does not have sufficient permissions to access destination workspace. Contact your system administrator. Confirm that you still have access to the destination workspace. It is also required that a System Admin account is present (used by an Integration Points Agent) and has access to the destination workspace.
Verify if a folder in destination workspace selected in the Integration Point exists or if a user has a proper permission to access it. Verify that the folder used for the integration point still exists. Also, check the permission settings for the folders in destination workspace to confirm that it's not secured for the user's group.
Verify if a Production Set used as the location in destination workspace exists or if a user has a proper permission to access it. Verify that the production set used as the location in the destination workspace still exists. Also, check the permission settings for productions in the destination workspace to confirm that it's not secured for the user's group.
Verify if a Production Set used as the location in destination workspace is in New status. Verify that the production set used as the location in the destination workspace still has a status of New. Only productions with a status of New can be used as a production set in an integration point. If the production set was already staged or run, it isn't a eligible to be used for the integration point.
Verify if production, which is the data source of this Integration Point, still exist or if user has required permissions for it. Check to see if the production used as the data source of the integration point still exists. Also, verify that you have the required permissions for the production used as the data source. If needed, contact your system administrator to verify your production permissions.
Verify if the user still has permission to create saved search on destination workspace. Verify that you have the required permissions for creating a saved search in the destination workspace for this integrated point job. If needed, contact your system administrator to verify your production permissions. One of missing permissions could be Search in Object Security and Advanced & Saved Searches in Other Settings, both of which are set in the group permission on destination workspace.

Retrying errors

Once you manually address the causes of the item-level errors that occurred in your integration point, you can retry those errors through the integration points console.

To do this, click Retry Errors. When you do this, you submit a retry job to the system, and the system starts to import data from the source provider.

(Click to expand)

After you start the error retry job, you have the option of monitoring its progress through the Status field at the bottom of the layout. The job you just started has a Job Type value of Retry Errors and is listed as the first job in the Status view.

Note the following about retrying errors:

  • The Retry Errors button is only be enabled when the previous job had an error to retry.
  • You're unable to retry errors for a job with a status of Stopped.
  • If you click Retry Errors and then click Stop on the retry job, Relativity marks both the current job's errors and those from the previous job as Expired.

Note the following overwrite behavior during error retry:

  • If the original integration points job had an overwrite mode of Append Only, then when you click Retry Errors, that retry job has an overwrite mode of Append/Overlay.
  • If the original integration points job had an overwrite mode of Append/Overlay, then when you click Retry Errors, that retry job keeps the overwrite mode of Append/Overlay.
  • If you click Run on an integration point that has errors without resolving those errors, Relativity retains the overwrite mode specified in the original job.

Viewing job history

To view all the audit actions associate with all integration points in your workspace, select the Job History tab.

(Click to expand)

job history tab in ECA

The default All Jobs view in the Job History tab provides the following fields:

  • Start Time (UTC) - the date and time at which an agent picked up the job.
  • Artifact ID - the identifier of the job.
  • Name - the name of the job. Clicking this takes you to the Job Details layout.
    (Click to expand)

  • Integration Point - the name of the integration point from which you ran the job.
  • Job Type - the type of job you ran. This displays one of the following values:
    • Run - the job was kicked off via the Run button on the integration point console.
    • Retry Errors - the job was kicked off via the Retry Errors button on the integration point console.
    • Scheduled Run - the job was kicked off via the Enable Scheduler field and subsequent time designation on the integration point layout.
  • Job Status - the current status of the job. This will display one of the following values:
    • Pending - the job has yet to be picked up by an agent.
    • Validation - an agent is validating the job to make sure it has the required settings, such as access to the source and destination objects (saved searches, workspaces, etc.). If validation fails, the status changes to "Validation failed," and Relativity logs an error. If the validation succeeds, the job moves on to a status of "Processing."
    • Validation failed - the job validation has failed, resulting in an error, the details of which are saved in the Job History Errors tab.
    • Processing - the agent has picked up the job and is in the process of completing it.
    • Completed - the job is complete, and no errors occurred.
    • Completed with errors - the job is complete and errors have occurred.
    • Error - job failed - a job-level error occurred and the job didn't complete because it failed.
  • Destination Workspace - the workspace to which you arranged to have documents promoted.
  • Items Imported - the number of documents successfully imported through the integration point. If errors occurred, this value is lower than the Total Items value and the difference appears in the Items with Errors field.
  • Total Items - the total number of documents included in the integration point. If this value is equal to the Items Imported, then no errors occurred. If errors have occurred, this value is higher than the Items Imported value and the difference between the two appears in the Items with Errors field.
  • Items with Errors - the number of documents that generated errors. If errors have occurred, this value is equal to the difference between the Items Imported and the Total Items fields.