Validate Attorneys

The Validate Attorneys pipeline step locates email addresses and other alias formats used to communicate within your data set, assigns them an individual owner or entity. It then determines attributes about the entity, and predicts if an entity is in a legal role that will drive privileged predictions on documents they communicate in.

The AI uses the Client Brain, Matter Brain, Known Attorneys, Domain Decision, and Equivalent Domain Decision to decide whether an entity plays a legal role. It also considers language from documents, extracted email signatures, and other information.

Requirements

Following are requirements for the Validate Attorneys pipeline step:

  • You must perform both the Run action and the Apply Annotations action on the Validate Attorneys pipeline step. The Run action will identify entities and their corresponding aliases and populate those entities on the Entities (Priv) tab.
  • Unlike other pipeline steps, a Prediction field value will not populate upon the completion of the Run action. A Prediction field value will populate after performing Apply Annotations action as the AI will learn from an initial round of annotations.
  • Because of this learning process, you must perform annotations and Apply Annotations up to four times for this step. Before performing the Apply Annotations action for the first time, you must annotate at least two entities as Legal Role, two entities as Not Legal Role, and a minimum of ten total entities that have an Aliases::Sent field value of greater than 0. Follow the conditions outlined in Annotate entities to complete annotations for this step.
  • After the fourth Apply Annotations action, entities not annotated on the Annotator Decision field will take their Prediction field value as truth to make privileged predictions.

Actions

To run Validate Attorneys:

  1. Navigate to the Projects (Priv) page.
  2. Select the project you would like to interact with.
  3. Locate the Validate Attorneys step in the Pipeline Steps category.
  4. Click the Run button within the Actions column next to the Validate Attorneys step.
    The Run button will only be available when you have successfully completed Match Equivalent Domains.

The Validate Attorneys step status will change to Run In Progress. The system will disable the Run button until the status updates.

Upon the successful completion of the Validate Attorneys Run action, the status on the step will change to Ready to Apply Annotations. The system enables the Apply Annotations action button.

Caution: Before completing this step, you must annotate the entities identified by the Validate Attorneys step to confirm or deny the Predictions on the Annotator Decision field. Quality check (QC) the decisions made by your team for accuracy. See Annotating entities for more details.

To apply annotations for Validate Attorneys:

  1. Navigate to the Projects (Priv) page.
  2. Select the project you would like to interact with.
  3. Locate the Validate Attorneys step in the Pipeline Steps category.
  4. Click the Apply Annotations button within the Actions column next to the Validate Attorneys step.
    The Apply Annotations button will only be available when you have successfully performed the Run action.
  5. A pop-up window displaying annotation details will appear. Review the Annotation Round, number of Total Entities Annotated and Entities Missing Annotator Decision.
    Click Confirm to indicate that you have performed annotations.
Note: At least 10 Entities with sent emails must be annotated before applying annotations. Entities missing an Annotator Decision after the final Annotation Round will use their Prediction field value as their decision. If the Prediction field value is incorrect, it may result in a Privilege Conferring individual being considered Privilege Neutral. This will lead to privileged documents being predicted as Not Privileged due to the lack of a Privilege Conferring individual presence (lower recall). The Final Privilege Status of an Entity can be adjusted in the Confirm Privilege Status step.
Note: Once you have applied annotations for the last annotation round, you will not be able to change the Annotator Decision.

The Validate Attorneys step status will change to Apply Annotations In Progress. The system will disable the Apply Annotations action button until the status updates.

Upon the successful completion of the Validate Attorneys Apply Annotations action, the status on the step will change to either of the following:

  • Ready to Apply Annotations—the AI has found new entities with a Prediction of legal role. You must apply more annotations.
  • Completed—you completed four annotation rounds and can move on to the next pipeline step.

Errors

If the status of the Validate Attorneys pipeline step becomes Run Failed or Apply Annotations Failed then there was an error in the action. Check the Error Details field within the pipeline steps category for more detailed error information including steps to resolve the issue.

Following is a list of the most common Validate Attorneys failures:

Error Cause Solution
Run Failed There is a system failure that is causing the operation to timeout. Retry the action, and if the issue persists contact Relativity support.
Apply Annotations Failed There is a system failure that is causing the operation to timeout. Retry the action, and if the issue persists contact Relativity support.
Apply Annotations Failed

There are missing annotations. You must annotate:

  • At least 10 Entities that have an Alias that has sent at least 1 email
  • At least 2 Annotation Decisions of Legal Role
  • At least 2 Annotation Decisions of Not Legal Role
 Perform more annotations to meet the minimum annotation requirement for this step and then try again.

Once you resolve any errors you can rerun the Validate Attorneys pipeline step.