Entities

Entities are the individuals communicating within the data set. They communicate by way of their aliases, which are most commonly full email addresses, user ids, short names, and appear within the documents being analyzed. By understanding what aliases belong to each entity, and the role each entity plays in the matter, you can determine scenarios where client-attorney privilege occurs across the documents.

During the Validate Attorneys pipeline step our AI will group the aliases in the data set under a specific entity and then make an AI prediction as to whether the entity is playing a Legal Role in the matter. This information is populated on the Entities (Priv) tab for you to annotate on the Annotator Decision field using the process outlined in Suggested annotation workflow, adjust aliases that belong to each entity using the Merge and Move operations, and populate more extensive entity information. Annotating each entity allows for the AI to understand who could be privilege conferring with the matter and what documents they communicated on driving client-attorney privilege predictions. Unlike other annotating workflows, for entities you will perform up to four (4) rounds of annotation. Each round consists of annotating entities and then performing the Apply Annotations Action as specified in Validate Attorneys. For the first round of annotations, the Prediction field will not be populated as the AI model requires some Annotation Decision field values to start its learn.

Note: Entity annotations should be performed by someone with the role of Annotator. For more information see Roles and security permissions.

Note: Use the Suggested annotation workflow to complete annotations quickly and accurately. Not using the Suggested annotation workflow could result in errors.

Requirements

Entities can only be annotated when the Validate Attorneys pipeline step has a status of Ready to Apply Annotations or when the Confirm Privilege Status pipeline step has a status of Ready to Apply Annotations.

Annotate entities

To annotate entities for the first round:

1. Navigate to the Annotation (Priv)>Entities (Priv) tab.
2. Filter the list to show Entities that you have not annotated by setting Annotator Decision to (Not Set).
3. Follow Round 1 of the Suggested annotation workflow.
4. Select the first Entity in the list that you would like to annotate.
5. Annotate these entities.
   1. Verify the linked aliases. Use the Move and Remove operations on the Alias list to remove aliases from the entity.
      Note: These actions are only currently available outside of Edit mode.
   2. Select either Legal Role, Not Legal Role, or Spammer on the Annotator Decision field.
      
   Note: The Entity Information section only needs to be updated when the Annotator Decision field is Legal Role.
   Note: Consider reviewing the Email Signature column within the Alias list or navigating to documents they sent or received to better understand who the entity is and their role in the matter.
   Note: You must annotate at least 10 entities that have sent at least 1 email. You must also have annotated at least 2 entities as Legal Role, and 2 entities as Not Legal Role. If these conditions are not met, the step will fail.

See Validate Attorneys to start the next round of annotations.

To annotate entities for subsequent rounds:

1. Navigate to the Annotation (Priv)>Entities (Priv) tab.
2. Filter the list to show Entities that you have not annotated by setting Annotator Decision to (Not Set).
3. Follow the subsequent round of the Suggested annotation workflow.
4. Select the first Entity in the list that you would like to annotate.
5. Review the Prediction field and Entity Information section.
6. Perform annotation on the entity:
   1. Verify the linked aliases.
   2. Use the Move and Remove operations on the Alias list to remove aliases from the entity.
   3. Select either Legal Role, Not Legal Role, or Spammer on the Annotator Decision field.
   4. Populate other Entity Information fields as required.
   5. If you have the context, you should also select a Final Privilege Status of either Privilege Conferring, Privilege Neutral, or Privilege Breaking. You will be able to re-visit or make this selection as part of the Confirm Privilege Status step.
      Note: If you select Privilege Conferring as the Final Privilege Status, the First Name and Last Name fields are required.
7. Select Save and Next to annotate the next entity.
   Note: To successfully annotate distribution lists as "Legal Role", add GenericLegalSupportWithinEntity to both the First Name and Last Name fields.

You have completed this round of entity annotations when all entities that have a Prediction field value of Legal Role have been annotated on the Annotator Decision field.

See Validate Attorneys to start the next round of annotations or complete the pipeline step if four annotation rounds have been performed.

Note: An entity that has been annotated on the Annotator Decision field will not receive updated Predictions or have different aliases linked to it between annotation rounds. You can think of this entity as locked-in; it can only be updated by you.

Mass entity annotation

To annotate Entities using the Mass Actions available:

1. Navigate to the Annotation (Priv)>Entities (Priv) tab.
2. Filter the list to show Entities that you have not annotated by setting Annotator Decision to (Not Set).
3. Scroll through the list of Entities and select all of the Entities you want to annotate as Spammer.
   See Suggested annotation workflow.
4. Use the Annotate as Spammer mass action button at the bottom of the page.
   
5. Confirm that you want to Annotate these Entities as Spammer. This will update the Annotator Decision field on the items selected.
6. Repeat steps 3-5 to mass annotate Entities as Not Legal Role using the Annotate as Not Legal Role mass action.

Note: You can only use mass actions to annotate Entities as Spammer or Not Legal Role. All other combinations require individual annotation as you must populate the Entity Information fields as part of the annotation.

Annotate entities fields

The following fields appear when annotating entities:

Field	Field type	Required	Description	Notes
Artifact ID	Fixed-length text	N	A unique identifier for this specific entity. This value will never change and is not editable.	By clicking on this value from a list, you will be directed to that Entity page where you can find more metadata on the included entity. This field can be used when trying to find a specific entity.
Full Name	Fixed-length text	N	A combination of the entity’s First Name, Last Name, and Reference ID (to always create uniqueness). This field is not directly editable but will update once saved if a change is made to the First Name or Last Name fields.	Multiple entities may have the same First Name and Last Name, but the Full Name will always be unique due to the addition of the Reference ID.
First Name	Fixed-length text	N	The entity’s first name. This field is editable.	If you select Privilege Conferring as the Final Privilege Status, this field is required.
Last Name	Fixed-length text	N	The entity’s last name. This field is editable.	If you select Privilege Conferring as the Final Privilege Status, this field is required.
Aliases	Multiple object	N	This field links all aliases belonging to the entity. This field is not directly editable but can be changed using the Move or Merge operations.	An alias may only be linked to one entity, but an entity may have multiple aliases.
Alias – Email Signature	Fixed-length text	N	This field will show in the Alias list and is populated for each alias. It contains the longest email signature belonging to the alias that was found in the data set. This value will never change and is not editable.	Use this field to help populate the Annotator Decision field and Entity Information section while performing entity annotations.
Alias - Sent	Whole number	N	This field will show in the Alias list and is populated for each alias. The number of documents sent by this alias. This value will never change and is not editable.
Alias - Received	Whole number	N	This field will show in the Alias list and is populated for each alias. The number of documents received by this alias. This value will never change and is not editable.
Cluster	Fixed-length text	N	An ID generated to group similar entities that our AI suspects should be merged but require a user’s input. If you believe entities with the same Cluster need to be merged, you can use the Merge mass action to accomplish this. This value will never change and is not editable.	Not all entities will have a value in this field. If you decide not to Merge entities with the same Cluster, ignore this field as it is not editable.
Source	Single choice	N	The origin of the entity’s creation. Entities with a Client Brain or Known Attorney source should already be marked as annotated. Annotation focus should be on entities marked as Newly Discovered.	Client Brain—this entity existed in a previous project that was run through aiR for Privilege. Known Attorney—this entity was added at the start of this project to the Known Attorney list. By Way of Domain—this entity is predicted to be an attorney as a result of the completion of Domain annotations. Newly Discovered—this entity is predicted to be an attorney by our Validate Attorneys AI model.
Prediction	Single choice	N	The prediction by the AI model as to whether the entity is playing a Legal Role or Not a Legal Role in this matter. This value will never change and is not editable.	This value will only be populated after the first Apply Annotation action is performed.
Annotator Decision	Single choice	Y	This field is the most important annotation field and allows for you to confirm or deny the Prediction of Legal Role, Not Legal Role, or Spammer.	Your selection on this field will change what fields require annotation.
Job Title	Fixed-length text	Required if Entity is Legal Role.	Populate this with the title of the individual within their organization. This information is used to provide more details for determining the entity’s privilege status in later steps.
LinkedIn URL	Fixed-length text	N	A web address found that supports your findings that the entity is in a Legal Role.	This is an optional field and is used by a Lead to QC the work before submitting.
Job Category	Single choice	Required if Entity is Legal Role..	Populate this with the legal job category of the individual within their organization. This information is used to provide more details for determining the entity’s privilege status in later steps. Choice of Compliance, Contracts, Counsel (External), Counsel (In- House/Internal), Legal Group Alias/Distribution List, Mergers & Acquisitions, Paralegal/Litigation & Legal Support, Patent/IP, Regulatory/Government Affairs & Relations, and Unspecified/Other Legal Role.
Maiden Name	Fixed-length text	N	The entity’s maiden name. This information is used to find all the ways this entity may have communicated or may have been mentioned in this data set.
Middle Name	Fixed-length text	N	The entity’s middle name. This information is used to find all the ways this entity may have communicated or may have been mentioned in this data set.
Nicknames (Comma-delimited)	Fixed-length text	N	Any nicknames or other names the entity might have been referred to as in the data set. This information is used to find all the ways this entity may have communicated or may have been mentioned in this data set.	Separate multiple nicknames by a comma.
Sent Documents	Multiple object	N	This field links to the documents that this specific entity sent. This field is not editable.	This information can be used to inform annotations. The documents linked will change when a Merge or Move operation is performed. This field links to the documents that this specific entity sent. It includes all email segments.
Received Documents	Multiple object	N	This field links to the documents that this specific entity received. This field is not editable.	This information can be used to inform annotations. The documents linked will change when a Merge or Move operation is performed. This field links to the documents that this specific entity received. It includes all email segments.
Annotated	Yes/No	N	Upon annotating and saving an entity this field will be populated with a checkmark. This field is not editable.	This field can be used in searches to quickly find entities you have or have not annotated . This field can also drive dashboard widgets for visual representation of your annotation progress.
Impacted by Merge/Move	Yes/No	N	This field is used to indicate if a Merge or Move operation was performed on this entity. When entities are merged, the resulting entity will have this field as Yes. When an alias is moved from one entity to another, both entities will have this field as Yes.	Our AI will no longer make changes on entities that have this field set as Yes.
Privilege Project	Multiple object	N	This field links to the project that found this specific entity. This field is not editable.	An entity may link to multiple projects if it appeared in multiple analyzed data sets.
Reference ID	Fixed-length text	N	A unique identifier used by the system. This value will never change and is not editable.
Final Privilege Status	Single choice	N	A field used during the Confirm Privilege Status pipeline step. Choice of Privilege Conferring, Privilege Neutral, or Privilege Breaker.	If you select Privilege Conferring as the Final Privilege Status, the First Name and Last Name fields are required. Can be ignored by Annotators.

Merge mass operation

The Merge mass operation exists on the Entity tab and lets you select multiple entities and move their linked aliases to a single entity. This will be used when the AI is conservative and creates multiple separate entities when there should have been a single entity.

To perform a Merge mass operation:

1. Navigate to the Annotation (Priv)>Entities (Priv) tab.
2. Identify and select the entities you’d like to merge using the check boxes on the left side of the Entity list.
3. Beneath the Entities list select the Merge operation within the Mass Operation drop-down menu.
4. A modal will appear with the list of entities you selected.
5. Select the entity you want all aliases merged into.
   All non-selected entities will be deleted after their aliases are moved over to the selected entity.
6. Select the Merge button to perform the operation.

Upon successful completion only one entity will remain with all aliases linked to it. All documents that were linked to the entities included in this operation will now be linked to the single resulting entity. That resulting entity will have a value of Yes on the Impacted by Merge/Move field and will no longer be editable by the AI during future Apply Annotation actions.

Note: You can only include Entities that are connected to your In-Progress project in the Merge mass action. Including Entities from previous projects in this workspace will error out.

Move and Remove operations

The Move and Remove operations exists on an individual entity page and allow you to select multiple aliases linked to that entity and move them to a different entity or remove them from this entity. This will be used when the AI links an alias to the wrong entity.

Note: The Move and Remove operations are not available in Edit mode. Exit Edit mode to find those buttons and perform those operations.

To perform a Move or Remove operation:

1. Navigate to the Annotation (Priv)>Entities (Priv) tab.
2. Select an individual entity that you want to move or remove aliases from.
3. Locate the Alias list on the Entity page.
4. Identify and select the aliases you want to move or remove using the checkboxes on the left side of the Alias list.
5. Above the alias list select the Move or Remove button.
6. For the Move operation, a modal will appear with the list of all entities in the workspace.
7. Select the entity you want to move these aliases to.
   Note: If you do not want to move these aliases to an existing entity, you can select the Remove button in the lower left corner of the modal to orphan them. In the next Apply Annotation action, these aliases will get assigned to a new entity by the AI.
8. Select the Move button to perform the operation.

Upon successful completion the aliases you selected will no longer be linked to this entity. All documents that were linked to this entity by way of those aliases will also be unlinked to this entity. Both the entities that you moved aliases from and to will have a value of Yes on the Impacted by Merge/Move field and will no longer be editable by the AI during future Apply Annotation actions.

Suggested annotation workflow

The suggested annotation workflow for Entities includes four rounds of annotation that must be completed in priority order. After every Round, you should navigate to your project and perform the Apply Annotations action on Validate Attorneys.

Note: If you've previously run a Privilege project in this workspace, you should also add a search condition of Privilege Project = current project to focus annotations on entities that are relevant to the current project.

Round 1

Following is the suggested workflow for the first round of entity annotation:

Step	Expected annotation	Content type	Example	How to locate
1	Merge (Priv) Mass Action	Clusters of Entities that should be merged	John Doe  jd@domain.com  John Do  John.doe@domain.com	Perform the following search across entities:    Review items that have the same Cluster ID and use the Merge (Priv) action to merge them if they are the same entity.
2	Legal Role	By way of domain	Jane Doe  janedoe@lawfirm.com	Perform the following search across entities:
3	Legal Role	Likely Legal  Email Signatures	Jane Doe   jd@client.com  Email signature: General Counsel  John Doe   john.doe@client.com  Email signature: Head of Compliance  Jim Doe  jim.doe@client.com  Email signature: Regulatory Affairs	Perform the following search across entities:
4	Not Legal Role	Email Signatures	Jane Doe   jd@client.com  Email signature: Project Manager  John Doe   john.doe@client.com  Email signature: Sales Associate  Jim Doe  jim.doe@client.com  Email signature: Engineering	Perform the following search across entities:
5	Legal Role	Top Communicators	Jane Doe   jd@lawfirm.com  Sent: 20	Perform the following search across entities:  Note: This image is an example of how Search Conditions should look. Document count varies depending on the size of your data set. Adjust the value of n until you get roughly 100 entities returned.

Round 2

Following is the suggested workflow for the second round of entity annotation:

Step	Expected Annotation	Content Type	Example	How to Locate
1	Legal Role	AI Predicted Legal Role	Jane Doe   jd@lawfirm.com  Prediction: Legal Role	Perform the following search across entities:
2	Not Legal Role	Top AI Predicted Not Legal Role	Jane Doe   jd@lawfirm.com  Prediction: Not Legal Role	Perform the following search across entities:  Note: This image is an example of how Search Conditions should look. Document count varies depending on the size of your data set.

Round 3

Following is the suggested workflow for the third round of entity annotation:

Step	Expected Annotation	Content Type	Example	How to Locate
1	Legal Role	AI Predicted Legal Role	Jane Doe   jd@lawfirm.com  Prediction: Legal Role	Perform the following search across entities:
2	Not Legal Role	Top AI Predicted Not Legal Role	Jane Doe   jd@lawfirm.com  Prediction: Not Legal Role	Perform the following search across entities:  Note: This image is an example of how Search Conditions should look. Document count varies depending on the size of your data set. Adjust the value of n until you get roughly 100 entities returned.

Round 4

Following is the suggested workflow for the fourth round of entity annotation:

Step	Expected Annotation	Content Type	Example	How to Locate
1	Legal Role	AI Predicted Legal Role	Jane Doe   jd@lawfirm.com  Prediction: Legal Role	Perform the following search across entities:
2	Not Legal Role	Top AI Predicted Not Legal Role	Jane Doe   jd@lawfirm.com  Prediction: Not Legal Role	Perform the following search across entities:  Note: This image is an example of how Search Conditions should look. Document count varies depending on the size of your data set. Adjust the value of n until you get roughly 400 entities returned.
3	Legal Role	Document Check	We are trying to find documents that include legal terms but where no attorney we’ve identified is participating. This is a method for finding missed entities.    You are reviewing these documents not to make a Privilege call, but to catch missed attorneys and annotate the entities as Legal Role.	Navigate to the Document Tab and perform the following search across documents:    It helps to create a view with the following fields, so you can jump into an entity and annotate them:  ·       Control Number  ·       Priv Sender Entities (all email segments)  ·       Priv Sender Entities (all email segments)::Annotator Decision  It helps to add the following field to your coding layout, so you can jump into an entity and annotate them:  ·       Priv Sender Entities (all email segments)  ·       Priv Recipient Entities (all email segments)
4	Merge (Priv) Mass Action	Spot Check Aliases	We are trying to find entities that should be merged into Entities that have been annotated with Legal Role.	Perform the following search across entities (remove the Annotator Decision IS NOT SET search conditions): Cluster IS SET, AND then SORT by the Cluster field     Scroll down the page and check the Annotator Decision field for entities that have an Annotator Decision = "Legal Role". For each of these entities, look at the other entities with the same Cluster Id (if they exist, they might have already been merged) and decide whether a merge should occur or not.   Then remove all conditions, AND then SORT by the Full Name field.   Perform the same action and scroll down the page checking the Annotator Decision field for entities that have an Annotator Decision = "Legal Role". This time look for the entities above and below these entities to see if they have similar Full Names and should be merged.