Entities

Entities are the individuals communicating within the data set. They communicate by way of their aliases, which are most commonly full email addresses, user ids, short names, and appear within the documents being analyzed. By understanding what aliases belong to each entity, and the role each entity plays in the matter, you can determine scenarios where client-attorney privilege occurs across the documents.

During the Validate Attorneys pipeline step, our AI will group the aliases in the data set under a specific entity and then make an AI prediction as to whether the entity is playing a Legal Role in the matter. This information is populated on the Entities (Priv) tab for you to annotate on the Annotator Decision field, adjust aliases that belong to each entity using the Merge and Move operations, and populate more extensive entity information. Annotating each entity allows for the AI to understand who could be privilege conferring with the matter and what documents they communicated on driving client-attorney privilege decisions. Unlike other annotating workflows, for entities you will perform up to four (4) rounds of annotation. Each round consists of annotating entities and then performing the Apply Annotations Action as specified in Validate Attorneys. For the first round of annotations, the Prediction field will not be populated as the AI model requires some Annotation Decision field values to start its learn.

Requirements

Entities can only be annotated when the Validate Attorneys pipeline step has a status of Ready to Apply Annotations or when the Confirm Privilege Status pipeline step has a status of Ready to Apply Annotations.

Annotate entities

To annotate entities for the first round:

Navigate to the Annotation (Priv)>Entities (Priv) tab.
Filter the list to show Entities that you have not annotated by setting Annotator Decision to (Not Set).
Follow Round 1 of the Suggested annotation workflow.
Select the first Entity in the list that you would like to annotate.
Annotate these entities.
1. Verify the linked aliases.
  Use the Move and Remove operations on the Alias list to remove aliases from the entity.
  Note: These actions are only currently available outside of Edit mode.
2. Select either Legal Role, Not Legal Role, or Spammer on the Annotator Decision field.
3. Select either Legal Degree or No Legal Degree on the Legal Degree field.
  - Populate other Entity Information section fields as required.
  - If you have the context, you should also select either Privilege Conferring, Privilege Neutral, Privilege Breaking, or Unknown.
    You will be able to re-visit or make this selection as part of the Confirm Privilege Status step.
Note: The Entity Information section only needs to be updated when the Annotator Decision field is Legal Role or Legal Degree field is Legal Degree.
Note: Consider reviewing the Email Signature column within the Alias list or navigating to documents they sent or received to better understand who the entity is and their role in the matter.
Note: You must annotate at least 10 entities that have sent at least 1 email. You must also have annotated at least 2 entities as Legal Role, and 2 entities as Not Legal Role. If these conditions are not met, the step will fail.

See Validate Attorneys to start the next round of annotations.

To annotate entities for subsequent rounds:

Navigate to the Annotation (Priv)>Entities (Priv) tab.
Filter the list to show Entities that you have not annotated by setting Annotator Decision to (Not Set).
Follow the subsequent round of the Suggested annotation workflow.
Select the first Entity in the list that you would like to annotate.
Review the Prediction field and Entity Information section.
Perform annotation on the entity:
1. Verify the linked aliases.
2. Use the Move and Remove operations on the Alias list to remove aliases from the entity.
3. Select either Legal Role, Not Legal Role, or Spammer on the Annotator Decision field.
4. Select either Legal Degree or No Legal Degree on the Legal Degree field.
5. Populate other Entity Information fields as required.
6. If you have the context, you should also select either Privilege Conferring, Privilege Neutral, Privilege Breaking, or Unknown. You will be able to re-visit or make this selection as part of the Confirm Privilege Status step.
Select Save and Next to annotate the next entity.

You have completed this round of entity annotations when all entities that have a Prediction field value of Legal Role have been annotated on the Annotator Decision field.

See Validate Attorneys to start the next round of annotations or complete the pipeline step if four annotation rounds have been performed.

Note: Entities that have been annotated on the Annotator Decision field will not receive updated Predictions or have different aliases linked to it between annotation rounds. You can think of this entity as locked-in where only you can update it.

Mass entity annotation

To annotate Entities using the Mass Actions available:

Navigate to the Annotation (Priv)>Entities (Priv) tab.
Filter the list to show Entities that you have not annotated by setting Annotator Decision to (Not Set).
Scroll through the list of Entities and select all of the Entities you want to annotate as Spammer.
See Suggested annotation workflow.
Use the Annotate as Spammer mass action button at the bottom of the page.
Confirm that you want to Annotate these Entities as Spammer. This will update the Annotator Decision field on the items selected.
Repeat steps 3-5 to mass annotate Entities as Not Legal Role/No Legal Degree using the Annotate as Not Legal Role/No Legal Degree mass action.

Note: You can only use mass actions to annotate Entities as Spammer or Not Legal Role/No Legal Degree. All other combinations require individual annotation as you must populate the Entity Information fields as part of the annotation.

Annotate entities fields

The following fields appear when annotating entities:

Field	Field type	Required	Description	Notes
Artifact ID	Fixed-length text	N	A unique identifier for this specific entity. This value will never change and is not editable.	By clicking on this value from a list, you will be directed to that Entity page where you can find more metadata on the included entity. This field can be used when trying to find a specific entities.
Full Name	Fixed-length text	N	A combination of the entity’s First Name, Last Name, and Reference ID (to always create uniqueness). This field is not directly editable but will update upon Save when a change is made to the First Name or Last Name fields.	Multiple entities may have the same First Name and Last Name, but the Full Name will always be unique due to the addition of the Reference ID.
First Name	Fixed-length text	N	The entity’s first name. This field is editable.
Last Name	Fixed-length text	N	The entity’s last name. This field is editable.
Aliases	Multiple object	N	This field links all aliases belonging to the entity. This field is not directly editable but can be changed using the Move or Merge operations.	An alias may only be linked to one entity, but an entity may have multiple aliases.
Alias – Email Signature	Fixed-length text	N	This field will show in the Alias list and is populated for each alias. It contains the longest email signature belonging to the alias that was found in the data set. This value will never change and is not editable.	Use this field to help populate the Annotator Decision field and Entity Information section while performing entity annotations.
Alias - Sent	Whole number	N	This field will show in the Alias list and is populated for each alias. The number of documents sent by this alias. This value will never change and is not editable.
Alias - Received	Whole number	N	This field will show in the Alias list and is populated for each alias. The number of documents received by this alias. This value will never change and is not editable.
Cluster	Fixed-length text	N	An ID generated to group similar entities that our AI suspects should be merged but require a user’s input. If you believe entities with the same Cluster need to be merged, you can use the Merge mass action to accomplish this. This value will never change and is not editable.	Not all entities will have a value in this field. If you decide not to Merge entities with the same Cluster, ignore this field as it is not editable.
Source	Single choice	N	The origin of the entity’s creation. Entities with a Client Brain or Known Attorney source should already be marked as annotated. Annotation focus should be on entities marked as Newly Discovered.	Client Brain—this entity existed in a previous project that was run through aiR for Privilege. Known Attorney—this entity was added at the start of this project to the Known Attorney list. By Way of Domain—this entity is predicted to be an attorney as a result of the completion of Domain annotations Newly Discovered—this entity is predicted to be an attorney by our Validate Attorneys AI model
Prediction	Single choice	N	The prediction by the AI model as to whether the entity is playing a Legal Role or Not a Legal Role in this matter. This value will never change and is not editable.	This value will only be populated after the first Apply Annotation action is performed.
Annotator Decision	Single choice	Y	This field is the most important annotation field and allows for you to confirm or deny the Prediction of Legal Role, Not Legal Role, or Spammer.	Your selection on this field will change what fields require annotation.
Legal Degree	Single choice	Y	This is used as another input to determine if an entity could be conferring privilege in this matter. Choice of Legal Degree or Not Legal Degree.	Your selection on this field will change what fields require annotation.
Job Title	Fixed-length text	Required if Entity is Legal Role or Legal Degree.	Populate this with the title of the individual within their organization. This information is used to provide more details for when determining the entity’s privilege status in later steps.
LinkedIn URL	Fixed-length text	N	A web address found that supports your findings that the entity is in a Legal Role and/or has a Legal Degree.	This is an optional field and is used by a Lead to QC the work before submitting.
Job Category	Single choice	Required if Entity is Legal Role or Legal Degree.	Populate this with the legal job category of the individual within their organization. This information is used to provide more details for when determining the entity’s privilege status in later steps. Choice of Compliance, Contracts, Counsel (External), Counsel (In- House/Internal), Legal Group Alias/Distribution List, Mergers & Acquisitions, Paralegal/Litigation & Legal Support, Patent/IP, Regulatory/Government Affairs & Relations, and Unspecified/Other Legal Role
Maiden Name	Fixed-length text	N	The entity’s maiden name. This information is used to find all the ways this entity may have communicated or been mentioned in this data set.
Middle Name	Fixed-length text	N	The entity’s middle name. This information is used to find all the ways this entity may have communicated or been mentioned in this data set.
Nicknames (Comma-delimited)	Fixed-length text	N	Any nicknames or other names the entity might have been referred to as in the data set. This information is used to find all the ways this entity may have communicated or been mentioned in this data set.	Separate multiple nicknames by a comma.
Sent Documents	Multiple object	N	This field links to the documents that this specific entity sent. This field is not editable.	This information can be used to inform annotations. The documents linked will change when a Merge or Move operation is performed. This field links to the documents that this specific entity sent. It includes all email segments.
Received Documents	Multiple object	N	This field links to the documents that this specific entity received on. This field is not editable.	This information can be used to inform annotations. The documents linked will change when a Merge or Move operation is performed. This field links to the documents that this specific entity received. It includes all email segments.”
Annotated	Yes/No	N	Upon annotating and saving an entity this field will be populated with a checkmark. This field is not editable.	This field can be used in searches to quickly find entities you’ve annotated or have not. This field can also drive dashboard widgets, for visual representation of your annotation progress.
Impacted by Merge/Move	Yes/No	N	This field is used to indicate if a Merge or Move operation was performed on this entity. When entities are merged, the resulting entity will have this field as Yes. When an alias is moved from one entity to another, both entities will have this field as Yes.	Our AI will no longer make changes on entities that have this field set as Yes.
Privilege Project	Multiple object	N	This field links to the project that found this specific entity. This field is not editable.	An entity may link to multiple projects if it was witnessed in multiple analyzed data sets.
Reference ID	Fixed-length text	N	A unique identifier used by the system. This value will never change and is not editable.
Final Privilege Status	Single choice	N	An aiR for Privilege Lead field used during the Confirm Privilege Status pipeline step. Choice of Privilege Conferring, Privilege Neutral, Privilege Breaker, and Unknown.	Can be ignored by aiR for Privilege Annotators.

Merge mass operation

The Merge mass operation exists on the Entity tab and lets you to select multiple entities and move their linked aliases to a single entity. This will be used when the AI is conservative and creates multiple separate entities when there should have been a single entity.

To perform a Merge mass operation:

Navigate to the Annotation (Priv)>Entities (Priv) tab.
Identify and select the entities you’d like to merge using the check boxes on the left side of the Entity list.
Beneath the Entities list select the Merge operation within the Mass Operation drop-down menu.
A modal will appear with the list of entities you selected.
Select the entity you want all aliases merged into.
Note: All non-selected entities will be deleted after their aliases are moved over to the selected entity.
Select the Merge button to perform the operation.

Upon successful completion only one entity will remain with all aliases linked to it. All documents that were linked to the entities included in this operation will now be linked to the single resulting entity. That resulting entity will have a value of Yes on the Impacted by Merge/Move field and will no longer be editable by the AI during future Apply Annotation actions.

Note: You can only include Entities that are connected to your In-Progress project in the Merge mass action. Including Entities from previous projects in this workspace will error out.

Move and Remove operations

The Move and Remove operations exists on an individual entity page and allow you to select multiple aliases linked to that entity and move them to a different entity or remove them from this entity. This will be used when the AI links an alias to the wrong entity.

Note: The Move and Remove operations are not available in Edit mode. Exit Edit mode to be able to find those buttons and perform those operations.

To perform a Move or Remove operation:

Navigate to the Annotation (Priv)>Entities (Priv) tab.
Select an individual entity that youwant to move or remove aliases from.
Locate the Alias list on the Entity page.
Identify and select the aliases you want to move or remove using the checkboxes on the left side of the Alias list.
Above the alias list select the Move or Remove button.
For the Move operation, a modal will appear with the list of all entities in the workspace.
Select the entity you want to move these aliases to.
Note: If you do not want to move these aliases to an existing entity, you can select the Remove button in the lower left corner of the modal to orphan them. In the next Apply Annotation action, these aliases will get assigned to a new entity by the AI.
Select the Move button to perform the operation.

Upon successful completion the aliases you selected will no longer be linked to this entity. All documents that were linked to this entity by way of those aliases will also be unlinked to this entity. Both the entities that you moved aliases from and to will have a value of Yes on the Impacted by Merge/Move field and will no longer be editable by the AI during future Apply Annotation actions.

Suggested annotation workflow

Following is the suggested annotation workflow for Entities. These steps should be completed in priority order:

Step	Expected Annotation	Content Type	Example	How to Locate
ROUND 1
1	Merge (Priv) Mass Action	Clusters of Entities that should be merged	John Doe jd@domain.com John Do John.doe@domain.com	Perform the following search across entities: Cluster IS SET Review items that have the same Cluster ID and use the Merge (Priv) action to merge them if they are the same entity.
2	Legal Role	By way of domain	Jane Doe janedoe@lawfirm.com	Perform the following search across entities: Source ANY OF THESE By Way of Domain
3	Legal Role	Likely Legal Email Signatures	Jane Doe jd@client.com Email signature: General Counsel John Doe john.doe@client.com Email signature: Head of Compliance Jim Doe jim.doe@client.com Email signature: Regulatory Affairs	Perform the following search across entities: Aliases::Email Signature IS LIKE Counsel Then, Aliases::Email Signature IS LIKE compliance Then, Aliases::Email Signature IS LIKE regulator
4	Not Legal Role	Email Signatures	Jane Doe jd@client.com Email signature: Project Manager John Doe john.doe@client.com Email signature: Sales Associate Jim Doe jim.doe@client.com Email signature: Engineering	Perform the following search across entities: Aliases::Email Signature IS SET AND Aliases::Sent IS GREATER THAN 0
3	Legal Role	Top Communicators	Jane Doe jd@lawfirm.com Sent: 20	Perform the following search across entities: Aliases::Sent IS GREATER THAN n (adjust the value of n until you get roughly 100 entities returned)
ROUND 2
1	Legal Role	AI Predicted Legal Role	Jane Doe jd@lawfirm.com Prediction: Legal Role	Perform the following search across entities: Prediction ANY OF THESE Legal Role
2	Not Legal Role	Top AI Predicted Not Legal Role	Jane Doe jd@lawfirm.com Prediction: Not Legal Role	Perform the following search across entities: Prediction ANY OF THESE Not Legal Role AND Aliases::Sent IS GREATER THAN n (adjust the value of n until you get roughly 100 entities returned)
ROUND 3
1	Legal Role	AI Predicted Legal Role	Jane Doe jd@lawfirm.com Prediction: Legal Role	Perform the following search across entities: Prediction ANY OF THESE Legal Role
2	Not Legal Role	Top AI Predicted Not Legal Role	Jane Doe jd@lawfirm.com Prediction: Not Legal Role	Perform the following search across entities: Prediction ANY OF THESE Not Legal Role AND Aliases::Sent IS GREATER THAN n (adjust the value of n until you get roughly 100 entities returned)
ROUND 4
1	Legal Role	AI Predicted Legal Role	Jane Doe jd@lawfirm.com Prediction: Legal Role	Perform the following search across entities: Prediction ANY OF THESE Legal Role
2	Not Legal Role	Top AI Predicted Not Legal Role	Jane Doe jd@lawfirm.com Prediction: Not Legal Role	Perform the following search across entities: Prediction ANY OF THESE Not Legal Role AND Aliases::Sent IS GREATER THAN n (adjust the value of n until you get roughly 400 entities returned)
3	Legal Role	Document Check	We are trying to find documents that include legal terms but where no attorney we’ve identified is participating. This is a method for finding missed entities. You are reviewing these documents not to make a Privilege call, but to catch missed attorneys and annotate the entities as Legal Role	Navigate to the Document Tab and perform the following search across documents: Index Search SEARCH TERMS counsel, attorney, legal, esq AND Priv Sender Entities (all email segments) IS SET AND Priv Sender Entities (all email segments) NOT THESE CONDITION – ANY OF THESE Legal Role, Not Legal Spammer It helps to create a view with the following fields, so you can easily jump into an entity and annotate them: · Control Number · Priv Sender Entities (all email segments) · Priv Sender Entities (all email segments)::Annotator Decision It helps to add the following field to your coding layout, so you can easily jump into an entity and annotate them: · Priv Sender Entities (all email segments) · Priv Recipient Entities (all email segments)
4	Merge (Priv) Mass Action	Spot Check Aliases	We are trying to find entities that should be merged into Entities that have been annotated with Legal Role	Perform the following search across entities (remove the Annotator Decision IS NOT SET search conditions): Annotator Decision ANY OF THESE Legal Role OR Legal Degree ANY OF THESE Legal Degree Pick the first entity, and copy a select part of their last name and search: Aliases IS LIKE [portion of last name] Then uncheck the first two conditions so you are just searching for Entities that have an Alias that includes part of the last name. Review Entities that are returned and merge if necessary (ALWAYS select the Entity where Annotator Decisions IS Legal Role). Perform these same steps for all Entities returned in initial search.