Developing prompt criteria

Considerations:

• To ensure optimal results, limit the Case Summary content to 20 or fewer sentences overall, and 20 or fewer entries each of People and Aliases, Noteworthy Organizations, and Noteworthy Terms. Although longer entries can be processed by the system, exceeding these limits may reduce result quality.
• If using prompt kickstarter to automatically generate initial prompt criteria, these fields will automatically populate based on the documents uploaded or text manually added using the Draft with AI feature. See Using prompt kickstarter for more information. Refer to this section as needed for field descriptions.

Fill out the following:

• Matter Overview—provide a concise overview of the case. Include the names of the plaintiff and defendant, the nature of the dispute, and other important case characteristics.
• People and Aliases—list the names and aliases of key custodians who authored or received the documents. Include their role and any other affiliations.
• Noteworthy Organizations—list the organizations and other relevant entities involved in the case. Highlight any key relationships, entities, or other notable characteristics.
• Noteworthy Terms—list and define any relevant words, phrases, acronyms, jargon, project code names, or slang that might be important to the analysis.
• Additional Context—list any additional information that does not fit the other fields. This section can be left blank.

Depending on which Analysis Type you chose when setting up the project, the remaining tabs will be Relevance, Key Documents, or Issues. Refer to the appropriate tab section below for more information on each one.

For best results when entering information in the Case Summary tab:

• Keep Case Summaries to 20 sentences or fewer. Longer summaries often include excess details, weakening prompt quality and increasing borderline documents. Concise, direct summaries aligned with best practices lead to better results.
• For People and Aliases, avoid listing every person in the case. The system picks up common information, such as work emails. Instead, focus on including people and aliases a typical person could not easily infer, such as unusual or personal email addresses, and only when those details matter for determining document relevance.

Fill out the following:

• Relevance Field—select a single-choice field that represents whether a document is relevant or non-relevant. This selection cannot be changed after the first job run.
• Relevant Choice—select the field choice you use to mark a document as relevant. This selection cannot be changed after the first job run.
• Relevance Criteria—summarize the criteria that determine whether a document is relevant. Include:
  • Keywords, phrases, legal concepts, parties, entities, and legal claims.
  • Any criteria that would make a document non-relevant, such as relating to a project that is not under dispute.
• Issues Field (Optional)—select a single-choice or multi-choice field that represents the issues in the case. This selection cannot be changed after the first job run.
• Choice Criteria—individually select each field Choice to be analyzed. A maximum of 10 choices can be analyzed at a time. To remove a selected choice, click the X in its row. For each choice, write criteria description in the text box listing the criteria that determine whether that issue applies to a document.
  • aiR does not make Issue predictions or coding during Relevance review, but you can use this field for reference when writing the Relevance Criteria. For example, you could tell aiR that documents containing any of the defined issues are relevant. If you want aiR to provide issue predictions, you must run another project using Issues as the analysis type on the documents.
  • The field choices cannot be changed after the first job run. However, you can still edit the criteria description in the text box.
  • If you delete or modify the name of a field Choice used in the project before running an analysis, an error message appears indicating the choices configured for the Issues Field are not valid. To resolve this, return to the Choice Criteria field within the prompt criteria tab and update it as necessary. For further details about deleting or modifying Choices, refer to Choices.

For best results when writing the Relevance Criteria:

• Limit it to 5-10 sentences.
• Do not use a plain number for the Choice Criteria name, such as 12.
• Do not paste in the original request for production (RFP), since those are often too long and complex to give good results. Instead, summarize it and include relevant excerpts.
• Group similar criteria together when you can. For example, if an RFP asks for “emails pertaining to X” and “documents pertaining to X,” write “emails or documents pertaining to X.”

Fill out the following:

• Key Document Field—select a single-choice field that represents whether a document is key to the case. This selection cannot be changed after the first job run.
• Key Document Choice—select the field choice you use to mark a document as key. This selection cannot be changed after the first job run.
• Key Document Criteria—summarize the criteria that determine whether a document is key. For best results, limit the Key Document Criteria to 5-10 sentences. Include:
  • Keywords, phrases, legal concepts, parties, entities, and legal claims.
  • Any criteria that would exclude a document from being key, such as falling outside a certain date range.

Fill out the following:

• Field—select a multi-choice field that represents the issues in the case.
• Choice Criteria—select each of the field Choices one by one to be analyzed. A maximum of 10 choices can be analyzed at a time. To remove a selected choice, click the X in its row. For each choice, write criteria description in the text box listing the criteria that determine whether that issue applies to a document, or this information can be populated using prompt kickstarter.
  • The field choices cannot be changed after the first job run. However, you can still edit the criteria description in the text box.
  • If you delete or modify the name of a field Choice used in the project before running an analysis, an error message appears indicating the choices configured for the Issues Field are not valid. To resolve this, return to the Choice Criteria field within the prompt criteria tab and update it as necessary. For further details about deleting or modifying Choices, refer to Choices.

For best results when writing the Choice Criteria:

• Include keywords, phrases, legal concepts, parties, entities, and legal claims.
• Include any criteria that would exclude a document from relating to that issue, such as falling outside a certain date range.
• Be clear and specific; avoid ambiguity.
• Limit the criteria description for each choice to 5-10 sentences.
• Each choice must have its own criteria. If a choice has no criteria, either fill it in or delete the choice.
• Do not use a plain number for the Choice Criteria name, such as 12.

Considerations:

• The selected types, their associated positive choice, and any additional details can be changed between job runs. However, the CBI Field and Positive Choice Rank cannot be changed.
• Confidentiality ranking is controlled by the order of choices on the selected CBI field in the workspace, not within the prompt criteria panel in the aiR for Review application.

Fill out the following:

• Confidential Business Information Field—select the desired CBI field from the list. This field represents whether a document is sensitive/confidential or not. This selection cannot be changed after the first job run.
• Positive Choice Rank—depending on which CBI Field was selected, this list displays the relevant options in the same order as they appear in the workspace, with top choice being most confidential/highly sensitive.
  • The ranking is crucial as it will impact document results. You can adjust these rankings by navigating to Fields in the workspace, opening the CBI field, and reordering the Choices.
  • Use the X icon to delete non-confidential choices.
  • The CBI field and Positive Choice Rank cannot be modified after the first job run.
• Information Types—select one or more Information Types that you are interested in identifying in your dataset. Only the selected types will be analyzed in your documents. Be selective in choosing these types to avoid overlap. If there are multiple hits, only one citation will be displayed for each Information Type in the Viewer. See Confidential Business Information types for descriptions of each one.
• Positive Choice—select the confidential designation corresponding to the Information Type chosen. The choices in this field reflect what you selected for Positive Choice Rank.
• Additional Details—we strongly recommend providing additional context in this box to further clarify and describe each Information Type. This information assists the model's ability to make more accurate classification predictions. Suggested guidelines:
  • Include matter-specific context on how to identify the documents you are looking for, especially key words and phrases.
  • Be clear and specific; avoid ambiguity.
  • Limit the context added for each type to 5-10 sentences.

  Confidential Business Information types

  The Information types eligible for reporting are described below. When a document contains multiple hits, only one citation per type is shown.

  • Business Plans & Strategy—relates to internal materials discussing business direction, priorities, or planning activities.
  • Customer Testimonials—includes customer-provided statements or feedback regarding products or services.
  • Financial Data—refers to non-public financial-related materials used internally by a business.
  • General Confidential Business Data—a broad category intended to capture internal business information that does not fit within a more specific category.
  • Intellectual Property (General)—relates to materials associated with a company's intellectual property or innovation efforts.
  • Marketing Plans & Campaigns—includes information related to marketing initiatives, planning, or promotional efforts.
  • Sales Data—refers to internal information related to sales activities or performance.
  • Technical Information—includes information describing technical aspects of products, services, or operations.
  • Trade Secrets—refers to sensitive business information that a company treats as secret.

1. On the prompt criteria panel, click the Draft with AI button. You can also click the three vertical ellipses More () icon and select Draft with AI.
   
   
   
   
2. Upload content using any of the methods below for AI to draft prompt criteria. The total number of selected files and uploaded characters counters are displayed at the top and update automatically as changes occur.
   The maximum number of documents is 10, with a combined total character limit of 300,000.
   
   • Drop file here or browse files—drag or upload document files. Supported file formats are .txt, .docx, and .pdf.
   • Add text manually—click to manually add text or to copy and paste from another source. You can also use this option to include additional case context for the AI assistant to use.
3. Select the Document Type for each uploaded file. Options include: Review Protocol, Request for Production, General Case Memo, Complaint, Key Document, and Other. If you select Other, enter a document type description in the Custom Type box. The number of characters in each file appears below the filename to help keep track of the 300,000 character limit.
   
   
4. Repeat steps 2-3 to upload more files.
5. To delete a file from the list, click its corresponding circle X icon.
6. To use prompt kickstarter to automatically draft Issue descriptions for your prompt criteria, do the following:
   1. For Issues projects, use the Select Issues Field drop-down list below the documents list. For Relevance or Relevance and Key projects, enable the Include Issues in Relevance Criteria toggle below the documents list.
      Case issues will be included as part of the criteria for a Relevance analysis. However, documents will not be labeled with issue choices as part of a Relevance analysis.
   2. Set up the Issues field by selecting a single-choice or multi-choice field to indicate the issues involved in the case. Each option should identify a separate issue for which prompt kickstarter will generate a description. See Issues tab for more details on the fields. For best results drafting Issue descriptions:
      • Name each issue choice concisely and descriptively to help the AI model understand and generate accurate outputs. For example, using the Choice Criteria name, Anti-bribery policy in Acme Company internal communication will help the AI generate a description such as Documents, policies, or internal communications describing or outlining Acme Company's corporate policies related to ant-bribery or anti-corruption with public officials.
      • Correct any misspellings.
      • Do not use a plain number for the Choice Criteria name, such as 12.
      • Upload only relevant documents, and if necessary, include additional case context using the Add text manually option. The AI assistant uses contextual cues from the uploaded files. Providing unrelated or contradictory materials may degrade the quality of the generated output.
      If the AI assistant is unable to find relevant information in the uploaded documents for at least one of the selected issues, a message appears. You can try to manually enter the choice criteria description. Alternatively, you can run Draft with AI again after updating the choice name or uploading more content-rich documents.
7. Click Draft to begin automatically drafting prompt criteria from the uploaded documents or the manually entered content. Typically results display within 1-2 minutes.
   You cannot run document analysis during the drafting process.
8. Review and edit the draft prompt criteria entered in the available tabs. Click Save to keep the changes or Discard to delete them.
   
   
9. Repeat these steps with other documents and information as needed until the desired set of prompt criteria is achieved.
10. Review the available tabs and enter any required information on each.
11. Once the desired prompt criteria is set, click Start Analysis to analyze documents. For more information on analyzing documents, see Running the analysis
    The Draft with AI option is no longer available after the analysis process begins.