Using project sets

Project sets make it easier to manage and refine prompt criteria for aiR for Review projects. They let you create new sets within a single aiR project so you can iterate without having to start over with a new aiR project. You can use these sets to change saved searches (data sources) during prompt criteria development, to switch between sets to review results, or to start prompt criteria validation in Review Center.

Example of drop-down list showing four versions of the project in the develop phase.

The project set workflow outlined below aligns with aiR for Review’s phases (see aiR for Review phases).

  1. Develop: This set type is used to develop your prompt criteria on a small document set. With develop sets, the data source (saved search) being used in the project can be changed to allow greater flexibility to run multiple jobs within a single aiR project.
  2. Validate: This set type is used to validate your prompt criteria leveraging Review Center.
  3. Apply: This set type is commonly used after validation to apply your prompt criteria to a larger document population.

The project set details strip contains the following:

Project sets bar with callouts for each section

1-the set type (Develop, Validate, Apply).

2-the saved search used for the set, number of documents analyzed, and number of documents in the saved search.

3-the version number of the prompt criteria for the set and date it ran or was saved.

4-click the down arrow to track the progress of your aiR for Review project and switch between project sets to review predictions and metrics.

5-click the + sign to create a new project set using a different data source, to validate the currently selected prompt criteria version, or to apply it to a larger document population.

Creating a new develop project set

Use the steps below to create a new develop project set within the current aiR for Review project. This enables you to change the data source (saved search) used to iterate on the prompt criteria and view results.

  1. Click the project sets + sign.
  2. Click I want to develop my prompt criteria using a different data source.
    Create a new set dialog with highlight box around the option, "I want to develop my prompt criteria using a different data source."
  3. Click Select and choose the data source (saved search) to use for this project set.
  4. Click Create Develop Set.
  5. Proceed with modifying the prompt criteria as needed.

Validating the prompt criteria

Use the steps below to validate the project set's prompt criteria within the current aiR for Review project.

  1. Within the desired project set, click the project sets + sign.
  2. Click I want to validate the prompt criteria on a target document population.
    Create a new set dialog with highlight box around the option, "I want to validate the prompt criteria on a target document population."
  3. Click Select and choose the data source (saved search) to use for this project set.
  4. Fill out the Validation Settings fields to determine the validation sample. For more information on each field, see Validation Settings fields.
  5. Review the confirmation summary modal and provide email addresses, if needed, then click Start Analysis.
  6. Click Go to Review Center to begin the human review and coding of the validation sample in Review Center. Refer to Prompt Criteria validation in Review Center for the steps to follow in Review Center.

For more information on validating prompt criteria, see Setting up aiR for Review prompt criteria validation.

Applying the prompt criteria

Use the steps below to apply the project set's prompt criteria within the current aiR for Review project.

  1. Within the desired project set, click the project sets + sign.
  2. Click I want to apply my prompt criteria to a document population.
    Create a new set dialog with highlight box around the option, "I want to apply the prompt criteria to a document population."
  3. Click Create Apply Set.
  4. Review the confirmation summary modal and provide email addresses, if needed, then click Start Analysis. For more information, see Running the analysis.
Conflicts are not displayed in the Project Metrics section of the dashboard after running Apply on a project set. See Version Metrics tab for more information on the Conflicts tab.

After the prompt criteria are applied to the document population, they cannot be changed. The only way to revise the criteria is by creating a new develop project set and updating it. See Creating a new develop project set for the details.

Please note that if the project set has already completed the prompt criteria validation process, you cannot change it by creating a new project set. Instead, you must start a new aiR for Review project. For additional guidance, see Validating prompt criteria and Setting up the project.