Review Center - Admin quick reference guide

This guide provides step-by-step instructions for creating and administering a Review Center queue, with a checklist at the end. This guide is intended as a quick reference, and it does not explain the features in depth.

For more detailed instructions, see the Review Center section of the documentation site.

Choose a queue type

  1. Saved Search—the queue serves up documents from a saved search, in the order determined by the saved search sort.

  2. Prioritized Review—the queue uses artificial intelligence to predict which documents are most likely to be coded positive, and it serves those up first. To train the predictions, you need to code at least one positive and one negative document in the data set before starting the queue. Coding more documents leads to more accurate predictions.

All queues can be set to serve entire relational groups to the same reviewer.

Create the required fields and choices

  • Review Field—a single-choice field that serves as the coding field for your queue. This field should have at least one positive choice (such as Responsive) and one negative choice (such as Not Responsive.) Any other choices will be considered neutral.

  • Rank Output (prioritized review queue only)— a decimal field that will hold the document ranks. Each prioritized review queue needs a separate Rank Output field on the Document object.

Set up the reviewer permissions group

  1. Decide which user group should contain the reviewers for the queue. You can either create a brand new group, or modify the permissions for an existing user group. You can assign multiple groups to the same queue.

  2. Add the group to the workspace.

  3. Assign the reviewer group these permissions:

    • Object Security: Document - View and Document - Edit

    • Object Security: Review Center Queue - View

    • Tab Visibility: Review Queues tab

Set up the layout

Create a coding layout that includes the queue’s review field. Reviewers will use this layout to code documents in the queue. You can also add any other fields you want such as rank output, issue coding, privilege coding, a field to indicate further review needed or technical issues, and more.

Create the saved search

Create a saved search that contains the documents you want in the Review Center queue.

Note: For prioritized review queues, the AI classifier only uses the extracted text field to make its predictions. Returning other fields in the saved search does not affect the results.

Create or customize the queue template

Review Center uses templates to speed up the process of setting up multiple queues. You can either customize one of the pre-made templates included with Review Center, or create your own.

If you already have a template customized for your workspace, you can skip this step.

  1. Navigate to the Review Library tab.

  2. Either click the Edit pencil next to an existing template, or click the New Review Center Queue button. Enter or edit the following information:

    • Name—the queue name reviewers will see.

    • Is Template—toggle this to On.

    • Template Description—enter notes about the template such as its intended use, comments about field settings, etc.

    • Queue Label—create and choose organizational labels that will apply to queues created from this template.

    • Reviewer Groups—this is not recommended for templates.

    • Queue Type—choose either Saved Search or Prioritized Review.

    • Data Source—select the saved search that contains the documents for your queue.

    • Rank Output (Prioritized Review only)—select the decimal field you created to hold the document rank scores.

    • Review Field—select the single choice field you created for review. This field must have two or more choices.

      • Positive Choice—select the choice that represents the positive or responsive designation.

      • Negative Choice—select the choice that represents the negative or non-responsive designation. Any remaining choices are considered neutral.

    • Relational Field—select a relational field for grouping documents in the queue. This makes reviewers receive related documents together, such as members of the same document family.

      Note: If you set a relational field on a template or queue, set the same field in the Related Items drop-down of the saved search Conditions tab. Only relational group members returned by the saved search will be included in the queue.

    • Allow Coded in Review (Saved Search only)—controls whether documents coded outside of the queue will still be served up in the queue.

      • Toggle this On to allow outside-coded documents to be served up.

      • Toggle this Off to exclude outside-coded documents.

        Note: Prioritized review queues use outside-coded documents to train their predictions, but they only show them to reviewers if the Relational Field is set.

    • Queue Display Options—select which statistics you want reviewers to see on the queue card in the Review Queues tab.

    • Reviewer Document View—select a view to control which fields reviewers see in the Documents panel of the Viewer.

    • Queue Refresh—controls whether the queue automatically refreshes after coding activity. This refresh includes re-running the saved search and checking for outside-coded documents. For prioritized review queues, this also re-trains the classifier with the latest coding and re-ranks documents in order of predicted relevance.

    • Email Notification Recipients—enter email addresses you want to be notified when a manual queue preparation is complete, a queue is empty, or a queue errors upon population. Separate multiple email addresses with a semicolon. Do not include a space.

  3. Click Save.

Create the queue

  1. Navigate to the Review Center tab.

  2. Click the Add Queue button.

  3. Select the template you want to use, then click Next.

  4. Under Reviewer Groups, choose one or more reviewer groups.

  5. In the other fields, check the default values filled in by the template. Change any values that should be different for this queue.

  6. Click Save.

The new queue appears as a tab in the banner at the top of the dashboard.

Note: After a queue has been created from a template, the two of them are no longer connected. You can edit the template without affecting the queue.

Activate and manage the queue

From the Review Center dashboard, you can start, track, manage, and edit all Review Center queues.

To start your queue, follow these steps:

  1. (Prioritized review queue only) Make sure at least one document in the data set is already coded positive, and at least one is coded negative. For even more efficient AI predictions, try having at least 10 positive-coded and 10 negative-coded documents.

  2. Navigate to the Review Center tab.

  3. If you have more than one queue, their names will be listed as tabs across the top. Click on the queue you want to start.

  4. (Optional) Click Prepare Only. This re-runs the saved search for all queues and retrains the AI classifier for prioritized review queues. It does not start the queue.

  5. Click Start Review. If you chose not to prepare the queue in advance, this will say Prepare and Start. After the queue has started, your reviewers can start reviewing.

After the queue has finished starting, a pause button will appear next to the words Active Queue. Clicking the button pauses the queue and stops reviewers from checking out more documents.

Update the queue and the classifier

If Queue Refresh is set to On, a queue with active reviewers will automatically re-run the saved search and update the classifier (for prioritized review queues) when there is coding activity in any queue. You can change this setting at any time.

If you need to trigger an immediate refresh, click on the words Auto Refresh to trigger an additional manual refresh. For example, if new documents have been added to the saved search, you can click this to add them to the queue quickly instead of waiting until the next auto-refresh. If Queue Refresh is turned off, this option will say Refresh Queue.

Review documents

  1. Navigate to the Review Queues tab.

  2. Each queue you are assigned to has a separate card. Locate the card with the same name as the queue you want.

  3. Click Start Review on the queue card.
    This brings you into the document viewer where you can review documents.

As you are reviewing, you can view and navigate to documents you already reviewed in the queue by clicking the word Documents on the left side of the viewer.

Review Center queue checklist

Can be done in your workspace template:

£ Create the review field.

£ Create at least two choices for the coding field.

£ For a prioritized review queue, create the rank output field. Each prioritized review queue needs its own rank output field.

£ Set up a review layout that includes the review field.

£ Create a saved search for documents you want in the queue.

£ Create the queue template.

Must be done in the workspace:

£ Create the reviewer permissions group.

£ Create a queue.

£ For a prioritized review queue, code at least 1 non-empty document positive and 1 non-empty document negative. The more documents you code in advance, the better the starting predictions will be.

£ (Optional) Prepare the queue in advance.

£ Start the queue