Coverage Review

Coverage Review serves up documents that are optimal for training the model. The goal of Coverage Review is to quickly separate your documents into the positive choice and negative choice categories. Unlike Prioritized Review, which serves up the highest ranking documents, Coverage Review is intended for quick production use cases where you want to train the model as fast as possible.

The documents that are served up during Coverage Review can be either relevant or non-relevant and are the most impactful to training the model. Coverage Review begins by serving the documents the model is most unsure about - these are documents with a rank near 50. Coverage Review continues serving up documents until there are no longer any documents to review.

Special considerations

In order to make the Coverage Review even more efficient, we recommend suppressing duplicate documents from your Active Learning project on project setup. For more information, see Project setup.

Starting the Coverage Review

Note: Starting the Coverage Review disables all other queues.

To start the review, click Start Review. If you haven't already added reviewers to your queue, you are prompted to add reviewers. For more information, see Adding Reviewers.

After the project begins, you can see the following items In the Coverage Review modal:

  • Coded - the number of documents coded or skipped in the Prioritized Review.
  • Active Reviewers - the number of reviewers added to the review.

Admins can pause the review at any point in the project by clicking the Pause Review button on the project modal. Once the review is paused, the access point on the document view is disabled. You can restart the review at any time. Documents coded on the review field while the review is paused are included the next time the model rebuilds. For more information, see Project Setup.

Adding Reviewers

From the project homepage, admins can add reviewers to each queue on a user-by-user basis or in bulk.

To add reviewers:

  1. Click Add Reviewers in the queue modal.
  2. Find and select an individual reviewer, or type to enter a reviewer name. You can also click All Users to select every user in the group.
  3. Click the green check mark to save your changes.
  4. Click the red X to cancel reviewer changes.

Note: We recommend no more than 20 concurrent reviewers per project. Concurrent reviewers are defined as reviewers making coding decisions in an Active Learning queue. There is no limit to how many reviewers you can add to a queue as long as the number of concurrent reviewers remains at 20 or fewer.