The functionality described on this site is available in Early Access only. It appears only in a preview environment and not in production. The content on this site is subject to change based on modifications made by product development and/or feedback provided by Relativity customers. Once we deploy Early Access functionality to RelativityOne production, this site will no longer display content until the next Early Access release is available in a preview environment.

Review statistics

The Review Statistics tab contains project statistics for each queue as well as a model update history.

This page contains the following information:

Review Summary

The Review Summary section contains four tabs:

Prioritized Review

The Prioritized Review tab shows the effectiveness of the Prioritized Review queue’s ability to locate relevant documents by reporting the review field breakdown and relevance rate each 200 documents. For every 200 documents that are coded, a new row appears in the Prioritized Review table. The first row in the table provides a summary for the entire project.

The Prioritized Review table contains the following columns:

  • Prioritized Review - the set of documents the statistics apply to.
  • # of Reviewers - the number of unique reviewers who reviewed documents in the Prioritized Review queue.
  • Coded [Positive Choice] - the number of documents coded with the positive designation on the review field.
  • Coded [Negative Choice] - the number of documents coded with the negative designation on the review field.
  • Skipped - the number of documents that were saved or had Save and Next selected with no coding decision supplied on the review field.
  • Index Health - the number of index health documents reviewed in the Prioritized Review queue. These documents are excluded from the relevance rate calculation.
  • Highest Ranked - the number of highly ranked documents reviewed in the Prioritized Review queue.
  • Highest Ranked Coded [Positive Choice] - the number of highly ranked documents that were coded with the positive designation in the Prioritized Review queue.
  • Relevance Rate - the percentage of documents that were predicted to be relevant that were then confirmed as relevant by reviewers' coding decisions. You can calculate the relevance rate manually using the following formula: Highest Ranked Coded [Positive Choice] / Highest Ranked.
  • Additional Reviewed by Family - the number of family documents reviewed in the Prioritized Review queue.
  • Additional Review by Family Coded [Positive Choice] - the number of family documents coded with the positive designation in the Prioritized Review queue.

(Click to expand)

Coverage Review

The Coverage Review tab shows the progress of the Coverage Review by reporting the review field breakdown every 200 documents. For every 200 documents that are coded, a new row appears in the Coverage Review table. The first row in the table provides a summary for the entire project.

The Coverage Review table contains the following columns:

  • Coverage Review - the set of documents the statistics apply to.
  • # of Reviewers - the number of unique reviewers who reviewed documents in the Coverage Review queue.
  • Coded [Positive Choice] - the number of documents coded with the positive designation on the review field.
  • Coded [Negative Choice] - the number of documents coded with the negative designation on the review field.
  • Skipped - the number of documents that were saved or had Save and Next selected with no coding decision supplied on the review field.

(Click to expand)

Elusion Test

Use the Elusion Test tab to monitor the progress to completing the Active Learning project. A new row is created when an Elusion Test starts and is populated with the available information about the Elusion Test. The last four columns populate as documents are coded and appear upon page refresh. If an Elusion Test is stopped before completion, the last four columns display values for what was coded.

The Elusion Test tab contains the following columns:

  • Review - each review is called Elusion Test plus a numeral. For example, the first Elusion Test is "Elusion Test 1" and it will increase by one each sequential Elusion Test.
  • Start Date - the UTC date and time the Elusion Test was started.
  • Rank Cutoff - user input when turning on the Elusion Test.
  • Discard Pile Size - Number of documents below the rank cutoff that are not coded.
  • Sample Size - number of documents in the Elusion Test. This number is computed when the Elusion Test is started.
  • Skipped - the number of documents that were saved or had Save and Next selected with no coding decision supplied on the review field.
  • Pending Document Count - number of documents with coding changes, including documents coded in the Elusion Test (not skipped documents in Elusion Test) and manually selected documents not contributing to the model's learning.
  • Confidence Level - user input or calculated when turning on the Elusion Test.
  • Margin of Error - user input when turning on the Elusion Test. Margin of error can change if documents were skipped in Elusion Test.
  • Coded [Positive choice] - the number of documents coded with the positive designation on the review field.
  • Coded [Negative choice] - the number of documents that were coded on the positive choice designation field outside of the project.
  • Elusion Rate - the Elusion Rate measured in percent. Please note this value is rounded to the nearest percent.
  • Eluded Document - the Elusion Rate measured in number of documents.

(Click to expand)

Manually-Selected

The Manually-selected Documents chart displays the number of document coding decisions made outside of the Active Learning queue on a given day.

The Manually-Selected tab contains the following columns:

  • Manually-selected Documents - the number of documents coded outside of the Active Learning queue.
  • Coded [Positive Choice] - the number of documents coded with the positive designation on the review field.
  • Coded [Negative Choice] - the number of documents coded with the negative designation on the review field.
  • Date submitted (UTC) - the date in UTC that the statistics were submitted.

(Click to expand)

Model Updates

The Model Updates section contains a history of Active Learning model builds. A new row is added each time the model builds, and the statistics are based off of the responsive cutoff set in the Project Settings. If you update the responsive cutoff at any point, the statistics will update accordingly.

  • Build Date - a timestamp indicating when the model build completed, displayed in local time.
  • Above or At Cutoff - the number of documents above or at the responsive cutoff.
  • Below Cutoff - the number of documents below the responsive cutoff.

The subsequent columns indicate the number of documents in each relevance rank range.

(click to expand)