Review statistics
The Review Statistics tab contains project statistics for each queue
This page contains the following information:
Review Summary
The Review Summary section contains four tabs:
Prioritized Review
The Prioritized Review tab shows the effectiveness of the Prioritized Review queue’s ability to locate relevant documents by reporting the review field breakdown and relevance rate each 200 documents. For every 200 documents that are coded (excluding additional reviewed by family), a new row appears in the Prioritized Review table. The first row in the table provides a summary for the entire project.
The Prioritized Review table contains the following columns:
- Prioritized Review - the set of documents the statistics apply to (excludes additional reviewed by family). The sum of the count of Coded [Positive Choice], Coded [Negative Choice], and Skipped documents should equal 200.
- # of Reviewers - the number of unique reviewers who reviewed documents in the Prioritized Review queue.
- Coded [Positive Choice] - the number of documents coded with the positive designation on the review field (excludes additional reviewed by family).
- Coded [Negative Choice] - the number of documents (excluding family) coded with the negative designation on the review field (excludes additional reviewed by family).
- Skipped - the number of documents that were saved or had Save and Next selected with no coding decision supplied on the review field (excludes additional reviewed by family).
- Index Health - the number of index health documents reviewed in the Prioritized Review queue. These documents are excluded from the relevance rate calculation.
- Highest Ranked - the number of highly ranked documents reviewed in the Prioritized Review queue.
- Highest Ranked Coded [Positive Choice] - the number of highly ranked documents that were coded with the positive designation in the Prioritized Review queue.
- Relevance Rate - the percentage of documents that were predicted to be relevant that were then confirmed as relevant by reviewers' coding decisions.
- Additional Reviewed by Family - the number of family documents reviewed in the Prioritized Review queue.
- Additional Review by Family Coded [Positive Choice] - the number of family documents coded with the positive designation in the Prioritized Review queue.
(Click to expand)
Coverage Review
The Coverage Review tab shows the progress of the Coverage Review by reporting the review field breakdown every 200 documents. For every 200 documents that are coded, a new row appears in the Coverage Review table. The first row in the table provides a summary for the entire project.
The Coverage Review table contains the following columns:
- Coverage Review - the set of documents the statistics apply to.
- # of Reviewers - the number of unique reviewers who reviewed documents in the Coverage Review queue.
- Coded [Positive Choice] - the number of documents coded with the positive designation on the review field.
- Coded [Negative Choice] - the number of documents coded with the negative designation on the review field.
- Skipped - the number of documents that were saved or had Save and Next selected with no coding decision supplied on the review field.
(Click to expand)
Elusion Test
Use the Elusion Test tab to monitor the progress to completing the Active Learning project. A new row is created when an Elusion Test starts and is populated with the available information about the Elusion Test. The last four columns populate as documents are coded and appear upon page refresh. If an Elusion Test is stopped before completion, the last four columns display values for what was coded.
The Elusion Test tab contains the following columns:
- Review - each review is called Elusion Test plus a numeral. For example, the first Elusion Test is "Elusion Test 1" and it will increase by one each sequential Elusion Test.
- Start Date - the UTC date and time the Elusion Test was started.
- Rank Cutoff - user input when turning on the Elusion Test.
- Discard Pile Size - Number of documents below the rank cutoff that are not coded.
- Sample Size - number of documents in the Elusion Test. This number is computed when the Elusion Test is started.
- Skipped - the number of documents that were saved or had Save and Next selected with no coding decision supplied on the review field.
- Pending Document Count - number of documents with coding changes, including documents coded in the Elusion Test (not skipped documents in Elusion Test) and manually selected documents not contributing to the model's learning.
- Confidence Level - user input or calculated when turning on the Elusion Test.
- Margin of Error - user input when turning on the Elusion Test. Margin of error can change if documents were skipped in Elusion Test.
- Coded [Positive choice] - the number of documents coded with the positive designation on the review field.
- Coded [Negative choice] - the number of documents coded with the negative designation on the review field.
- Elusion Rate - the percentage of documents coded relevant in the elusion sample. Please note this value is rounded to the nearest percent.
- Eluded Document - the Elusion Rate measured in number of documents.
(Click to expand)
Manually-Selected
The Manually-Selected tab contains the following columns:
- Manually-selected Documents - the number of documents coded outside of the Active Learning queue.
- Coded [Positive Choice] - the number of documents coded with the positive designation on the review field.
- Coded [Negative Choice] - the number of documents coded with the negative designation on the review field.
- Date submitted (UTC) - the date in UTC that the statistics were submitted.
(Click to expand)
Model Updates
The Model Updates section contains a history of Active Learning model builds. A new row is added each time the model builds, and the statistics are based off of the responsive cutoff set in the Project Settings. If you update the responsive cutoff at any point, the statistics will update accordingly.
- Build Date - a timestamp indicating when the model build completed, displayed in local time.
- Above or At Cutoff - the number of documents above or at the responsive cutoff.
- Below Cutoff - the number of documents below the responsive cutoff.
The subsequent columns indicate the number of documents in each relevance rank range.
(click to expand)
