Detector QC

Detector QC is the process by which humans adjust detectors for precision and recall before the review team begins their review. You can use the following suggested workflow to complete this task.

QCing batches containing a certain PI type

QC batches typically fall into one of two categories, batches containing PI and batches not containing PI.

To QC documents containing PI:

  1. From the Project Dashboard tab, select a document to view.
  2. Click the Annotations List tab on the right-hand side of the document viewer. This will show all the PI predictions in the document.
    Note: To navigate to the exact location of the PI on a document, click on the PI value in the Annotation List.

    An image of the Annotation List tab.
  3. If the PI prediction is a false positive, we recommend taking note of the over-prediction in a way that makes sense to you, without logging the actual PI. You can utilize the note when you actually make the adjustment to detectors or add to the blocklist.
  4. After determining the accuracy of predicted PI, review the rest of the document for misses.
    If you see missed PI take note of the under-prediction in your records, as you will likely want to update a detector to include the miss.
    An example of PI annotations.
  5. When you are satisfied reviewing the document, click the > arrow in the upper-middle portion of the screen to move to the next document.
    Note: Do not click Mark Complete on the right side of the screen. Documents marked Complete are no longer eligible for batching by a first-level review. However, any document marked Complete can have its status updated using the Mass Edit functionality. For more information on batching, see Batching and batch management.
  6. Continue through the documents in your batch. Look out for the document ID you started with. When you see it again, you have finished the batch.
  7. To submit the batch, navigate to the annotation tab, select the three dots next to your batch name and select Finish Batch.

QCing batches containing no PI

If you are QCing a No PI batch, review the document to get an idea of what type of document you are handling. Some types of documents, such as training manuals, advertisements, business receipts, or academic papers are less likely to contain PI. Emails, internal reports, and customer-related spreadsheets are more likely to contain PI.

You can use the Search feature in Native View to look up common phrases that are typically near PI. Phrases like “SSN,” “DOB,” “cell,” “mobile” and more can be used to find potential under-predictions.

To search, click the magnifying glass icon on the right-hand side of the screen in Native view.
This opens a search field where you can type in words or phrases you believe could appear in the document.
If they are found, click the results to navigate to that part of the document.
An image of the viewer with the Search icon highlighted.

Note: If you see any missed PI, take note of the underprediction so that you can update the appropriate detector.

Continue with this workflow until you have viewed all documents in your batch. To submit the batch, navigate to the annotation tab, select the three dots next to your batch name and select Finish Batch.