Detector QC

Before the review team begins their review, you can adjust detectors for precision and recall. Use the following suggested workflow to complete this process.

QC batches that contain PI

QC batches typically fall into one of two categories, batches containing PI and batches not containing PI. The following steps assume you have created Review Queues as outlined in Quality control.

To QC documents that contain PI:

1. Navigate to the Review Queues tab and locate the card for the queue you want to review.
2. Click Start Review to open the document viewer.
3. Open the PI Detection panel on the left side of the document viewer. The panel displays all PI predictions in the document.
4. Review each detected PI prediction.
   • If a prediction is a false positive, take note of the over-prediction without logging the actual PI. You can reference this note when you adjust detectors or add terms to the blocklist.
5. Review the rest of the document for any PI that the detectors did not capture.
   • If you identify missed PI, take note of the under-prediction so you can update the appropriate detector.

6. Click Save and Next.

   Do not lock the document. Locking prevents detector changes from being applied the next time you run Data Analysis.

7. Continue through the remaining documents in the batch. When you see the document ID you started with, you have completed the batch.

QC batches that do not contain PI

When you QC a batch with no identified PI, start by identifying the document type. The following types of documents are less likely to contain PI:

• Training manuals
• Advertisements
• Business receipts
• Academic papers

The following types are more likely to contain PI:

• Emails
• Internal reports
• Customer-related spreadsheets

Use the Search feature to look up common phrases that typically appear near PI, such as:

• "SSN"
• "DOB"
• "cell"
• "mobile"

If you identify PI that a detector did not capture, take note of the under-prediction so you can update the appropriate detector.