Report sample-based statistics

This script generates reports for the deprecated sample-based learning feature. If you have legacy sample-based learning projects in your workspace, you can select one and generate several types of reports based on its contents.

Because the sample-based learning feature is now deprecated, you cannot modify the data within the projects or create new ones. These reports show each legacy project as-is, regardless of whether the project was completed.

Report types

The available reports are:

  • Project Summary—this provides a consolidated set of quality metrics from the entire project so that you can see the state of the project based on the last round completed. It includes manually coded documents, project categorization results, project overturn results, and control set statistics.

  • Overturn Summary—this provides a high-level view of a project’s overturn percentages and tallies from round to round. The accompanying table gives even greater detail, including a breakdown of responsive and non-responsive overturn rates.

  • Round Summary—this report is split into two graphs. The first graph provides a categorization breakdown of each category per round, allowing you to track categorization results as the project progressed. The second graph displays the volatility of each category per round, while accompanying tables provide additional information such as document tallies and population size changes.

  • Issue Summary—this provides a breakdown of issue coding categorization results. The table version of the report also displays document tallies and previous round information.

  • Designation Issue Comparison—this provides insight into designation and issue coding categorization results and how they relate to one another.

  • Rank Distribution—this provides insight into categorization results according to their rank score. The accompanying table also provides document tallies.

  • Control Set Statistics—this displays a project’s accuracy trends from round to round. The report displays precision, recall, and F1 scores for each round in a single line chart.

Inputs

  1. Navigate to the Scripts tab.

  2. Select the Report Sample-Based Statistics script. If the script is not present in the workspace's Script tab, you must add it.

  3. Click Run.

  4. In the pop-up window, fill out:

    1. Project—select the legacy project you want the report based on.

    2. Report Type—select the type of report.

  5. Click Run.

  6. Click Accept on the pop-up confirmation.

This script may take a few minutes to process.

Results

After the script finishes, a custom URL will appear in the Script Output pane. To view the report, copy and paste the URL into a new browser tab.

This report can be exported into a variety of formats using the drop-down menu in the upper right corner.