Active Learning Useful Dashboards

Recipe Overview

This recipe describes interesting dashboards that can be useful when doing an Active Learning project.

Directions

Note: The fields starting with "CSR" are created only when you run "update ranks" for the first time in your Active Learning project.

Make sure the following fields have Group By and Pivot On turned on:

  • AL Designation = Designation Field Name - the field used for coding by the reviewers.
  • CSR- Project Name Cat. Set::Category Rank - the field named after the project that contains the rank score for the classification.
  • Categories – Project Name Cat. Set - the field named after a project that contains the category applied by the system. This value is the positive choice if Rank >= cutoff and the other choice is Rank < cutoff. The user chooses the cutoff when they Import Ranks.
  • Active Learning Project Reviewers::User - the Reviewer ID captured during save of record.

The following widgets can be created with the fields referenced above.

Designation Field Coding

This widget tracks Documents Manually Coded by Reviewers. The AL Designation field could be named Responsive or Relevance but we have created a single choice field named AL Designation.

Fill out the following fields when creating the widget:

  • Group by: AL Designation
  • Group By Results Returned: All
  • Pivot on: <Grand Total>
  • Sort on: <Grand Total> (Descending)
  • Default Display Type: Pie Chart
  • Show Blank
  • Show Legend

Active Learning Results Summary

This widget will help you understand the system identified responsiveness.

Fill out the following fields when creating the widget:

  • Group by: Categories - Active Learning Project Cat. Set
  • Group By Results Returned: All
  • Pivot on: <Grand Total>
  • Sort on: <Grand Total> (DESCending)
  • Default Display Type: Pie Chart
  • Show Legend

Machine Classification against Coding Values

This widget compares the human coding values against machine classification. This is like the project bar chart.

Fill out the following fields when creating the widget:

  • Group by: CSR - Active Learning Project Cat. Set::Category Rank
  • Group By Results Returned: All
  • Pivot on: AL Designation
  • Pivot on Results Returned: All
  • Sort on: CSR - Active Learning Project Cat. Set::Category Rank (ASCending)
  • Default Display Type: Stacked Bar Chart
  • Vertical Orientation
  • Show Blank
  • Show Legend

User Coding Tracker

This widget will show the number of documents coded per reviewer. This is handy to understand who is coding what and track progress.

Fill out the following fields when creating the widget:

  • Group by: Active Learning Project Reviewers::User
  • Group By Results Returned: All
  • Pivot on: <Grand Total>
  • Sort on: <Grand Total> (DESCending)
  • Default Display Type: Bar Chart (Horizontal Orientation)
  • Show Legend

Keep the following in mind when creating the widget:

  • Select the Responsive items coded by reviewers. Select the Non Responsive classified items. Hopefully this doesn’t return any documents. This shows conflicts or RAR like overturns. You can also consult the Reviewers::User widget to see who coded these documents.
  • Select the Non Responsive items coded by reviewers. Select the Responsive classified items. Once again hopefully this doesn’t return any documents. This shows conflicts or RAR like overturns.
  • Select a reviewer. See how the coding values change to understand how the coding breakdown occurs per person. Is someone coding everything as Responsive or nothing as Responsive. How do reviewers compare.