A pre-coded seed round only includes documents that have already been coded during manual review for the purpose of expediting training and are not included as part of another round. RAR identifies sample documents based on those found in the specified saved search with a value set on the Designation field.
- Documents included in the pre-coded seed round sample MUST be reviewed using the proper
Sample-Based LearningProtocol. See Sample-Based Learning standards and protocol.
- Ensure that the documents are coded using the same designation field as is set on the project. Also, ensure that you don't have an excessive amount (thousands) of documents in the sample when you start the round.
This page contains the following sections:
- Executing a pre-coded seed round
- Reviewing documents for a pre-coded seed round
- Finishing a pre-coded seed round
- Reviewing Sample-Based Learning reports after a pre-coded seed round
- It is important that a pre-coded seed round contain documents with both responsive and not responsive designations to give the system enough information to properly categorize documents in your project.
- The system does not consider the Use as an example field in determining eligibility. Therefore, if a document is coded on the designation field and is not currently in another round in the project, that document is eligible for inclusion in a pre-coded seed set even if the Use as example field is set to No or blank (not set).
- If the documents used in a Pre-coded seeds round are coded after the project is created, the Seed Reviewer field will be filled for any overturns caused by these pre-coded seeds. However, if the documents were coded via Mass Edit, the Seed Reviewer field will be empty.
- If the documents used in a Pre-coded seeds round were coded before the project was created, the Seed Reviewer field will be empty for any overturns caused by these pre-coded seeds.
- You can't use pre-coded sets for quality control. If documents in the pre-coded seed rounds have already been categorized, they won't create overturns.
To execute a pre-coded seed round:
- Click Start Round on the console.
- Select Pre-coded seeds as the Round Type.
- For the Saved search for sampling, select a search that contains all of the previously coded documents which you would like to include. If the search also includes documents that haven’t been coded, they are automatically excluded when the sample is created.
- Specify your desired Sampling Methodology settings. The sample set is the randomly-selected group of documents produced by to be used for manual review as a means of training the system.
Note: The fields in the Sampling Methodology section are defaulted to the values on the project settings; however, if you select Training as the round type, you override those default values.
- Statistical sampling- creates a sample set based on statistical sample calculations, which determines how many documents your reviewers need to code in order to get results that reflect the project universe as precisely as needed. Selecting this option makes the Margin of error field required.
- Confidence level - the probability that the rate in the sample is a good measure of the rate in the project universe. This is used in the round to calculate the overturn range as well as the sample size, if you use statistical sampling.
- Margin of error - the predicted difference between the observed rate in the sample and the true rate in the project universe. This is used in the round to calculate the overturn range as well as the sample size, if you use statistical sampling.
- Percentage - creates a sample set based on a specific percentage of documents from the project universe. Selecting this option makes the Sampling percentage field required.
- Sampling percentage - the percentage of the eligible sample population used to create the sample size.
- Fixed sample size - creates a sample set based on a specific number of documents from the project universe. Selecting this option makes the second Fixed sample size field required.
- Fixed sample size - the number of documents you want to include in your sample size.
Clicking Calculate sample displays the number of documents in the saved search selected for the round and the number of documents in the sample. If the values for the sample and/or saved search are unexpected, you can change any setting in the Start Round layout and re-calculate before clicking Go. You can't calculate the sample if you don't have access to the saved search selected for the round. This button is disabled if you've selected Stratified sampling as the sampling type.
- Automatically create batches - determines whether or not a batch set and batches are automatically created for this round's sample set. By default, this field is set to whatever value was specified in the project settings. Once the sample set has been created, you can view and edit the corresponding batch set in the Batch Sets tab.
- Maximum batch size - the maximum number of documents that the automatically-created batches will contain. This is required if the Automatically create batches field above is set to Yes. This value must be greater than zero or an error appears when you attempt to save the project. The batch set and batches created from this project are editable after you create the project. By default, this field is set to whatever value was specified in the project settings.
Note: When the round is initially created, the field specified as the Use as an example field is set to Yes by default for documents included in the round. If you delete a round, RAR reverts the Use as an example field value to Not Set (null).
Documents for a pre-coded seed round do not need to be reviewed by a set of reviewers. See Sample-Based Learning document review for more information on protocol for assigning documents out and reviewing documents during a round.
Note: If you're done using a project, it's better for workspace performance if you finish the round rather than leaving it in a status of either Review in Progress or Review complete.
Once all of the documents in the sample set have been coded, you should finish the round. You also have the option of finishing a round before all of the sample set documents have been coded.
To finish a pre-coded seed round:
- Click Finish Round on the console.
- Specify whether you want to categorize documents when you finish the round. You have two options depending on your project:
- Categorize for designation - categorize all documents in the project based on their designation coding.
- Categorize for issues - categorize all documents in the project based on their issue coding. This is only available if you have added a key issue field to the project and a reviewer has issue-coded at least one document in the sample set.
- Specify whether you want to save categorization results from the previous round when you finish the current round. You may have two options depending on your project:
- Save designation results - save the results of designation coding from the previous categorization. This is useful because when categorization runs, the previous results are cleared in order to apply the new category values. You can't save designation results if you did not categorize designations in a previous round.
- Save issue results - save the results of issue coding from the previous categorization. This is only available if you have added a key issue field to the project. You can only save issue results if you categorized issues in a previous round.
Note: You shouldn't save results at the end of every round. Saving results, especially for larger cases, can add several hours to the time it takes to finish the round.
- Enter the naming for your categorization results.
- Categorization results set name - the name of the categorization results set. By default, this is the name of the previous round. This is only available for editing if you are saving designation and/or issue results.
- Categorization results set description - a description of the categorization results. This is only available for editing if you are saving designation and/or issues results.
- Click Go. If you choose to both categorize and save results, the saving of results is performed first, then categorization.
The following reports should be reviewed after a pre-coded seed round:
- Round Summary report – useful after categorization because it shows the changes in categorization percentage from round to round and also provides categorization volatility. See Round Summary report
Note: If issues are also being categorized by