Clustering

Analytics uses clustering to create groups of conceptually similar documents. With clusters, you can identify conceptual groups in a workspace or subset of documents using an existing Analytics index. Unlike categorization, clustering doesn't require much user input. You can create clusters based on your selected documents without requiring example documents or category definitions. You can view the clusters identified by the index in the Cluster browser available on the Documents tab.

When you submit documents for clustering, the Analytics engine determines the positions of the documents in the conceptual index. Depending on the conceptual similarity, the index identifies the most logical groupings of documents and places them into clusters. Once the Analytics engine creates these clusters, it runs a naming algorithm to label each node in the hierarchy appropriately to refer to the conceptual content of the clustered documents.

Clustering is useful when working with unfamiliar data sets. However, because clustering is unsupervised, the Analytics engine doesn't indicate which concepts are of particular interest to you. Use investigative features such as Sampling, Searching, or Pivot in order to find the clusters that are of most interest.

Note: Clustering doesn't perform conceptual culling of irrelevant documents, but there is a group created for documents without searchable text. The name of the group is UNCLUSTERED. This group can be used to find documents that don’t have enough searchable data. All documents with conceptual data get clustered somewhere once. Clustering may be used on any list of documents. For example: a saved search based on custodians, search term results, date ranges or the entire workspace.

After you've created clusters, you can use those clusters to create batches to speed up a linear review.

See these related pages:

• Cluster visualization

Creating or replacing a cluster

You can create new clusters or replace existing ones. The steps for these tasks are similar, except that you need to select an existing cluster when you perform a replacement.

To automatically create a cluster when creating a conceptual index, leave the Cluster Documents option set to Yes. For more information, see the following:

• Creating an Analytics index
• Default settings for automatically created clusters

To manually create a new cluster or replace an existing one:

1. Navigate to the Documents tab and find the documents that you wish to cluster. This might be a saved search, folder, or all documents in the workspace.
2. Select the documents you want to cluster. You can click the checkboxes of individual documents in the list and select Checked from the drop-down menu. Alternatively, you can select All to include all documents.
3. Select Cluster from the Mass Operations drop-down menu.
   The Cluster X Documents pop-up displays.
4. Complete the fields on the Cluster dialog. See Fields.
5. Once you've completed the fields on the Cluster dialog, click Submit for Clustering.

To view the cluster visualization on the Documents tab, do the following:

1. On the Documents tab, click the Add Widget button and select Cluster Visualization from the drop-down menu.
2. Select the desired Cluster Set from the drop-down menu.
3. Click Apply.
   The selected cluster visualization displays on the Documents tab.

Fields

The following fields display on the Cluster window:

• Mode - determines whether this is a new cluster or a replacement
  • Create New Cluster - creates a new cluster
  • Replace Existing Cluster - replaces a cluster that already exists. See Replacing an existing cluster.
• Name - the name of the new cluster. If you are replacing an existing cluster this field becomes Cluster, in which you select the cluster you'd like to replace from a drop-down list of existing ones.
• Relativity Analytics Index - the Analytics index that is used to cluster your documents. The index you select here must have queries enabled for you to be able to submit the selected documents for clustering. All of the selected documents must be in the data source of this index; otherwise, they end up as Not Clustered.

Click + to display the following Advanced Options:

• Title Format - determines how the cluster appears in the browser tree.
  • Outline only - displays the assigned number of the cluster along with the number of documents in the cluster and its coherence.
  • Title only - displays the title of the cluster (up to 10 terms that best represent the cluster) along with the number of documents in the cluster and its coherence.
  • Outline and title - displays the assigned number, title, number of documents in the cluster, and its coherence.
• Maximum Hierarchy Depth - the number of levels in a cluster hierarchy (e.g., a value of 1 results in the creation of only top-level clusters). Documents can only be in one cluster at each level. Lower level clusters are more tightly conceptually-related than higher level clusters. The maximum value for this field is 5. The default value is set to 3.
  Note: When Maximum Hierarchy Depth is set to be greater than 1, a maximum of 16 top-level clusters will be created.
• Minimum Coherence - the level of conceptual correlation that items must have to be included in the same cluster. The default value is 0.7. This affects the depth of the cluster tree, but it does so only in conjunction with the Maximum Hierarchy Depth setting. Each cluster has a coherence, which is a measure of tightness. A loose cluster node has a lower coherence, while that of a tighter cluster node of more-related documents is higher. When the Analytics engine sees a cluster node that it has built, it decides whether to break it into subclusters by looking at its coherence. If it is at or above the minimum coherence, it leaves it alone. If it’s below the minimum coherence, it breaks it up, but only if the maximum depth has not yet been reached. Cluster nodes that don't reach the full depth of the tree have a coherence higher than the Minimum Coherence setting, with rare exceptions.
• Generality - determines how vague (general) or specific the clusters will be at each level. Values range from 0 (most specific) to 1 (most general). The default value is 0.5. Low generality (closer to 0.0) results in more clusters that are tighter at each level of the cluster tree, including the root. High generality (closer to 1.0) gives you fewer and broader clusters . A low generality means that many top-level clusters are created, while a high generality means that few are created. However, as noted above, with Maximum Hierarchy Depth greater than 1, the number of top-level clusters is typically 16. As you move further down the tree, the same general principle applies. A low generality expands each level vertically.
• Create Cluster Score - checking this checkbox creates a decimal field on the Document object that stores the document's coherence score within the cluster. Enabling this will increase the duration of the cluster job significantly. This is unchecked by default.

See Deleting a cluster.

Default settings for automatically created clusters

If you choose to automatically create a cluster when setting up a new conceptual index, the cluster will behave as follows:

• It will be called "Default Cluster [index name]."
•  It will be created immediately after the conceptual index has been activated.
•  It will be linked to the chosen conceptual index.
•  The cluster settings will have the following values:
  • Title Format: Outline and Title
  • Maximum Hierarchy Depth: 3
  • Minimum Coherence: 0.7
  • Generality: 0.5
  • Create Cluster Score Field: No

You can replace the default cluster at any time through the Mass Operations menu. For more information, see Creating or replacing a cluster.

Replacing an existing cluster

Replace existing cluster is the same as Creating or replacing a cluster, except the results replace existing clustering options. When you select Replace Existing Cluster, you're prompted to select the existing cluster set you would like to replace.

Viewing clusters

Note: The Cluster browser has been deprecated. You can access clustering by using the Cluster visualization. To learn more, see Cluster Visualization.

Deleting a cluster

To delete a cluster, perform the following steps:

1. Navigate to the Analytics Indexes tab, and click on the Analytics index that was used to create the cluster.
2. Select the checkbox next to the cluster you want to delete from the cluster list at the bottom of the Analytics index console.
3. Click Delete.