Importing raw unprocessed data for Processing via Import/Export

Note: Starting September 1, 2024, we’re streamlining our Staging boundaries to more effectively separate staging data from workspace and system data. With this change, you will no longer be able to write to or access data outside of the defined staging area. Folders that will remain within the Staging area: ARM, ProcessingSource, StructuredData, and TenantVM. Folders that will be removed: FTA, Temp, Export, and dtSearch, in addition to any other folders that you manually created. Refer to the Staging Area topic and Staging Area FAQ in Community for more details.

Note: Effective August 20, 2024, Import/Export uploads raw (unprocessed) data to the ProcessingSource folder and triggers the Processing application. The ProcessingSource folder is included in your Staging threshold. To avoid additional storage costs, you may want to delete the uploaded raw (unprocessed) data from this folder once it is no longer needed.

You can use Import/Export to import native, unprocessed/unstructured files, including Slack data, into RelativityOne that require processing and it will automatically trigger a Processing job in one operation.

    Notes:
  • Import/Export is not a replacement for the Processing application; however it is a simplified workflow when setting up a processing set. Using the Processing application is recommended when setting up large, complex processing jobs. Also, within the Processing application, you can use Quick Create Sets, Inventory, check and update Processing Profiles, and customize data source options.
  • Processing profiles must already exist in RelativityOne before using them in the steps below.
  • For optimal performance, we recommend using Import/Export with Express Transfer and organizing data in such a way that there are as few Data Sources as possible in a single job.

If you want to use Import/Export with Express Transfer active, which is the recommended workflow for data transfer, refer to Express Transfer overview for more information.

To import raw unprocessed/unstructured files for Processing:

  1. Use tabs, search, or the quick nav bar to navigate to the Import/Export Job page.
  2. On the Import/Export Job page, click the New Import/Export Job button to start a new job.
  3. On the Choose workflow dialog, the Import button is selected by default. Next, click the Raw (Unprocessed) Data workflow box to import native (raw) files into your workspace.

Choose workflow raw unprocessed data highlighted

Note: If Express Transfer has been activated, the Express Transfer section will display a status of Active. See Activating and deactivating Express Transfer for more information.

  1. Select the data files to import as follows depending on the activation status of Express Transfer:
    • With Express Transfer Active: Click Select on the Select data to Import dialog to browse to your local drive and select the folder with the files that you want to upload. In the case of universal naming convention (UNC) paths (for example, \\ServerName\SharedResourceName\FilePath), you can copy and paste the path into the Select Folder address bar at the top of the dialog box in order to access the desired folder. You also have the option on the dialog to choose from file share by clicking Choose from File Share, selecting the import source folder, and clicking Select Folder. Once the folder has been selected, click Continue.

Note: When you select a folder to upload, Express Transfer will transfer all the data from the selected folder, including sub-folders, to the staging area, regardless of the specific job configuration, like the custodian mapping settings. This also applies to the load file import workflows.

Select data to import

  • With Express Transfer not Active: On the Select file dialog, do one of the following below to upload a file. Data files must be zipped when using the first two options mentioned below. Single PST files do not require zipping.

Import export select file dialog

  • Drag and drop the zipped data file or single PST file into the upload area.
  • Click browse for a file to select a zipped data or single PST file from your local drive.
  • Click Choose From Staging to upload files directly from the Staging Area. Then, select the correct ProcessingSource folder and click Select. You do not need to zip files when importing with this option.
  1. On the Create Processing Job dialog, complete the following fields:

Create processing job dialog

  • Job name—provide a descriptive job name so that you can quickly identify and track that job on the Import/Export Job page. The Job name field accepts up to 50 characters.
  • Profile—select the desired Processing profile. The Processing profile must already exist in Relativity.

Note: If you are importing Slack data, please refer to Importing Slack short message data for Processing via Import/Export below.

  • Destination folder—click the Select button, and do one of the following:
    • Choose an existing folder.
    • Create a new workspace folder by right-clicking on an existing folder, click Create, enter a new folder name, click out of the new folder to save it, and then click Select.
  • (Optional) Notification Addresses—enable email notifications by providing the email addresses of those who should get notified once the job completed, failed, or canceled.
  1. Next, use the Select Data Sources section to select the Data Sources and assign Custodians. A maximum of 70 data sources (folders with data, loose files) can be selected and assigned custodians. See recommendations below for preparing data sources.
      Notes: Recommendations for preparing data sources.
    • Each selected file is treated as a separate data source. Consider organizing loose files into folders per custodian.
    • If you import data for a single custodian, you may want to create a top-level folder and place all the data into that folder to speed up the configuration and execution of the Processing job.
    • To exclude Microsoft Office temporary files, which are those file names starting with a ~ (tilde) sign, see Exclude temporary Microsoft Office files from data import for more information.
    • Be sure your data sources have unique names. Data sources are not case sensitive. If you have data sources with duplicate names, you may see processing errors. For example, \\[file.share]\Processing Source[SOURCE.NAME] and \\[file.share]\Processing Source[Source.Name] would be considered duplicate names since the system does not differentiate between capital and lower case letters. To make them unique, you could rename the folders [SOURCE.NAME1] and [Source.Name2].

Choose the Data Source by selecting the checkbox next to the folder name. Then, assign the Custodian manually or automatically, or by adding a new custodian now, as described below:

  • Manually assign custodians: Select the proper custodian from the Custodian drop-down list and click the Assign button.
  • Automatically assign custodians: Click the Auto Map Custodians button to automatically assign custodians to data source folders and files based on the naming pattern. Only the Last name, First name naming pattern is supported at this time. If you want to modify the automatic assignment, click Clear next to the custodian’s name. The application reads data source names and checks if matching custodians have already been added to the Entity object. Depending on the result, one of the following occurs:
  • Custodian(s) already exists in the Entity: If the custodian(s) already exists in the Entity, the application automatically assigns corresponding custodians to folders and files.
  • New custodian automatically created: If the application cannot find the corresponding custodian in the Entity object, it creates a new one based on the data source name. Newly created custodians are marked with a blue dot in the Select Data Sources pane. New custodians will be added to the Entity object after you click Run Job in the next step. However, if you click Cancel instead of running the job, the new custodians will not be created.
  • Add a new custodian now: If you want to add a new custodian to the drop-down list from here instead of through the Entity object, click the Add Custodian link, and do the following on the Add Custodian dialog:

Add Custodian

  1. Select the custodian Type:
  • Person—select this option to enter a First Name and Last Name of the individual acting as custodian of the data to be processed.
  • Other—select this option if the custodian of the data to be processed is not an individual but is, for example, an entity or company name. Then, enter the name in the Full Name field. You can also select this option if you wish to enter an individual's full name without having that name include a comma once you export the data associated with it.
  1. Click the Add Custodian button. The newly created custodian will be added to the Custodian drop-down list. Then, you can select and assign the custodian to the data source.

 

  1. Click Run Job to start importing data.

Note: Import/Export will upload all the zipped data to the Staging area, extract it, and create a new Processing job based on the data source selection. Only data sources assigned to custodians will be processed and imported to your workspace.

Note: Effective August 20, 2024, Import/Export uploads raw (unprocessed) data to the ProcessingSource folder and triggers the Processing application. The ProcessingSource folder is included in your Staging threshold. To avoid additional storage costs, you may want to delete the uploaded raw (unprocessed) data from this folder once it is no longer needed.

  1. Review the importing progress on the Importing & Processing dialog. Real-time details regarding the number of transferred files and data size being processed displays during the Uploading and Extracting steps. Do any of the following:

Note:  Only records that imported without errors will appear in the workspace.

Importing and processing progress bar dialog

  • Click Cancel to stop the importing process before it completes.
  • Click Close when the importing process is complete.

Caution: If you imported data without activating Express Transfer, you can close the dialog before the import process finishes without causing an interruption in data flow as long as it's closed after the Uploading step is completed. Closing the dialog earlier will terminate the data transfer (upload) process.

  • Click Go to Processing when the Discovery step begins to go to the Processing Sets page of the Processing application.

Import/Export utilizes the Processing application to process native files, extract metadata, and create documents. Depending on the processing profile configuration, the following processing phases will be triggered:

  • Discovery
  • Discovery and publish (if the auto-publish option is enabled in your processing profile)
    Notes:
  • We recommend using the Processing application directly to determine global dedupe order creating processing sets.
  • The Inventory phase of processing is always skipped when you import unstructured data. If Inventory is needed, use Staging Explorer to import data into RelativityOne, and then run Processing directly.

Please refer to Processing in Relativity documentation for more information about using the Processing application to ingest raw data directly into your workspace.

For more information on where Import/Export saves files, see the Community article, Import/Export: Where Files Are Saved By The Import/Export Application.

Note: If you ran the job with Express Transfer active, you are able to close the Import/Export application while the job is running, if desired, and the transfer job will continue and not be affected. You can monitor the progress and history of jobs within Express Transfer (see Express Transfer Job tracking and history). You do not have to be logged into RelativityOne to view information within the Express Transfer application.

Importing Slack short message data for Processing via Import/Export

All commercial customers can now leverage Import/Export to upload Slack data for Processing.

To import Slack short messages for Processing via Import/Export:

  1. Copy the zip file containing the Slack data into a folder on your drive.
  2. Use tabs, search, or the quick nav bar to navigate to the Import/Export Job page.
  3. On the Import/Export Job page, click the New Import/Export Job button to start a new job.
  4. Activate Express Transfer. See Activating and deactivating Express Transfer for more information.

Note: If your organization is not permitted to use Express Transfer and you need to leverage the web mode of Import/Export to import Slack data, you must zip the folder in which the zip file is located from Step 1. Be aware that this will create an additional zip level (the zip file from Step 1 within a zipped folder file) for transport purposes only. For this reason, we recommend using Express Transfer for all your imports and exports.

  1. On the Choose workflow dialog, the Import button is selected by default.
  2. Click the Raw (Unprocessed) Data workflow box.
  3. Provide the Job Name, select a Destination Folder, and select the Processing Profile.

Note: The following settings must already be enabled in the Processing Profile: Convert Slack to RSMF and Download Attachments. See Processing Profile Fields for more information.

  1. For Data Source, select the folder that contains the zipped Slack data, then assign a Custodian.
  2. Click Run Job. Import/Export uploads the data and passes the job configuration to the Processing application. Please refer to Processing in Relativity documentation for more information about using Processing to ingest raw data directly into your workspace.

Note: All the RSMF conversion is done in Processing automatically. See Processing an RSMF file for more information.