General recommendations for structured data import and export jobs

Note: As of September 1, 2024, we’ve streamlined our Staging boundaries to more effectively separate staging data from workspace and system data. With this change, you will no longer be able to write to or access data outside of the four defined staging area folders. The four folders that comprise the Staging area are ARM, ProcessingSource, StructuredData, and TenantVM. Folders that were removed include FTA, Temp, Export, and dtSearch, in addition to any other folders that you manually created. Refer to the Staging Area topic and Staging Area FAQ in Community for more details.

Load file import recommendations

  • The Pre-check Load File feature enables validation of load file data before executing the import job. We strongly recommend using this feature to ensure that the import job doesn’t encounter metadata errors.
  • When using Import/Export in web only mode, ensure that the full path listed in the load file is captured in the ZIP file.
  • Try to limit the number of fields in a single import job to 100 to minimize risk of job failure due to SQL exception related issues.
  • When importing new choice fields, you may experience the unexpected behaviors, such as errors occurring when importing choice fields or slow import jobs for choice fields. To help prevent these issues, try to limit the number of choice values in a single import job to 100. This number indicates the maximum of new values in all choice type fields across the entire set of records.
  • Extracted Text import:
    • There is no limitation or performance impact related to size of extracted text when the import is performed to Data Grid.
    • Performance and stability are affected for extracted texts import to SQL. If there is no other option and you must use SQL, please import extracted text separately from other data fields.
    • Depending on the job configuration, the extracted text import to Data Grid is 60-80% faster than similar import to SQL.
    • Performance improves when extracted text files use UTF-16 encoding.

Load file export recommendations

  • Depending on the exported data structure, a Workspace folder structure can degrade performance significantly, up to 30 times slower in edge cases, so we recommend using the File Type folder structure.
  • Setting too high value of the volume and sub-directory files can also cause significant performance degradation. We recommend using the default settings: Max Volume size=6000MB, Max number of files per folder=2000.
  • Exporting images as PDFs or multi-page TIFFs is slower than exporting them as single page images.
  • Exporting to a ZIP archive using Import/Export in web only mode is significantly slower than exporting data with Express Transfer or exporting data to the Staging only.
  • Exporting text files in Unicode (UTF-16) encoding is faster than in other encodings.