Import ORC
Description
Section titled “Description”Import Apache ORC files from PlaidCloud Document. ORC is a columnar storage format written by Hive and Spark, with the schema carried in the file.
The step form has three tabs:
- Import Parameters — the Source File (a Document account and the file or folder path) and the Target Table, with Remove non-ASCII Characters (
ascii_only), Delete file(s) after import (delete_file) and Skip re-import if source files unchanged (skip_if_unchanged). - Table Data Selection — maps the source columns to the target columns, with Summarize and Distinct. Populate reads the columns and their types from the schema carried in the ORC file.
- Data Filters — filters the rows before and after summarizing.
When the file is missing or empty, Populate answers Nothing to populate with a warning naming the path and keeps the mapping you have.
Migrated Prophecy projects use this step for ORC datasets.
Examples
Section titled “Examples”No examples yet…