What Affects Performance
Rowpath uses different execution paths depending on the file type, operation, and current dataset state. Some work can happen immediately, while other work must scan rows or produce a new result.
Fast Operations
These operations are typically fast:
- Opening a large-file preview.
- Renaming columns.
- Hiding columns.
- Reordering columns.
- Removing or editing compatible Working View steps.
- Running limited preview queries.
Column layout changes are fast because they do not require scanning or rewriting the dataset.
Operations That May Take Longer
These operations can take longer on large datasets:
- Duplicate Analysis.
- Validation counts.
- Missing-value summaries.
- Aggregates.
- Full-file SQL queries.
- Full Working View exports.
- Building optimized backing data.
- Loading into memory.
Preview Vs Full Execution
Rowpath often previews a limited result first. SQL preview SELECT queries use a 1,000-row limit when needed. Full execution and exports may run over the full dataset or current Working View.
Improving Performance
- Use previews while building a query or workflow.
- Use Load Into Memory when repeated full-file operations are expected and memory estimates are acceptable.
- Use Working Views to avoid rewriting the source file while iterating.
- Use derived results or exports when you need a persisted output.
- Limit SQL results while exploring.
Notes
- Large files may start in preview mode and later use optimized local backing data.
- Performance depends on file size, file format, operation type, available memory, and whether the data is already optimized or loaded into memory.
Related pages: