File Connector

The File Connector imports structured data from files into the DataPool. It supports CSV, Excel, JSON, and XML formats with three delivery methods.

Delivery Methods

Method	Description	Best For
Upload	Upload a file directly through the UI	One-time or infrequent updates
URL Polling	Fetch a file from a URL on a schedule	Files hosted on web servers or cloud storage
Watch Folder	Monitor a local directory for new files	Automated file drops from other systems

Format Configuration

CSV

Setting	Description	Default
Delimiter	Column separator	`,` (comma)
Has Header Row	First row contains column names	Yes
Encoding	File character encoding	UTF-8
Quote Character	Character for quoting fields	`"`

Example CSV:


Region,Sales,Target
North,125000,100000
South,98000,110000
East,142000,130000

Setting	Description	Default
Sheet Name	Which worksheet to read	First sheet
Header Row	Row number containing column names	1
Data Start Row	Row number where data begins	2
Column Range	Columns to include (e.g. A:F)	All columns

JSON

Setting	Description	Default
Root Path	JSONPath to the data array	`$` (root)
Item Key	Property that uniquely identifies items	First property

Example JSON:


{
  "data": [
    { "region": "North", "sales": 125000, "target": 100000 },
    { "region": "South", "sales": 98000, "target": 110000 }
  ]
}

With root path: $.data

XML

Setting	Description	Default
Root Element	XPath to the repeating element	Required
Item Key	Element or attribute for unique ID	First element

Example XML:


<report>
  <region name="North">
    <sales>125000</sales>
    <target>100000</target>
  </region>
</report>

With root element: //region

URL Polling Configuration

Setting	Description	Default
URL	Full URL to the file	Required
Polling Interval	How often to fetch the file	300 seconds
Authentication	HTTP Basic, Bearer token, or none	None
Headers	Custom HTTP headers	None

The File Connector checks if the file has changed before processing. If the file content is identical to the last fetch, no DataPool update occurs. This prevents unnecessary processing and data churn.

Watch Folder Configuration

Setting	Description	Default
Path	Local directory path to watch	Required
File Pattern	Glob pattern for matching files	`.`
Process Action	What to do after processing	Move to archive folder
Archive Path	Where to move processed files	Required if action is Move

Best Practices

Use URL polling for data that updates regularly from a known location
Set the item key correctly to avoid duplicate records
For CSV files, ensure consistent column ordering across file versions
Test with a sample file before configuring automated delivery
Use the archive action for watch folders to prevent reprocessing