This function can be defined as sync or async function. The second leverages Python's asynchronous programming capabilities for memory efficient data processing. This is paramount when working with large amounts of data (e.g. importing several GBs sized files).

When using the function as synchronous function it is important to keep in mind that any data processing done in that way requires all the data to be present in-memory for the time of the computation. This can put the system under considerable amount of stress and may slow down other processes on the system. The synchronous version should thus only be used if it is certain that the operation is never executed with large amounts of data. In practice this applies only to a limited set of use cases, e.g. for the task of defining a start table schema for a data node.

The signatures are the following:

Function arguments

• params: TIS Parameters of the current project as dictionary mapping parameter name → parameter value,

• types: data types of columns in the result table as dictionary mapping column index → type,

• columns: column names of the result table as dictionary mapping column name → column index,

• files: an object that provides access to the TIS File I/O data streams. The usage of this parameter is described in detail here.

The function returns a list that replaces the result table. Do not assign new objects to the types and columns arguments.