civis.io.civis_file_to_table

civis.io.civis_file_to_table(file_id, database, table, client=None, max_errors=None, existing_table_rows='fail', diststyle=None, distkey=None, sortkey1=None, sortkey2=None, table_columns=None, primary_keys=None, last_modified_keys=None, escaped=False, execution='immediate', delimiter=None, headers=None, credential_id=None, polling_interval=None, hidden=True)

Upload the contents of one or more Civis files to a Civis table. All provided files will be loaded as an atomic unit in parallel, and should share the same columns in the same order, and be in the same format.

Note

Civis files must be in a CSV-like delimiter separated format.

Parameters

file_idint or list[int]

Civis file ID or a list of Civis file IDs.

databasestr or int

Upload data into this database. Can be the database name or ID.

tablestr

The schema and table you want to upload to. E.g., 'scratch.table'.

clientcivis.APIClient, optional

If not provided, an civis.APIClient object will be created from the CIVIS_API_KEY.

max_errorsint, optional

The maximum number of rows with errors to remove from the import before failing. If multiple files are provided, this limit applies across all files combined.

existing_table_rowsstr, optional

The behaviour if a table with the requested name already exists. One of 'fail', 'truncate', 'append', 'drop', or 'upsert'. Defaults to 'fail'.

diststylestr, optional

The distribution style for the table. One of 'even', 'all' or 'key'.

distkeystr, optional

The column to use as the distkey for the table.

sortkey1str, optional

The column to use as the sortkey for the table.

sortkey2str, optional

The second column in a compound sortkey for the table.

table_columnslist[Dict[str, str]], optional

A list of dictionaries, ordered so each dictionary corresponds to a column in the order that it appears in the source file. Each dict should have a key “name” that corresponds to the column name in the destination table, and a key “sql_type” corresponding to the intended column data type in the destination table. The “sql_type” key is not required when appending to an existing table. The table_columns parameter is required if the table does not exist, the table is being dropped, or the columns in the source file do not appear in the same order as in the destination table. Example: [{"name": "foo", "sql_type": "INT"}, {"name": "bar", "sql_type": "VARCHAR"}]

primary_keys: list[str], optional

A list of the primary key column(s) of the destination table that uniquely identify a record. These columns must not contain null values. If existing_table_rows is “upsert”, this field is required. Note that this is true regardless of whether the destination database itself requires a primary key.

last_modified_keys: list[str], optional

A list of the columns indicating a record has been updated. If existing_table_rows is “upsert”, this field is required.

escaped: bool, optional

A boolean value indicating whether or not the source file(s) escape quotes with a backslash. Defaults to false.

execution: string, optional, default “immediate”

One of “delayed” or “immediate”. If “immediate”, refresh column statistics as part of the run. If “delayed”, flag the table for a deferred statistics update; column statistics may not be available for up to 24 hours. In addition, if existing_table_rows is “upsert”, delayed executions move data from staging table to final table after a brief delay, in order to accommodate multiple concurrent imports to the same destination table.

delimiterstring, optional

The column delimiter. One of ',', '\t' or '|'. If not provided, will attempt to auto-detect.

headersbool, optional

Whether or not the first row of the file should be treated as headers. The default, None, attempts to autodetect whether or not the first row contains headers.

credential_idstr or int, optional

The ID of the database credential. If None, the default credential will be used.

polling_intervalint or float, optional

Number of seconds to wait between checks for job completion.

hiddenbool, optional

If True (the default), this job will not appear in the Civis UI.

Returns

resultsCivisFuture

A CivisFuture object.

Raises

CivisImportError

If multiple files are given and determined to be incompatible for import. This may be the case if their columns have different types, their delimiters are different, headers are present in some but not others, or compressions do not match.

Examples

>>> file_id = 100
>>> fut = civis.io.civis_file_to_table(file_id,
...                                    'my-database',
...                                    'scratch.my_data')
>>> fut.result()