civis.io.dataframe_to_civis

civis.io.dataframe_to_civis(df, database, table, api_key=None, client=None, max_errors=None, existing_table_rows='fail', diststyle=None, distkey=None, sortkey1=None, sortkey2=None, headers=None, credential_id=None, polling_interval=None, archive=False, hidden=True, **kwargs)[source]

Upload a pandas DataFrame into a Civis table.

The DataFrame’s index will not be included. To store the index along with the other values, use df.reset_index() instead of df as the first argument to this function.

Parameters:
df : pandas.DataFrame

The DataFrame to upload to Civis.

database : str or int

Upload data into this database. Can be the database name or ID.

table : str

The schema and table you want to upload to. E.g., 'scratch.table'. Schemas or tablenames with periods must be double quoted, e.g. 'scratch."my.table"'.

api_key : DEPRECATED str, optional

Your Civis API key. If not given, the CIVIS_API_KEY environment variable will be used.

client : civis.APIClient, optional

If not provided, an civis.APIClient object will be created from the CIVIS_API_KEY.

max_errors : int, optional

The maximum number of rows with errors to remove from the import before failing.

existing_table_rows : str, optional

The behaviour if a table with the requested name already exists. One of 'fail', 'truncate', 'append' or 'drop'. Defaults to 'fail'.

diststyle : str, optional

The distribution style for the table. One of 'even', 'all' or 'key'.

distkey : str, optional

The column to use as the distkey for the table.

sortkey1 : str, optional

The column to use as the sortkey for the table.

sortkey2 : str, optional

The second column in a compound sortkey for the table.

headers : bool, optional [DEPRECATED]

Whether or not the first row of the file should be treated as headers. The default, None, attempts to autodetect whether or not the first row contains headers.

This parameter has no effect in versions >= 1.11 and will be removed in v2.0. Tables will always be written with column names read from the DataFrame. Use the header parameter (which will be passed directly to to_csv()) to modify the column names in the Civis Table.

credential_id : str or int, optional

The ID of the database credential. If None, the default credential will be used.

polling_interval : int or float, optional

Number of seconds to wait between checks for job completion.

archive : bool, optional (deprecated)

If True, archive the import job as soon as it completes.

hidden : bool, optional

If True (the default), this job will not appear in the Civis UI.

**kwargs : kwargs

Extra keyword arguments will be passed to pandas.DataFrame.to_csv().
Returns:
fut : CivisFuture

A CivisFuture object.

See also

to_csv()

Examples

>>> import pandas as pd
>>> df = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]})
>>> fut = civis.io.dataframe_to_civis(df, 'my-database',
...                                   'scratch.df_table')
>>> fut.result()