Data Import and Export¶
The civis.io namespace provides several functions for moving data in and
out of Civis.
Tables¶
Often, your data will be in structured format like a table in a relational
database, a CSV, or a dataframe. The following functions handle moving
structured data to and from Civis. When using these functions, it is
recommended to have pandas installed and to pass use_pandas=True in
the appropriate functions. If pandas is not installed, data returned
from Civis will all be treated as strings.
civis_to_csv(filename, sql, database[, …]) |
Export data from Civis to a local CSV file. |
civis_to_multifile_csv(sql, database[, …]) |
Unload the result of SQL query and return presigned urls. |
civis_file_to_table(file_id, database, table) |
Upload the contents of one or more Civis files to a Civis table. |
csv_to_civis(filename, database, table[, …]) |
Upload the contents of a local CSV file to Civis. |
dataframe_to_civis(df, database, table[, …]) |
Upload a pandas DataFrame into a Civis table. |
read_civis(table, database[, columns, …]) |
Read data from a Civis table. |
read_civis_sql(sql, database[, use_pandas, …]) |
Read data from Civis using a custom SQL string. |
export_to_civis_file(sql, database[, …]) |
Store results of a query to a Civis file |
split_schema_tablename(table) |
Split a Redshift ‘schema.tablename’ string |
Files¶
These functions will pass flat files to and from Civis. This is useful if you have data stored in binary or JSON format. Any type of file can be stored in platform via the files endpoint.
civis_to_file(file_id, buf[, api_key, client]) |
Download a file from Civis. |
dataframe_to_file(df[, name, expires_at, client]) |
Store a DataFrame as a CSV in Civis Platform |
file_id_from_run_output(name, job_id, run_id) |
Find the file ID of a File run output with the name “name” |
file_to_civis(buf[, name, api_key, client]) |
Upload a file to Civis. |
file_to_dataframe(file_id[, compression, client]) |
Load a DataFrame from a CSV stored in a Civis File |
file_to_json(file_id[, client]) |
Restore JSON stored in a Civis File |
json_to_file(obj[, name, expires_at, client]) |
Store a JSON-serializable object in a Civis File |
Databases¶
These functions move data from one database to another and expose an interface
to run SQL in the database. Use query_civis() when you need to
execute SQL that does not return data (for example, a GRANT or
DROP TABLE statement).
transfer_table(source_db, dest_db, …[, …]) |
Transfer a table from one location to another. |
query_civis(sql, database[, api_key, …]) |
Execute a SQL statement as a Civis query. |