Clusters

class Clusters(session_kwargs, client, return_type='raw')

Examples

>>> import civis
>>> client = civis.APIClient()
>>> client.clusters.list_kubernetes(...)

Methods

delete_kubernetes_partitions(id, ...)

Delete a Cluster Partition

get_kubernetes(id, *[, include_usage_stats])

Describe a Kubernetes Cluster

get_kubernetes_instance_configs(...[, ...])

Describe an Instance Config

get_kubernetes_partitions(id, ...[, ...])

Describe a Cluster Partition

list_kubernetes(*[, organization_id, ...])

List Kubernetes Clusters

list_kubernetes_compute_hours(id, *[, ...])

List compute hours for a Kubernetes Cluster

list_kubernetes_deployment_stats(id)

Get stats about deployments associated with a Kubernetes Cluster

list_kubernetes_deployments(id, *[, ...])

List the deployments associated with a Kubernetes Cluster

list_kubernetes_instance_configs_active_workloads(id, *)

List active workloads in an Instance Config

list_kubernetes_instance_configs_historical_graphs(...)

Get graphs of historical resource usage in an Instance Config

list_kubernetes_instance_configs_user_statistics(...)

Get statistics about the current users of an Instance Config

list_kubernetes_partitions(id, *[, ...])

List Cluster Partitions for given cluster

patch_kubernetes_partitions(id, ...[, ...])

Update a Cluster Partition

post_kubernetes_partitions(id, ...)

Create a Cluster Partition for given cluster

delete_kubernetes_partitions(id: int, cluster_partition_id: int)

Delete a Cluster Partition

Parameters:
idint

The ID of the cluster which this partition belongs to.

cluster_partition_idint

The ID of this cluster partition.

Returns:
None

Response code 204: success

get_kubernetes(id: int, *, include_usage_stats: bool = None)

Describe a Kubernetes Cluster

Parameters:
idint
include_usage_statsbool, optional

When true, usage stats are returned in instance config objects. Defaults to false.

Returns:
civis.response.Response
  • idint

    The ID of this cluster.

  • organization_idstr

    The id of this cluster’s organization.

  • organization_namestr

    The name of this cluster’s organization.

  • organization_slugstr

    The slug of this cluster’s organization.

  • raw_cluster_slugstr

    The slug of this cluster’s raw configuration.

  • custom_partitionsbool

    Whether this cluster has a custom partition configuration.

  • cluster_partitionsList[dict]

    List of cluster partitions associated with this cluster.

    • cluster_partition_idint

      The ID of this cluster partition.

    • namestr

      The name of the cluster partition.

    • labelsList[str]

      Labels associated with this partition.

    • instance_configsList[dict]

      The instances configured for this cluster partition.

      • instance_config_idint

        The ID of this InstanceConfig.

      • instance_typestr

        An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

      • min_instancesint

        The minimum number of instances of that type in this cluster.

      • max_instancesint

        The maximum number of instances of that type in this cluster.

      • instance_max_memoryint

        The amount of memory (RAM) available to a single instance of that type in megabytes.

      • instance_max_cpuint

        The number of processor shares available to a single instance of that type in millicores.

      • instance_max_diskint

        The amount of disk available to a single instance of that type in gigabytes.

      • usage_statsdict
        • pending_memory_requestedint

          The sum of memory requests (in MB) for pending deployments in this instance config.

        • pending_cpu_requestedint

          The sum of cpu requests (in millicores) for pending deployments in this instance config.

        • running_memory_requestedint

          The sum of memory requests (in MB) for running deployments in this instance config.

        • running_cpu_requestedint

          The sum of cpu requests (in millicores) for running deployments in this instance config.

        • pending_deploymentsint

          The number of pending deployments in this instance config.

        • running_deploymentsint

          The number of running deployments in this instance config.

    • default_instance_config_idint

      The id of the InstanceConfig that is the default for this partition.

  • is_nat_enabledbool

    Whether this cluster needs a NAT gateway or not.

  • hoursfloat (float)

    The number of hours used this month for this cluster.

get_kubernetes_instance_configs(instance_config_id: int, *, include_usage_stats: bool = None)

Describe an Instance Config

Parameters:
instance_config_idint

The ID of this instance config.

include_usage_statsbool, optional

When true, usage stats are returned in instance config objects. Defaults to false.

Returns:
civis.response.Response
  • instance_config_idint

    The ID of this InstanceConfig.

  • instance_typestr

    An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

  • min_instancesint

    The minimum number of instances of that type in this cluster.

  • max_instancesint

    The maximum number of instances of that type in this cluster.

  • instance_max_memoryint

    The amount of memory (RAM) available to a single instance of that type in megabytes.

  • instance_max_cpuint

    The number of processor shares available to a single instance of that type in millicores.

  • instance_max_diskint

    The amount of disk available to a single instance of that type in gigabytes.

  • usage_statsdict
    • pending_memory_requestedint

      The sum of memory requests (in MB) for pending deployments in this instance config.

    • pending_cpu_requestedint

      The sum of cpu requests (in millicores) for pending deployments in this instance config.

    • running_memory_requestedint

      The sum of memory requests (in MB) for running deployments in this instance config.

    • running_cpu_requestedint

      The sum of cpu requests (in millicores) for running deployments in this instance config.

    • pending_deploymentsint

      The number of pending deployments in this instance config.

    • running_deploymentsint

      The number of running deployments in this instance config.

  • cluster_partition_idint

    The ID of this InstanceConfig’s cluster partition

  • cluster_partition_namestr

    The name of this InstanceConfig’s cluster partition

get_kubernetes_partitions(id: int, cluster_partition_id: int, *, include_usage_stats: bool = None)

Describe a Cluster Partition

Parameters:
idint

The ID of the cluster which this partition belongs to.

cluster_partition_idint

The ID of this cluster partition.

include_usage_statsbool, optional

When true, usage stats are returned in instance config objects. Defaults to false.

Returns:
civis.response.Response
  • cluster_partition_idint

    The ID of this cluster partition.

  • namestr

    The name of the cluster partition.

  • labelsList[str]

    Labels associated with this partition.

  • instance_configsList[dict]

    The instances configured for this cluster partition.

    • instance_config_idint

      The ID of this InstanceConfig.

    • instance_typestr

      An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

    • min_instancesint

      The minimum number of instances of that type in this cluster.

    • max_instancesint

      The maximum number of instances of that type in this cluster.

    • instance_max_memoryint

      The amount of memory (RAM) available to a single instance of that type in megabytes.

    • instance_max_cpuint

      The number of processor shares available to a single instance of that type in millicores.

    • instance_max_diskint

      The amount of disk available to a single instance of that type in gigabytes.

    • usage_statsdict
      • pending_memory_requestedint

        The sum of memory requests (in MB) for pending deployments in this instance config.

      • pending_cpu_requestedint

        The sum of cpu requests (in millicores) for pending deployments in this instance config.

      • running_memory_requestedint

        The sum of memory requests (in MB) for running deployments in this instance config.

      • running_cpu_requestedint

        The sum of cpu requests (in millicores) for running deployments in this instance config.

      • pending_deploymentsint

        The number of pending deployments in this instance config.

      • running_deploymentsint

        The number of running deployments in this instance config.

  • default_instance_config_idint

    The id of the InstanceConfig that is the default for this partition.

list_kubernetes(*, organization_id: int = None, organization_slug: str = None, raw_cluster_slug: str = None, exclude_inactive_orgs: bool = None, limit: int = None, page_num: int = None, order: str = None, order_dir: str = None, iterator: bool = None)

List Kubernetes Clusters

Parameters:
organization_idint, optional

The ID of this cluster’s organization. Cannot be used along with the organization slug.

organization_slugstr, optional

The slug of this cluster’s organization. Cannot be used along with the organization ID.

raw_cluster_slugstr, optional

The slug of this cluster’s raw configuration.

exclude_inactive_orgsbool, optional

When true, excludes KubeClusters associated with inactive orgs. Defaults to false.

limitint, optional

Number of results to return. Defaults to its maximum of 50.

page_numint, optional

Page number of the results to return. Defaults to the first page, 1.

orderstr, optional

The field on which to order the result set. Defaults to organization_id. Must be one of: organization_id, created_at.

order_dirstr, optional

Direction in which to sort, either asc (ascending) or desc (descending) defaulting to asc.

iteratorbool, optional

If True, return a generator to iterate over all responses. Use when more results than the maximum allowed by limit are needed. When True, limit and page_num are ignored. Defaults to False.

Returns:
civis.response.PaginatedResponse
  • idint

    The ID of this cluster.

  • organization_idstr

    The id of this cluster’s organization.

  • organization_namestr

    The name of this cluster’s organization.

  • organization_slugstr

    The slug of this cluster’s organization.

  • raw_cluster_slugstr

    The slug of this cluster’s raw configuration.

  • custom_partitionsbool

    Whether this cluster has a custom partition configuration.

  • cluster_partitionsList[dict]

    List of cluster partitions associated with this cluster.

    • cluster_partition_idint

      The ID of this cluster partition.

    • namestr

      The name of the cluster partition.

    • labelsList[str]

      Labels associated with this partition.

    • instance_configsList[dict]

      The instances configured for this cluster partition.

      • instance_config_idint

        The ID of this InstanceConfig.

      • instance_typestr

        An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

      • min_instancesint

        The minimum number of instances of that type in this cluster.

      • max_instancesint

        The maximum number of instances of that type in this cluster.

      • instance_max_memoryint

        The amount of memory (RAM) available to a single instance of that type in megabytes.

      • instance_max_cpuint

        The number of processor shares available to a single instance of that type in millicores.

      • instance_max_diskint

        The amount of disk available to a single instance of that type in gigabytes.

      • usage_statsdict
        • pending_memory_requestedint

          The sum of memory requests (in MB) for pending deployments in this instance config.

        • pending_cpu_requestedint

          The sum of cpu requests (in millicores) for pending deployments in this instance config.

        • running_memory_requestedint

          The sum of memory requests (in MB) for running deployments in this instance config.

        • running_cpu_requestedint

          The sum of cpu requests (in millicores) for running deployments in this instance config.

        • pending_deploymentsint

          The number of pending deployments in this instance config.

        • running_deploymentsint

          The number of running deployments in this instance config.

    • default_instance_config_idint

      The id of the InstanceConfig that is the default for this partition.

  • is_nat_enabledbool

    Whether this cluster needs a NAT gateway or not.

list_kubernetes_compute_hours(id: int, *, include_usage_stats: bool = None)

List compute hours for a Kubernetes Cluster

Parameters:
idint
include_usage_statsbool, optional

When true, usage stats are returned in instance config objects. Defaults to false.

Returns:
civis.response.Response
  • total_normalized_hoursint

    The total number of normalized hours used by this cluster.

  • normalized_hours_by_instance_typestr

    Denotes the instance type the normalized hours are attributed to.

  • updated_at : str (time)

  • month_and_yearstr

    The month and year the normalized hours are attributed to.

list_kubernetes_deployment_stats(id: int)

Get stats about deployments associated with a Kubernetes Cluster

Parameters:
idint

The ID of this cluster.

Returns:
civis.response.Response
  • base_typestr

    The base type of this deployment

  • statestr

    State of the deployment

  • countint

    Number of deployments of base type and state

  • total_cpuint

    Total amount of CPU in millicores for deployments of base type and state

  • total_memoryint

    Total amount of Memory in megabytes for deployments of base type and state

list_kubernetes_deployments(id: int, *, base_type: str = None, state: str = None, limit: int = None, page_num: int = None, order: str = None, order_dir: str = None, iterator: bool = None)

List the deployments associated with a Kubernetes Cluster

Parameters:
idint

The id of the cluster.

base_typestr, optional

If specified, return deployments of these base types. It accepts a comma- separated list, possible values are ‘Notebook’, ‘Service’, ‘Run’.

statestr, optional

If specified, return deployments in these states. It accepts a comma- separated list, possible values are pending, running, terminated, sleeping

limitint, optional

Number of results to return. Defaults to its maximum of 50.

page_numint, optional

Page number of the results to return. Defaults to the first page, 1.

orderstr, optional

The field on which to order the result set. Defaults to created_at. Must be one of: created_at.

order_dirstr, optional

Direction in which to sort, either asc (ascending) or desc (descending) defaulting to asc.

iteratorbool, optional

If True, return a generator to iterate over all responses. Use when more results than the maximum allowed by limit are needed. When True, limit and page_num are ignored. Defaults to False.

Returns:
civis.response.PaginatedResponse
  • idint

    The id of this deployment.

  • namestr

    The name of the deployment.

  • base_idint

    The id of the base object associated with the deployment.

  • base_typestr

    The base type of this deployment.

  • statestr

    The state of the deployment.

  • cpuint

    The CPU in millicores required by the deployment.

  • memoryint

    The memory in MB required by the deployment.

  • disk_spaceint

    The disk space in GB required by the deployment.

  • instance_typestr

    The EC2 instance type requested for the deployment.

  • authordict
    • idint

      The ID of this user.

    • namestr

      This user’s name.

    • usernamestr

      This user’s username.

    • initialsstr

      This user’s initials.

    • onlinebool

      Whether this user is online.

  • max_memory_usagefloat (float)

    If the deployment has finished, the maximum amount of memory used during the deployment, in MB.

  • max_cpu_usagefloat (float)

    If the deployment has finished, the maximum amount of cpu used during the deployment, in millicores.

  • created_at : str (time)

  • updated_at : str (time)

list_kubernetes_instance_configs_active_workloads(id: int, *, state: str = None)

List active workloads in an Instance Config

Parameters:
idint

The id of the instance config.

statestr, optional

If specified, return workloads in these states. It accepts a comma- separated list, possible values are pending, running

Returns:
civis.response.Response
  • idint

    The id of this deployment.

  • base_typestr

    The base type of this deployment.

  • base_idint

    The id of the base object associated with this deployment.

  • base_object_namestr

    The name of the base object associated with this deployment. Null if you do not have permission to read the object.

  • job_typestr

    If the base object is a job run you have permission to read, the type of the job. One of “python_script”, “r_script”, “container_script”, or “custom_script”.

  • job_idint

    If the base object is a job run you have permission to read, the id of the job.

  • job_cancel_requested_atstr (time)

    If the base object is a job run you have permission to read, and it was requested to be cancelled, the timestamp of that request.

  • statestr

    The state of this deployment.

  • cpuint

    The CPU in millicores requested by this deployment.

  • memoryint

    The memory in MB requested by this deployment.

  • disk_spaceint

    The disk space in GB requested by this deployment.

  • userdict
    • idint

      The ID of this user.

    • namestr

      This user’s name.

    • usernamestr

      This user’s username.

    • initialsstr

      This user’s initials.

    • onlinebool

      Whether this user is online.

  • created_atstr (time)

    The timestamp of when the deployment began.

  • cancellablebool

    True if you have permission to cancel this deployment.

list_kubernetes_instance_configs_historical_graphs(instance_config_id: int, *, timeframe: str = None)

Get graphs of historical resource usage in an Instance Config

Parameters:
instance_config_idint

The ID of this instance config.

timeframestr, optional

The span of time that the graphs cover. Must be one of 1_day, 1_week.

Returns:
civis.response.Response
  • cpu_graph_urlstr

    URL for the graph of historical CPU usage in this instance config.

  • mem_graph_urlstr

    URL for the graph of historical memory usage in this instance config.

list_kubernetes_instance_configs_user_statistics(instance_config_id: int, *, order: str = None, order_dir: str = None)

Get statistics about the current users of an Instance Config

Parameters:
instance_config_idint

The ID of this instance config.

orderstr, optional

The field on which to order the result set. Defaults to running_deployments. Must be one of pending_memory_requested, pending_cpu_requested, running_memory_requested, running_cpu_requested, pending_deployments, running_deployments.

order_dirstr, optional

Direction in which to sort, either asc (ascending) or desc (descending). Defaults to desc.

Returns:
civis.response.Response
  • user_idstr

    The owning user’s ID

  • user_namestr

    The owning user’s name

  • pending_deploymentsint

    The number of deployments belonging to the owning user in “pending” state

  • pending_memory_requestedint

    The sum of memory requests (in MB) for deployments belonging to the owning user in “pending” state

  • pending_cpu_requestedint

    The sum of CPU requests (in millicores) for deployments belonging to the owning user in “pending” state

  • running_deploymentsint

    The number of deployments belonging to the owning user in “running” state

  • running_memory_requestedint

    The sum of memory requests (in MB) for deployments belonging to the owning user in “running” state

  • running_cpu_requestedint

    The sum of CPU requests (in millicores) for deployments belonging to the owning user in “running” state

list_kubernetes_partitions(id: int, *, include_usage_stats: bool = None)

List Cluster Partitions for given cluster

Parameters:
idint
include_usage_statsbool, optional

When true, usage stats are returned in instance config objects. Defaults to false.

Returns:
civis.response.Response
  • cluster_partition_idint

    The ID of this cluster partition.

  • namestr

    The name of the cluster partition.

  • labelsList[str]

    Labels associated with this partition.

  • instance_configsList[dict]

    The instances configured for this cluster partition.

    • instance_config_idint

      The ID of this InstanceConfig.

    • instance_typestr

      An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

    • min_instancesint

      The minimum number of instances of that type in this cluster.

    • max_instancesint

      The maximum number of instances of that type in this cluster.

    • instance_max_memoryint

      The amount of memory (RAM) available to a single instance of that type in megabytes.

    • instance_max_cpuint

      The number of processor shares available to a single instance of that type in millicores.

    • instance_max_diskint

      The amount of disk available to a single instance of that type in gigabytes.

    • usage_statsdict
      • pending_memory_requestedint

        The sum of memory requests (in MB) for pending deployments in this instance config.

      • pending_cpu_requestedint

        The sum of cpu requests (in millicores) for pending deployments in this instance config.

      • running_memory_requestedint

        The sum of memory requests (in MB) for running deployments in this instance config.

      • running_cpu_requestedint

        The sum of cpu requests (in millicores) for running deployments in this instance config.

      • pending_deploymentsint

        The number of pending deployments in this instance config.

      • running_deploymentsint

        The number of running deployments in this instance config.

  • default_instance_config_idint

    The id of the InstanceConfig that is the default for this partition.

patch_kubernetes_partitions(id: int, cluster_partition_id: int, *, instance_configs: List[dict] = None, name: str = None, labels: List[str] = None)

Update a Cluster Partition

Parameters:
idint

The ID of the cluster which this partition belongs to.

cluster_partition_idint

The ID of this cluster partition.

instance_configsList[dict], optional

The instances configured for this cluster partition.

  • instance_typestr

    An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

  • min_instancesint

    The minimum number of instances of that type in this cluster.

  • max_instancesint

    The maximum number of instances of that type in this cluster.

namestr, optional

The name of the cluster partition.

labelsList[str], optional

Labels associated with this partition.

Returns:
civis.response.Response
  • cluster_partition_idint

    The ID of this cluster partition.

  • namestr

    The name of the cluster partition.

  • labelsList[str]

    Labels associated with this partition.

  • instance_configsList[dict]

    The instances configured for this cluster partition.

    • instance_config_idint

      The ID of this InstanceConfig.

    • instance_typestr

      An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

    • min_instancesint

      The minimum number of instances of that type in this cluster.

    • max_instancesint

      The maximum number of instances of that type in this cluster.

    • instance_max_memoryint

      The amount of memory (RAM) available to a single instance of that type in megabytes.

    • instance_max_cpuint

      The number of processor shares available to a single instance of that type in millicores.

    • instance_max_diskint

      The amount of disk available to a single instance of that type in gigabytes.

    • usage_statsdict
      • pending_memory_requestedint

        The sum of memory requests (in MB) for pending deployments in this instance config.

      • pending_cpu_requestedint

        The sum of cpu requests (in millicores) for pending deployments in this instance config.

      • running_memory_requestedint

        The sum of memory requests (in MB) for running deployments in this instance config.

      • running_cpu_requestedint

        The sum of cpu requests (in millicores) for running deployments in this instance config.

      • pending_deploymentsint

        The number of pending deployments in this instance config.

      • running_deploymentsint

        The number of running deployments in this instance config.

  • default_instance_config_idint

    The id of the InstanceConfig that is the default for this partition.

post_kubernetes_partitions(id: int, instance_configs: List[dict], name: str, labels: List[str])

Create a Cluster Partition for given cluster

Parameters:
idint

The ID of the cluster which this partition belongs to.

instance_configsList[dict]

The instances configured for this cluster partition.

  • instance_typestr

    An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

  • min_instancesint

    The minimum number of instances of that type in this cluster.

  • max_instancesint

    The maximum number of instances of that type in this cluster.

namestr

The name of the cluster partition.

labelsList[str]

Labels associated with this partition.

Returns:
civis.response.Response
  • cluster_partition_idint

    The ID of this cluster partition.

  • namestr

    The name of the cluster partition.

  • labelsList[str]

    Labels associated with this partition.

  • instance_configsList[dict]

    The instances configured for this cluster partition.

    • instance_config_idint

      The ID of this InstanceConfig.

    • instance_typestr

      An EC2 instance type. Possible values include t2.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m5.12xlarge, and c5.18xlarge.

    • min_instancesint

      The minimum number of instances of that type in this cluster.

    • max_instancesint

      The maximum number of instances of that type in this cluster.

    • instance_max_memoryint

      The amount of memory (RAM) available to a single instance of that type in megabytes.

    • instance_max_cpuint

      The number of processor shares available to a single instance of that type in millicores.

    • instance_max_diskint

      The amount of disk available to a single instance of that type in gigabytes.

    • usage_statsdict
      • pending_memory_requestedint

        The sum of memory requests (in MB) for pending deployments in this instance config.

      • pending_cpu_requestedint

        The sum of cpu requests (in millicores) for pending deployments in this instance config.

      • running_memory_requestedint

        The sum of memory requests (in MB) for running deployments in this instance config.

      • running_cpu_requestedint

        The sum of cpu requests (in millicores) for running deployments in this instance config.

      • pending_deploymentsint

        The number of pending deployments in this instance config.

      • running_deploymentsint

        The number of running deployments in this instance config.

  • default_instance_config_idint

    The id of the InstanceConfig that is the default for this partition.