projectId STRING Required

Required. The ID of the Google Cloud Platform project that the cluster belongs to

region STRING Required

Required. The Cloud Dataproc region in which to handle the request

filter STRING

Optional. A filter constraining the clusters to list. Filters are case-sensitive and have the following syntax:field = value AND field = value ...where field is one of status.state, clusterName, or labels.[KEY], and [KEY] is a label key. value can be * to match all values. status.state can be one of the following: ACTIVE, INACTIVE, CREATING, RUNNING, ERROR, DELETING, or UPDATING. ACTIVE contains the CREATING, UPDATING, and RUNNING states. INACTIVE contains the DELETING and ERROR states. clusterName is the name of the cluster provided at creation time. Only the logical AND operator is supported; space-separated items are treated as having an implicit AND operator.Example filter:status.state = ACTIVE AND clusterName = mycluster AND labels.env = staging AND labels.starred = *

pageToken STRING

Optional. The standard List page token

pageSize INTEGER

Optional. The standard List page size

clusters[] OBJECT

Describes the identifying information, config, and status of a cluster of Compute Engine instances

clusters[].labels OBJECT

Optional. The labels to associate with this cluster. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a cluster

clusters[].labels.customKey.value STRING

Optional. The labels to associate with this cluster. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a cluster

clusters[].metrics OBJECT

Contains cluster daemon metrics, such as HDFS and YARN stats.Beta Feature: This report is available for testing purposes only. It may be changed before final release

clusters[].metrics.hdfsMetrics OBJECT

The HDFS metrics

clusters[].metrics.hdfsMetrics.customKey.value INTEGER

The HDFS metrics

clusters[].metrics.yarnMetrics OBJECT

The YARN metrics

clusters[].metrics.yarnMetrics.customKey.value INTEGER

The YARN metrics

clusters[].status OBJECT

The status of a cluster and its instances

clusters[].status.detail STRING

Output only. Optional details of cluster's state

clusters[].status.state ENUMERATION

Output only. The cluster's state

clusters[].status.stateStartTime ANY

Output only. Time when this state was entered

clusters[].status.substate ENUMERATION

Output only. Additional state information that includes status reported by the agent

clusters[].statusHistory[] OBJECT

The status of a cluster and its instances

clusters[].statusHistory[].detail STRING

Output only. Optional details of cluster's state

clusters[].statusHistory[].state ENUMERATION

Output only. The cluster's state

clusters[].statusHistory[].stateStartTime ANY

Output only. Time when this state was entered

clusters[].statusHistory[].substate ENUMERATION

Output only. Additional state information that includes status reported by the agent

clusters[].config OBJECT

The cluster config

clusters[].config.workerConfig OBJECT

Optional. The config settings for Compute Engine resources in an instance group, such as a master or worker group

clusters[].config.workerConfig.instanceNames[] STRING

clusters[].config.workerConfig.numInstances INTEGER

Optional. The number of VM instances in the instance group. For master instance groups, must be set to 1

clusters[].config.workerConfig.isPreemptible BOOLEAN

Optional. Specifies that this instance group contains preemptible instances

clusters[].config.workerConfig.imageUri STRING

Optional. The Compute Engine image resource used for cluster instances. It can be specified or may be inferred from SoftwareConfig.image_version

clusters[].config.workerConfig.machineTypeUri STRING

Optional. The Compute Engine machine type used for cluster instances.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/us-east1-a/machineTypes/n1-standard-2 projects/[project_id]/zones/us-east1-a/machineTypes/n1-standard-2 n1-standard-2Auto Zone Exception: If you are using the Cloud Dataproc Auto Zone Placement feature, you must use the short name of the machine type resource, for example, n1-standard-2

clusters[].config.gceClusterConfig OBJECT

Common config settings for resources of Compute Engine cluster instances, applicable to all instances in the cluster

clusters[].config.gceClusterConfig.tags[] STRING

clusters[].config.gceClusterConfig.serviceAccount STRING

Optional. The service account of the instances. Defaults to the default Compute Engine service account. Custom service accounts need permissions equivalent to the following IAM roles: roles/logging.logWriter roles/storage.objectAdmin(see https://cloud.google.com/compute/docs/access/service-accounts#custom_service_accounts for more information). Example: [account_id]@[project_id].iam.gserviceaccount.com

clusters[].config.gceClusterConfig.subnetworkUri STRING

Optional. The Compute Engine subnetwork to be used for machine communications. Cannot be specified with network_uri.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/regions/us-east1/subnetworks/sub0 projects/[project_id]/regions/us-east1/subnetworks/sub0 sub0

clusters[].config.gceClusterConfig.networkUri STRING

Optional. The Compute Engine network to be used for machine communications. Cannot be specified with subnetwork_uri. If neither network_uri nor subnetwork_uri is specified, the "default" network of the project is used, if it exists. Cannot be a "Custom Subnet Network" (see Using Subnetworks for more information).A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/regions/global/default projects/[project_id]/regions/global/default default

clusters[].config.gceClusterConfig.zoneUri STRING

Optional. The zone where the Compute Engine cluster will be located. On a create request, it is required in the "global" region. If omitted in a non-global Cloud Dataproc region, the service will pick a zone in the corresponding Compute Engine region. On a get request, zone will always be present.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone] projects/[project_id]/zones/[zone] us-central1-f

clusters[].config.gceClusterConfig.internalIpOnly BOOLEAN

Optional. If true, all instances in the cluster will only have internal IP addresses. By default, clusters are not restricted to internal IP addresses, and will have ephemeral external IP addresses assigned to each instance. This internal_ip_only restriction can only be enabled for subnetwork enabled networks, and all off-cluster dependencies must be configured to be accessible without external IP addresses

clusters[].config.gceClusterConfig.metadata OBJECT

The Compute Engine metadata entries to add to all instances (see Project and instance metadata (https://cloud.google.com/compute/docs/storing-retrieving-metadata#project_and_instance_metadata))

clusters[].config.gceClusterConfig.metadata.customKey.value STRING

The Compute Engine metadata entries to add to all instances (see Project and instance metadata (https://cloud.google.com/compute/docs/storing-retrieving-metadata#project_and_instance_metadata))

clusters[].config.gceClusterConfig.serviceAccountScopes[] STRING

clusters[].config.softwareConfig OBJECT

Specifies the selection and config of software inside the cluster

clusters[].config.softwareConfig.imageVersion STRING

Optional. The version of software inside the cluster. It must be one of the supported Cloud Dataproc Versions, such as "1.2" (including a subminor version, such as "1.2.29"), or the "preview" version. If unspecified, it defaults to the latest Debian version

clusters[].config.softwareConfig.properties OBJECT

Optional. The properties to set on daemon config files.Property keys are specified in prefix:property format, for example core:hadoop.tmp.dir. The following are supported prefixes and their mappings: capacity-scheduler: capacity-scheduler.xml core: core-site.xml distcp: distcp-default.xml hdfs: hdfs-site.xml hive: hive-site.xml mapred: mapred-site.xml pig: pig.properties spark: spark-defaults.conf yarn: yarn-site.xmlFor more information, see Cluster properties

clusters[].config.softwareConfig.properties.customKey.value STRING

Optional. The properties to set on daemon config files.Property keys are specified in prefix:property format, for example core:hadoop.tmp.dir. The following are supported prefixes and their mappings: capacity-scheduler: capacity-scheduler.xml core: core-site.xml distcp: distcp-default.xml hdfs: hdfs-site.xml hive: hive-site.xml mapred: mapred-site.xml pig: pig.properties spark: spark-defaults.conf yarn: yarn-site.xmlFor more information, see Cluster properties

clusters[].config.softwareConfig.optionalComponents[] ENUMERATION

clusters[].config.masterConfig OBJECT

Optional. The config settings for Compute Engine resources in an instance group, such as a master or worker group

clusters[].config.masterConfig.instanceNames[] STRING

clusters[].config.masterConfig.numInstances INTEGER

Optional. The number of VM instances in the instance group. For master instance groups, must be set to 1

clusters[].config.masterConfig.isPreemptible BOOLEAN

Optional. Specifies that this instance group contains preemptible instances

clusters[].config.masterConfig.imageUri STRING

Optional. The Compute Engine image resource used for cluster instances. It can be specified or may be inferred from SoftwareConfig.image_version

clusters[].config.masterConfig.machineTypeUri STRING

Optional. The Compute Engine machine type used for cluster instances.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/us-east1-a/machineTypes/n1-standard-2 projects/[project_id]/zones/us-east1-a/machineTypes/n1-standard-2 n1-standard-2Auto Zone Exception: If you are using the Cloud Dataproc Auto Zone Placement feature, you must use the short name of the machine type resource, for example, n1-standard-2

clusters[].config.secondaryWorkerConfig OBJECT

Optional. The config settings for Compute Engine resources in an instance group, such as a master or worker group

clusters[].config.secondaryWorkerConfig.instanceNames[] STRING

clusters[].config.secondaryWorkerConfig.numInstances INTEGER

Optional. The number of VM instances in the instance group. For master instance groups, must be set to 1

clusters[].config.secondaryWorkerConfig.isPreemptible BOOLEAN

Optional. Specifies that this instance group contains preemptible instances

clusters[].config.secondaryWorkerConfig.imageUri STRING

Optional. The Compute Engine image resource used for cluster instances. It can be specified or may be inferred from SoftwareConfig.image_version

clusters[].config.secondaryWorkerConfig.machineTypeUri STRING

Optional. The Compute Engine machine type used for cluster instances.A full URL, partial URI, or short name are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/zones/us-east1-a/machineTypes/n1-standard-2 projects/[project_id]/zones/us-east1-a/machineTypes/n1-standard-2 n1-standard-2Auto Zone Exception: If you are using the Cloud Dataproc Auto Zone Placement feature, you must use the short name of the machine type resource, for example, n1-standard-2

clusters[].config.encryptionConfig OBJECT

Encryption settings for the cluster

clusters[].config.encryptionConfig.gcePdKmsKeyName STRING

Optional. The Cloud KMS key name to use for PD disk encryption for all instances in the cluster

clusters[].config.securityConfig OBJECT

Security related configuration, including Kerberos

clusters[].config.initializationActions[] OBJECT

Specifies an executable to run on a fully configured node and a timeout period for executable completion

clusters[].config.configBucket STRING

Optional. A Google Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Google Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Cloud Dataproc staging bucket)

clusters[].clusterName STRING

Required. The cluster name. Cluster names within a project must be unique. Names of deleted clusters can be reused

clusters[].clusterUuid STRING

Output only. A cluster UUID (Unique Universal Identifier). Cloud Dataproc generates this value when it creates the cluster

clusters[].projectId STRING

Required. The Google Cloud Platform project ID that the cluster belongs to

nextPageToken STRING

Output only. This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent ListClustersRequest