Get Metrics
|
|||||
|
|
||||
Request the job status.
To request the status of a job, we recommend using
projects.locations.jobs.getMetrics with a [regional endpoint]
(https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using
projects.jobs.getMetrics is not recommended, as you can only request the
status of jobs that are running in us-central1
Authorization
To use this building block you will have to grant access to at least one of the following scopes:
- View and manage your data across Google Cloud Platform services
- View and manage your Google Compute Engine resources
- View your Google Compute Engine resources
- View your email address
Input
This building block consumes 4 input parameters
| Name | Format | Description |
|---|---|---|
projectId Required |
STRING |
A project id |
location Required |
STRING |
The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id |
jobId Required |
STRING |
The job to get messages for |
startTime |
ANY |
Return only metric data that has changed since this time. Default is to return all information about all metrics for the job |
= Parameter name
= Format
|
projectId STRING Required A project id |
|
location STRING Required The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id |
|
jobId STRING Required The job to get messages for |
|
startTime ANY Return only metric data that has changed since this time. Default is to return all information about all metrics for the job |
Output
This building block provides 17 output parameters
| Name | Format | Description |
|---|---|---|
metricTime |
ANY |
Timestamp as of which metric values are current |
metrics[] |
OBJECT |
Describes the state of a metric |
metrics[].scalar |
ANY |
Worker-computed aggregate value for aggregation kinds "Sum", "Max", "Min", "And", and "Or". The possible value types are Long, Double, and Boolean |
metrics[].meanCount |
ANY |
Worker-computed aggregate value for the "Mean" aggregation kind. This holds the count of the aggregated values and is used in combination with mean_sum above to obtain the actual mean aggregate value. The only possible value type is Long |
metrics[].meanSum |
ANY |
Worker-computed aggregate value for the "Mean" aggregation kind. This holds the sum of the aggregated values and is used in combination with mean_count below to obtain the actual mean aggregate value. The only possible value types are Long and Double |
metrics[].updateTime |
ANY |
Timestamp associated with the metric value. Optional when workers are reporting work progress; it will be filled in responses from the metrics API |
metrics[].name |
OBJECT |
Identifies a metric, by describing the source which generated the metric |
metrics[].name.origin |
STRING |
Origin (namespace) of metric name. May be blank for user-define metrics; will be "dataflow" for metrics defined by the Dataflow service or SDK |
metrics[].name.name |
STRING |
Worker-defined metric name |
metrics[].name.context |
OBJECT |
Zero or more labeled fields which identify the part of the job this metric is associated with, such as the name of a step or collection. For example, built-in counters associated with steps will have
context['step'] = |
metrics[].name.context.customKey.value |
STRING |
Zero or more labeled fields which identify the part of the job this metric is associated with, such as the name of a step or collection. For example, built-in counters associated with steps will have
context['step'] = |
metrics[].distribution |
ANY |
A struct value describing properties of a distribution of numeric values |
metrics[].set |
ANY |
Worker-computed aggregate value for the "Set" aggregation kind. The only possible value type is a list of Values whose type can be Long, Double, or String, according to the metric's type. All Values in the list must be of the same type |
metrics[].gauge |
ANY |
A struct value describing properties of a Gauge. Metrics of gauge type show the value of a metric across time, and is aggregated based on the newest value |
metrics[].internal |
ANY |
Worker-computed aggregate value for internal use by the Dataflow service |
metrics[].cumulative |
BOOLEAN |
True if this metric is reported as the total cumulative aggregate value accumulated since the worker started working on this WorkItem. By default this is false, indicating that this metric is reported as a delta that is not associated with any WorkItem |
metrics[].kind |
STRING |
Metric aggregation kind. The possible metric aggregation kinds are "Sum", "Max", "Min", "Mean", "Set", "And", "Or", and "Distribution". The specified aggregation kind is case-insensitive. If omitted, this is not an aggregated value but instead a single metric sample value |
= Parameter name
= Format
|
metricTime ANY Timestamp as of which metric values are current |
|
metrics[] OBJECT Describes the state of a metric |
|
metrics[].scalar ANY Worker-computed aggregate value for aggregation kinds "Sum", "Max", "Min", "And", and "Or". The possible value types are Long, Double, and Boolean |
|
metrics[].meanCount ANY Worker-computed aggregate value for the "Mean" aggregation kind. This holds the count of the aggregated values and is used in combination with mean_sum above to obtain the actual mean aggregate value. The only possible value type is Long |
|
metrics[].meanSum ANY Worker-computed aggregate value for the "Mean" aggregation kind. This holds the sum of the aggregated values and is used in combination with mean_count below to obtain the actual mean aggregate value. The only possible value types are Long and Double |
|
metrics[].updateTime ANY Timestamp associated with the metric value. Optional when workers are reporting work progress; it will be filled in responses from the metrics API |
|
metrics[].name OBJECT Identifies a metric, by describing the source which generated the metric |
|
metrics[].name.origin STRING Origin (namespace) of metric name. May be blank for user-define metrics; will be "dataflow" for metrics defined by the Dataflow service or SDK |
|
metrics[].name.name STRING Worker-defined metric name |
|
metrics[].name.context OBJECT Zero or more labeled fields which identify the part of the job this metric is associated with, such as the name of a step or collection. For example, built-in counters associated with steps will have
context['step'] = |
|
metrics[].name.context.customKey.value STRING Zero or more labeled fields which identify the part of the job this metric is associated with, such as the name of a step or collection. For example, built-in counters associated with steps will have
context['step'] = |
|
metrics[].distribution ANY A struct value describing properties of a distribution of numeric values |
|
metrics[].set ANY Worker-computed aggregate value for the "Set" aggregation kind. The only possible value type is a list of Values whose type can be Long, Double, or String, according to the metric's type. All Values in the list must be of the same type |
|
metrics[].gauge ANY A struct value describing properties of a Gauge. Metrics of gauge type show the value of a metric across time, and is aggregated based on the newest value |
|
metrics[].internal ANY Worker-computed aggregate value for internal use by the Dataflow service |
|
metrics[].cumulative BOOLEAN True if this metric is reported as the total cumulative aggregate value accumulated since the worker started working on this WorkItem. By default this is false, indicating that this metric is reported as a delta that is not associated with any WorkItem |
|
metrics[].kind STRING Metric aggregation kind. The possible metric aggregation kinds are "Sum", "Max", "Min", "Mean", "Set", "And", "Or", and "Distribution". The specified aggregation kind is case-insensitive. If omitted, this is not an aggregated value but instead a single metric sample value |