DATAPROC

dataproc/Check Autoscaling Policy

Verify autoscaling policies.

dataproc/Check Bq Connector

Check for issues related to BigQuery connector such as version dependency conflicts.

dataproc/Check Cluster Network

Verify that the nodes in the cluster can communicate with each other.

dataproc/Check Cluster Network Connectivity

Verify that the nodes in the cluster can communicate with each other.

dataproc/Check Cluster Quota

Verify if the Dataproc cluster has quota issues.

dataproc/Check Cluster Stock Out

Verify if Dataproc cluster has stockout issue.

dataproc/Check Cluster Version

Verify if the cluster version is supported.

dataproc/Check Gc Pause

Verify if STW GC Pause has happened.

dataproc/Check Gcs Connector

Check for non-default GCS connector and for errors in logs connected to Cloud Storage.

dataproc/Check If Job Failed

Verify if dataproc job failed.

dataproc/Check Init Script Failure

Verify if dataproc cluster init script failed.

dataproc/Check Job Throttling

Verify the presence of Job Throttling logs…

dataproc/Check Killing Orphaned Application

Verify if the killing of Orphaned applications has happened.

dataproc/Check Logs Exist

Checks if specified logs messages exist in the Dataproc cluster.

dataproc/Check Master Oom

Check if OOM has happened on master.

dataproc/Check Permissions

Check if the permissions are set correctly.

dataproc/Check Port Exhaustion

Verify if the port exhaustion has happened.

dataproc/Check Preemptible

Verify preemptibility.

dataproc/Check Private Google Access

Check if the subnetwork of the cluster has private google access enabled.

dataproc/Check Python Import Failure

Check if the python import failure has happened.

dataproc/Check Shared Vpc Roles

Verify if dataproc cluster is using Shared VPC.

dataproc/Check Shuffle Failures

Check for logs indicating shuffle failures.

dataproc/Check Shuffle Service Kill

Verify the presence of shuffle service kill related logs.

dataproc/Check Stackdriver Setting

Check if Stackdriver is enabled for the cluster.

dataproc/Check Sw Preemption

Verify if secondary worker preemption has happened.

dataproc/Check Task Not Found

Verify if dataproc job failed due to task not found.

dataproc/Check Worker Disk Usage Issue

Verify if worker disk usage issue has happened.

dataproc/Check Worker Oom

Verify if OOM has happened on worker nodes.

dataproc/Check Yarn Runtime Exception

Verify presence of CheckYarnRuntimeException logs…

dataproc/Cluster Creation End

The end step of the runbook

dataproc/Cluster Details

Gathers cluster parameters needed for further investigation.

dataproc/Cluster Exists

Prepares the parameters required for the dataproc/cluster-creation runbook.

dataproc/Cluster In Error

Verifies if the cluster is in Error state and gathers additional parameters.

dataproc/Data Proc Cluster Exists

Verify if cluster exists in Dataproc UI.

dataproc/Internal Ip Gateway

Check if the cluster is using internal IP only.

dataproc/Job Exists

Prepares the parameters required for the dataproc/spark_job_failures runbook.

dataproc/Service Account Exists

Validating service account and permissions in Dataproc cluster project or another project.

dataproc/Spark Job End

The end step of the runbook.