dataproc/Check Sw Preemption

Product: Cloud Dataproc
Step Type: COMPOSITE STEP

Description

None

Failure Reason

Found logs messages related to secondary worker preemption on the cluster: {cluster_name}.

Failure Remediation

This error occurs when secondary nodes are preempted. Please confirm if you are using secondary workers with preemptible instances. (The default Dataproc secondary worker type is a standard preemptible VM.)

You can recreate a cluster configured with non-preemptible secondary workers to ensure the secondary workers are not preemptible. [1] https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms#non-preemptible_workers

Success Reason

Didn’t find logs messages related to secondary worker preemption on the cluster: {cluster_name}.