dataflow/ERR/2023_002
Dataflow job does not fail during execution due to IP space exhaustion
Product: Dataflow
Rule class: ERR - Something that is very likely to be wrong
Description
A Dataflow job runs successfully if subnet has enough ip space for all workers in job, otherwise it fails with IP_SPACE_EXHAUSTED error.
You can search in the Logs Explorer for such jobs with the logging query:
resource.type="dataflow_step"
log_id("dataflow.googleapis.com/job-message")
severity=ERROR
textPayload=~"IP_SPACE_EXHAUSTED"
Remediation
- If dataflow job failed because of IP_SPACE_EXHAUSTED error, then this might be because of exhaustion of subnet IP address space. The possible resolution can be either by reducing the number of workers requesting IP addresses, increasing the subnet IP address space, or running enough jobs to not reach this limitation.