dataflow/WARN/2023_003
Dataflow job doen’t stuck at draining state for more than 3 hours
Product: Dataflow
Rule class: WARN - Something that is possibly wrong
Description
A Dataflow job might got stuck at draining as draining doesn’t fix stuck pipelines.
Remediation
Draining doesn’t fix stuck pipelines. If data movement is blocked, the pipeline remains stuck after the drain command. To address a stuck pipeline, use the update command to update the pipeline with code that resolves the error that is creating the problem. You can also cancel stuck jobs, but canceling jobs might result in data loss.