composer/ERR/2023_003

Cloud Composer Dags are not getting timed out by the Dag Processor

Product: Cloud Composer
Rule class: ERR - Something that is very likely to be wrong

Description

Sometimes we can see dag processor logs with task time out error. “Processor for/home/airflow/gcs/dags/exampledagname.py with PID 12345678 started at has timed out, killing it.” In an ideal composer environment this error shouldnot occur as it is a cause of scheduler resource constarint and complex DAGparsing implementation.

Remediation

Use the dag code best practices so that parse time get improved. Make sure scheduler is working at its best in terms of the resources. You may start reducing the schdeuler’s work by increasing the min_file_process_interval This page will help to mask the issue by increasing the parsing timeout value, but it will not prevent the issue. Please follow the above points to identify and fix the DAG parsing issue.

Further information

Please visit this page for more information about DAG parsing inspection.