composer/BP/2023_003

Cloud Composer does not override the StatsD configuration

Product: Cloud Composer
Rule class: BP - Best practice, opinionated recommendation

Description

Metrics from Cloud Composer like scheduler heartbeat, number of completed tasks and pods are collected via the StatsD daemon. If you override the default StatsD configuration, it will cause missing metrics in the monitoring pages and components including airflow-scheduler that depend on Statsd metrics for healthcheck will be marked as unhealthy.

Remediation

Remove the overridden StatsD Airflow configurations, otherwise the monitoring page will never work.

Note that there is no way to make both the composer default settings and your customized StatsD settings work.

Further information