datafusion/WARN/2024_003
Scaling down is disabled for the Compute Profile for Dataproc.
Product: Cloud Data Fusion
Rule class: WARN - Something that is possibly wrong
Description
Autoscaling is not recommended for scaling down. Decreasing the cluster size with autoscaling removes nodes that hold intermediate data, which might cause your pipelines to run slowly or fail in datafusion.