bigquery/ERR/2022_004
BigQuery jobs are not failing due to shuffle operation resources exceeded
Product: BigQuery
Rule class: ERR - Something that is very likely to be wrong
Description
The query job failed because the maximum disk and memory limit available for shuffle operations was exceeded for the project or organization.
Remediation
The limit cannot be increased for users - it is linearly correlated to the number of slots available in the project. Provision more slots for this to be increased.
Further recommendations that can help:
- Reduce query concurrency
- Optimize the query to use less shuffle resources
- Query less data within a single query job
- Materialize intermediate results