bigquery/ERR/2022_003

BigQuery jobs are not failing while accessing data in Drive due to a permission issue

Product: BigQuery
Rule class: ERR - Something that is very likely to be wrong

Description

BigQuery jobs are failing because the authentication token is missing the Google Drive access scope or the user/service account is not granted at least the Viewer role on the Drive file

Remediation

Further information

The drive scope can be applied at a different level as well:

gcloud auth login --enable-gdrive-access
  • If you user credentials are used as Application Default Credentials, you can specify scopes the following way (--scopes can be used to add any scope, not limited to drive or bigquery):
gcloud auth application-default login --scopes openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/drive,https://www.googleapis.com/auth/bigquery
  • Using the bq tool when running bq query:
bq query --enable_gdrive=true 'SELECT ......'
gcloud container node-pools create {POOL-NAME} \
--cluster={CLUSTER-NAME} \
--disk-type=pd-ssd --machine-type=n1-standard-16 --enable-autoscaling --max-nodes=3 --min-nodes=0 --num-nodes=0 --preemptible --zone=us-west1-a \
--scopes=cloud-platform,https://www.googleapis.com/auth/drive
gcloud compute instances create {INSTANCE-NAME} \
--zone=us-west1-a --image-family=debian-9 --image-project=debian-cloud --machine-type=n1-standard-16 \
--scopes=cloud-platform,https://www.googleapis.com/auth/drive