dataflow/ERR/2024_005

Dataflow and its controller service account have the necessary permissions to interact with Pub/Sub topics.

Product: Dataflow
Rule class: ERR - Something that is very likely to be wrong

Description

Dataflow jobs rely on IAM permissions to access Pub/Sub topics.While the Dataflow job itself needs the ‘pubsub.subscriber’ role to receive messages, the controller service account also requires permission to view topic details (‘pubsub.topics.get’). This permission is usually included in the broader ‘pubsub.viewer’ role. If the controller service account lacks ‘pubsub.topics.get’ permission, it will fail to create subscriptions, resulting in a ‘GETTING_PUBSUB_SUBSCRIPTION_FAILED’ error.

Remediation

The Controller service account also needs pubsub.topics.get permission included in role pubsub.viewer in addition to role pubsub.subscriber.

Further information

Public documentation - Accessing Pub/Sub topics and subscriptions