GCP Service Account roles do not work correctly

Question

When granting roles to my service account, those roles do not give me the permissions they say they do.

I am using Terraform. I have created a new service account like so:

gcloud iam service-accounts create terraform \ --display-name "Terraform admin account"

I created a new JSON credentials for this account:

gcloud iam service-accounts keys create service account.json \ --iam-account terraform@PROJECT_ID.iam.gserviceaccount.com

I have granted the Compute Instance Admin role to this service account as described here: https://cloud.google.com/iam/docs/understanding-roles#role_types

gcloud projects add-iam-policy-binding PROJECT_ID \ --member serviceAccount:terraform@PROJECT_ID.iam.gserviceaccount.com \ --role roles/compute.instanceAdmin

I can see the service account has the right permissions:

gcloud projects get-iam-policy PROJECT_ID  \
--flatten="bindings[].members" \
--format='table(bindings.role)' \
--filter="bindings.members:terraform"
ROLE
roles/compute.instanceAdmin

And I am STILL getting the following errors, over and over, even when I grant owner or editor roles.

Error: Error reading ComputeNetwork "network-2pal-network": googleapi: Error 403: Required 'compute.networks.get' permission for 'projects/PROJECT_ID/global/networks/mars-production-network-2pal-network', forbidden

Error: Error reading ComputeGlobalAddress "mars-production-lb-ip": googleapi: Error 403: Required 'compute.globalAddresses.get' permission for 'projects/PROJECT_ID/global/addresses/production-lb-ip', forbidden

I am completely stuck. Does anyone have any idea how to get past this. Its like GCP isn't even checking the credentials. It's madness.

Thank you

Answer 1

Have you by any chance deleted the terraform service account before and just to re-create it later? Recreated service accounts with the same name show unexpected behaviors. This is a known issue in GCP IAM - deleting_and_recreating_service_accounts

Per the documentation linked above -

To avoid confusion, we suggest using unique service account names.
If this is not possible, you can grant a role to the new service account by:

1. Explicitly removing all bindings granting that role to the old service account.
2. Re-granting those roles to the new service account.

Answer 2

If the user will be managing virtual machine instances that are configured to run as a service account, you must also grant the roles/iam.serviceAccountUser role. Users granted the Service Account User role on a service account can use it to indirectly access all the resources to which the service account has access.

For example, if a service account has been granted the Compute Admin role (roles/compute.admin), a user that has been granted the Service Account Users role (roles/iam.serviceAccountUser) on that service account can act as the service account to start a Compute Engine instance. In this flow, the user impersonates the service account to perform any tasks using its granted roles and permissions.