Understanding workload identity federation

2021-12-22

Google Cloud’s Workload Identity Federation allows you to use your ambient credentials from AWS,Azure or in any arbitrary OIDC providers to access Google Cloud Resources.

The existing documentation primarily covers what it is, how to configure it and then use it. What the documentation omits is the internal flows invovled in getting the federated token from google.

This pages covers some slides I’ve used to describe how federation actually works interms of dataflow and configuration. Use this to understand for the various components that go into the flow and what an attribute mapping ’looks like'

If you want to just work on a demo, you can find the samples here

This article covers the flows for OIDC and AWS but the concepts and mapping are the same. It also further subdivides and dives into “impersonated” vs “federated” credentials which you are already using behind the scenes anyway.

This article will not cover how you get OIDC or AWS credentials but focus on what you do with those creds to access GCP.

Note, this article digs into the details …far more than you would need to know for casual use…

Flow Diagrams

As mentioned, there two types of flows: Impersonated and Federated.

  • Impersonated: with impersonated the external identity is mapped to an actual GCP service account and that service account needs permissions on the target resource
  • Federated: federated identities allows your intermediate credential direct access to the resource. Your federated principal:// or group must have IAM permission on the target resource

The high level flow is like

  1. Acquire ambient AWS|OIDC|Azure credentials
  2. Exchange that ambient token with Google for an intermediate federated token
  3. Depending on the service:
    • If the service you are accessing on GCP accepts the federated token, use it directly. (this is federated)
    • If the service you are access does not accept federated tokens, exchange the federated token for a GCP Service Account token. Use the Service Account token to access GCP (this is impersonated)

So..wait, what do you mean exchange….well, this whole capability hinges on a token-broker type service you may not have heard of: Security Token Service

GCP Security Token Service

“The Security Token Service exchanges Google or third-party credentials for a short-lived access token to Google Cloud resources.”

So thats it…GCP has a service that based on your administrators configuration for your domain, it will accept an arbitrary token from a 3rd party provider and then return back a federated token “google knows about”. The last part is critical: GCP (or any resource anywhere), should know something about the principal requesting access to a protected resource. With Workload identity federation and the STS server, your administrator defines what providers and identities GCP should ‘know about’.

Lets take a closer look at the request-response from this STS Server API:

The API accepts the standard JSON structure defined in rfc8693: OAuth 2.0 Token Exchange where the parameters define which Workload Identity Federation configuration to use (audience) and and the actual ambient token 3rd party token to exchange (subjectToken). In the example below, we are using an OIDC provider so the subjectToken is a JWT.

The exchange request/response looks like this

  • with POST Payload request.json
{
  "audience": "//iam.googleapis.com/projects/248066739582/locations/global/workloadIdentityPools/oidc-pool-1/providers/oidc-provider-1",
  "grantType": "urn:ietf:params:oauth:grant-type:token-exchange",
  "requestedTokenType": "urn:ietf:params:oauth:token-type:access_token",
  "scope": "https://www.googleapis.com/auth/cloud-platform",
  "subjectTokenType": "urn:ietf:params:oauth:token-type:jwt",
  "subjectToken": "eyJhbGciOiJSUzI1NiIsImtpZCI6I...."
}
  • POST to
curl -v -X POST -H "Content-Type: application/json; charset=utf-8" -d @request.json https://sts.googleapis.com/v1/token

Gives response

{
  "access_token": "ya29.d.Kp4DCwi13u77SSq21Sh8_Jv6dffJAP",
  "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
  "token_type": "Bearer",
  "expires_in": 3092
}

This exchange is pretty straightforward: its plain request-response:

images/wif_5.png

There we have it, the access_token. Critically, this is the federated token

To jump ahead a bit, few services directly accept this STS-issued token in an IAM binding. Currently (as of 12/23/21), Just GCS and iamcredentials GCP apis accepts this token as-is. What that means is you can take this and access GCS directly as the principal (assuming the principal as the IAM binding described later on in the documentation). An audit log entry for GCS would show the principal (note the principalSubject:// is all that is known to google;there’s no mapped GCP service account)

images/wif_9.png

As you can probably tell, the impersonated flow already uses the federated token!…i,e to get the service account’s access_token, the federated token is used to call iamcredentials.generateAccessToken(). You can also exchange this federated token a given service accounts id_token or to sign arbitrary bytes as that service account. More on that later

If you really want to know more about STS server (and a potential way you one with Terraform if you happens to have one handy), see

OIDC

Lets start off OIDC based federation where the identity provider could be any arbitrary OIDC system you happen to run. The git sample above and the screenshots below just happens to use Google’s Identity Platform as a provider but you could use any one you choose.

As mentioned, there are two flows and its easiest in a diagrams

OIDC Impersonated

Note the steps in orange would have been done by your administrator as part of configuring workload identity federation.

images/wif_1.png

One step to call out is 8: how does our authN, Cloud SDK libraries transparently do all this. The answer is describe here where Application Default Credentials now supports doing all the legwork for you if it knows about the configuration and service account you intend to use.

For example, if you run any Cloud SDK library after setting export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sts_adc.json with the following contents, the google auth library will automatically read the oidc token contained within credential_source.file= and use that with the STS server (credential_source.url is also supported). Once it has the STS server’s token, it will exchange that token with iamcredentials.generateAccessToken() endpoint and get the service_accont’s token for oidc-federated@fabled-ray-104117.iam.gserviceaccount.com. Then finally use to access any service.

  • sts_adc.json
{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/248066739582/locations/global/workloadIdentityPools/oidc-pool-1/providers/oidc-provider-1",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/tmp/oidccred.txt"
  },
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/oidc-federated@fabled-ray-104117.iam.gserviceaccount.com:generateAccessToken"
}

All this happens in the background for you and when the SDK client actually access the service, an audit log entry would look like this:

images/wif_6.png

Note the principalSet:// and principalEmail: values. The former is the mapped identity while the latter is the service account thats tied to the user.

To allow this flow, the federated identity mut be allowed to acquire the service account’s token and the service account must have permission on the resource

gcloud iam service-accounts add-iam-policy-binding oidc-federated@$PROJECT_ID.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/oidc-pool-1/subject/alice@domain.com"

gsutil iam ch serviceAccount:oidc-federated@$PROJECT_ID.iam.gserviceaccount.com:objectViewer gs://$PROJECT_ID-test

OIDC Federated

Many GCP services now “understand” the token you get from the STS service directly…you don’t have to impersonate a service account to access a service.

For that flow (which is recommended), simply omit the service_account_impersonation_url variable in the sts_adc.json file:

  • sts_adc.json:
{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/248066739582/locations/global/workloadIdentityPools/oidc-pool-1/providers/oidc-provider-1",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/tmp/oidccred.txt"
  }
}

images/wif_2.png

then just the mapped identity must have access to the resource

gcloud projects add-iam-policy-binding $PROJECT_ID  \
 --member "principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/oidc-pool-1/subject/alice@domain.com" \
 --role roles/storage.objectAdmin

Once the resource is accessed, the logs would show the principal:// only (no impersonation)

images/wif_9.png

users groups principal principalSet

We should probably discuss a bit about what a principal:// and principalSet:// are and how you can map your users to them properly.

At a high level, these are users and groups, respectively. Your admin configure the workload pools to map the claims on your presented identity to these principals and principalSet value. In OIDC, we will map claims embedded in the token to users and groups GCP uderstands. For more info, see Mapping Scenarios.

Identities Member format
Single identity principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/subject/SUBJECT_NAME
All identities in a group principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/group/GROUP_NAME
All identities with a specific attribute value principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.ATTRIBUTE_NAME/ATTRIBUTE_VALUE
All identities in a pool principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.ATTRIBUTE_NAME/ATTRIBUTE_VALUE

To map an individual user (subject) with OIDC, your admin will first define the attribute mapping and claims that will correspond back to whats presented in the oidc_token to STS and some GCP services.

  • principal:// The section below on the left column shows the gcloud configuration your admin would setup for the pool and the IAM bindings on the resource thats needed. Just to clarify,the Federated IAM binding done on GCS directly only applies to certain services with GCP

images/wif_7.png

The right hand column shows the oidc_token Alice uses. Notice how the clams are mapped in orange and the subject in green. This mapping the admin does establishes the control about who access what.

  • principalSet://

This is a group and similar to above, this is done by the mapping in the claims to a group pool defined by the admin. In the case below, it basically states, “use the claim ‘mygroup’ in the oidc_token as the indication of groups alice belows to”. The IAM binding on the resouce states, “only allow peopel in “group1” access to this resource

images/wif_8.png

AWS

Federating with AWS is very similar to OIDC and others but the distinction happens in how the mapping takes place and where the AWS token comes from.

For AWS, consider getting ambient credentials directly using env-vars or ambient credentials

images/wif_3.png images/wif_4.png

  • env-var
$ export AWS_ACCESS_KEY_ID=AK...

$ export AWS_SECRET_ACCESS_KEY=tW...

$ aws sts get-caller-identity
{
    "UserId": "AIDAUH3H6EGKDO36JYJH3",
    "Account": "291738886548",
    "Arn": "arn:aws:iam::291738886548:user/svcacct1"
}
  • on ec2
[root@ip-172-31-28-179 test]# aws sts get-caller-identity
{
    "Account": "291738886548", 
    "UserId": "AROAUH3H6EGKM3W5BCPKR:i-01eb8a107a2026dcd", 
    "Arn": "arn:aws:sts::291738886548:assumed-role/ec2role/i-01eb8a107a2026dcd"
}

Your administrator would have to map these identities using the to the appropriate type. For the first, the mapping is done as an individual user (principal://), the second one is any system that can assume a specific role (ec2role, inthis case). The latter could be a group of VMs on AWS

gcloud iam service-accounts add-iam-policy-binding aws-federated@$PROJECT_ID.iam.gserviceaccount.com   \
    --role roles/iam.workloadIdentityUser \
    --member "principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-pool-1/subject/arn:aws:iam::291738886548:user/svcacct1"

gcloud iam service-accounts add-iam-policy-binding aws-federated@$PROJECT_ID.iam.gserviceaccount.com   \
    --role roles/iam.workloadIdentityUser \
    --member "principalSet://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-pool-2/attribute.aws_role/arn:aws:sts::291738886548:assumed-role/ec2role"

One the mapping is done,the same support for ’easy’ Application Default Credentials can happen with any Cloud SDK.

Just define the file for either impersonated or federated credentials and access the service:

$ export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json

$ export TOKEN=`gcloud auth application-default print-access-token`

$ curl -v -H "Authorization: Bearer $TOKEN" \
     https://storage.googleapis.com/storage/v1/b/$PROJECT_ID-mybucket/o/foo.txt?alt=media

AWS Impersonated

The sts-creds.json for impersonated access would necessarily require the service_account_impersonation_url mapping and requires the service account being mapped aws-federated@fabled-ray-104117.iam.gserviceaccount.com to have access to the GCP resource (the GCS bucket here)

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/248066739582/locations/global/workloadIdentityPools/aws-pool-1/providers/aws-provider-1",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  },
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/aws-federated@fabled-ray-104117.iam.gserviceaccount.com:generateAccessToken"
}

The logs would show the original principal:// and the mapped service account images/wif_10.png

AWS Federated

For federated use, you can omit the service_account_impersonation_url file and directly use the STS-issued credentials.

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/248066739582/locations/global/workloadIdentityPools/aws-pool-1/providers/aws-provider-1",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  }
 }

Again, this should be a rare configuration.

The audit logs would show the principal:// accessing the resource.

images/wif_11.png

Using Federated Tokens to sign signJWT or get gcp idtokens

A final word on using the federated tokens. The ADC just exchanges the STS-issued token for an access_token by default. It does this by calling the iamcredentials.generateAccessToken() endpoint:

However, the federated token is ALSO capable of using any other endpoint

What this implies is you can use the credentials to generate a google issued oidc_tokens for a given audience. You can then use this token to access services like Cloud Run, Cloud Functions or to generate signedURLs.

However, you will need to manually invoke the IAM api to do so like this:

Assume you have GCP WIF setup for oidc as shown here

First allow the mapped principal permissions to directly impersonate a given SA:

gcloud iam service-accounts add-iam-policy-binding     target-serviceaccount@fabled-ray-104117.iam.gserviceaccount.com  \
    --member='principal://iam.googleapis.com/projects/1071284184436/locations/global/workloadIdentityPools/oidc-pool-1/subject/alice@domain.com' \
	--role='roles/iam.serviceAccountTokenCreator'

The iam.serviceAccountTokenCreator role includes concentric set of permission (see Concentric IAMCredentials Permissions: The secret life of signBlob. If you just need to generate an access_token or an id_token, consider generating a custom role with that specific capability.

in this case, federated token alice has is capable of directly getting an OIDC token for target-serviceaccount.

then if alice’s original federating token is in the file /tmp/oidccred.txt and your the ADC config is:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/1071284184436/locations/global/workloadIdentityPools/oidc-pool-1/providers/oidc-provider-1",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/tmp/oidccred.txt"
  }
}

then for each example below, remember to set

export GOOGLE_APPLICATION_CREDENTIALS=/path/to//sts-creds.json

#!/usr/bin/python
from google.auth import credentials
from google.cloud import  iam_credentials_v1

import google.auth
from google.auth.transport import requests


project_id = 'fabled-ray-104117'

client = iam_credentials_v1.services.iam_credentials.IAMCredentialsClient()

target_credentials = 'target-serviceaccount@{}.iam.gserviceaccount.com'.format(project_id)

name = "projects/-/serviceAccounts/{}".format(target_credentials)
id_token = client.generate_id_token(name=name,audience='https://foo.bar', include_email=True)

print(id_token.token)

package main

import (
	"context"
	"log"

	credentials "cloud.google.com/go/iam/credentials/apiv1"
	credentialspb "google.golang.org/genproto/googleapis/iam/credentials/v1"
)

var ()

const ()

func main() {
	ctx := context.Background()

	c, err := credentials.NewIamCredentialsClient(ctx)
	if err != nil {
		log.Fatalf("%v", err)
	}
	defer c.Close()

	req := &credentialspb.GenerateIdTokenRequest{
		Name:         "projects/-/serviceAccounts/target-serviceaccount@fabled-ray-104117.iam.gserviceaccount.com",
		Audience:     "https://foo.bar",
		IncludeEmail: true,
	}
	resp, err := c.GenerateIdToken(ctx, req)
	if err != nil {
		log.Fatalf("%v", err)
	}

	log.Printf("IdToken %v", resp.Token)

}

package com.test;

import com.google.cloud.iam.credentials.v1.GenerateIdTokenRequest;
import com.google.cloud.iam.credentials.v1.GenerateIdTokenResponse;
import com.google.cloud.iam.credentials.v1.IamCredentialsClient;
import com.google.cloud.iam.credentials.v1.ServiceAccountName;
public class TestApp {
	public static void main(String[] args) {
		TestApp tc = new TestApp();
	}

	private static String targetServiceAccount = "target-serviceaccount@fabled-ray-104117.iam.gserviceaccount.com";
	private static String audience = "https://foo.bar";

	public TestApp() {
		try {

			// GoogleCredentials sourceCredentials =
			// GoogleCredentials.getApplicationDefault();
			IamCredentialsClient iamCredentialsClient = IamCredentialsClient.create();

			String name = ServiceAccountName.of("-", targetServiceAccount).toString();

			GenerateIdTokenRequest request = GenerateIdTokenRequest.newBuilder().setName(name).setAudience(audience)
					.setIncludeEmail(true).build();
			GenerateIdTokenResponse response = iamCredentialsClient.generateIdToken(request);
			System.out.println("IDToken " + response.getToken());

		} catch (Exception ex) {
			System.out.println("Error:  " + ex.getMessage());
		}
	}

}
  • pom.xml:
<?xml version="1.0" encoding="UTF-8"?>


<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>

  <groupId>com.test.TestApp</groupId>
  <artifactId>TestApp</artifactId>

  <properties>
  </properties>

  <dependencies>

<!-- https://mvnrepository.com/artifact/com.google.cloud/google-cloud-iamcredentials -->
<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-iamcredentials</artifactId>
    <version>2.0.6</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.auth/google-auth-library-credentials -->
<dependency>
    <groupId>com.google.auth</groupId>
    <artifactId>google-auth-library-credentials</artifactId>
    <version>1.3.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.auth/google-auth-library-oauth2-http -->
<dependency>
    <groupId>com.google.auth</groupId>
    <artifactId>google-auth-library-oauth2-http</artifactId>
    <version>1.3.0</version>
</dependency>


 </dependencies>
  <build>
    <plugins>
     <plugin>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>2.0.2</version>
      <configuration>
        <source>1.8</source>
        <target>1.8</target>
      </configuration>
     </plugin>

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.2.1</version>
        <executions>
          <execution>
            <goals>
              <goal>java</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <mainClass>com.test.TestApp</mainClass>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>


const { IAMCredentialsClient } = require('@google-cloud/iam-credentials');

async function main() {

  const targetCredentials = 'target-serviceaccount@fabled-ray-104117.iam.gserviceaccount.com'

  const iam_client = new IAMCredentialsClient();
  const [resp] = await iam_client.generateIdToken({
    name: `projects/-/serviceAccounts/${targetCredentials}`,
    audience: 'https://foo.bar',
    includeEmail: true
  });
  console.info(resp.token);

}

main().catch(console.error);

using System;
using System.IO;
using System.Text;
using System.Threading.Tasks;

using Google.Apis.Auth;
using Google.Apis.Http;
using Google.Apis.Auth.OAuth2;
using System.Net.Http;
using System.Net.Http.Headers;
using Google.Cloud.Iam.Credentials.V1;


namespace Program
{
    public class Program
    {
        [STAThread]
        static void Main(string[] args)
        {
            try
            {
                //Google.ApplicationContext.RegisterLogger(new ConsoleLogger(LogLevel.All, true));
                new Program().Run().Wait();
            }
            catch (AggregateException ex)
            {
                foreach (var err in ex.InnerExceptions)
                {
                    Console.WriteLine("ERROR: " + err.Message);
                }
            }
        }

        public async Task<string> Run()
        {
            string targetAudience = "https://foo.bar";
            string targetPrincipal = "target-serviceaccount@fabled-ray-104117.iam.gserviceaccount.com";

            GoogleCredential sourceCredential = await GoogleCredential.GetApplicationDefaultAsync();

            IAMCredentialsClient client = IAMCredentialsClient.Create();
            GenerateIdTokenResponse resp = client.GenerateIdToken(new GenerateIdTokenRequest()
            {
                Name = "projects/-/serviceAccounts/" + targetPrincipal,
                Audience = targetAudience,
                IncludeEmail = true
            });

            Console.WriteLine("ID TOken " + resp.Token);
            return null;
        }
    }
}

This site supports webmentions. Send me a mention via this form.