gcp

Using JWT AccessTokens with Google Cloud Client Libraries

2021-12-15

Back in 2018 I stumbled across an obscure addendum in Google’s oauth2 documentation that described a unique type of authentication flow Google supported: JWT Access Tokens. You know what a JWT is and you probably know what a generic oauth2 access_token is too. However, in this case, its just a custom authentication flow certain GCP services honor.

Its basically a way for a service account to authenticate to a Google Cloud Service without needing an extra round trip that provides an oauth2 access_token. This flow also has the benefit of not depending on the intermediate google oauth2 server that would return the access_token (i,e less components –> slightly better reliability).

There is a catch with this flow: the caller must be able directly or indirectly sign a JWT using the service account’s private key. Having gcp service account keys is fairly problematic by itself (see Best practices for securing service accounts). Essentially, while this capability is nice, its of limited practical use.

For now, lets assume you have secured the service account’s key within hardware like a Trusted Platform Module or the private key was bound to KMS or bound inside HashiCorp Vault. In these scenarios, you could generate a JWTAccessToken and use that for authentication. The articles below cites the various mechanism you can use to acquire the raw token.

Google Cloud libraries implementations shown below still require references to the raw service account key to do anything. This significantly diminishes this features capability.

The intent of this article is to explain the merits of using this type of flow and the disadvantages. If you can secure the raw key into hardware,this flow maybe of use.

project='your-project-id'

from google.auth import jwt
from google.cloud import pubsub_v1

audience = 'https://pubsub.googleapis.com/google.pubsub.v1.Publisher'
credentials = jwt.Credentials.from_service_account_file(
    '/path/to/svc_account.json',
    audience=audience)

publisher = pubsub_v1.PublisherClient(credentials=credentials)
project_path = f"projects/{project}"
for topic in publisher.list_topics(request={"project": project_path}):
  print(topic.name)
package main

import (
	"fmt"
	"io/ioutil"

	pubsub "cloud.google.com/go/pubsub"
	"golang.org/x/net/context"
	"golang.org/x/oauth2/google"
	"google.golang.org/api/iterator"
	"google.golang.org/api/option"
)

const (
	projectID = "your-project-id"
)

func main() {

	ctx := context.Background()

	keyBytes, err := ioutil.ReadFile("/path/to/svc_account.json")
	if err != nil {
		panic(err)
	}

	aud := "https://pubsub.googleapis.com/google.pubsub.v1.Publisher"
	tokenSource, err := google.JWTAccessTokenSourceFromJSON(keyBytes, aud)
	if err != nil {
		panic(err)
	}

	//creds, err := google.JWTAccessTokenSourceWithScope()
	pubsubClient, err := pubsub.NewClient(ctx, projectID, option.WithTokenSource(tokenSource))
	if err != nil {
		fmt.Printf("pubsub.NewClient: %v", err)
		return
	}
	defer pubsubClient.Close()

	pit := pubsubClient.Topics(ctx)
	for {
		topic, err := pit.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			fmt.Printf("pubssub.Iterating error: %v", err)
			return
		}
		fmt.Printf("Topic Name: %s\n", topic.ID())
	}
}
package com.test;

import com.google.cloud.pubsub.v1.TopicAdminClient;
import com.google.cloud.pubsub.v1.TopicAdminClient.ListTopicsPagedResponse;
import com.google.cloud.pubsub.v1.TopicAdminSettings;

import com.google.pubsub.v1.ListTopicsRequest;
import com.google.pubsub.v1.ProjectName;
import com.google.pubsub.v1.Topic;
import com.google.api.gax.core.FixedCredentialsProvider;
import com.google.auth.oauth2.ServiceAccountJwtAccessCredentials;
import java.io.File;
import java.io.InputStream;
import java.net.URI;
import java.io.FileInputStream;

public class TestApp {
	public static void main(String[] args) {
		TestApp tc = new TestApp();
	}

	public TestApp() {
		try {

			String serviceAccuntKey = "/path/to/svc_account.json";

			URI audience = new URI("https://pubsub.googleapis.com/google.pubsub.v1.Publisher");

			InputStream credentialsStream = new FileInputStream(new File(serviceAccuntKey));
			ServiceAccountJwtAccessCredentials credentials = ServiceAccountJwtAccessCredentials
					.fromStream(credentialsStream, audience);
			TopicAdminClient topicClient = TopicAdminClient
					.create(TopicAdminSettings.newBuilder()
							.setCredentialsProvider(FixedCredentialsProvider.create(credentials)).build());

			ListTopicsRequest listTopicsRequest = ListTopicsRequest.newBuilder()
					.setProject(ProjectName.format("your-project-id"))
					.build();

			ListTopicsPagedResponse response = topicClient.listTopics(listTopicsRequest);
			Iterable<Topic> topics = response.iterateAll();
			for (Topic topic : topics)
				System.out.println(topic);

		} catch (Exception ex) {
			System.out.println("Error: " + ex);
		}
	}

}

TODO:

var log4js = require("log4js");
var logger = log4js.getLogger();

const {GoogleAuth, JWTAccess, OAuth2Client} =  require('google-auth-library');
const {PubSub} = require('@google-cloud/pubsub');

key = require("/path/to/svc_account.json");
const projectId = 'your-project-id';
const aud = "https://pubsub.googleapis.com/google.pubsub.v1.Publisher";
const client = new JWTAccess();
client.fromJSON(key);

const md = client.getRequestHeaders(aud);

console.log(md);

// TODO, attach the client to pubsub somehow

// https://cloud.google.com/nodejs/docs/reference/pubsub/latest/pubsub/clientconfig
// const pubsub = new PubSub({
//   projectId: projectId,
// });
// pubsub.getTopics((err, topic) => {
// 	if (err) {
// 		console.log(err);
// 		return;
// 	}
// 	topic.forEach(function(entry) {
//     logger.info(entry.name);
// 	});
// });

TODO

using System;

using Google.Cloud.PubSub.V1;
using System.Threading.Tasks;
using System.IO;
using Google.Api.Gax.ResourceNames;
using Google.Api.Gax.Grpc;
using Google.Apis;
using Google.Apis.Auth.OAuth2;
using Grpc.Auth;

namespace main
{
    class Program
    {
        const string projectID = "your-project-id";
        [STAThread]
        static void Main(string[] args)
        {
            new Program().Run().Wait();
        }

        private Task Run()
        {
            var _audience = "https://pubsub.googleapis.com/google.pubsub.v1.Publisher";
            var keyFile = "/path/to/svc_account.json";
            var stream = new FileStream(keyFile, FileMode.Open, FileAccess.Read);
            ServiceAccountCredential cred = ServiceAccountCredential.FromServiceAccountData(stream);
            var token = cred.GetAccessTokenForRequestAsync(_audience).Result;
            Console.WriteLine(token);

            // i don't know how to attach the credential to the client here...
            
            PublisherServiceApiClient publisher = PublisherServiceApiClient.Create();
            ProjectName projectName = ProjectName.FromProject(projectID);

            foreach (Topic t in publisher.ListTopics(projectName))
                Console.WriteLine(t.MessageStoragePolicy);
            return Task.CompletedTask;
        }
    }
}

For what its worth, the original article i wrote on this topic is cited below but I’m writing this up as an update since many google cloud client libraries currently supports this flow. This article describes how you can acquire these JWT access tokens and use them in a client library.

Faster and more Reliable ServiceAccount authentication for Google Cloud Platform APIs


This site supports webmentions. Send me a mention via this form.