Encrypting Google Application Default and gcloud credentials with GPG SmardCard

2018-05-14

You use Google Cloud SDK gcloud all day long. You also occasionally make use of scripts that rely on GCP Application Default Credentials. All good. However, did you know those credentials by default gets stored in your home directory unencrypted in a sqlite3 db?

I get it, its not that big of a deal because its in your own home directory and there is a sense of trust in running trusted code in gcloud (if you don’t trust gcloud and you run it locally, there’s a whole other set of problems beyond unencrypted credentials…).

Anyway, I wanted to tinker with gcloud and GnuPG last year after someone kindly gave me a “YubiKey NEO” after GCP Next'17. What i did with that is what is detailed in this article: I encrypted the credentials for gcloud and ADC with a GPG keypair where the private part for decryption is only stored on a SmartCard. The private key never leaves the card and all decryption is handled on hardware.

What that means is when you initialize gcloud, your credentials and access_token gets encrypted and stored in sqllite. When any application needs to access it, you MUST insert your USB SmartCard into your computer!!!

Now, while the gcloud credentials saved on your computer is encrypted, it will get decrypted in userspace (meaning, its not that secure if there is a malicious program already running)

NOTE: What I did here is pretty impractical: I hacked gcloud and overrode some specific files just to make this work…Its not supported and will promptly break the moment you upgrade the gcloud SDK!!!

For amusement only

References

Default gcloud and ADC Credential Store

You didn’t believe me the gcloud SDK isn’t encrypted? ok, lets see:

sudo apt-get install sqlite3
cd ~/.config/gcloud/ 
$ sqllite3

sqlite> .open access_tokens.db
sqlite> .schema
  CREATE TABLE IF NOT EXISTS "access_tokens" (account_id TEXT PRIMARY KEY, access_token TEXT, token_expiry TIMESTAMP, rapt_token TEXT);
sqlite> select * from access_tokens;

$ sqlite3 credentials.db
sqlite> .schema 
   CREATE TABLE IF NOT EXISTS "credentials" (account_id TEXT PRIMARY KEY, value BLOB);
sqlite> .open credentials.db
sqlite> select * from credentials;

what did you see?

GPG SmartCard

Ok, i’ve skipped the very involved steps of setting up GPG with a SmartCard (it wasn’t trivial…especially loading the private keys on the card!). Some of the links cited in the references section above describe how to set this up…you ofcourse need a writeable smartcard (e.g. YubiKey NEO not regular YubiKey Nano, etc).

Assuming you’ve got a smartcard setup with private keys and your local laptop with the public key, we can proceed with configuring gcloud. Just for reference though, i’ll show how the config for the GPG and SmartCard I’ve got setup in the Appendix

Verify GPG-Python uses SmartCard

The sample outlined in this article uses python-gnupg so lets make sure that part works First step is to make sure gnupg is installed and you actually have the public keys (the private ones should be on the crypto card)

Download the keys (the following keyID is mine)

apt-get install -y gnupg

$ gpg2 --keyserver pgp.mit.edu --recv-keys 616BF511
gpg: key 97341836616BF511: "Salmaan Rashid <salrashid123@gmail.com>"

Then setup Trust for this key:

gpg2 --edit-key  616BF511

Now Install python-gnupg

sudo apt-get install python-gnupg 
virtualenv env
source env/bin/activate
pip install -r requirements.txt

Then execute the main.py script to check if the SmartCard is used: main.py (yeah, i should upgrade to python3)

import gnupg
from pprint import pprint
# pip3 install python-gnupg

gpg = gnupg.GPG(gnupghome='/home/srashid/.gnupg')
public_keys = gpg.list_keys()
private_keys = gpg.list_keys(True)
pprint(private_keys)

unencrypted_string = 'Lorem Ipsum'

public_keys = gpg.list_keys()
encrypted_data = gpg.encrypt(data=unencrypted_string, recipients=public_keys[0][
'fingerprint'])
encrypted_string = str(encrypted_data)

print("Encrypted ")
print(encrypted_string)

encrypted_string = str(encrypted_string)
decrypted_data = gpg.decrypt(encrypted_string)
print("Decrypted: ")
print(decrypted_data)

What you should see is first the encrypted form of the plainText without any prompting because the public key exists in your trust store locally:

$ python main.py 
Encrypted 
-----BEGIN PGP MESSAGE-----
hQEMA6NnAtzEGT8eAQgAuO8UZ9Q+CAcheojc7hXEYyfVSjVYxFUjY7AWylg5KuQr
sBtWjxx6RnIKhp2xq82t3A/wqEruB0hOWBm37hDYPeYcjyLQZ/gNbpPMHVZxRNaQ
a62/KeANEcjTRDveMjjPQ0nctwgxtHNMF3i27g45ubtSK3xIpjVkMwvRkNSHOe3x
Y7pvLV6YxqsOF+4PnPIE+KGBBSQGsuF4AzAspA7cHik+MWvcT80KDBDp/kIR7pSG
JSXLwpcyCA+5L+FMmL6rPJ6o+DJiLK28NwUu0RMRWAogX7tAioDNeApgGeZbAzNP
/SVylWGTk4WOw/64Mt/tgNioAQ17I7PNSybrU6u2n9JGAbC9MEdq6kHpGZScIeGh
GuUwLccBPLenANqBJ/W0UiOjvijakDpdKsSBy+RWaJFfVeYFeS1zZzRWN9fDttAn
xIXDQjcjpg==
=adqB
-----END PGP MESSAGE-----

But at that point, for decryption, you should get prompted for insert the smartcard and enter the PIN:

images/pgp_1.png

then enter PIN challenge: images/pgp_2.png

then GPG is allowed to access the private key to decrypt:

Decrypted: 
Lorem Ipsum

Encrypted Credentials and Token DB

Wow…ok, so now wev’e got GPG and SmartCard setup and working…now to hack at gcloud

First backup your SDK and existing credential store (since we’re gonna do an overwrite and encrypt everything…this is just a test; i didn’t care to account for existing creds store)

tar cvf gcloud_backup.tar ~/.config/gcloud
rm -rf ~/.config/gcloud

Then in your gcloud SDK installation, edit google-cloud-sdk/lib/googlecloudsdk/core/credentials/creds.py

and add on the CryptoFunctions() class anywhere

then in override the Load and Store methods in the following classes which just encryptes the tokens and credentials prior to store

Also modify the AccessTokenCache

Once you make those edits, just run

gcloud auth login

Once you run through that, check out the sqllite database column values

they’re encrypted!

So now use gcloud and see what happens: gcloud auth print-access-token

What you’ll see is a prompt to enter in your SmartCard!!:

images/pgp_3.png

then enter in PIN to continue

If you don’t have the smartcard or don’t enter the PIN in correctly, gcloud can’t decrypt the token or any token!

Application Default Credentials

Now we’ve just setup gcloud to use a smartcard, lets see what happens with Cloud API libraries that uses gcloud as Application Default Credentials:

Consider the following sample:

from google.cloud import storage
client = storage.Client()
buckets = client.list_buckets()
for bkt in buckets:
  print(bkt)

If you run that sample gcs.py, what you’ll see is a Prompt to enter in the smartCard and PIN:

For example, the following screenshot shows running the script with the smart card plugged in, the unplugged and rerun: i get prompted the second time to enter in the Card:

images/pgp_4.png

Conclusion

There was no real practical reason for me to go through this exercise since I don’t have a formal proposal or solution here.

It was just a way for me to tinker with gcloud and GPG and the nifty Yubikey i got. I can imagine someone formalizing this or some company that doens’t want to save gcloud credentials unencrypted somewhere or…in a high security environment to only bootstrap if a hardware token is required to run any operation.

Appendix

GPG SmartCard Setup First initialize GPG to read in the public keys (eg. as if you’re on a new laptop)

$ gpg2 --keyserver pgp.mit.edu --recv-keys 616BF511
gpg: key 97341836616BF511: public key "Salmaan Rashid <salrashid123@gmail.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1

on Local (searhorse): images/pgp_5.png

On pgp.mit.edu keyserver: images/pgp_6.png

Now setup trust for this key:

List the keys you’ve got…since we setup separate keys for Encryption, Sign and Authentication, we’ve got different subkeys (three of them)

$ gpg2 --list-keys
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
/home/srashid/.gnupg/pubring.kbx
--------------------------------
pub   rsa2048 2017-03-12 [SC]
      5D8EA7261718FE5728BA937C97341836616BF511
uid           [ultimate] Salmaan Rashid <salrashid123@gmail.com>
sub   rsa2048 2017-03-12 [E]
sub   rsa2048 2017-03-12 [A]
sub   rsa2048 2017-03-12 [S]

Now plugin in your YubiKey to the USB port and use gpg2 to read what is on the card

Again, this shows the three keys are actually on the card, not locally.

As mentioned, setting up the YubiKey with GPG keys wasn’t trivial..i’ve provided some links in the References section above to get that going.

This site supports webmentions. Send me a mention via this form.