When working with Google Cloud Storage (GCS), service account JSON keys provide a more secure and manageable authentication method compared to traditional access keys. The JSON key contains all necessary credentials in a structured format:
{
"type": "service_account",
"project_id": "your-project",
"private_key_id": "unique_id",
"private_key": "-----BEGIN PRIVATE KEY-----\n...",
"client_email": "service-account@project.iam.gserviceaccount.com",
"client_id": "client-id-number",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account%40project.iam.gserviceaccount.com"
}
There are two primary methods to configure gsutil with your JSON key:
Method 1: Using Environment Variable
The most straightforward approach is setting the GOOGLE_APPLICATION_CREDENTIALS environment variable:
# Linux/macOS
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json"
# Windows
set GOOGLE_APPLICATION_CREDENTIALS="C:\path\to\key.json"
After setting the variable, gsutil commands will automatically use this credential:
gsutil ls gs://your-bucket-name
Method 2: Configuring .boto File
For more persistent configuration, edit the .boto configuration file:
[Credentials]
gs_service_key_file = /path/to/key.json
gs_service_account_name = service-account@project.iam.gserviceaccount.com
When using JSON keys with gsutil, you might encounter:
- Permission Errors: Ensure the service account has proper IAM roles (Storage Admin or Storage Object Admin)
- Key Format Issues: Verify the JSON structure and that the private key contains proper line breaks
- Expired Keys: Service account keys don't expire by default but can be rotated
For scripts requiring temporary access, generate short-lived credentials:
gcloud auth activate-service-account --key-file=key.json
gcloud config set project your-project-id
This creates a temporary authentication token valid for one hour.
When working with JSON service account keys:
- Restrict file permissions (chmod 400 key.json)
- Never commit keys to version control
- Consider using Workload Identity Federation for production environments
- Rotate keys regularly using gcloud iam service-accounts keys
When working with Google Cloud Storage buckets via gsutil
, the standard documentation often focuses on traditional key/secret pairs. However, many modern Google Cloud setups use JSON key files for service account authentication. Here's how to properly configure this:
First, ensure you have the Google Cloud SDK installed and initialized:
# Install Google Cloud SDK
curl https://sdk.cloud.google.com | bash
exec -l $SHELL
gcloud init
There are two primary methods to use your JSON key:
Method 1: Set Default Application Credentials
Export the environment variable pointing to your key file:
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json"
gsutil ls gs://your-bucket-name
Method 2: Configure gcloud with the Service Account
gcloud auth activate-service-account --key-file=key.json
gcloud config set account client_email_from_json_file@project.iam.gserviceaccount.com
gsutil ls gs://your-bucket-name
Check your active authentication:
gcloud auth list
gsutil config -a
If you encounter permission errors, ensure:
- The service account has proper IAM permissions (Storage Admin or similar)
- The JSON file hasn't expired or been revoked
- There are no extra characters or formatting issues in the JSON
For projects requiring multiple service accounts:
gcloud auth activate-service-account account1@project.iam.gserviceaccount.com --key-file=key1.json
gsutil -i account1@project.iam.gserviceaccount.com ls gs://bucket1
gcloud auth activate-service-account account2@project.iam.gserviceaccount.com --key-file=key2.json
gsutil -i account2@project.iam.gserviceaccount.com ls gs://bucket2