GCP PubSub

This guide explains how to set up GCP PubSub listener in order to allow Just In Time Rotation for GCP Secrets Manager.

How it Works

reloader will listen for audit log events from a given GCP PubSub, and, based on pattern matching, will trigger an ExternalSecret reconciliation if that object queries the GCP Secret Manager key that had its version updated. The diagram below gives an overview of the setup: Architecture Light

Setting Up GCP

In order to set up GCP, four steps are needed:

Create a Topic and a Subscription on GCP Pub/Sub
Create a Log Router to send information from Audit Logs to PubSub
Create a Service Account & Permissions for reloader
Install reloader in your cluster

The following terraform code contains a recipe for everything needed within GCP using service account keys. We recommend using it with a local kind cluster for a quick onboarding experience - use workload identity in production environments.

The only action remaining is to install reloader within your cluster.

[Optional] Enable Service APIs

PROJECT_ID=your-project-id 
gcloud services enable pubsub.googleapis.com --project $PROJECT_ID
gcloud services enable secretmanager.googleapis.com --project $PROJECT_ID
gcloud services enable logging.googleapis.com --project $PROJECT_ID

Create a Topic and a Subscription on GCP Pubsub

 PROJECT_ID=your-project-id 
 TOPIC_ID=your-topic-id
 SUBSCRIPTION_ID=your-subscription-id
 # Create a Pub/Sub Topic
 gcloud pubsub topics create $TOPIC_ID --project $PROJECT_ID
 # Create a Pub/Sub Subscription
 gcloud pubsub subscriptions create $SUBSCRIPTION_ID \
--topic $TOPIC_ID \
--project $PROJECT_ID

Create a Log router for the PubSub Topic

 PROJECT_ID=your-project-id 
 TOPIC_ID=your-topic-id
 SUBSCRIPTION_ID=your-subscription-id
 SINK_NAME=secret-manager-addsecretversion-sink
 # Create a Log Router to send Audit Logs to Pub/Sub filtering SecretManager AddSecretVersion events
gcloud logging sinks create $SINK_NAME \
pubsub.googleapis.com/projects/$PROJECT_ID/topics/$TOPIC_ID \
--log-filter 'protoPayload.methodName=~"google.cloud.secretmanager.v1.SecretManagerService.AddSecretVersion"' \
--project $PROJECT_ID
# Get the sink service account
SINK_SERVICE_ACCOUNT=$(gcloud logging sinks describe $SINK_NAME \
--format 'value(writerIdentity)' \
--project $PROJECT_ID)
# Grant Pub/Sub Publisher role to the sink service account
gcloud pubsub topics add-iam-policy-binding $TOPIC_ID \
--member $SINK_SERVICE_ACCOUNT \
--role roles/pubsub.publisher \
--project $PROJECT_ID

Create Service Account & Permissions (Service Account Keys)

SA_NAME=reloader-sa
PROJECT_ID=your-project-id 
# Create a Service Account for reloader
gcloud iam service-accounts create $SA_NAME \
--display-name "Service Account for Reloader" \
--project $PROJECT_ID
# Grant Pub/Sub Subscriber role to the Service Account
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member "serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
--role roles/pubsub.subscriber
# Create and download the Service Account key
gcloud iam service-accounts keys create key.json \
--iam-account "$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com"
echo "now, make sure you install this key in the cluster so that reloader can use it"

Create Service Account & Permissions (Workload Identity)

SA_NAME=reloader-sa
PROJECT_ID=your-project-id 
NAMESPACE=service-account-namespace
KSA_NAME=service-account-name
# Create a Service Account for reloader
gcloud iam service-accounts create $SA_NAME \
--display-name "Service Account for Reloader" \
--project $PROJECT_ID
# Grant Pub/Sub Subscriber role to the Service Account
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member "serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
--role roles/pubsub.subscriber
# Create Workload Identity Binding
gcloud iam service-accounts add-iam-policy-binding $SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \
--member "serviceAccount:$PROJECT_ID.svc.id.goog[$NAMESPACE/$KSA_NAME]" \
--role roles/iam.workloadIdentityUser
# Create a Kubernetes Service Account
kubectl create serviceaccount $KSA_NAME --namespace $NAMESPACE
# Annotate the Kubernetes Service Account with the GCP Service Account email
kubectl annotate serviceaccount $KSA_NAME \
--namespace $NAMESPACE \
iam.gke.io/gcp-service-account=$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com

After all these steps - we are ready to install & use reloader

Example Configuration

Before applying this manifest, be sure to have installed reloader first.

apiVersion: reloader.external-secrets.io/v1alpha1
kind: Config
metadata:
  name: gcp-sample
spec:
  notificationSources:
    - type: GooglePubSub
      googlePubSub:
        subscriptionID: value-from-subscription-id
        projectID: value-from-project-id
        auth:
        # If using service account keys
          secretRef:
            secretAccessKeySecretRef:
              name: secret-name
              namespace: secret-namespace
              key: creds_json
        # If using Workload Identity
          workloadIdentity:
            clusterName: name-of-your-cluster-if-on-different-project
            clusterLocation: us-east1
            clusterProjectID: cluster-project-if-different
            serviceAccountRef:
              name: service-account-name
              namespace: service-account-namespace
  destinationsToWatch:
    - type: ExternalSecret
      externalSecret:
        labelSelectors:
          matchLabels:
            app: my-app

After that, every new AddSecretVersion message will automatically update your Kubernetes Secret values!

Getting Started

Getting Started

Configuration

Reference

How it Works

Setting Up GCP

Example Configuration

Getting Started

Getting Started

Configuration

Reference

​How it Works

​Setting Up GCP

​Example Configuration

How it Works

Setting Up GCP

Example Configuration