Skip to main content
The External Secrets Enterprise Edition is product suite is a premium product. It requires a specific subscription. Contact us for more information.
This example demonstrates a common use case for ESI Federation: a central cluster managing secrets in Vault, and a client cluster consuming these secrets via esi-cli without having direct access to Vault.

​
Scenario Overview

  • Federation Server Cluster (cluster-main): Hosts the ESI Federation Server and a Vault instance. Secrets are stored in Vault.
  • Client Cluster (client-cluster-alpha): Hosts an application (Prometheus, in this example) that needs secrets from cluster-main’s Vault. esi-cli will be used to fetch these secrets.

​
Configuration Steps

​
1. On the Federation Server Cluster (cluster-main)

A. Install and Configure ESE: Ensure External Secrets Enterprise Edition is installed and configured to act as a Federation Server. This typically involves enabling federation features in its deployment. B. Set up ClusterSecretStore for Vault: A ClusterSecretStore named vault-prod is configured to connect to the local Vault instance. 
# Example: vault-prod ClusterSecretStore on cluster-main
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: vault-prod
spec:
  provider:
    vault:
      server: "http://vault.vault-ns.svc.cluster.local:8200"
      path: "secret"
      version: "v2"
      auth:
        # Appropriate Vault authentication method for ESE
        kubernetes:
          mountPath: "kubernetes-main"
          role: "external-secrets"
          serviceAccountRef:
            name: "ese-sa"
            namespace: "external-secrets-operator-ns"
C. Create KubernetesFederation CR for the client cluster: This CR registers client-cluster-alpha with the Federation Server. 
# Filename: kf-client-cluster-alpha.yaml on cluster-main
apiVersion: identity.federation.external-secrets.io/v1alpha1
kind: KubernetesFederation
metadata:
  name: client-cluster-alpha
spec:
  # The Kubernetes API Server URL of client-cluster-alpha
  url: https://kubeapi.client-alpha.example.com
Apply this CR: kubectl apply -f kf-client-cluster-alpha.yaml D. Create Authorization CR for the client’s Service Account: This CR grants the prometheus-esi-client ServiceAccount from client-cluster-alpha permission to access the vault-prod ClusterSecretStore. 
# Filename: auth-allow-alpha-monitoring.yaml on cluster-main
apiVersion: federation.external-secrets.io/v1alpha1
kind: Authorization
metadata:
  name: allow-alpha-monitoring
spec:
  federationRef:
    kind: KubernetesFederation
    name: client-cluster-alpha # Links to the KubernetesFederation CR above
  subject:
    # The JWT issuer URL for client-cluster-alpha's SA tokens
    issuer: https://kubernetes.default.svc.cluster.local 
    # The subject claim for the specific SA in client-cluster-alpha
    subject: system:serviceaccount:monitoring:prometheus-esi-client 
  allowedClusterSecretStores:
    - vault-prod # Grants access to this store on cluster-main
Apply this CR: kubectl apply -f auth-allow-alpha-monitoring.yaml

​
2. On the Client Cluster (client-cluster-alpha)

A. Create the Service Account: Ensure the prometheus-esi-client ServiceAccount exists in the monitoring namespace. 
# Filename: sa-prometheus-esi-client.yaml on client-cluster-alpha
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus-esi-client
  namespace: monitoring
Apply this CR: kubectl apply -f sa-prometheus-esi-client.yaml B. Configure Application Pod to use esi-cli: The Prometheus application pod will use esi-cli in an init container (or as the main container if Prometheus is launched by esi-cli) to fetch secrets before Prometheus starts. Here’s an example snippet of a Pod manifest using esi-cli as an init container: 
# Part of Prometheus Pod manifest on client-cluster-alpha
apiVersion: v1
kind: Pod
metadata:
  name: prometheus-with-federated-secrets
  namespace: monitoring
spec:
  serviceAccountName: prometheus-esi-client # Crucial: Pod uses the authorized SA
  initContainers:
    - name: esi-init
      image: ghcr.io/external-secrets-inc/esi-cli:latest # Use the appropriate esi-cli image
      args:
        - "--mode=init"
        - "--binary-path=/usr/local/bin/esi-noop" # Placeholder if only file injection is needed, or path to a script
        # If env var injection is primary, --binary-path would be the main app, e.g., /bin/prometheus
        - "--federated-server-url=https://esi-federation.cluster-main.example.com" # URL of Federation Server
        - "--federated-store=vault-prod" # Target store on Federation Server
        # Example: Fetch 'db-credentials' (interpreted by Federation Server via 'vault-prod')
        # and inject its 'password' key as an env var MY_APP_DB_PASSWORD
        - "--inject-on-env=MY_APP_DB_PASSWORD=db-credentials.password"
        # Example: Fetch 'api-keys' and write 'service-a.token' to a file
        - "--inject-on-file=/etc/secrets/service-a.token=api-keys.service-a-token"
      volumeMounts:
        - name: secrets-volume
          mountPath: /etc/secrets
  containers:
    - name: prometheus
      image: prom/prometheus:latest
      # Prometheus would be configured to read /etc/secrets/service-a.token
      # or use environment variable MY_APP_DB_PASSWORD
      env:
        - name: MY_APP_DB_PASSWORD # This will be populated by esi-cli
          value: "placeholder" # Will be overwritten
      volumeMounts:
        - name: secrets-volume
          mountPath: /etc/secrets
          readOnly: true
  volumes:
    - name: secrets-volume
      emptyDir: {}
Alternative esi-cli invocation if esi-cli directly runs Prometheus: 
# If esi-cli is the main container entrypoint or part of a script run by --binary-path
/usr/local/bin/esi-cli --mode=init \
  --binary-path=/bin/prometheus \
  --args="--config.file=/etc/prometheus/prometheus.yml" \
  --federated-server-url="https://esi-federation.cluster-main.example.com" \
  --federated-store="vault-prod" \
  --inject-on-env="PROMETHEUS_DB_USER=db-credentials.username,PROMETHEUS_DB_PASS=db-credentials.password" \
  # Assuming 'db-credentials' is an ExternalSecret data template on the federated store 'vault-prod'

​
Flow of Secret Retrieval

  1. The prometheus-with-federated-secrets pod starts on client-cluster-alpha.
  2. The esi-init init container (or esi-cli as main process) starts, using the prometheus-esi-client ServiceAccount.
  3. esi-cli sends its SA token (from system:serviceaccount:monitoring:prometheus-esi-client) and its cluster’s CA cert to https://esi-federation.cluster-main.example.com.
  4. The Federation Server on cluster-main validates the token using the JWKS from https://kubeapi.client-alpha.example.com (derived from the KubernetesFederation CR).
  5. The Federation Server checks the Authorization CR allow-alpha-monitoring. It confirms that system:serviceaccount:monitoring:prometheus-esi-client from client-cluster-alpha (issuer https://kubernetes.default.svc.cluster.local) is allowed to access vault-prod.
  6. The Federation Server fetches the requested secret data (e.g., db-credentials.password or api-keys.service-a-token) from its vault-prod store.
  7. The secret data is returned to esi-cli on client-cluster-alpha.
  8. esi-cli injects the secrets as environment variables or files (e.g., sets MY_APP_DB_PASSWORD or writes to /etc/secrets/service-a.token).
  9. The init container completes, and the main Prometheus container starts with the secrets available.
This setup successfully allows client-cluster-alpha to consume secrets managed by cluster-main’s Vault instance without client-cluster-alpha (or its applications) needing any direct credentials or network access to Vault. The Federation Server acts as a secure intermediary.
This example illustrates the power and flexibility of ESI Federation for managing secrets in a distributed environment.