Skip to content

Secrets with KSOPS

For customers with a dedicated ArgoCD instance, for an additional cost, we can offer the Kustomize SOPS plugin which can be used to decrypt resources encrypted with SOPS.

On the Cluster, Redpill Linpro will:

  • Deploy ArgoCD instance
  • Generate a private/public key with the age binary
  • Create a secret in the ArgoCD namespace of the customer that stores the public and private keys
  • Customize the Argo CD instance to use the Kustomize SOPS plugin
  • Push the Public key into the git repository/or send it securely to the customer

Developers/the customer will:

  • Create a Secret on their local console
  • Download the public key to encrypt the secret with SOPS CLI
  • Generate KSOPS YAML with the encrypted secret and push to their Git repository

Requirements for the developer/Customer

Download the following binaries to your client machine: (Decrypting locally is only possible if you have the private key stored on your client machine, which is not advised)

  1. sops
  2. kustomize

Configure SOPS

  1. SOPS is configured via .sops.yaml

    For this example and testing, KSOPS relies on the SOPS creation rules defined in .sops.yaml. To make encrypted secrets more readable, we suggest using the following encryption regex to only encrypt data and stringData values. This leaves non-sensitive fields, like the secret's name, unencrypted and human readable.

    cat <<EOF > .sops.yaml
    creation_rules:
      - encrypted_regex: "^(data|stringData)$"
        age: <your public key>
    EOF
    
  2. Create a Resource (local Kubernetes Secret)

    cat <<EOF > secret.yaml
    apiVersion: v1
    kind: Secret
    metadata:
      name: mysecret
    type: Opaque
    data:
      username: YWRtaW4=
      password: MWYyZDFlMmU2N2Rm
    EOF
    
  3. Encrypt the Resources with SOPS CLI (Specific SOPS configuration in .sops.yaml)

    All changes to the yaml file after encryption will give data integrity errors, so re-encrypt if you want to add stuff!

    sops -e secret.yaml > secret.enc.yaml
    
  4. Define KSOPS kustomize Generator(Create a local Kubernetes Secret)

    cat <<EOF > secret-generator.yaml
    apiVersion: viaduct.ai/v1
    kind: ksops
    metadata:
    # Specify a name
      name: secret-generator
      annotations:
        config.kubernetes.io/function: |
          exec:
            path: ksops
    files:
      - ./secret.enc.yaml
    EOF
    
  5. Create an kustomization.yaml (example)

    cat <<EOF > kustomization.yaml
    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    generatorOptions:
      disableNameSuffixHash: true
    namespace: yournamespace
    
    generators:
      - ./secret-generator.yaml
    EOF
    
  6. Run ArgoCD

Troubleshooting

Sanity Checks

  1. Validate ksops command is in your path

    # Should output a path to KSOPS executable
    command -v ksops
    
  2. If you prefer to not install ksops to your path, make sure the path to the executable in the generator manifest is relative to the manifests files

  3. Running kustomize build --enable-alpha-plugins --enable-exec . | oc diff -f - will not work locally since you need the private key to decrypt your encrypted values. A typical error output of this would be:

    failed to evaluate function: error decrypting file "secret.sops.yaml" from manifest.Files: trouble decrypting file: Error getting data key: 0 successful groups required, got 0unable to generate manifests: error decrypting file "secret.sops.yaml" from manifest.Files: trouble decrypting file: Error getting data key: 0 successful groups required, got 0Error: couldn't execute function: exit status 1
    
  4. If the following annotation is not configured with correct indentation

    metadata:
      annotations:
        config.kubernetes.io/function: |
          exec:
            path: ksops
    

    you will get the following error output:

    Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = Manifest generation error (cached): `kustomize build <path to cached source>/k8s --enable-alpha-plugins --enable-exec` failed exit status 1: Error: accumulating resources: accumulation err='accumulating resources from 'test': '<path to cached source>/' must resolve to a file': recursed accumulation of path '<path to cached source>/k8s/test': loading generator plugins: failed to load generator: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('/.config/kustomize/plugin'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)