Simplifying TLS Certificate Management in Kubernetes with Cert-manager and Vault

Simplifying TLS Certificate Management in Kubernetes with Cert-manager and Vault

Introduction

Cert-manager creates TLS certificates for workloads in your Kubernetes or OpenShift cluster and renews the certificates before they expire. Cert-manager can obtain certificates from a variety of certificate authorities, including: Let's Encrypt, HashiCorp Vault, Venafi and private PKI.

In this blog I am using Hashicorp Vault as Certificate Issuer with Cert-Manager

Login to Vault( I am running vault in my cluster)

root@master:~/vault# kubectl exec -it vault-0 -- /bin/sh

Enable the PKI secrets engine at its default path.

/ $ vault secrets enable pki
Success! Enabled the pki secrets engine at: pki/

Configure the max lease time-to-live

/ $ vault secrets tune -max-lease-ttl=8760h pki
Success! Tuned the secrets engine at: pki/

Generate a self-signed certificate

/ $ vault write pki/root/generate/internal \
>     common_name=arobyte.tech \
>     ttl=8760h
Key              Value
---              -----
certificate      -----BEGIN CERTIFICATE-----
MIIDODCCAiCgAwIBAgIUekLUNWVLV3am8DTRk33Y9KX0t8kwDQYJKoZIhvcNAQEL
BQAwFzEVMBMGA1UEAxMMYXJvYnl0ZS50ZWNoMB4XDTI0MDMxMzE1NDU0NVoXDTI1
MDMxMzE1NDYxNVowFzEVMBMGA1UEAxMMYXJvYnl0ZS50ZWNoMIIBIjANBgkqhkiG

-----END CERTIFICATE-----
expiration       1741880775
issuing_ca       -----BEGIN CERTIFICATE-----
MIIDODCCAiCgAwIBAgIUekLUNWVLV3am8DTRk33Y9KX0t8kwDQYJKoZIhvcNAQEL
BQAwFzEVMBMGA1UEAxMMYXJvYnl0ZS50ZWNoMB4XDTI0MDMxMzE1NDU0NVoXDTI1
zw4bj+X2hQyMqu5QHdFF4n58s9I9M5oq9IIBlMqxQqQdN79UirJc/LTk71roOKi7
PD1A3HmuNnWt04+0f8maI9txbUToWq15t8d5zBoM85sF2AGc04OmQmXvL+cGqImJ
9+RIo+iKIJnLiAMt
-----END CERTIFICATE-----
serial_number    7a:42:d4:35:65:4b:57:76:a6:f0:34:d1:93:7d:d8:f4:a5:f4:b7:c9

Configure the PKI secrets engine certificate issuing and certificate revocation list (CRL) endpoints to use the Vault service in the default namespace.

/ $ vault write pki/config/urls \
>     issuing_certificates="http://vault.vault.svc.cluster.local:8200/v1/pki/ca" \
>     crl_distribution_points="http://vault.vault.svc.cluster.local:8200/v1/pki/crl"
Success! Data written to: pki/config/urls

Configure a role named arobyte-role-tech that enables the creation of certificates robyte.tech domain with any subdomains.

/ $ vault write pki/roles/arobyte-role-tech \
>     allowed_domains=arobyte.tech \
>     allow_subdomains=true \
>     max_ttl=72h
Success! Data written to: pki/roles/arobyte-role-tech

Create a policy named pki that enables read access to the PKI secrets engine paths.

/ $ vault policy write pki - <<EOF
> path "pki*"                        { capabilities = ["read", "list"] }
> path "pki/sign/arobyte-role-com"    { capabilities = ["create", "update"] }
> path "pki/issue/arobyte-role-com"   { capabilities = ["create"] }
> EOF
Success! Uploaded policy: pki

Enable the Kubernetes authentication method.

/ $ vault auth enable kubernetes
Success! Enabled kubernetes auth method at: kubernetes/

Configure the Kubernetes authentication method to use location of the Kubernetes API.

/ $ vault write auth/kubernetes/config \
>     kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
Success! Data written to: auth/kubernetes/config

Create a Kubernetes authentication role named issuer that binds the pki policy with a Kubernetes service account named issuer

/ $ vault write auth/kubernetes/role/issuer \
>     bound_service_account_names=issuer \
>     bound_service_account_namespaces=default \
>     policies=pki \
>     ttl=20m
Success! Data written to: auth/kubernetes/role/issuer

Lets Install Cert-manager

root@master:~# kubectl create namespace cert-manager

Install Jetstack's cert-manager's version 1.12.3 resources.

root@master:~# kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.12.3/cert-manager.crds.yaml
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
root@master:~# helm repo add jetstack https://charts.jetstack.io
"jetstack" has been added to your repositories
root@master:~# helm repo update
...Successfully got an update from the "jetstack" chart repository
Update Complete. ⎈Happy Helming!⎈

Install the cert-manager

root@master:~# helm install cert-manager \
    --namespace cert-manager \
    --version v1.12.3 \
  jetstack/cert-manager

NAME: cert-manager
LAST DEPLOYED: Wed Mar 13 21:26:19 2024
NAMESPACE: cert-manager
STATUS: deployed

Check the status

root@master:~# kubectl get pods --namespace cert-manager
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-65dfbdf7d6-qp5fk              1/1     Running   0          3m44s
cert-manager-cainjector-79f5dbffcf-lh4d6   1/1     Running   0          3m44s
cert-manager-webhook-77b984cc67-8nxhh      1/1     Running   0          3m44s

Create a service account named issuer within the default namespace.

root@master:~# kubectl create serviceaccount issuer
serviceaccount/issuer created

Create a secret definition

root@master:~# cat >> issuer-secret.yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: issuer-token
  annotations:
    kubernetes.io/service-account.name: issuer
type: kubernetes.io/service-account-token
EOF
root@master:~# kubectl apply -f issuer-secret.yaml
secret/issuer-token created
root@master:~# kubectl get secrets
NAME                 TYPE                                  DATA   AGE
issuer-token   kubernetes.io/service-account-token   3      7s

Define an Issuer, named vault-issuer, that sets Vault as a certificate issuer.

root@master:~# cat > vault-issuer.yaml <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: vault-issuer
  namespace: default
spec:
  vault:
    server: http://vault.vault.svc.cluster.local:8200
    path: pki/sign/arobyte-role-tech
    auth:
      kubernetes:
        mountPath: /v1/auth/kubernetes
        role: issuer
        secretRef:
          name: issuer-token
          key: token
EOF
root@master:~# kubectl apply --filename vault-issuer.yaml
issuer.cert-manager.io/vault-issuer created

Create the arobyte-tech certificate

root@master:~# cat arobyte-tech-cert.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: arobyte-tech
  namespace: default
spec:
  secretName: arobyte-role-tech
  issuerRef:
    name: vault-issuer
  commonName: www.arobyte.tech
  dnsNames:
  - www.arobyte.tech
root@master:~# kubectl apply --filename arobyte-tech-cert.yaml

View the details of the arobyte-tech certificate

root@master:~# kubectl describe certificate.cert-manager arobyte-tech -n default
Name:         arobyte-tech
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Certificate


Events:
  Type    Reason     Age    From                                       Message
  ----    ------     ----   ----                                       -------
  Normal  Issuing    2m55s  cert-manager-certificates-trigger          Issuing certificate as Secret does not exist
  Normal  Generated  2m55s  cert-manager-certificates-key-manager      Stored new private key in temporary Secret resource "arobyte-tech-gwkjp"
  Normal  Requested  2m54s  cert-manager-certificates-request-manager  Created new CertificateRequest resource "arobyte-tech-fdvjr"
  Normal  Issuing    27s    cert-manager-certificates-issuing          The certificate has been successfully issued


The certificate reports that it has been issued successfully.

Verify the Certificates

root@master:~# kubectl get certificate
NAME           READY   SECRET              AGE
arobyte-tech   True    arobyte-role-tech   95m


root@master:~# kubectl describe secrets arobyte-role-tech
Name:         arobyte-role-tech
Namespace:    default
Labels:       controller.cert-manager.io/fao=true
Annotations:  cert-manager.io/alt-names: www.arobyte.tech
              cert-manager.io/certificate-name: arobyte-tech
              cert-manager.io/common-name: www.arobyte.tech
              cert-manager.io/ip-sans:
              cert-manager.io/issuer-group:
              cert-manager.io/issuer-kind: Issuer
              cert-manager.io/issuer-name: vault-issuer
              cert-manager.io/uri-sans:

Type:  kubernetes.io/tls

Data
====
tls.key:  1675 bytes
ca.crt:   1176 bytes
tls.crt:  1419 bytes

root@master:~# kubectl describe certificate arobyte-tech
Name:         arobyte-tech
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Certificate
Metadata:

Spec:
  Common Name:  www.arobyte.tech
  Dns Names:
    www.arobyte.tech
  Issuer Ref:
    Name:       vault-issuer
  Secret Name:  arobyte-role-tech
Status:
  Conditions:
    Last Transition Time:  2024-03-13T17:43:48Z
    Message:               Certificate is up to date and has not expired
    Observed Generation:   1
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2024-03-16T17:43:47Z
  Not Before:              2024-03-13T17:43:17Z
  Renewal Time:            2024-03-15T17:43:37Z
  Revision:                1
Events:                    <none>