Configure SSL/TLS connection to Kafka

Enforcing SSL/TLS on the connection to Kafka ensures the communication between the brokers and the clients is encrypted, which is a recommended practice.

Kafka cluster configuration

The specific Kafka cluster configuration is out of scope of the ESS configuration. Depending on the Kafka provider you chose to support your ESS deployment, configuration details may vary.

After enabling SSL/TLS on your Kafka cluster, obtain the Certificate Authority (CA) public root certificate bundle in PEM format. Clients need this certificate bundle to encrypt traffic to the cluster.

Some Kafka providers use public trust repository as their CA. For major cloud providers, these public CAs are often included in the JVM's default trust store, such as AWS's MSK or GCP Managed Kafka. If your provider's CA is already part of the default JVM trust store, you can skip the manual import steps below.

Adding the Kafka cluster CA to the JVM trust store

If your Kafka provider's CA certificate bundle is not part of the default JVM trust store, you need to add it manually so that clients can trust the cluster's certificate upon connection.

See Add Custom Certificates to ESS Services for intructions.

Configuring the clients to use SSL

The ESS kustomizer includes a component that configures the Kafka clients to use SSL.

To enable SSL for Kafka clients:

1. Ensure your Kafka cluster is configured to support SSL

2. Ensure the cluster's certificate is available in the JVM trust store

3. Add ../release/ess/deployment/kubernetes/components/kafka-clients-ssl/ to your root kustomization.yaml file

If the Kafka cluster is not configured to support SSL or the cluster's certificate is not available in the JVM trust store before modifying the kustomization.yaml file, the deployment will fail.