ESS’ Kafka Configuration

ESS’ services communicate with each other by sending messages through Kafka. For example:

• Pod storage service sends resource notification events through Kafka and WebSocket Notification service consumes these events.

• ESS services (including the Auditing service) send audit events through Kafka and the Auditing service consumes these events.

The following discusses some key Kafka configurations.

For more information on Kafka configuration, see https://quarkus.io/guides/kafka#kafka-configuration.

Tip

Kafka MUST be configured with topic auto-creation enabled (i.e. auto.create.topics.enable = true). See Custom MSK Configurations.

Configuring Bootstrap Kafka Brokers

You can configure ESS services to connect with Kafka either globally or per channel.

Global Configuration

To configure globally such that all the ESS’ message channels use the same Kafka instance, you can set KAFKA_BOOTSTRAP_SERVERS configuration.

Note

Inrupt-provided Kustomize overlays include, in the kafka-credentials.env file, KAFKA_BOOTSTRAP_SERVERS as an input to update. Set its value as part of updating inputs for your deployment.

Per Channel Configuration

To configure per channel such that the message channel uses a separate Kafka instance, for the services that use the channel, configure the corresponding input and output bootstrap servers for that channel MP_MESSAGING_[INCOMING|OUTGOING]_[CHANNEL]_BOOTSTRAP_SERVERS.

Configuring Password for Encryption and Decryption of Messages

By default, Inrupt enables data encryption for all data that pass through the Kafka messaging system.

As part of updating the inputs for your deployment, set the Kafka data encryption passwords in the kafka-credentials.env file (downloaded as part of the installation).

Tip

You MUST set the data encryption key values to a strong password.