Pod Provisioning Service#

Starting in ESS 2.0, Pod provisioning and storage is handled by a separate Pod Provisioning Service and Pod Storage Service.

Pod Provisioning/Creation#

ESS’ Pod provisioning service manages the creation of Pods, using the following URL format:

https://storage.{ESS Domain}/{Unique Root Container}/

Prior to version 2.0, ESS Pods used URLs of the form https://{ESS Domain}/{username}/.

Default Resource (Extended Profile)#

When creating a Pod, ESS creates an extended profile resource. The extended profile resource is separate from the public WebID profile. The extended profile resource, unlike the public WebID profile, is hosted in the user’s Pod, and by default, is private. Users can grant or deny access to their extended profile like any other resource in their Pod.

Initial ACP Policies#

When a Pod is created, like any other Pod resource, an Access Control Resource is also created for the Pod Root. The ACR is initialized with the default ACP policies for the Pod Owner and for Access Grant enablement:

  • Initial Pod Owner policies give the Pod Owner read and write access to the Pod. These policies also specify a client matcher as well if the Authorization service’s configuration for the initial client allow list is set:

    Note

    Starting in 2.1, ESS uses the values in its Authorization service’s INRUPT_AUTHORIZATION_DEFAULT_ACR_CLIENT_ID_ALLOW_LIST (at the time of Pod creation) to create the client matcher for the initial ACP policies. If the configuration is unset, ESS uses the values in its Authorization service’s INRUPT_AUTHORIZATION_CLIENT_ID_ALLOW_LIST (at the time of Pod creation).

    Using the value of the Pod owner’s WebID and an initial client allow list, ESS creates the initial policies of the form:

    If allOf(AgentMatcher and ClientMatcher) evaluates to true, Then allow (Read and Write).

    Specifically, ESS creates:

    Policy 1 for the Pod Root:

    If the agent matches the Pod owner’s WebID, and if the client application’s Client ID has a match in the initial client allow list, allow Read and Write access.

    Policy 2 for the Pod Root’s Initial Member Policies:

    If the agent matches the Pod owner’s WebID, and if the client application’s Client ID has a match in the initial client allow list, allow Read and Write access.

    For more information on a Container’s Member Policies, see Member Policies.

    Disambiguation

    Both Authorization Service and Pod Storage Service have a INRUPT_AUTHORIZATION_CLIENT_ID_ALLOW_LIST setting.

    Only the Authorization Service setting affects which clients are allowed. The Pod Storage Service is for Discovery purposes only.

  • Initial Access Grant Enablement policies allow the use of Access Grants that grant read/write/append access to the Pod resources. New in Version 2.2

    If allOf(VC Matcher) evaluates to true, Then allow (Read and Write and Append).

    Specifically, ESS creates:

    Policy 3 for the Pod Root:

    If a presented VC matches the specified type, allow its use for Read, Write, and Append access.

    Policy 4 for the Pod Root’s Initial Member Policies:

    If a presented VC matches the specified type, allow its use for Read, Write, and Append access.

    See also INRUPT_AUTHORIZATION_DEFAULT_ACR_ACCESS_GRANTS_ALLOWED_MODES.

    Important

    The policies only enable the use of Access Grants for the allowed access modes. To determine the access for an agent using an access grant, ESS uses the intersection of:

    • The allowed access specified by the policy, and

    • The granted access specified in the Access Grant (for the resource specified in the Access Grant).

Note

A Pod’s initial Policies are set when the Pod is provisioned. As such, updates to the various INRUPT_AUTHORIZATION_DEFAULT_ACR_* options do not affect existing Pods.

That is, once a Pod’s initial policies have been created, changes to the various INRUPT_AUTHORIZATION_DEFAULT_ACR_* options are not reflected in that Pod’s policies.

Provisioning Endpoints#

Create a New Pod#

The ESS Pod provisioning service provides an endpoint that client applications can use to create new Pods.

Important

Access to this endpoint requires users to be authenticated.

The following configurations, if set, may affect the behavior of this endpoint:

To create a new Pod (and the extended profile resource), a client will issue an authenticated POST request to the endpoint.

Method:

POST

Endpoint:

https://provision.{ESS domain}/

Body:

N/A

Content-Type:

N/A

Upon successful creation (status 201), the endpoint returns a Location header with the location of the new Pod. In addition, the response contains a JSON payload with information about the newly created Pod:

{
   "@context":{
      "id":"@id",
      "storage":{
         "@type":"@id",
         "@id":"http://www.w3.org/ns/pim/space#storage"
      },
      "profile":{
         "@type":"@id",
         "@id":"http://xmlns.com/foaf/0.1/isPrimaryTopicOf"
      }
   },
   "id":"{WebID}",
   "profile":"https://storage.{ESS Domain}/{Root Container}/profile",
   "storage":"https://storage.{ESS Domain}/{Root Container}/"
}

@context

Contains the following context for Pod information fields:

{
   "id":"@id",
   "storage":{
      "@type":"@id",
      "@id":"http://www.w3.org/ns/pim/space#storage"
   },
   "profile":{
      "@type":"@id",
      "@id":"http://xmlns.com/foaf/0.1/isPrimaryTopicOf"
   }
}

id

The WebID of the authenticated user for whom the Pod has been created.

profile

The URL of an extended profile resource stored on the Pod. The URL has the form https://storage.{ESS Domain}/{Root Container}/profile.

Note

The extended profile resource is separate from the public WebID Profile Document. As with any resource in a user’s Pod, the extended profile resource is private by default.

storage

The URL root of the newly created Pod. The URL has the form https://storage.{ESS Domain}/{Root Container}.

This payload can be used to update the WebID Profile with the Pod information.

List Pods for a User#

The ESS Pod provisioning service provides an endpoint that client applications can use to list a user’s Pods.

Important

Access to this endpoint requires users to be authenticated.

To list Pods for the logged in user, a client can issue an authenticated GET request to the endpoint.

Method:

GET

Endpoint:

https://provision.{ESS domain}/list

Body:

N/A

Content-Type:

N/A

The endpoint returns an array of the unique Root Containers (relative to the Storage base URL), prefixed with a slash “/”; e.g.,

[
  "/973ef337-ce21-4762-975b-671ac6ebc180/",
  "/e3fefa9f-4fe0-4e4c-a5b6-81be0f12fe9c/"
]

Using the appropriate programming language URL builder/constructor, the client can construct the Pod URL using the Storage base URL value (for example, https://storage.{ESS domain}) and the returned Root Containers.

To determine the base URL value, see INRUPT_STORAGE_HTTP_BASE_URL.

Configuration#

As part of the installation process, Inrupt provides base Kustomize overlays and associated files that require deployment-specific configuration inputs.

The following configuration options are available for the service and may be set as part of updating the inputs for your deployment. The Inrupt-provided base Kustomize overlays may be using updated configuration values that differ from the default values.

Provision Options#

INRUPT_STORAGE_HTTP_BASE_URL#

The base URL of the storage service.

Important

INRUPT_STORAGE_MAX_PODS_PER_OWNER#

Default: 10

For Pod Provision Service Only

The maximum number of Pods owned by a specific WebID.

Important

The INRUPT_STORAGE_MAX_PODS_PER_OWNER value must equal value must equal Authorization Service’s INRUPT_AUTHORIZATION_MAX_POD_COUNT value. When changing the INRUPT_STORAGE_MAX_PODS_PER_OWNER value, ensure you also update INRUPT_AUTHORIZATION_MAX_POD_COUNT to the same value.

Solid-OIDC Issuer Configuration Options#

INRUPT_JWT_ALLOWED_SIGNATURE_ALGORITHMS#

Default: ES256, RS256

A comma-separated list that specifies the allowed encryption algorithms used to sign ID tokens.

INRUPT_JWT_ISSUER_ALLOW_LIST#

A comma-separated list [1] of trusted issuers of Solid-OIDC tokens.

See also INRUPT_JWT_ISSUER_DENY_LIST.

Tip

If your application, such as a start app, provisions the Pod and updates the WebID with the provisioned Pod information, ensure that the WebID service’s INRUPT_JWT_ISSUER_ALLOW_LIST overlaps with the INRUPT_JWT_ISSUER_ALLOW_LIST value set for this service.

INRUPT_JWT_ISSUER_DENY_LIST#

A comma-separated list [1] of disallowed issuers of Solid-OIDC tokens.

Logging Configuration#

QUARKUS_LOG_LEVEL#

Default: INFO

Logging level.

Kafka Configuration#

INRUPT_KAFKA_AUDITV1EVENTSENCRYPTED_CIPHER_PASSWORD#

The strong cipher key to use when running auditing with encrypted messages over the auditv1eventsencrypted topic.

INRUPT_KAFKA_AUDITV1EVENTSPRODUCERENCRYPTED_CIPHER_PASSWORD#

The strong cipher key to use when running auditing with encrypted messages over the auditv1eventsproducerencrypted topic.

Added in version 2.2.0.

INRUPT_KAFKA_SOLIDRESOURCE_CIPHER_PASSWORD#

The symmetric key to use when encrypting messages (see MP_MESSAGING_OUTGOING_SOLIDRESOURCE_VALUE_SERIALIZER).

KAFKA_BOOTSTRAP_SERVERS#

Default: localhost:9092

Comma-delimited list of Kafka broker servers for use by ESS services, including this service.

Setting KAFKA_BOOTSTRAP_SERVERS configures ESS to use the same Kafka instance(s) for all its Kafka message channels (e.g., solidresource and auditv1out message channels). The Pod management services use the solidresource and auditv1out message channels.

Note

Inrupt-provided overlays default to using KAFKA_BOOTSTRAP_SERVERS.

To use a different Kafka instance for the solidresource and auditv1out channels, use MP_MESSAGING_OUTGOING_SOLIDRESOURCE_BOOTSTRAP_SERVERS and MP_MESSAGING_OUTGOING_AUDITV1OUT_BOOTSTRAP_SERVERS instead.

See also ESS’ Kafka Configuration.

MP_MESSAGING_OUTGOING_AUDITV1OUT_BOOTSTRAP_SERVERS#

Default: localhost:9092

Comma-delimited list of Kafka broker servers used for the outgoing audit v1 messages.

These messages are sent over the auditv1out message channel.

Note

To configure ESS to use the same Kafka instances for all its Kafka message channels, use KAFKA_BOOTSTRAP_SERVERS option instead. Inrupt-provided overlays default to using KAFKA_BOOTSTRAP_SERVERS.

MP_MESSAGING_OUTGOING_SOLIDRESOURCE_BOOTSTRAP_SERVERS#

Default: localhost:9092

Comma-delimited list of Kafka broker servers used for the outgoing resource notification messages.

These messages are sent over the solidresource message channel.

Note

To configure ESS to use the same Kafka instances for all its Kafka message channels, use KAFKA_BOOTSTRAP_SERVERS option instead. Inrupt-provided overlays default to using KAFKA_BOOTSTRAP_SERVERS.

MP_MESSAGING_OUTGOING_SOLIDRESOURCE_VALUE_SERIALIZER#

Default: org.apache.kafka.common.serialization.StringSerializer

The serializer used for the notification messages the service sends to Kafka.

Supported values are:

  • org.apache.kafka.common.serialization.StringSerializer

    When set to this value, notification messages sent to Kafka are unencrypted.

    Services that consume these messages (e.g., WebSocket Notification Service) will need to set their MP_MESSAGING_INCOMING_SOLIDRESOURCE_VALUE_DESERIALIZER to the corresponding deserializer value org.apache.kafka.common.serialization.StringDeserializer.

  • com.inrupt.components.kafka.encryption.EncryptMessageSerializer

    When set to this value, notification messages sent to Kafka are encrypted. Services that consume these encrypted messages (e.g., WebSocket Notification Service) will need to set their MP_MESSAGING_INCOMING_SOLIDRESOURCE_VALUE_DESERIALIZER configuration to the corresponding deserializer value com.inrupt.components.kafka.encryption.DecryptMessageDeserializer.

AWS Options#

INRUPT_STORAGE_S3_BUCKET_NAME#

Default: inrupt.ess.storage

The name of the S3 bucket used for storage.

QUARKUS_S3_AWS_CREDENTIALS_STATIC_PROVIDER_ACCESS_KEY_ID#

AWS Access key id.

QUARKUS_S3_AWS_CREDENTIALS_STATIC_PROVIDER_SECRET_ACCESS_KEY#

AWS Secret access key.

QUARKUS_S3_AWS_REGION#

An Amazon Web Services region that hosts the S3 Bucket.

QUARKUS_S3_ENDPOINT_OVERRIDE#

Override S3 endpoint URL.

OpenTelemetry Options#

QUARKUS_OTEL_EXPORTER_OTLP_ENDPOINT#

The URL to which the OpenTelemetry exporter sends data.

QUARKUS_OTEL_TRACES_SAMPLER_ARG#

Default: 0.0d

A double compatible value between 0.0d and 1.0d to determine the sampling rate of the OpenTelemetry exporter. A value of 0.0d results in disabling OpenTelemetry exports.

Configuration Logging#

Starting in 2.2, ESS services log their startup configuration.

INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW#

Default: inrupt,smallrye.jwt.expiration.grace,mp.jwt.verify.clock.skew,smallrye.jwt.always-check-authorization,smallrye.jwt.token.decryption.kid,smallrye.jwt.token.schemes,smallrye.jwt.require.named-principal,smallrye.jwt.time-to-live,smallrye.jwt.jwks.refresh-interval,smallrye.jwt.jwks.forced-refresh-interval,smallrye.jwt.required.claims,mp.jwt.verify.audiences

A comma-separated list of configuration property prefixes (case-sensitive) that determine which configurations are logged:

When specifying the prefixes, you can specify the prefixes using one of two formats:

Warning

Use the same format for both INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW and INRUPT_LOGGING_CONFIGURATION_PREFIX_DENY.

For example, if you change the format of INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW, change the format of INRUPT_LOGGING_CONFIGURATION_PREFIX_DENY as well.

Tip

To avoid allowing more than desired configurations, specify as much of the prefix as possible. If the prefix specifies the complete prefix term, include the term delineator. For example:

  • If using dot-notation, if you want to match configuration properties of the form foobar.<xxxx>..., specify foobar. (including the dot .) instead of, for example, foo or foobar.

  • If using converted form, if you want to match configuration properties of the form FOOBAR_<XXXX>..., specify FOOBAR_ (including the underscore _) instead of, for example, FOO or FOOBAR.

Added in version 2.2.0.

INRUPT_LOGGING_CONFIGURATION_PREFIX_DENY#

Default: inrupt.kafka

A comma-separated list of configuration name prefixes (case-sensitive) that determines which configurations (that would otherwise match the INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW) are not logged. That is, INRUPT_LOGGING_CONFIGURATION_PREFIX_DENY acts as a filter on INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW. For example:

  • If foobar. is an allowed prefix, to suppress foobar.private.<anything>, you can specify foobar.private. to the deny list.

  • If foobar. is not an allowed prefix, no property starting with foobar. is logged. As such, you do not need to specify foobar.private to the deny list.

When specifying the prefixes, you can specify the prefixes using one of two formats:

Warning

Use the same format for both INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW and INRUPT_LOGGING_CONFIGURATION_PREFIX_DENY.

For example, if you change the format of INRUPT_LOGGING_CONFIGURATION_PREFIX_ALLOW, change the format of INRUPT_LOGGING_CONFIGURATION_PREFIX_DENY as well.

Added in version 2.2.0.

Logging Redaction#

INRUPT_LOGGING_REDACTION_NAME_ACTION#

Default: REPLACE

Type of the redaction to perform. Supported values are:

Action

Description

REPLACE

Default. Replaces the matching text with a specified replacement.

PLAIN

Leaves the matching field unprocessed. Only available if the redaction target is a field (i.e., INRUPT_LOGGING_REDACTION_{NAME}_FIELD).

DROP

Suppresses the matching field. Only available if the redaction target is a field (i.e., INRUPT_LOGGING_REDACTION_{NAME}_FIELD).

PRIORITIZE

Changes the log level of the matching message.

SHA256

Replaces the matching text with its hash.

  • If the action is REPLACE (default), see also INRUPT_LOGGING_REDACTION_{NAME}_REPLACEMENT.

  • If the action is to PRIORITIZE, see also INRUPT_LOGGING_REDACTION_{NAME}_LEVEL.

For more information on log redaction, see Logging Redaction.

Added in version 2.2.0.

INRUPT_LOGGING_REDACTION_NAME_ENABLED#

Default: true

A boolean that determines whether the redaction configurations with the specified INRUPT_LOGGING_REDACTION_{NAME}_ prefix is enabled.

For more information on log redaction, see Logging Redaction.

Added in version 2.2.0.

INRUPT_LOGGING_REDACTION_NAME_EXCEPTION#

Fully qualified name of the exception class to match in the log messages (includes inner exception). Configure to target an exception message class.

For more information on log redaction, see Logging Redaction.

Added in version 2.2.0.

INRUPT_LOGGING_REDACTION_NAME_FIELD#

Exact name of the field to match in the log messages. Configure to target a specific log message field for redaction.

For more information on log redaction, see Logging Redaction.

Added in version 2.2.0.

INRUPT_LOGGING_REDACTION_NAME_LEVEL#

A new log level to use for the log message if the INRUPT_LOGGING_REDACTION_{NAME}_ACTION is PRIORITIZE.

Added in version 2.2.0.

INRUPT_LOGGING_REDACTION_NAME_PATTERN#

A regex (see Java regex pattern) to match in the log messages. Configure to target log message text that matches a specified pattern.

For more information on log redaction, see Logging Redaction.

Added in version 2.2.0.

INRUPT_LOGGING_REDACTION_NAME_REPLACEMENT#

Replacement text to use if the INRUPT_LOGGING_REDACTION_{NAME}_ACTION is REPLACE.

If unspecified, defaults to [REDACTED].

For more information on log redaction, see Logging Redaction.

Added in version 2.2.0.

Application-Defined Metadata Propagation#

INRUPT_AUDIT_PRODUCER_REQUEST_METADATA_ALLOW#

A comma-separated list [1] of application-defined properties that can be included in the associated audit events (unless specified in the corresponding INRUPT_AUDIT_PRODUCER_REQUEST_METADATA_DENY).

This configuration is case-sensitive to the propagated properties in the baggage.

Tip

To include a propagated property that was added via the INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW configuration, ensure that the cases of these properties match.

See:

Added in version 2.2.0.

INRUPT_AUDIT_PRODUCER_REQUEST_METADATA_DENY#

A comma-separated list [1] of application-defined properties to exclude from the associated audit messages. This setting takes precedence over INRUPT_AUDIT_PRODUCER_REQUEST_METADATA_ALLOW.

This configuration is case-sensitive to the propagated properties in the baggage.

Tip

To exclude a propagated property that was added via the INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW configuration, ensure that the cases of these properties match.

See:

Added in version 2.2.0.

INRUPT_LOGGING_REQUEST_METADATA_ALLOW#

A comma-separated list [1] of application-defined properties that can be included in the associated log messages (unless specified in the corresponding INRUPT_LOGGING_REQUEST_METADATA_DENY).

This configuration is case-sensitive to the propagated properties in the baggage.

Tip

To include a propagated property that was added via the INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW configuration, ensure that the cases of these properties match.

See:

Added in version 2.2.0.

INRUPT_LOGGING_REQUEST_METADATA_DENY#

A comma-separated list [1] of application-defined properties to exclude from the associated log messages. This setting takes precedence over INRUPT_LOGGING_REQUEST_METADATA_ALLOW.

This configuration is case-sensitive to the propagated properties in the baggage.

Tip

To exclude a propagated property that was added via the INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW configuration, ensure that the cases of these properties match.

See:

Added in version 2.2.0.

INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW#

A comma-separated list [1] of non-baggage request headers to add to the baggage (unless specified in the corresponding INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_DENY); i.e., include these non-baggage request headers as application-defined properties.

The configuration is case-insensitive; i.e., the listed headers do not need to match the case of the client request headers. For example, a list that includes x-correlation-id can match x-correlation-id header, X-CoRrElAtIoN-Id header, etc.

See:

Added in version 2.2.0.

INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_DENY#

A comma-separated list [1] of non-baggage request headers to exclude from being added to the baggage; i.e., excludes these headers as application-defined properties. This setting takes precedence over INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW.

The configuration is case-insensitive; i.e., the listed headers do not need to match the case of the client request headers. For example, a list that includes x-correlation-id can match (and exclude) x-correlation-id header, X-CoRrElAtIoN-Id header, etc.

See:

Added in version 2.2.0.

INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_OVERRIDES#

A flag that determines ESS behavior when metadata property is defined both as a header and as a baggage entry:

  • If true, ESS updates/overrides the baggage entry with the header value.

  • If false (the default), ESS keeps the baggage entry.

For details, Duplicate Property Definition.

Added in version 2.2.0.

INRUPT_REQUEST_METADATA_REFLECTOR_HEADER_ALLOW#

A comma-separated list [1] of application-defined properties that can return as response headers (unless specified in the corresponding INRUPT_REQUEST_METADATA_REFLECTOR_HEADER_DENY).

This configuration is case-sensitive to the propagated properties in the baggage.

Tip

See:

Added in version 2.2.0.

INRUPT_REQUEST_METADATA_REFLECTOR_HEADER_DENY#

A comma-separated list [1] of application-defined properties to exclude from returning as response headers. This setting takes precedence over INRUPT_REQUEST_METADATA_REFLECTOR_HEADER_ALLOW.

This configuration is case-sensitive to the propagated properties in the baggage.

Tip

To exclude a propagated property that was added via the INRUPT_REQUEST_METADATA_PROPAGATOR_HEADER_ALLOW configuration, ensure that the cases of these properties match.

See:

Added in version 2.2.0.

Purge Configuration#

The Storage service contains user data, and as such it can be purged upon user request. See the Purger Application documentation for more information about the data being purged.

INRUPT_PURGE_BATCH_SIZE#

Default: 100

The maximum number of resources that the purge task will purge in each batch. This must be a non-zero, positive integer.

Added in version 2.3.0.

INRUPT_PURGE_CLEANUP_TASK_EVERY#

Default: PT5H

Frequency at which a task goes through stored purge statuses to clear any which are beyond their retention window.

Added in version 2.3.0.

INRUPT_PURGE_IN_PROGRESS_TIMEOUT_SECONDS#

Default: 120

Timeout after which an ongoing purge task is considered stale. Stale tasks are picked up by an ESS background process to be taken to completion. By keeping track of a purge task’s state (active or stale) the service can ensure that a purge which was started will eventually reach completion, even if the system is disrupted whilst the asynchronous purge process is ongoing.

Added in version 2.3.0.

INRUPT_PURGE_PROCESS_TASK_EVERY#

Default: PT5M

Frequency at which an ESS background process goes through ongoing purges to pick up the incomplete stale ones. See INRUPT_PURGE_IN_PROGRESS_TIMEOUT_SECONDS for additional details.

Added in version 2.3.0.

INRUPT_PURGE_STATUS_RETENTION_WINDOW#

Default: P2D

Duration after which a purge task status will be cleared from storage. The purge task contains some Personally Identifying Data (such as the WebID), so ensuring it is cleared after a purge is required for compliance.

Added in version 2.3.0.

INRUPT_PURGE_SYSTEM_STATUS_ROOT#

Default: /system/purge/status/

The path for storing purge statuses as system resources.

Added in version 2.3.0.

Additional Information#

See also Quarkus Configuration Options.