# Pruning ESS provides a pruning feature to perform hard deletes of [soft-deleted resources](https://docs.inrupt.com/ess/pod-resources#resource-deletion) and [orphan data](https://docs.inrupt.com/ess/pod-resources#modification-of-resource-content). ## Pruning (Hard Deletes) Prune consists of two [Kubernetes CronJobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) : * [One](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#pruning-soft-deleted-resources) to delete “prunable” resources. Prunable resources are resources that have been marked for deletion (i.e., soft-deleted) and are past their [**`INRUPT_STORAGE_PRUNE_RETENTION_WINDOW`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_retention_window) . * [One](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#pruning-orphan-data) to delete orphan data. Orphan data are resource data without associated metadata. {% hint style="warning" %} **Important**\ Pruning operations may negatively affect performance. If possible, schedule the CronJob to run at times when you can minimize its impact. To configure the Prune jobs, see [Modify Prune Configuration](https://docs.inrupt.com/ess/2.3/installation/customize-configurations/customization-pod-maintenance/modify-prune) . {% endhint %} ## Configuration ### Configuration to Prune Soft-Deleted Resources For [soft-deleted resources](https://docs.inrupt.com/ess/pod-resources#resource-deletion), Prune has the following configurations: * [**`INRUPT_STORAGE_PRUNE_RETENTION_WINDOW`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_retention_window) * Required. * Determines which soft-deleted resources are “prunable”. * Specify the value in a format supported by Java [Duration.parse()](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/time/Duration.html#parse\(java.lang.CharSequence\)) method. * [**`INRUPT_STORAGE_PRUNE_PRUNABLE_BATCH_SIZE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_prunable_batch_size) * Required. * Limits the number of results returned when querying the metadata. * Set to an integer value. * [**`INRUPT_STORAGE_PRUNE_ORPHAN_BATCH_SIZE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_orphan_batch_size) * Required. * Set to **`0`** when pruning soft-deleted resources. * [**`COM_INRUPT_STORAGE_METADATA_JDBC_CONNECTIONLIMITER_OPENCONNECTION_TIMEOUT_VALUE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#com_inrupt_storage_metadata_jdbc_connectionlimiter_openconnection_timeout_value) * Required. * Determines how long to keep the connection to the metadata database open. * Set to an integer value. Adjust the value to accommodate changes in * [**`INRUPT_STORAGE_PRUNE_RETENTION_WINDOW`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_retention_window) * [**`INRUPT_STORAGE_PRUNE_PRUNABLE_BATCH_SIZE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_prunable_batch_size) To configure the Prune jobs, see [Modify Prune Configuration](https://docs.inrupt.com/ess/2.3/installation/customize-configurations/customization-pod-maintenance/modify-prune) ### Configuration to Prune Orphan Data For [orphan data](https://docs.inrupt.com/ess/pod-resources#modification-of-resource-content), Prune has the following configurations: * [**`INRUPT_STORAGE_PRUNE_RETENTION_WINDOW`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_retention_window) * Optional. * No impact on pruning orphan data. * [**`INRUPT_STORAGE_PRUNE_PRUNABLE_BATCH_SIZE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_prunable_batch_size) * Required. * Set to **`0`** when pruning orphan data. * [**`INRUPT_STORAGE_PRUNE_ORPHAN_BATCH_SIZE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_orphan_batch_size) * Required. * Determines the maximum number of data identifiers selected by Prune during [orphan data](https://docs.inrupt.com/ess/pod-resources#modification-of-resource-content) cleanup. * Set to an integer value. * [**`COM_INRUPT_STORAGE_METADATA_JDBC_CONNECTIONLIMITER_OPENCONNECTION_TIMEOUT_VALUE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#com_inrupt_storage_metadata_jdbc_connectionlimiter_openconnection_timeout_value) * Required. * Determines how long to keep the connection to the metadata database open. * Adjust the value to accommodate change to [**`INRUPT_STORAGE_PRUNE_ORPHAN_BATCH_SIZE`**](https://docs.inrupt.com/ess/services/service-pod-management/service-pod-storage#inrupt_storage_prune_orphan_batch_size). To configure the Prune jobs, see [Modify Prune Configuration](https://docs.inrupt.com/ess/2.3/installation/customize-configurations/customization-pod-maintenance/modify-prune) ## Observability {% tabs %} {% tab title="Default Logging" %} {% hint style="warning" %} Logging for Pruning jobs share a consistent pattern where the **`messageId`** has the prefix **`STORAGEPRUNE`** . {% endhint %} ```json { "timestamp":, "sequence":, "loggerClassName":, "loggerName":, "level":, "message": "STORAGEPRUNE: ", "threadName":, "threadId":, "hostName":, "processName":, "processId":, "messageId": "STORAGEPRUNE" // additional relevant fields, if any } ``` For pruning jobs, the additional fields include: * an **`mdc`** (managed diagnostic context) field that can be used for correlation; * various pruning metrics. The following lists the various pruning metrics that appear in the **`INFO`** level log messages (listed by the **`messageId`** ): * **`STORAGEPRUNE000001`** (associated with the pruning start process)
FieldDescription
retentionWindowMillisecondsThe value of the configured retention window.
prunableBatchSizeThe value of the configured prunable batch size.
orphanBatchSizeThe value of the configured orphan batch size.
* **`STORAGEPRUNE000002`** (associated with finding prunable objects process)
resourceCountThe number of Solid resource metadata entries found to fall outside the retention window.
contentCountThe number of Solid resource data entries found to belong to metadata entries that fall outside the retention window.
durationMillisecondsTime taken to find prunable resource metadata.
* **`STORAGEPRUNE000005`** (associated with finding prunable resource data in persistence)
resultCountThe number of Solid resource data entries listed from S3.
durationMillisecondsTime taken to list resource data.
* **`STORAGEPRUNE000007`** (associated with finding prunable orphan data in persistence)
resultCountThe number of orphan data entries listed from S3.
durationMillisecondsTime taken to list orphan data.
* **`STORAGEPRUNE000009`** (associated with pruning/deletion process from persistence)
resultCountThe number of Solid resource data entries deleted from S3.
durationMillisecondsTime taken to delete resource data.
* **`STORAGEPRUNE000010`** (associated with pruning/deletion process from metadata)
durationMillisecondsTime taken to delete resource metadata.
{% endtab %} {% tab title="Prometheus" %} Prune emits Prometheus metrics with the following labeled names. All of the following are prefixed with **`application_com_inrupt_storage_prune_`**.
{type="retentionWindow"}The value of the configured retention window.
{type="prunableBatchSize"}The value of the configured prunable batch size.
{type="orphanBatchSize"}The value of the configured orphan batch size.
{type="resource", status="prunable"}The number of Solid resource metadata entries found to fall outside the retention window.
{type="data", status="prunable"}The number of Solid resource data entries found to belong to metadata entries that fall outside the retention window.
{type="data", status="listed"}The number of Solid resource data entries listed from S3.
{type="data", status="orphan"}The number of Solid resource data entries (out of the total listed) found to lack corresponding listed metadata entries.
{type="data", status="deleted"}The number of Solid resource data entries deleted from S3.
Pruner_findPrunableTime (in milliseconds) taken to find prunable resource metadata (i.e., metadata associated with soft-deleted resources).
Pruner_listDataTime (in milliseconds) taken to list resource data.
Pruner_findOrphansTime (in milliseconds) taken to identify orphan resource data.
Pruner_deleteDataTime (in milliseconds) taken to delete resource data.
Pruner_pruneMetadataTime (in milliseconds) taken to delete resource metadata.
{% endtab %} {% tab title="OpenTelemetry" %} When OpenTelemetry is configured, the application emits a single span named **`prune`** with the following attributes.
retentionWindowMillisecondsThe value of the configured retention window.
prunableBatchSizeThe value of the configured prunable batch size.
orphanBatchSizeThe value of the configured orphan batch size.
resourceCountThe number of Solid resource metadata entries found to fall outside the retention window.
dataCountThe number of Solid resource data entries found to belong to metadata entries that fall outside the retention window.
{% endtab %} {% endtabs %} --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.inrupt.com/ess/2.3/administration/resource-deletions.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.