Storage Metrics#
Starting in 2.1, ESS includes a Storage Metrics feature to gather the following metrics:
The total number of Pods.
The number of “created” Pods (i.e., Pods that have been created and confirmed).
The number of “deleted” Pods (i.e., Pods that have been soft-deleted, i.e., marked as deleted).
The Storage Metrics is run as a Kubernetes CronJobs . To modify its schedule, see Modify Storage Metrics Schedule.
Observability#
As part of the ESS 2.2 Logging Enhancements, logging for Pruning
jobs share the consistent pattern where the messageId
has the
prefix STORAGEMETRICS
or STORAGE
:
{
"timestamp":<value>,
"sequence":<value>,
"loggerClassName":<value>,
"loggerName":<value>,
"level":<value>,
"message": "<STORAGEMETRICS|STORAGE><number>: <description>",
"threadName":<value>,
"threadId":<value>,
"hostName":<value>,
"processName":<value>,
"processId":<value>,
"messageId": "STORAGEMETRICS|STORAGE<number>"
// additional relevant fields, if any
}
For the storage metrics features, the additional fields include:
an
mdc
(managed diagnostic context) field that can be used for correlation;various storage metrics.
The following lists the various storage metrics that appear in
the INFO
level log messages (listed by the messageId
):
Note
The log messages that include the storage metrics detail use
the STORAGE
prefix in the messageIds.
STORAGE000128
(associated with the storage counts)resultCount
Total number of Pods (both “created” and “deleted”)
STORAGE000130
(associated with the “created” storage counts)resultCount
The number of Pods that have been created and confirmed.
Tip
For the count of “deleted” Pods (i.e., soft-deleted or marked as deleted), you can subtract from the
STORAGE000128
messageresultCount
theresultCount
from the theSTORAGE000130
message; alternatively, you can set the Log Level toDEBUG
(and return the level toINFO
afterwards).
Prune emits Prometheus metrics with the following labeled names.
All of the following are prefixed with application_com_inrupt_storage_metrics_MetricsCollector_
.
|
Total number of Pods (both “created” and “deleted”) |
|
The number of Pods that have been created and confirmed. |
|
The number of Pods that have been marked as deleted. |
|
Time taken to run the queries to gather the metrics. |
When OpenTelemetry is configured, the application emits a single
span named collect
with the following attributes.
|
Total number of Pods (both “created” and “deleted”) |
|
The number of Pods that have been created and confirmed. |
|
The number of Pods that have been marked as deleted. |
|
Time taken to run the queries to gather the metrics. |