Appendix: OpenTelemetry

ESS supports the use of OpenTelemetry for observability. Specifically, ESS has been instrumented to support OpenTelemetry. For an overview on OpenTelemetry, see: https://opentelemetry.io/docs/specs/otel/overview/.

ESS also uses OpenTelemetry to propagate application-defined properties that are defined in the client requests.

Correlation

By TraceId

When associated with a span:

• ESS log messages can include OpenTelemetry information in the mdc field:

  "mdc": {"spanId":<value>,"traceId":<value>,"sampled":"true"}

  The traceId field can be used to correlate log messages across services that are associated with a client request. However, as the traceId is subject to how the client specifies the traceId for its requests, correlation by traceId may not be suitable for audit trail purposes.

• ESS audit events can include OpenTelemetry information in the instrument field:

  {
     ...
     "instrument" : [
        // ...
        {
           "spanId" : <value>,
           "traceId" : <value>,
           "parentId" : <value>,
           "name" : "OpenTelemetry Span Context",
           "isSampled" : true,
           "type" : [
              "SpanContext"
           ]
        }
     ],
  }

  The traceId field can be used to correlate audit events across services that are associated with a client request. However, as the traceId is subject to how the client specifies the traceId for its requests, correlation by traceId may not be suitable for audit trail purposes.

By Application-Defined Property

ESS can propagate application-defined properties sent in client’s request header to include in associated log messages, associated audit events, and associated response to the request.

Depending upon configuration:

• ESS log messages can include the requestMetadata:

  "requestMetadata":{"property1":"<value1>","property2":"<value2>", ...}

  Correlation by client defined properties may not be suitable for audit trail purposes as these properties are subject to how the client manages these properties as well as the ESS configuration.

• ESS audit events can include the application-defined request metadata in the instrument field:

  "instrument" : [
     // ...
     {
       "name" : "Application-Defined Request Metadata",
       "items": [
          {
             "mediaType" : "text/plain",
             "name" : "x-correlation-id",
             "content" : "2049875809728750827498245084"
          },
          {
             "mediaType":"text/plain",
             "name":"my-client-version",
             "content":"1.0.3"
          }
       ],
       "type":[
          "urn:uuid:1a05e301-4013-40c9-bae7-5d719b7151c8"
       ]
     }
     // ...
  ],

  Correlation by client defined properties may not be suitable for audit trail purposes as these properties are subject to how the client manages these properties as well as the ESS configuration.

• ESS’ response can include the properties as headers.