# Appendix: OpenTelemetry

ESS supports the use of [OpenTelemetry](https://opentelemetry.io/docs) for observability. Specifically, ESS has been [instrumented](https://opentelemetry.io/docs/concepts/instrumentation/) to support OpenTelemetry. For an overview on OpenTelemetry, see: <https://opentelemetry.io/docs/specs/otel/overview/>.

ESS also uses OpenTelemetry to propagate [application-defined properties](/ess/2.3/administration/application-defined-metadata.md) that are defined in the client requests.

## Correlation

### By TraceId

When associated with a [span](https://opentelemetry.io/docs/concepts/signals/traces/#spans):

* ESS log messages can include OpenTelemetry information in the **`mdc`** field:

  ```json
  "mdc": {"spanId":<value>,"traceId":<value>,"sampled":"true"}
  ```

  The **`traceId`** field can be used to correlate log messages across services that are associated with a client request. However, as the **`traceId`** is subject to how the client specifies the **`traceId`** for its requests, correlation by **`traceId`** may not be suitable for audit trail purposes.
* ESS audit events can include OpenTelemetry information in the **`instrument`** field:

  ```json
  {
     ...
     "instrument" : [
        // ...
        {
           "spanId" : <value>,
           "traceId" : <value>,
           "parentId" : <value>,
           "name" : "OpenTelemetry Span Context",
           "isSampled" : true,
           "type" : [
              "SpanContext"
           ]
        }
     ],
  }
  ```

  The **`traceId`** field can be used to correlate audit events across services that are associated with a client request. However, as the **`traceId`** is subject to how the client specifies the **`traceId`** for its requests, correlation by **`traceId`** may not be suitable for audit trail purposes.

### By Application-Defined Property

ESS can propagate [application-defined properties](/ess/2.3/administration/application-defined-metadata.md) sent in client’s request header to include in associated log messages, associated audit events, and associated response to the request.

Depending upon configuration:

* ESS log messages can include the **`requestMetadata`**:

  ```json
  "requestMetadata":{"property1":"<value1>","property2":"<value2>", ...}
  ```

  Correlation by client defined properties may not be suitable for audit trail purposes as these properties are subject to how the client manages these properties as well as the ESS [configuration](/ess/2.3/services/service-oidc/service-application-registration.md#configuration).
* ESS audit events can include the application-defined request metadata in the **`instrument`** field:

  ```json
  "instrument" : [
     // ...
     {
       "name" : "Application-Defined Request Metadata",
       "items": [
          {
             "mediaType" : "text/plain",
             "name" : "x-correlation-id",
             "content" : "2049875809728750827498245084"
          },
          {
             "mediaType":"text/plain",
             "name":"my-client-version",
             "content":"1.0.3"
          }
       ],
       "type":[
          "urn:uuid:1a05e301-4013-40c9-bae7-5d719b7151c8"
       ]
     }
     // ...
  ],
  ```

  Correlation by client defined properties may not be suitable for audit trail purposes as these properties are subject to how the client manages these properties as well as the ESS configuration.
* ESS’ response can include the properties as headers.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.inrupt.com/ess/2.3/reference/appendix-opentelemetry.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.