Auditing#

ESS services support the auditing of their activities.

Note

To ensure that audit messages are delivered, the Audit service may send the message more than once.

For instance, multiple instances of the Audit service may exist when applying configuration changes or when performing a rolling upgrade. During this time, duplicate messages (i.e., messages with the same unique id value) can occur if the old Audit instance processes a message but shuts down before acknowledging the message on Kafka, and then, the new Audit instance processes the unacknowledged message.

Audit Events#

The following events are audited:

Services

Event Name

Notes

Most Services [4]

  • service-started

  • service-shutdown

Service Startup/Shutdown.

Access Grant Service

  • access-grant-created

  • access-grant-queried

  • access-grant-read

  • access-grant-revoked

  • access-grant-verified

  • access-request-created

  • access-request-read

  • access-request-revoked

  • access-request-verified

  • access-denial-created

  • access-denial-read

  • access-denial-revoked

  • access-denial-verified

  • request-authorized [1]

  • revocation-status-read [2]

Access Request/Grant/Denial lifecycle events.

Added in version 2.2.0: [3]

Authorization Service

  • acr-created

  • acr-updated

  • acr-deleted

  • provisioned-pod-access-control

  • deprovisioned-pod-access-control

ACR Lifecycle events.

Pod Storage Service

  • resource-created

  • resource-updated

  • resource-deleted

Resource Lifecycle events.

  • resource-read

Successful resource read events (GET and HEAD operations).

To enable, see INRUPT_STORAGE_AUDIT_RESOURCE_READ_ENABLED configuration for Pod Storage Service.

Added in version 2.1.

Pod Provision Service

  • pod-provisioned

Pod Provisioned event.

Query Service

  • query-succeeded

Query events.

Query Indexer

  • ingest-succeeded

  • ingest-failed

Query Indexer events.

Solid OIDC Broker Service

  • openid-backend-idp-login

  • openid-token-requested

  • openid-authorization-initialized

Authentication/Authorization flow.

UMA Service

  • uma-token-created

UMA Grant Flow.

WebID Service

  • webid-created

  • webid-updated

  • webid-deleted

WebID Profile events.

Note

ESS uses asynchronous messaging for auditing.

Audit Event Message Internal Format#

Internally, ESS’ audit event messages are in RDF and serialized as ActivityStreams 2.0 JSON-LD documents:

Note

Although the following document shows all possible fields for an event message, the specific events determine which fields appear.

Starting in version 2.2, Pod-related event messages include:

  • the actor information (WebID)

  • the application information (Client ID).

  • the Pod information (Pod root url and Pod data subject)

{
   "@context":[
      "https://www.w3.org/ns/activitystreams",
      "https://schema.inrupt.com/audit/v1.jsonld"
   ],
   "id":"<UUID of the event>",
   "type": [ "Activity", <type2>,... ],
   "name":"<event name>",
   "summary": "<event description>",
   "generator": <JSON document identifying the software producing the event>,
   "actor": [ <JSON document identifying the actor associated with the event>, ... ],
   "object": [ <JSON document identifying the object associated with the event>, ... ],
   "instrument": [ <JSON document identifying the client/application associated with the event>,
                   <JSON document with associated OpenTelemetry data>,
                   <JSON document with associated application-defined metadata>,
                   <JSON document identifying the Pod root and data subject>, ... ],
   "result": [ <JSON document containing associated results for the event, if any> ],
   "published": "<datetime>",
   "identifier":"<identifier to use for correlated events>"
}

@context

Specifies the JSON-LD contexts.

id

Universally Unique IDentifier (UUID) for the event.

type

An array of event types; e.g., [ "Activity", "Create" ].

name

Name that denotes the event; e.g., service-started, openid-backend-idp-login, resource-created, access-grant-created, etc.

See Audit Events for a list of audited events names.

summary

Short description associated with the message name.

generator

JSON document identifying the software (e.g., service) producing the event. For example, the generator for a Pod provision event:

"generator": {
   "id": "<service URL>"
   "type": ["SoftwareApplication"],
   "name": "<application name>",
   "qualifiedAssociation": "<processId>",
   "wasAssociatedWith": "<Kubernetes pod name>"
}

actor

An array of JSON documents that identify the agents, if any, associated with the event. The actor’s identity can be denoted by various combination of fields, such as (list below is not exhaustive):

  • id and type fields;

  • name and type fields.

The field can also be empty [] for events not initiated by a user (such as service start events, etc.).

Note

Starting in version 2.2, id and name fields are associated with a type field. In earlier versions, the id field and name field may appear without a clarifying type.

For example:

  • For an Access Request/Grant event, which include the id and the type:

    "actor" : [
       {
          "id" : "https://id.example.com/someusername",
          "type" : [
             "Agent"
          ]
       }
    ],
    
  • For IdP login events:

    "actor": [
       {
          "name": "someusername",
          "type" : [
             "Agent"
          ]
       }
    ]
    

object

An array of JSON documents that identify the objects associated with the event; that is, the object(s) against which the action is performed.

The object can be denoted by various combination of fields, such as (list below is not exhaustive):

  • id and name fields;

  • id and type fields;

  • name and qualifiedAssociation fields;

For example:

  • A Pod provisioned event:

    "object": [
       { "type": [ "Storage" ], "id": "<Pod Root URL>" }
    ]
    
  • For access request/grant/denial creation events (available starting in version 2.1.5), the object field contains the created access request/grant/denial:

    • Starting in 2.2, the object for the create access request/grant/denial events contains the created request/grant/denial in a document with:

      • the content field that contains the access request/grant/denial as string and

      • the mediaType field.

      Previously, the object contained the created access request/grant/denial directly as an element.

    • Starting in 2.2, the object contains a new element that identifies the Pod resource.

    "object": [
       {
         "content" : "{\"id\":\"https://vc.example.com/vc/79288a3 ... }",
         "mediaType": "application/ld+json"
       },
    
       {
         "type" : [
            "Resource"
         ],
         "id" : "https://storage.example.com/ad3b.../some/resource"
      }
     ]
    
  • For Pod resource CRUD events ( resource-created, resource-read, resource-updated, and resource-deleted) events, the object field include an object that identifies the Pod resource. [5]

    For example: [6]

    "object": [
       {
          "generated" : "6cba8240-9f79-40db-a129-c6b8edddb840",
          "invalidated" : "4e12ed06-c4e5-41bc-bcd3-0f6a821c08ca",
          "type" : [
             "Resource"
          ],
          "id" : "https://storage.example.com/ad3b.../some/resource"
       }
    ]
    

instrument

An array of JSON documents that identify:

  • The clients associated with the event, if any. For example:

    "instrument" : [
       {
          "id" : "https://start.example.com/app/id",
          "summary" : "Client identifier",
       }
    ],
    

    Note

    Starting in version 2.2, instrument objects that identify the clients have an associated summary field with the value "Client identifier".

    In earlier versions, the id field may appear without a clarifying summary or may have included "name" : "Client Application" instead of summary.

    Some events such as service-started do not have associated clients, and thus may have empty instrument array.

  • The associated OpenTelemetry instrument info (New in version 2.2)

    For example:

    "instrument" : [
       // ...
       {
          "traceId" : "7decd3657a9efffc010a4b6a4b3da586",
          "spanId" : "91123fce3c668451",
          "isSampled" : true,
          "name" : "OpenTelemetry Span Context",
          "type" : [
             "SpanContext"
          ]
       },
       // ...
    ],
    

    OpenTelemetry traceId field can be used to correlate messages associated with a request. See Appendix: Audit Event Correlation for more information.

  • The associated Pod information (New in version 2.2):

    • hasDataSubject contains the Pod data subject information (Pod Data Subject refers to the agent who created the Pod. )

    • hasStorage contains the Pod root URL.

    For example:

    "instrument" : [
       // ...
    
       {
          "hasDataSubject" : {
             "id" : "https://id.example.com/someusername",
             "type" : [
                "https://w3id.org/dpv#DataSubject"
             ]
          },
          "hasStorage" : "https://storage.example.com/root/",
          "type" : [
             "http://www.w3.org/2004/02/skos/core#Concept"
          ]
       }
       // ...
    
    ],
    
  • Application-defined metadata sent in client requests (New in version 2.2)

    For example:

    "instrument" : [
       // ...
       {
    
         "name" : "Application-Defined Request Metadata",
         "items": [
            {
               "mediaType" : "text/plain",
               "name" : "x-correlation-id",
               "content" : "2049875809728750827498245084"
            },
            {
               "mediaType":"text/plain",
               "name":"my-client-version",
               "content":"1.0.3"
            }
         ],
         "type":[
            "urn:uuid:1a05e301-4013-40c9-bae7-5d719b7151c8"
         ]
       }
       // ...
    
    ],
    

    See Application-Defined Metadata for the configuration needed to include in audit events.

result

An array of JSON documents that contains associated results. For example:

  • an access-request-verified event includes the results of the verification, or

  • an access-grant-revoked event includes the updated status.

Added in version 2.1.5.

published

The timestamp of the event.

identifier

Identifier to use for correlated events from a single service that have occurred within the same request. To correlate events across services for a request, use the OpenTelemetry traceId in the instrument field.

For examples, see Appendix: Audit Events Examples.

Integration with External Logging Systems#

The ESS Auditing service can log to:

Syslog#

When integrating with Syslog, ESS audit events are converted to Syslog message format:

<priority>version timestamp hostname service processId messageId message

Where:

  • hostname, service, processId and messageId values are taken extracted from the audit event agent field.

  • message is the full ESS audit event in JSON.

For more information, see Integration with Syslog and

Microsoft Sentinel#

When integrating with Microsoft Sentinel, the ESS audit events are converted into a Sentinel-specific format and POST’ed to the Sentinel service.

For more information, see Integration with Sentinel.