Audit logs

[kpsherva]

RFC: https://github.com/inveniosoftware/rfcs/blob/master/rfcs/framework-0080-audit-logs.md

InvenioRDM should come with audit logs capabilities. Concrete use cases:

• who/when removed a record from a community
• who/when removed a member from a community
• who/when included a record to a community

[ppanero]

• record added to a community

[ntarocco]

Related use case: inveniosoftware/invenio-app-rdm#2285

[mesemus]

Use case from CESNET: we are legally bound to collect and keep the following information:

• User login/logout, including unsuccessful attempts
• All events that can affect permissions
  • User community membership operations
  • Inclusion/Exclusion of record in community
  • Changing embargoes
  • Publishing/unpublishing/deleting records
• Service availability status (server start, stop, ...)
• All operations performed by people with administration-level permissions
• Unhandled errors from the application

We need to log at least:

• date and time
• network identification (ip address)
• user identification (if known)
• event type
• event outcome (success, denied access)

We need to pass the log to external central logging facility via GELF
protocol with mutual certificate authentication.

[ntarocco]

Thank you for sharing this, very useful! The idea is to implement the mechanism first, and a few actions. Then, enrich it progressively with more actions.
However, I have a couple of remarks:

• user identification: do you mean the user id/username in the Invenio db? Or something else?
• ip address: this is tricky. GDPR or similar laws require services to clearly document was is collected, and for how long it is retained. Plus, there is the right to be forgotten to take into account (delete personal data on demand). Is this a hard requirement for you?

[mesemus]

• User identification: In our repositories, users always log in with e-infra AAI (oauth protocol), so ideally we'd like to pass the external identification.
• IP address - NUKIB's requirements (National Cyber and Security Information Agency) state this as a MUST, so it is a hard requirement for us. Under GDPR it is taken as a legitimate interest. We have a strict 6 months retention policy for security-related logs, users are informed about this in terms & conditions of services and the logs are of course confidential - they are collected on dedicated hardware and handled by a special department (without the development/devops having access to them).

Would a pluggable mechanism work for you? By default, it would generate the audit messages you need, but it would also allow us to enrich the payload and use our own GELF sender if it isn't supported by default.

I should also add that we need to send the audit logs in a structured form, that is a json payload, not a serialization to a string that would need to be parsed.

[ntarocco]

User identification

We are planning to store the user id (from the Invenio db) in the logs, I am not sure that it is enough for you.

IP address... We have a strict 6 months retention policy ...

Exactly that. Does it mean that you will have to remove the logs after 6 months?

Would a pluggable mechanism work for you? By default, it would generate the audit messages you need, but it would also allow us to enrich the payload and use our own GELF sender if it isn't supported by default.

I suggest you reading the RFC and adding your comments there, we will share it soon.

I should also add that we need to send the audit logs in a structured form, that is a json payload, not a serialization to a string that would need to be parsed.

It will be structured logging, not string.

[ntarocco]

This feature has been implemented in v13. It is now possible to extend the implementation and add new actions.
I will close this discussion.