Background

Matomo's archiving process relies on invalidation records to determine which data needs to be (re-)processed. Whenever an invalidation occurs, a corresponding record is created, and the archiving process identifies the next available invalidation for processing. During this process, the status of the invalidation is updated, and a timestamp is set to indicate when processing began.

Problem Statement

In a multi-tenant environment with multiple archiving servers, it would be beneficial to track which server is processing a specific invalidation. Currently, there is no built-in mechanism to determine which server is handling a given invalidation, which can make troubleshooting and monitoring more challenging.

Proposed Solution

To address this, we propose extending the invalidation table to include a field for storing the hostname of the archiving server as well as the current process id when an invalidation is marked as "started." This enhancement would provide several benefits, including:

• Improved Monitoring and Debugging:

  • Administrators can easily identify which server is processing a specific invalidation.
  • Administrators can easily identify if an invalidation is still processed by looking for a process with the stored id
  • Helps in diagnosing issues, such as stuck or long-running invalidations.

• Easier Failure Recovery:

  • In cases where an archiving server crashes, the invalidation record may still be marked as "started."
  • By knowing which server was responsible, administrators can quickly determine whether an invalidation needs to be reset.

Use Case Example

If a server crashes, invalidations assigned to it would remain in the "started" state indefinitely. With the hostname stored in the invalidation record, administrators can easily identify affected entries and take appropriate action to reset or reprocess them.

Implementation Considerations

• Introduce a new columns (e.g., processing_host, process_id) in the invalidation table.
• Ensure the hostname and the current process id is set when an invalidation is marked as "started."