Development to 0.6.0

.github/workflows/python-app.yml

@@ -9,7 +9,7 @@ on:
   pull_request:
 
 permissions:
-  contents: read
+  contents: write
 
 jobs:
   build:
@@ -23,6 +23,10 @@ jobs:
           activate-environment: true
           enable-cache: true
 
+      - uses: actions/checkout@v6
+        with:
+          ref: ${{ github.event.pull_request.head.ref }}
+
       - name: Set timezone
         run: sudo timedatectl set-timezone Europe/Vienna
 
@@ -32,6 +36,20 @@ jobs:
       - name: Test with pytest
         run: uv run pytest -s --run-ignored
 
+      - name: Commit and Push Changes
+        env:
+          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
+        run: |
+          git add .
+          if git diff --cached --quiet; then
+            echo "No tracked files changed"
+            exit 0
+          fi
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git commit -m "Automated documentation update"
+          git push origin "HEAD:$PR_BRANCH"
+
       # integration tests for DetectMateService
       - name: Checkout DetectMateService
         uses: actions/checkout@v4

.pre-commit-config.yaml

@@ -73,3 +73,13 @@ repos:
   hooks:
     - id: python-check-blanket-noqa
     - id: python-no-log-warn
+
+# Repair mangled MkDocs snippet includes (`--8 < --"docs` -> `--8<-- "docs`)
+- repo: local
+  hooks:
+    - id: fix-snippet-markers
+      name: fix mkdocs snippet markers
+      language: system
+      entry: sed -i -E
+      args: ['s/--8 *< *-- *"docs/--8<-- "docs/g']
+      files: ^docs/.*\.md$

docs/advanced/overall_architecture.md

@@ -0,0 +1,42 @@
+# Overall architecture
+
+## How things work and where data goes
+
+This document describes the high-level design of the DetectMateLibrary, how components interact, the data contracts they use, and guidance for deploying and extending the system. The library is built around small, composable components that operate on streaming log data and exchange strongly-typed Schema objects.
+
+Key goals:
+
+- Clear separation of concerns (reading, parsing, detection, output).
+- Stream-friendly processing with minimal buffering.
+- Well-defined schema contracts so components can be composed or run as microservices.
+- Easy extensibility: add new readers, parsers or detectors by subclassing core base classes.
+
+## Components flow
+
+The pipeline is strictly directional:
+
+- Parser: consumes raw logs and produces parsed log objects (structured fields, timestamps, variables).
+- Detector: consumes parsed logs and generates alerts/findings when rules or models match anomalous behavior.
+- Alert Aggregation: consumes alerts and aggregates them.
+
+Each arrow represents a stream of Schema objects. Components are designed to run in the same process for lightweight setups or as separate services for scalable deployments.
+
+![Components flow](../img/diagrams_structure.png)
+
+## Components architecture
+
+All components inherit from a `CoreComponent` class. This class provides all the essential functionality required for DetectMate to operate (see UML diagram below). Every `Detector` must inherit from `CoreDetector`, every `AlertAggregator` must inherit from `CoreAlertAggregation` and every `Parser` must inherit from `CoreParser` to ensure compatibility with DetectMate.
+
+Each component's arguments must be stored in its corresponding configuration class. These config classes follow the same design pattern as their components and must inherit from `CoreConfig`.
+
+![Architecture](../img/uml_structure.png)
+
+## Components methods
+
+Each Core* base class exposes a small, stable API that implementations must implement or may override.
+
+```python
+--8<-- "docs/examples/others/components_methods.py:read"
+```
+
+Go back [Index](../index.md)

docs/alert_aggregator.md

@@ -15,7 +15,7 @@ This document explains expected APIs, how to implement a parser, testing tips an
 - `CoreParser.run()` handles lifecycle and calls `aggregate_alerts()` for each input; implement pure alert aggregation logic inside `aggregate_alerts()` where possible.
 - Use a typed `Config` class (subclass of `CoreAlertAggregationConfig`) to hold runtime parameters.
 
-## CoreParser — minimal API
+## CoreParser  --  minimal API
 
 Recommended signatures and behavior:
 

docs/auxiliar/persistency.md

@@ -31,20 +31,20 @@ Two families ship today:
   raw rows. Very storage heavy and *not recommended* for production-ready detectors.
 - **Tracker backends** (`EventStabilityTracker`) keep only derived features
   (e.g. "this variable has been constant for the last 10k events") that are relevant for the detector. Use these
-  when you only need a summary or a subset of the log's information, not the raw history — they cost a fraction of
+  when you only need a summary or a subset of the log's information, not the raw history  --  they cost a fraction of
   the memory.
 
 All backends implement the same four-method contract: `add_data`, `get_data`,
 `dump`, `load`. That contract is what `EventPersistency` and
-`PersistencySaver` rely on — anything you add later only has to follow it.
+`PersistencySaver` rely on  --  anything you add later only has to follow it.
 
 ### 3. Saver lifecycle (`PersistencySaver`)
 
 `EventPersistency` itself is in-memory. To survive a process restart, the
 state has to be written somewhere. `PersistencySaver` wraps an
 `EventPersistency` and:
 
-- writes to disk (or any `fsspec` URI) on two triggers — a wall-clock interval
+- writes to disk (or any `fsspec` URI) on two triggers  --  a wall-clock interval
   and an event-count threshold;
 - optionally `auto_load`s previously saved state during construction;
 - exposes `start()` / `stop()` so the background timer can be torn down
@@ -108,10 +108,10 @@ ep[event_id]                       # alias for get_event_data
 | Class | Use when |
 |---|---|
 | `persistency.EventDataFrame` | You need history and a Pandas DataFrame is the natural shape. |
-| `persistency.ChunkedEventDataFrame` | High-volume / streaming workloads — Polars-backed with row-retention and automatic compaction. |
+| `persistency.ChunkedEventDataFrame` | High-volume / streaming workloads  --  Polars-backed with row-retention and automatic compaction. |
 | `persistency.EventStabilityTracker` | You only care about how variables behave over time (`STATIC` / `STABLE` / `UNSTABLE` / `RANDOM`). Cheapest memory footprint. |
 
-All three are re-exported from the top of the package — `persistency.X` is the
+All three are re-exported from the top of the package  --  `persistency.X` is the
 canonical import; the deeply nested submodules are an implementation detail.
 
 ### Persisting to disk
@@ -134,7 +134,14 @@ saver.stop()    # final flush, stops the background timer
 
 `PersistencySaver.save()` is thread-safe, and `stop()` is idempotent. The two
 save triggers (`save_interval_seconds` and `events_until_save`) are
-independent — whichever fires first wins.
+independent  --  whichever fires first wins.
+
+If writing to storage fails (unwritable path, full disk, lost credentials),
+`save()` and `stop()` raise `persistency.PersistencySaveError`, and
+`events_since_save` keeps counting the unsaved events. Saves fired by the two
+triggers cannot raise to a caller, so they log the failure at `ERROR` instead;
+the timer keeps running, and after a failed save the event-count trigger waits
+another `events_until_save` events before retrying.
 
 #### Restoring state
 
@@ -147,21 +154,21 @@ saver = persistency.PersistencySaver(
 ```
 
 If `auto_load=True` and no saved state exists, the constructor raises
-`persistency.PersistencyLoadError` immediately — fail-fast rather than
+`persistency.PersistencyLoadError` immediately  --  fail-fast rather than
 silently starting empty.
 
 #### Exporting and importing state on demand
 
-For one-shot transfers — e.g. moving trained state to a new environment, or
-taking a manual snapshot — use the standalone functions directly:
+For one-shot transfers  --  e.g. moving trained state to a new environment, or
+taking a manual snapshot  --  use the standalone functions directly:
 
 ```python
 from detectmatelibrary.utils import persistency
 
 # Export to a file URI
 persistency.save(ep, "./snapshots/trained-state")
 
-# Export to bytes (no disk I/O — useful when sending state over a network API)
+# Export to bytes (no disk I/O  --  useful when sending state over a network API)
 data: bytes = persistency.save(ep)
 
 # Import from a file URI
@@ -185,7 +192,7 @@ when a saver is active.
 #### Detector-level export and import
 
 When working through a detector (the typical path for DetectMateService), use
-the methods on the detector object directly — no need to access
+the methods on the detector object directly  --  no need to access
 `EventPersistency` internals:
 
 ```python