Skip to content

Commit 1bbd683

Browse files
author
Paweł Kędzia
committed
Add Auditing section to main README and create detailed auditor subsystem README
1 parent 66e9056 commit 1bbd683

File tree

2 files changed

+195
-0
lines changed

2 files changed

+195
-0
lines changed

README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,22 @@ metrics for monitoring and alerting.
108108
LLM_ROUTER_MINIMUM=1 python3 -m llm_router_api.rest_api
109109
```
110110

111+
---
112+
113+
## 🔐 Auditing
114+
115+
The router can record request‑level events (guard‑rail checks, payload masking, custom logs) in a tamper‑evident,
116+
encrypted form.
117+
All audit entries are written by the **auditor** module and stored under `logs/auditor/` as GPG‑encrypted files.
118+
119+
For a complete guide—including key generation, encryption workflow, and decryption utilities—see the dedicated
120+
documentation:
121+
122+
➡️ **[Auditing subsystem documentation](llm_router_api/core/auditor/README.md)**
123+
124+
125+
---
126+
111127
## 📦 Docker
112128

113129
Run the container with the default configuration:
Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# Auditing subsystem – `llm-router`
2+
3+
The **auditor** package provides a pluggable, tamper‑evident audit‑log system for the
4+
LLM‑router. All audit entries are written as JSON, encrypted with GPG and stored
5+
under `logs/auditor`. The subsystem is used by the router to record:
6+
7+
* request guard‑rail decisions
8+
* payload masking operations
9+
* custom audit events emitted by the application (e.g. business‑logic logs)
10+
11+
The implementation is deliberately lightweight so it can be swapped out for a
12+
different storage backend (database, cloud bucket, …) without touching the rest
13+
of the code base.
14+
15+
---
16+
17+
## 📁 Directory layout
18+
19+
```
20+
21+
llm_router_api/
22+
└─ core/
23+
└─ auditor/
24+
├─ __init__.py # package marker
25+
├─ auditor.py # public API – AnyRequestAuditor
26+
└─ log_storage/
27+
├─ __init__.py
28+
├─ log_storage_interface.py # abstract storage contract
29+
└─ gpg.py # GPG‑backed storage implementation
30+
```
31+
32+
* **`auditor.py`** – high‑level helper that forwards audit records to a
33+
storage backend. The default backend is `GPGAuditorLogStorage`.
34+
* **`log_storage_interface.py`** – defines the `AuditorLogStorageInterface`
35+
protocol (`store_log(audit_log, audit_type)`).
36+
* **`gpg.py`** – concrete implementation that encrypts each log entry with the
37+
public GPG key located at `resources/keys/llm-router-auditor-pub.asc` and
38+
writes the encrypted payload to a timestamped file
39+
`logs/auditor/<audit_type>__<timestamp>.audit`.
40+
41+
---
42+
43+
## 🛠️ How the auditor works
44+
45+
1. **Endpoint code** (e.g. `endpoint_i.py`) creates an `AnyRequestAuditor`
46+
instance with the router’s logger.
47+
2. When an auditable event occurs, the endpoint builds a dictionary that
48+
contains at least the keys `audit_type` and `payload`.
49+
3. `AnyRequestAuditor.add_log()` forwards the dictionary to the configured
50+
storage backend.
51+
4. `GPGAuditorLogStorage.store_log()`
52+
* JSON‑serialises the dictionary (pretty‑printed).
53+
* Encrypts the JSON string with the imported public key.
54+
* Writes the encrypted ASCII‑armored data to `logs/auditor/`.
55+
5. The resulting files have the extension `.audit`. They are **confidential**
56+
and **tamper‑evident** – any modification breaks the GPG decryption.
57+
58+
---
59+
60+
## 🔐 GPG key management
61+
62+
The repository ships two helper scripts under `scripts/`:
63+
64+
| Script | Purpose |
65+
|---------------------------|------------------------------------------------------------------------------------------------------------------------------|
66+
| `gen_and_export_gpg.sh` | Generates a 4096‑bit RSA key pair (no interactive prompts) and exports the public (`*.asc`) and private (`*-priv.asc`) keys. |
67+
| `decrypt_auditor_logs.sh` | Decrypts all `*.audit` files in `logs/auditor/` and writes the resulting JSON to `*.json`. |
68+
69+
### 1️⃣ Generate a new key pair
70+
71+
``` bash
72+
cd scripts
73+
./gen_and_export_gpg.sh
74+
```
75+
76+
The script will:
77+
78+
* Prompt for an email address (used as the GPG user ID).
79+
* Prompt for a passphrase (protects the private key).
80+
* Create a key pair in the local GPG keyring.
81+
* Export the **public** key to `llm-router-auditor-pub.asc`.
82+
* Export the **private** key to `llm-router-auditor-priv.asc`.
83+
84+
> **Important:** Keep the private key (`*-priv.asc`) and its passphrase safe.
85+
> Only the public key is required by the router at runtime.
86+
87+
### 2️⃣ Place the public key where the router expects it
88+
89+
``` bash
90+
mkdir -p resources/keys
91+
cp llm-router-auditor-pub.asc resources/keys/
92+
```
93+
94+
The `GPGAuditorLogStorage` class automatically imports the key from this
95+
location when the application starts.
96+
97+
### 3️⃣ Decrypt audit logs
98+
99+
``` bash
100+
cd scripts
101+
./decrypt_auditor_logs.sh
102+
```
103+
104+
For each file `logs/auditor/<type>__<timestamp>.audit` the script produces a
105+
human‑readable `*.json` file next to it:
106+
107+
```
108+
logs/auditor/request__20231129_123456.789012.audit → request__20231129_123456.789012.json
109+
```
110+
111+
You will be prompted for the passphrase of the **private** key if it is
112+
encrypted.
113+
114+
---
115+
116+
## 📚 Example: Auditing a request guard‑rail decision
117+
118+
```python
119+
from llm_router_api.core.auditor.auditor import AnyRequestAuditor
120+
import logging
121+
122+
logger = logging.getLogger("router")
123+
auditor = AnyRequestAuditor(logger)
124+
125+
# Somewhere inside an endpoint, after a guard‑rail check:
126+
audit_record = {
127+
"audit_type": "guardrail_request",
128+
"payload": {
129+
"user_id": "12345",
130+
"input": "",
131+
"decision": "blocked",
132+
"reason": "PII detected"
133+
}
134+
}
135+
auditor.add_log(audit_record)
136+
```
137+
138+
The record is encrypted and persisted as e.g.:
139+
140+
```
141+
logs/auditor/guardrail_request__20231129_141530.123456.audit
142+
```
143+
144+
---
145+
146+
## 🧩 Extending the auditor
147+
148+
If you need a different storage backend (e.g. a database), implement a new
149+
class that inherits from `AuditorLogStorageInterface` and provides a
150+
`store_log(self, audit_log, audit_type)` method. Then change the constant
151+
`DEFAULT_AUDITOR_STORAGE_CLASS` in `auditor.py` to point to your class.
152+
153+
---
154+
155+
## 📖 Further reading
156+
157+
* **`llm_router_api/core/auditor/gpg.py`** – source code of the GPG storage
158+
implementation.
159+
* **`scripts/`** – full scripts for key generation and decryption.
160+
* **`README.md`** – overall project documentation.
161+
162+
---
163+
164+
*Happy auditing!* 🎯
165+
166+
167+
---
168+
00/000000
169+
170+
## 📄 Update to the main `README.md`
171+
172+
Add the following section near the **Features** or **Monitoring & Metrics** part of the main README:
173+
174+
```
175+
176+
Place the snippet where you describe optional runtime features (e.g., after *Monitoring & Metrics*). This gives users a
177+
quick overview and a direct link to the full audit guide.
178+
179+
```

0 commit comments

Comments
 (0)