Compare commits
5 Commits
e31f6f0ca1
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| cb117e1008 | |||
| 6aec4f6082 | |||
| 39ba6bd7da | |||
| b2f1baaac5 | |||
| 7864d2ed18 |
25
.gitignore
vendored
Normal file
25
.gitignore
vendored
Normal file
@@ -0,0 +1,25 @@
|
||||
|
||||
evidence/
|
||||
.local_keys
|
||||
*.pyc
|
||||
__pycache__/
|
||||
tests/.dbg*
|
||||
tests/.e2e/
|
||||
|
||||
# Local credentials and runtime data
|
||||
.env
|
||||
.env.*
|
||||
*.key
|
||||
*.pem
|
||||
*.sqlite
|
||||
*.sqlite-*
|
||||
*.sock
|
||||
*.sock.lock
|
||||
*.pcap
|
||||
.venv/
|
||||
.pytest_cache/
|
||||
|
||||
# Editor backup files
|
||||
*.bkp
|
||||
|
||||
fig
|
||||
94
README.md
94
README.md
@@ -1 +1,93 @@
|
||||
# Hi
|
||||
# LEAF
|
||||
|
||||
LEAF connects a passive Suricata sensor to VyOS firewall enforcement. It reads
|
||||
alerts, applies a threshold policy, stores block decisions in SQLite, and
|
||||
updates a dedicated router address group through the VyOS API.
|
||||
|
||||
Suricata receives mirrored traffic. Enforcement happens asynchronously on the
|
||||
router, so packets can pass before a block is committed.
|
||||
|
||||
```text
|
||||
Mirrored traffic -> Suricata -> Unix socket -> policy -> SQLite
|
||||
|
|
||||
reconciliation worker
|
||||
|
|
||||
VyOS API + readback
|
||||
|
|
||||
firewall drop rules
|
||||
|
||||
Suricata stats -> EVE file on tmpfs -> metrics
|
||||
Controller decisions -> persistent audit log
|
||||
```
|
||||
|
||||
## Policy
|
||||
|
||||
Rules must carry a `leaf_action` metadata value before they can produce a block:
|
||||
|
||||
| Metadata | Target | Requirement |
|
||||
|---|---|---|
|
||||
| `leaf_action block` | Destination IPv4 address | Policy permits the SID and address |
|
||||
| `leaf_action block_src` | Source IPv4 address | Also requires `policy.block_src: true` |
|
||||
|
||||
The lab configuration allows SIDs **1000001** and **1000030**, requires two alerts
|
||||
for the same SID and address within 300 seconds, and assigns a 3600-second TTL.
|
||||
Qualifying alerts refresh the TTL. The default cap is 50 active addresses.
|
||||
Private, loopback, link-local, multicast and configured trusted networks are
|
||||
protected. The included rules are examples for controlled TEST-NET traffic.
|
||||
|
||||
One worker serializes router changes and verifies group membership after each
|
||||
commit. SQLite retains desired blocks during API failures. Expiry and drift
|
||||
reconciliation run independently of new alerts. Source and destination hooks
|
||||
share the same address group; the stored `kind` records the initial reason.
|
||||
|
||||
## Repository
|
||||
|
||||
| Path | Contents |
|
||||
|---|---|
|
||||
| `controller/leaf_controller/` | Ingestion, policy, state, metrics and API client |
|
||||
| `controller/leaf-controller.service` | Controller service |
|
||||
| `config/prototype-vyos.yaml` | Lab controller configuration |
|
||||
| `controller/config.yaml` | Generic configuration example |
|
||||
| `suricata/` | Sensor configuration, rules and retention units |
|
||||
| `scripts/` | Installation, maintenance and read-only diagnostics |
|
||||
| `tests/` | Unit tests, mock integration and bounded live tests |
|
||||
| `vyos/router-leaf.snippet.txt` | Illustrative enforcement hooks |
|
||||
| `fig/` | Architecture figures |
|
||||
|
||||
## Getting started
|
||||
|
||||
The sensor requires Python 3, the packages in `controller/requirements.txt`,
|
||||
Suricata 7, systemd and a capture interface receiving the intended mirror.
|
||||
Review the interface names, API endpoint, TLS trust, protected networks and
|
||||
firewall hooks before installing.
|
||||
|
||||
```bash
|
||||
sudo apt install python3 python3-requests python3-yaml
|
||||
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p 'test_*.py' -v
|
||||
bash tests/run_e2e.sh
|
||||
```
|
||||
|
||||
See [Installation and operations](docs/INSTALLATION.md) for deployment,
|
||||
[Architecture](docs/ARCHITECTURE.md) for the controller design, and
|
||||
[Testing](docs/TESTING.md) for offline and live checks.
|
||||
|
||||
## Limits
|
||||
|
||||
LEAF is an IPv4 lab prototype. The sample rules do not establish production
|
||||
detection quality, and no high-throughput capacity is claimed. Router commits
|
||||
can take tens of seconds.
|
||||
|
||||
Socket alerts are not a durable journal. A process crash, disconnection or
|
||||
Suricata send-buffer pressure can lose alerts before they reach SQLite. A zero
|
||||
controller queue-drop count does not establish that Suricata delivered every
|
||||
alert. Graceful shutdown persists accepted queued alerts; previously stored
|
||||
block decisions survive restarts.
|
||||
|
||||
Live counter tests establish detection and firewall-rule hits. Ingress capture
|
||||
alone does not prove destination receipt or application connectivity. Sustained
|
||||
throughput and full sensor/router reboot recovery require separate testing.
|
||||
|
||||
|
||||
|
||||
|
||||

|
||||
|
||||
52
config/prototype-vyos.yaml
Normal file
52
config/prototype-vyos.yaml
Normal file
@@ -0,0 +1,52 @@
|
||||
# LEAF-PROTO prototype config — run on the LEAF VM (eth1 P2P link to VIPERA).
|
||||
# Secrets stay in env: export LEAF_VYOS_KEY=... before starting the controller.
|
||||
vyos:
|
||||
url: "https://10.0.249.0:8443"
|
||||
key_env: LEAF_VYOS_KEY
|
||||
# Router serves a self-signed CN=localhost cert with no IP SAN, so plain
|
||||
# CA/hostname verification cannot work; trust is anchored on the pinned
|
||||
# fingerprint below (regenerate with:
|
||||
# echo | openssl s_client -connect 10.0.249.0:8443 2>/dev/null | openssl x509 -noout -fingerprint -sha256)
|
||||
tls_cert_pin: "21:3A:73:9A:B5:EE:8A:83:F8:50:E8:F9:4E:0C:DD:28:81:29:75:1F:33:18:F9:C9:0A:9C:AB:F0:91:EF:3C:C4"
|
||||
timeout: 60
|
||||
retries: 2
|
||||
block_group: IDS-BLOCK
|
||||
ensure_primitives: true
|
||||
adopt_foreign_members: false # this dedicated group is owned by durable controller state
|
||||
save_config: false # transient membership; SQLite is the durable desired state
|
||||
eve:
|
||||
path: /var/log/suricata-leaf/eve.json
|
||||
checkpoint: true # resume at the sqlite-persisted offset after restarts
|
||||
alert_socket:
|
||||
path: /run/suricata/alerts.sock # AF_UNIX listener owned by this controller;
|
||||
# suricata unix_stream connects as the client
|
||||
queue_size: 1000
|
||||
audit:
|
||||
max_bytes: 16777216 # rotate decisions.jsonl at 16 MiB...
|
||||
keep: 3 # ...keeping three rotated files
|
||||
policy:
|
||||
mode: threshold
|
||||
min_hits: 2
|
||||
window_seconds: 300
|
||||
ttl_seconds: 3600
|
||||
max_blocks: 50
|
||||
sids_only: [1000001, 1000030] # lab-only automatic actions; expand after validating rules
|
||||
block_src: true # permit source actions for the explicitly allowed lab SIDs
|
||||
allow_cidrs: # never block these (management/fabric floor)
|
||||
- 10.0.249.0/31 # VIPERA<->sensor P2P link
|
||||
- 192.168.88.0/27 # uplink transit
|
||||
- 192.168.1.0/24 # management LAN (my workstation path)
|
||||
- 192.168.100.0/24 # LEAF management LAN (sensor itself)
|
||||
state_db: /var/lib/leaf-controller/state.sqlite
|
||||
audit_log: /var/log/leaf-controller/decisions.jsonl
|
||||
|
||||
poll_interval: 0.2
|
||||
router_interval: 2
|
||||
batch_delay: 0.2
|
||||
metrics: # stats records: counting only, never policy input
|
||||
reconcile_interval: 30 # = suricata stats period; one health line per window
|
||||
capture_interval: 3600 # capture.log metrics line cadence (s)
|
||||
capture_log: /var/log/leaf-controller/capture.log
|
||||
capture_max_bytes: 10485760 # rotate at 10 MiB...
|
||||
capture_log_keep: 6 # ...keeping six rotated files
|
||||
log_level: INFO
|
||||
32
controller/config.yaml
Normal file
32
controller/config.yaml
Normal file
@@ -0,0 +1,32 @@
|
||||
# Generic deployment example. The lab installer seeds config/prototype-vyos.yaml.
|
||||
vyos:
|
||||
url: https://10.66.0.1 # VyOS management address
|
||||
key_env: LEAF_VYOS_KEY # API key read from this env var (never stored here)
|
||||
ca_bundle: /etc/leaf-controller/vyos-ca.pem # CA that signed router cert (see snippet)
|
||||
tls_verify: true
|
||||
timeout: 10
|
||||
retries: 3
|
||||
|
||||
block_group: IDS-BLOCK
|
||||
ensure_primitives: true
|
||||
adopt_foreign_members: true
|
||||
save_config: false # true = persist config on router after each change
|
||||
|
||||
eve:
|
||||
path: /var/log/suricata-leaf/eve.json
|
||||
|
||||
policy:
|
||||
mode: threshold # threshold | immediate
|
||||
min_hits: 2
|
||||
window_seconds: 300
|
||||
ttl_seconds: 3600
|
||||
max_blocks: 50
|
||||
allow_cidrs: [] # additional never-block CIDRs
|
||||
sids_only: [] # non-empty = only act on listed sids
|
||||
sids_deny: []
|
||||
block_src: false # honor metadata leaf_action=block_src
|
||||
|
||||
state_db: /var/lib/leaf-controller/state.sqlite
|
||||
audit_log: /var/log/leaf-controller/decisions.jsonl
|
||||
poll_interval: 0.2
|
||||
sync_interval: 300.0
|
||||
30
controller/leaf-controller.service
Normal file
30
controller/leaf-controller.service
Normal file
@@ -0,0 +1,30 @@
|
||||
[Unit]
|
||||
Description=LEAF alert controller and VyOS reconciliation worker
|
||||
# ordering contract: the controller binds the alert socket BEFORE the
|
||||
# sensor unit starts (suricata is the unix_stream CLIENT); do not reorder.
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
RequiresMountsFor=/var/log/suricata-leaf
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
NotifyAccess=main
|
||||
TimeoutStartSec=30
|
||||
EnvironmentFile=/etc/leaf-controller/env
|
||||
Environment=PYTHONDONTWRITEBYTECODE=1
|
||||
ExecStart=/usr/bin/python3 -m leaf_controller.main --config /etc/leaf-controller/config.yaml
|
||||
WorkingDirectory=/opt/leaf-controller/controller
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=180
|
||||
User=leafctrl
|
||||
Group=leafctrl
|
||||
UMask=0027
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=strict
|
||||
ProtectHome=yes
|
||||
ReadWritePaths=/var/lib/leaf-controller /var/log/leaf-controller /run/suricata
|
||||
PrivateTmp=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
2
controller/leaf_controller/__init__.py
Normal file
2
controller/leaf_controller/__init__.py
Normal file
@@ -0,0 +1,2 @@
|
||||
"""leaf_controller — Suricata EVE -> VyOS dynamic block controller (prototype)."""
|
||||
__version__ = "0.1.0"
|
||||
170
controller/leaf_controller/alertsock.py
Normal file
170
controller/leaf_controller/alertsock.py
Normal file
@@ -0,0 +1,170 @@
|
||||
"""Bounded AF_UNIX EVE ingress. Accepted queues are volatile until policy persists them.
|
||||
|
||||
Suricata can also lose records before delivery on EAGAIN. Local counters do
|
||||
not prove end-to-end losslessness; this is a lab control plane, not a journal.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import fcntl
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import queue
|
||||
import socket
|
||||
import stat
|
||||
import threading
|
||||
|
||||
log = logging.getLogger('leaf.sock')
|
||||
MAX_RECORD = 1 << 20
|
||||
|
||||
|
||||
class AlertSocket:
|
||||
def __init__(self, path, queue_size=1000, rcvbuf=1 << 20):
|
||||
if int(queue_size) <= 0:
|
||||
raise ValueError('alert_socket.queue_size must be positive')
|
||||
self.path = path
|
||||
self.q = queue.Queue(maxsize=int(queue_size))
|
||||
self.counts = dict(connections=0, records=0, parse_errors=0, queue_drops=0, other=0)
|
||||
self.available = threading.Event()
|
||||
self._rcvbuf = int(rcvbuf)
|
||||
self._sock = self._conn = self._thread = self._lock = self._identity = None
|
||||
self._stop = threading.Event()
|
||||
|
||||
def bind(self):
|
||||
os.makedirs(os.path.dirname(os.path.abspath(self.path)), exist_ok=True)
|
||||
self._lock = open(self.path + '.lock', 'a')
|
||||
try:
|
||||
fcntl.flock(self._lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
if os.path.lexists(self.path):
|
||||
st = os.lstat(self.path)
|
||||
if not stat.S_ISSOCK(st.st_mode):
|
||||
raise OSError('refusing to replace a non-socket path')
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as probe:
|
||||
probe.settimeout(0.2)
|
||||
try:
|
||||
probe.connect(self.path)
|
||||
except (ConnectionRefusedError, FileNotFoundError):
|
||||
os.unlink(self.path)
|
||||
else:
|
||||
raise OSError('another listener owns this socket')
|
||||
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self._sock = sock
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, self._rcvbuf)
|
||||
sock.settimeout(0.2)
|
||||
sock.bind(self.path)
|
||||
st = os.lstat(self.path)
|
||||
self._identity = (st.st_dev, st.st_ino)
|
||||
os.chmod(self.path, 0o660)
|
||||
sock.listen(1)
|
||||
except BaseException:
|
||||
self.close()
|
||||
raise
|
||||
|
||||
def start(self):
|
||||
self._thread = threading.Thread(target=self._accept_loop, name='alert-sock', daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _accept_loop(self):
|
||||
while not self._stop.is_set():
|
||||
try:
|
||||
conn, _ = self._sock.accept()
|
||||
except socket.timeout:
|
||||
continue
|
||||
except OSError:
|
||||
break
|
||||
self._conn = conn
|
||||
self.counts['connections'] += 1
|
||||
try:
|
||||
conn.settimeout(0.2)
|
||||
conn.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, self._rcvbuf)
|
||||
self._read_conn(conn)
|
||||
except OSError:
|
||||
pass
|
||||
finally:
|
||||
conn.close()
|
||||
self._conn = None
|
||||
|
||||
def _read_conn(self, conn):
|
||||
pending = bytearray()
|
||||
oversized = False
|
||||
while not self._stop.is_set():
|
||||
try:
|
||||
data = conn.recv(65536)
|
||||
except socket.timeout:
|
||||
continue
|
||||
if not data:
|
||||
if pending or oversized:
|
||||
self.counts['parse_errors'] += 1
|
||||
return
|
||||
for part in data.splitlines(keepends=True):
|
||||
complete = part.endswith(b'\n')
|
||||
if not oversized:
|
||||
pending.extend(part)
|
||||
if len(pending) > MAX_RECORD:
|
||||
pending.clear()
|
||||
oversized = True
|
||||
if complete:
|
||||
if oversized:
|
||||
self.counts['parse_errors'] += 1
|
||||
else:
|
||||
self._enqueue(pending)
|
||||
pending.clear()
|
||||
oversized = False
|
||||
|
||||
def _enqueue(self, line):
|
||||
if len(line) > MAX_RECORD:
|
||||
self.counts['parse_errors'] += 1
|
||||
return
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except (ValueError, RecursionError, UnicodeError):
|
||||
self.counts['parse_errors'] += 1
|
||||
return
|
||||
if not isinstance(rec, dict) or rec.get('event_type') != 'alert':
|
||||
self.counts['other'] += 1
|
||||
return
|
||||
try:
|
||||
self.q.put_nowait(rec)
|
||||
self.counts['records'] += 1
|
||||
self.available.set()
|
||||
except queue.Full:
|
||||
self.counts['queue_drops'] += 1
|
||||
log.error('alert queue full: dropped %d', self.counts['queue_drops'])
|
||||
|
||||
def drain(self, limit=256):
|
||||
self.available.clear()
|
||||
out = []
|
||||
while len(out) < limit:
|
||||
try:
|
||||
out.append(self.q.get_nowait())
|
||||
except queue.Empty:
|
||||
break
|
||||
if not self.q.empty():
|
||||
self.available.set()
|
||||
return out
|
||||
|
||||
def close(self):
|
||||
self._stop.set()
|
||||
if self._conn:
|
||||
try:
|
||||
self._conn.shutdown(socket.SHUT_RDWR)
|
||||
except OSError:
|
||||
pass
|
||||
if self._sock:
|
||||
self._sock.close()
|
||||
if self._thread:
|
||||
self._thread.join(timeout=2)
|
||||
if self._thread.is_alive():
|
||||
raise RuntimeError('socket reader did not stop')
|
||||
self._thread = None
|
||||
self._sock = None
|
||||
if self._identity:
|
||||
try:
|
||||
st = os.lstat(self.path)
|
||||
if (st.st_dev, st.st_ino) == self._identity:
|
||||
os.unlink(self.path)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
self._identity = None
|
||||
if self._lock:
|
||||
self._lock.close()
|
||||
self._lock = None
|
||||
155
controller/leaf_controller/config.py
Normal file
155
controller/leaf_controller/config.py
Normal file
@@ -0,0 +1,155 @@
|
||||
"""Config loading + defaults. YAML file, secrets via env var only."""
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
DEFAULTS: dict[str, Any] = {
|
||||
"vyos": {
|
||||
"url": "https://10.66.0.1",
|
||||
"key_env": "LEAF_VYOS_KEY",
|
||||
"ca_bundle": None, # path to CA/pinned cert; None -> system CAs
|
||||
"tls_verify": True,
|
||||
"tls_cert_pin": "", # sha256 cert fingerprint; pins TLS trust (IP-hosted self-signed router cert)
|
||||
"timeout": 10.0,
|
||||
"retries": 3,
|
||||
},
|
||||
"block_group": "IDS-BLOCK",
|
||||
"ensure_primitives": True, # verify group/rule hook exists on startup
|
||||
"adopt_foreign_members": True, # members found on router but not in state db
|
||||
"save_config": False, # POST /config-file save after every mutation
|
||||
"eve": {
|
||||
"path": "/var/log/suricata-leaf/eve.json",
|
||||
"checkpoint": True, # resume at persisted offset across restarts
|
||||
},
|
||||
"alert_socket": {
|
||||
# AF_UNIX listener for suricata's unix_stream output: alerts arrive
|
||||
# through the socket instead of eve.json. empty path disables the
|
||||
# socket path entirely (offline/mock runs keep pure file tailing).
|
||||
"path": "/run/suricata/alerts.sock",
|
||||
"queue_size": 1000,
|
||||
},
|
||||
"audit": {
|
||||
"max_bytes": 16777216, # rotate decisions.jsonl at 16 MiB
|
||||
"keep": 3, # rotated files retained
|
||||
},
|
||||
"metrics": {
|
||||
# stats records are counting-only: never reach state/audit/API.
|
||||
"reconcile_interval": 30.0, # one journal line per stats window
|
||||
"capture_interval": 3600.0, # capture.log line cadence
|
||||
"capture_log": None, # default: capture.log next to audit_log
|
||||
"capture_max_bytes": 10485760, # rotate capture.log at 10 MiB...
|
||||
"capture_log_keep": 6, # ... keeping six rotated files
|
||||
"log_level": "INFO",
|
||||
},
|
||||
"policy": {
|
||||
"mode": "threshold", # threshold | immediate
|
||||
"min_hits": 2,
|
||||
"window_seconds": 300,
|
||||
"ttl_seconds": 3600,
|
||||
"max_blocks": 50,
|
||||
"allow_cidrs": [], # extra never-block networks
|
||||
"sids_deny": [], # never act on these sids
|
||||
"sids_only": [], # if non-empty: act ONLY on these sids
|
||||
},
|
||||
"always_allow_cidrs": [
|
||||
# hard floor: never block routers/LAN/mgmt/mcast-ish space no matter what
|
||||
"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16",
|
||||
"0.0.0.0/8", "127.0.0.0/8", "240.0.0.0/4",
|
||||
"100.64.0.0/10", "169.254.0.0/16", "224.0.0.0/4",
|
||||
"fc00::/7", "fe80::/10",
|
||||
],
|
||||
"state_db": "/var/lib/leaf-controller/state.sqlite",
|
||||
"audit_log": "/var/log/leaf-controller/decisions.jsonl",
|
||||
"poll_interval": 0.2,
|
||||
"sync_interval": 300.0,
|
||||
}
|
||||
|
||||
|
||||
def _merge(base: dict, override: dict) -> dict:
|
||||
out = dict(base)
|
||||
for k, v in override.items():
|
||||
if isinstance(v, dict) and isinstance(out.get(k), dict):
|
||||
out[k] = _merge(out[k], v)
|
||||
else:
|
||||
out[k] = v
|
||||
return out
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
raw: dict = field(default_factory=dict)
|
||||
|
||||
def get(self, dotted: str, default: Any = None) -> Any:
|
||||
cur: Any = self.raw
|
||||
for part in dotted.split("."):
|
||||
if not isinstance(cur, dict) or part not in cur:
|
||||
return default
|
||||
cur = cur[part]
|
||||
return cur
|
||||
|
||||
# --- derived helpers ---
|
||||
def vyos_url(self) -> str:
|
||||
return self.get("vyos.url").rstrip("/")
|
||||
|
||||
def vyos_key(self) -> str:
|
||||
env = self.get("vyos.key_env", "LEAF_VYOS_KEY")
|
||||
key = os.environ.get(env, "")
|
||||
if not key and not self.get("dry_run"):
|
||||
raise SystemExit(f"missing API key: set env var {env}")
|
||||
return key
|
||||
|
||||
def allow_networks(self) -> list:
|
||||
nets = list(DEFAULTS["always_allow_cidrs"]) + list(self.get("always_allow_cidrs", [])) + list(self.get("policy.allow_cidrs", []))
|
||||
return [ipaddress.ip_network(c, strict=False) for c in nets]
|
||||
|
||||
def is_allowed_ip(self, ip: str) -> bool:
|
||||
try:
|
||||
addr = ipaddress.ip_address(ip)
|
||||
except ValueError:
|
||||
return True # malformed / non-ip (eg:66 etc) -> treat as allowed, never block
|
||||
return any(addr in net for net in self.allow_networks())
|
||||
|
||||
|
||||
def load_config(path: str) -> Config:
|
||||
with open(path) as fh:
|
||||
user = yaml.safe_load(fh) or {}
|
||||
if not isinstance(user, dict):
|
||||
raise SystemExit(f"{path}: expected a mapping")
|
||||
cfg = Config(raw=_merge(DEFAULTS, user))
|
||||
import math
|
||||
for name in ('alert_socket.queue_size', 'policy.min_hits', 'policy.max_blocks',
|
||||
'policy.window_seconds', 'policy.ttl_seconds', 'poll_interval',
|
||||
'sync_interval', 'router_interval', 'vyos.timeout', 'vyos.retries'):
|
||||
value = cfg.get(name, 2 if name == 'router_interval' else None)
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value) or value <= 0:
|
||||
raise ValueError(name + ' must be a positive finite number')
|
||||
for name in ('alert_socket.queue_size', 'policy.min_hits', 'policy.max_blocks', 'vyos.retries'):
|
||||
if int(cfg.get(name)) != cfg.get(name):
|
||||
raise ValueError(name + ' must be an integer')
|
||||
for name in ('audit.max_bytes', 'audit.keep', 'metrics.capture_max_bytes', 'metrics.capture_log_keep'):
|
||||
v = cfg.get(name)
|
||||
if isinstance(v,bool) or not isinstance(v,int) or v < 0:
|
||||
raise ValueError(name + ' must be a non-negative integer')
|
||||
if cfg.get('policy.mode') not in ('threshold', 'immediate'):
|
||||
raise ValueError('policy.mode must be threshold or immediate')
|
||||
cfg.allow_networks() # Validate every configured CIDR at startup.
|
||||
return cfg
|
||||
|
||||
|
||||
def read_key_env(path, name='LEAF_VYOS_KEY'):
|
||||
"""Read the simple quoted assignment emitted by the credential tool; never eval."""
|
||||
import shlex
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
key, sep, value = line.strip().partition('=')
|
||||
if sep and key == name:
|
||||
parts = shlex.split(value, comments=True)
|
||||
if len(parts) != 1 or not parts[0]:
|
||||
raise ValueError('invalid API key assignment')
|
||||
return parts[0]
|
||||
raise ValueError('API key assignment missing')
|
||||
106
controller/leaf_controller/eve.py
Normal file
106
controller/leaf_controller/eve.py
Normal file
@@ -0,0 +1,106 @@
|
||||
"""Read complete EVE records, preserving partial writes and handling rotation."""
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
log = logging.getLogger("leaf.eve")
|
||||
|
||||
class EveTailer:
|
||||
def __init__(self, path, poll_interval=0.2, from_start=False, checkpoint=None, idle_wait=None):
|
||||
self.path, self.poll, self.from_start = path, poll_interval, from_start
|
||||
# (dev, ino, pos) persisted while the controller was running; if it
|
||||
# matches the file on disk we resume there instead of skipping ahead.
|
||||
self.checkpoint = checkpoint
|
||||
self.idle_wait = idle_wait or time.sleep
|
||||
self._fh = None
|
||||
self._ino = None
|
||||
self._pos = 0
|
||||
|
||||
def _open(self):
|
||||
try:
|
||||
fh = open(self.path, "rb")
|
||||
except FileNotFoundError:
|
||||
# A file created after startup must be read from its beginning.
|
||||
self.from_start = True
|
||||
return None
|
||||
st = os.fstat(fh.fileno())
|
||||
self._ino = (st.st_dev, st.st_ino)
|
||||
if not self.from_start:
|
||||
ck = self.checkpoint
|
||||
if ck and ck[:2] != self._ino:
|
||||
log.warning("eve checkpoint inode mismatch on %s; processing "
|
||||
"current file from start", self.path)
|
||||
fh.seek(0)
|
||||
elif ck and ck[2] <= st.st_size:
|
||||
fh.seek(ck[2])
|
||||
elif ck:
|
||||
log.warning("eve checkpoint pos %d beyond size %d; rereading %s from 0",
|
||||
ck[2], st.st_size, self.path)
|
||||
fh.seek(0)
|
||||
else:
|
||||
# First run ever: never replay pre-existing history.
|
||||
fh.seek(0, os.SEEK_END)
|
||||
self.from_start = True
|
||||
self._fh, self._pos = fh, fh.tell()
|
||||
log.info("opened eve file %s (pos=%d)", self.path, self._pos)
|
||||
return fh
|
||||
|
||||
def _needs_reopen(self):
|
||||
try:
|
||||
st = os.stat(self.path)
|
||||
except FileNotFoundError:
|
||||
return True
|
||||
return (st.st_dev, st.st_ino) != self._ino or st.st_size < (self._fh.tell() if self._fh else self._pos)
|
||||
|
||||
def _read_ready(self, fh):
|
||||
# Never consume a partial record: the writer may append its remainder.
|
||||
out = []
|
||||
for _ in range(1000):
|
||||
pos = fh.tell()
|
||||
line = fh.readline()
|
||||
if not line.endswith(b"\n"):
|
||||
fh.seek(pos)
|
||||
break
|
||||
out.append((line.decode("utf-8", errors="replace"), fh.tell()))
|
||||
return out
|
||||
|
||||
@staticmethod
|
||||
def _parse(line):
|
||||
if isinstance(line, tuple):
|
||||
line = line[0]
|
||||
try:
|
||||
event = json.loads(line)
|
||||
return event if isinstance(event, dict) else None
|
||||
except (ValueError, RecursionError):
|
||||
log.warning("skipping invalid EVE JSON record")
|
||||
return None
|
||||
|
||||
def events(self):
|
||||
try:
|
||||
while True:
|
||||
fh = self._fh or self._open()
|
||||
if fh is None:
|
||||
self.idle_wait(self.poll)
|
||||
yield None
|
||||
continue
|
||||
lines = self._read_ready(fh)
|
||||
for line, end in lines:
|
||||
self._pos = end
|
||||
parsed = self._parse(line)
|
||||
if parsed is not None:
|
||||
yield parsed
|
||||
if self._needs_reopen() and not lines:
|
||||
fh.close()
|
||||
self._fh = None
|
||||
continue
|
||||
if not lines:
|
||||
self.idle_wait(self.poll)
|
||||
yield None
|
||||
finally:
|
||||
self.close()
|
||||
|
||||
def close(self):
|
||||
if self._fh:
|
||||
self._fh.close()
|
||||
self._fh = None
|
||||
403
controller/leaf_controller/main.py
Normal file
403
controller/leaf_controller/main.py
Normal file
@@ -0,0 +1,403 @@
|
||||
"""Durable alert decisions with a separate, serialized VyOS reconciliation worker."""
|
||||
from __future__ import annotations
|
||||
import argparse
|
||||
import fcntl
|
||||
import glob
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import socket
|
||||
import threading
|
||||
import tempfile
|
||||
import time
|
||||
from .alertsock import AlertSocket
|
||||
from .config import load_config
|
||||
from .eve import EveTailer
|
||||
from .metrics import Metrics
|
||||
from .policy import Policy
|
||||
from .state import State
|
||||
from .vyos_client import VyosApiError, VyosClient
|
||||
|
||||
log = logging.getLogger('leaf')
|
||||
|
||||
class Audit:
|
||||
def __init__(self, path, max_bytes=0, keep=3):
|
||||
os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True)
|
||||
self.path, self.max_bytes, self.keep = path, int(max_bytes), int(keep)
|
||||
self.fh = open(path, 'a')
|
||||
self.lock = threading.Lock()
|
||||
def write(self, obj):
|
||||
with self.lock:
|
||||
self.fh.write(json.dumps(dict(ts=time.time(), **obj)) + '\n')
|
||||
self.fh.flush()
|
||||
if self.max_bytes and self.fh.tell() >= self.max_bytes:
|
||||
self._rotate_locked()
|
||||
def _rotate_locked(self):
|
||||
"""Rotate at the configured size and retain the newest archives."""
|
||||
self.fh.close()
|
||||
stamp = str(time.time_ns()).zfill(20)
|
||||
dst = self.path + '.' + stamp
|
||||
n = 2
|
||||
while os.path.exists(dst): # tolerate a frozen/coarse clock
|
||||
dst = self.path + '.' + stamp + f'.{n:08d}'
|
||||
n += 1
|
||||
os.replace(self.path, dst)
|
||||
self.fh = open(self.path, 'a')
|
||||
rotated = sorted(glob.glob(self.path + '.*'), key=lambda f: (os.stat(f).st_mtime_ns, f))
|
||||
doomed = rotated[:-self.keep] if self.keep else rotated
|
||||
for old in doomed:
|
||||
try:
|
||||
os.remove(old)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
log.info('rotated audit log, keeping %d rotated file(s)', self.keep or 0)
|
||||
def close(self):
|
||||
self.fh.close()
|
||||
|
||||
class Controller:
|
||||
def __init__(self, cfg, dry_run=False):
|
||||
self.cfg, self.dry_run = cfg, dry_run
|
||||
self._closed = False
|
||||
self.group = cfg.get('block_group', 'IDS-BLOCK')
|
||||
self._scratch = tempfile.TemporaryDirectory(prefix='leaf-dry-run-') if dry_run else None
|
||||
state_path = os.path.join(self._scratch.name, 'state.sqlite') if self._scratch else cfg.get('state_db')
|
||||
os.makedirs(os.path.dirname(os.path.abspath(state_path)), exist_ok=True)
|
||||
self._lock = open(state_path + '.lock', 'a')
|
||||
try:
|
||||
fcntl.flock(self._lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError:
|
||||
self._lock.close()
|
||||
raise SystemExit('another controller owns this state database')
|
||||
self.state = State(state_path)
|
||||
self.policy = Policy(cfg, self.state)
|
||||
self.audit = Audit(os.path.join(self._scratch.name, "audit.jsonl") if dry_run else cfg.get("audit_log"),
|
||||
int(cfg.get("audit.max_bytes", 16777216)),
|
||||
int(cfg.get("audit.keep", 3)))
|
||||
# metrics: counting only; stats records never touch state/audit/API.
|
||||
# health line once per full stats window, capture.log every
|
||||
# metrics.capture_interval seconds (same bounded scheme as audit).
|
||||
self.metrics = Metrics(float(cfg.get("metrics.reconcile_interval", 30.0)))
|
||||
self._metrics_log = logging.getLogger("leaf.metrics")
|
||||
self._metrics_log.setLevel(getattr(
|
||||
logging, str(cfg.get("metrics.log_level", "INFO")).upper(), logging.INFO))
|
||||
self._capture_path = (cfg.get("metrics.capture_log")
|
||||
or os.path.join(os.path.dirname(
|
||||
os.path.abspath(cfg.get("audit_log"))), "capture.log"))
|
||||
if dry_run:
|
||||
self._capture_path = os.path.join(self._scratch.name, "capture.jsonl")
|
||||
self._capture_interval = max(0.0, float(cfg.get("metrics.capture_interval", 3600)))
|
||||
self._capture_log = None
|
||||
self._next_capture = 0.0
|
||||
self.client = None if dry_run else VyosClient(
|
||||
cfg.vyos_url(), cfg.vyos_key(), ca_bundle=cfg.get('vyos.ca_bundle'),
|
||||
tls_verify=cfg.get('vyos.tls_verify', True),
|
||||
cert_pin=cfg.get('vyos.tls_cert_pin') or None,
|
||||
timeout=float(cfg.get('vyos.timeout', 75)),
|
||||
retries=int(cfg.get('vyos.retries', 2)))
|
||||
self.stop = threading.Event()
|
||||
self.wake = threading.Event()
|
||||
self._last_sync = 0.0
|
||||
self._initial_sync = True
|
||||
self._worker = None
|
||||
# socket ingestion: suricata unix_stream output connects to our
|
||||
# listener; empty path disables (offline/mock runs keep file tailing)
|
||||
sock_path = cfg.get('alert_socket.path')
|
||||
self._sock = None
|
||||
if sock_path and not dry_run:
|
||||
self._sock = AlertSocket(
|
||||
sock_path, int(cfg.get('alert_socket.queue_size') or 1000))
|
||||
|
||||
def router_members(self):
|
||||
return set(self.client.op_with_retry(self.client.group_members, self.group))
|
||||
|
||||
def startup(self):
|
||||
if self.dry_run:
|
||||
return
|
||||
info = self.client.info()
|
||||
log.info('connected to VyOS API: version=%s', info.get('version', '?'))
|
||||
if self.cfg.get('ensure_primitives', True):
|
||||
# Both metadata actions use one group; validate the corresponding hooks.
|
||||
self.client.verify_enforcement(self.group, self.cfg.get('policy.block_src', False))
|
||||
self.sync_router_state()
|
||||
|
||||
def sync_router_state(self):
|
||||
"""One batched commit, then read back before acknowledging any change."""
|
||||
if self.dry_run:
|
||||
return
|
||||
now = time.time()
|
||||
members = self.router_members()
|
||||
rows = {r.ip: r for r in self.state.all_blocks()}
|
||||
# Only adopt on startup, never resurrect known expired entries.
|
||||
foreign = members - rows.keys()
|
||||
if self._initial_sync and self.cfg.get('adopt_foreign_members', True):
|
||||
for ip in sorted(foreign):
|
||||
if self.cfg.is_allowed_ip(ip):
|
||||
log.warning('leaving protected foreign router member unowned: %s', ip)
|
||||
continue
|
||||
if self.state.count_active() >= int(self.cfg.get('policy.max_blocks', 50)):
|
||||
log.warning('foreign member adoption capped; leaving unowned: %s', ip)
|
||||
continue
|
||||
self.state.upsert_hit(ip, 'dest', 0, float(self.cfg.get('policy.ttl_seconds', 3600)), now)
|
||||
self.state.set_applied(ip)
|
||||
self.audit.write({'action': 'adopt', 'ip': ip})
|
||||
log.info('adopted foreign member %s into state', ip)
|
||||
rows = {r.ip: r for r in self.state.all_blocks()}
|
||||
self._initial_sync = False
|
||||
active = {ip for ip, r in rows.items() if r.expires > now and not self.cfg.is_allowed_ip(ip)}
|
||||
expired = rows.keys() - active
|
||||
adds, deletes = active - members, set(expired) & members
|
||||
if not self.cfg.get('adopt_foreign_members', True):
|
||||
deletes |= foreign
|
||||
ops = [self.client.member_op('set', self.group, ip) for ip in sorted(adds)]
|
||||
ops += [self.client.member_op('delete', self.group, ip) for ip in sorted(deletes)]
|
||||
if ops:
|
||||
# A timeout may mean the commit succeeded: next pass reads before retrying.
|
||||
try:
|
||||
self.state.set_router_dirty(True)
|
||||
self.client.configure(ops)
|
||||
except Exception:
|
||||
self.audit.write({'action': 'reconcile', 'ok': False,
|
||||
'adds': sorted(adds), 'deletes': sorted(deletes)})
|
||||
raise
|
||||
members = self.router_members()
|
||||
if not adds <= members or deletes & members:
|
||||
raise VyosApiError('router readback does not match requested membership changes')
|
||||
if self.cfg.get('save_config') and self.state.router_dirty():
|
||||
self.client.save_config()
|
||||
self.state.set_router_dirty(False)
|
||||
for ip in active & members:
|
||||
row = rows[ip]
|
||||
if not row.applied or ip in adds:
|
||||
self.state.set_applied(ip)
|
||||
self.audit.write({'action': 'block', 'ip': ip, 'kind': row.kind,
|
||||
'sigs': row.sigs, 'expires': row.expires,
|
||||
'ok': True, 'verified': True})
|
||||
log.info('BLOCKED %s (%s), router membership verified', ip, row.kind)
|
||||
for ip in expired - members:
|
||||
row = rows[ip]
|
||||
# Do not delete a fresh decision made during the slow router request.
|
||||
removed = self.state.remove(ip, expired_before=now)
|
||||
if removed:
|
||||
self.audit.write({'action': 'unblock', 'ip': ip, 'kind': row.kind,
|
||||
'reason': 'ttl-expired', 'ok': True, 'verified': True})
|
||||
log.info('UNBLOCKED %s, router absence verified', ip)
|
||||
elif self.cfg.is_allowed_ip(ip):
|
||||
# A newly protected address must not remain locally desired.
|
||||
self.state.remove(ip)
|
||||
else:
|
||||
self.state.set_applied(ip, 0)
|
||||
self.wake.set()
|
||||
self._last_sync = time.time()
|
||||
|
||||
def _route(self, event):
|
||||
"""Dispatch by event_type; only alerts can act. Stats records are
|
||||
counting-only: they must never reach policy/state/audit/API."""
|
||||
etype = event.get('event_type')
|
||||
if etype == 'alert':
|
||||
self.metrics.note_alert()
|
||||
self.handle_event(event)
|
||||
elif etype == 'stats':
|
||||
try:
|
||||
self.metrics.note_stats(event)
|
||||
except (TypeError, ValueError, AttributeError, OverflowError, RecursionError):
|
||||
log.warning('ignoring malformed stats record')
|
||||
self.metrics.note_other()
|
||||
else:
|
||||
self.metrics.note_other()
|
||||
|
||||
def _sock_desc(self):
|
||||
if self._sock is None:
|
||||
return 'off(eve-file)'
|
||||
c = self._sock.counts
|
||||
return ('rec:%d conn:%d qdrop:%d perr:%d other:%d'
|
||||
% (c['records'], c['connections'], c['queue_drops'],
|
||||
c['parse_errors'], c['other']))
|
||||
|
||||
def _emit_metrics(self, now):
|
||||
"""One health line per full stats window; capture.log line on cadence.
|
||||
Read-only against controller state (count_active is a query)."""
|
||||
rep = self.metrics.report(now)
|
||||
if rep is None:
|
||||
return
|
||||
if rep['init']:
|
||||
self._metrics_log.info('metrics baseline: %s',
|
||||
json.dumps(rep['stats'], separators=(',', ':')))
|
||||
return
|
||||
flag = ' (counters reset: suricata restart)' if rep['restart'] else ''
|
||||
self._metrics_log.info(
|
||||
'health: alerts=%d cps=%.1f eps=%.1f dropped+=%d errors+=%d '
|
||||
'invalid+=%d suppressed+=%d overflow+=%d blocks=%d uptime=%ds sock=%s%s',
|
||||
rep['alerts'], rep['cps'], rep['eps'], rep['dropped'],
|
||||
rep['delta']['capture_errors'], rep['delta']['invalid'],
|
||||
rep['delta']['suppressed'], rep['delta']['queue_overflow'],
|
||||
self.state.count_active(), rep['stats']['uptime'],
|
||||
self._sock_desc(), flag)
|
||||
if now >= self._next_capture:
|
||||
self._next_capture = now + self._capture_interval
|
||||
if self._capture_log is None:
|
||||
self._capture_log = Audit(
|
||||
self._capture_path,
|
||||
int(self.cfg.get('metrics.capture_max_bytes', 10485760)),
|
||||
int(self.cfg.get('metrics.capture_log_keep', 6)))
|
||||
self._capture_log.write({'window': rep['window'],
|
||||
'alerts': rep['alerts'],
|
||||
'cps': rep['cps'], 'eps': rep['eps'],
|
||||
'dropped': rep['dropped'],
|
||||
'delta': rep['delta'],
|
||||
'stats': rep['stats']})
|
||||
|
||||
def handle_event(self, event):
|
||||
try:
|
||||
dec = self.policy.evaluate(event)
|
||||
except (TypeError, ValueError, AttributeError, OverflowError, RecursionError):
|
||||
log.warning('ignoring malformed alert record')
|
||||
return
|
||||
if not dec:
|
||||
return
|
||||
existing = self.state.get(dec.ip)
|
||||
if (not existing or existing.expires <= time.time()) and self.state.count_active() >= int(self.cfg.get('policy.max_blocks', 50)):
|
||||
self.audit.write({'action': 'guard', 'ip': dec.ip, 'sid': dec.sid, 'reason': 'max_blocks'})
|
||||
return
|
||||
row = self.state.upsert_hit(dec.ip, dec.kind, dec.sid,
|
||||
float(self.cfg.get('policy.ttl_seconds', 3600)))
|
||||
if not row.applied:
|
||||
self.audit.write({'action': 'decision', 'ip': dec.ip, 'kind': dec.kind,
|
||||
'sid': dec.sid, 'expires': row.expires, 'dry_run': self.dry_run})
|
||||
if self.dry_run:
|
||||
if not row.applied:
|
||||
self.audit.write({'action': 'block', 'ip': dec.ip, 'kind': dec.kind, 'dry_run': True})
|
||||
self.state.set_applied(dec.ip)
|
||||
else:
|
||||
self.wake.set()
|
||||
|
||||
def housekeeping(self):
|
||||
now = time.time()
|
||||
self.state.prune_hits(float(self.cfg.get('policy.window_seconds', 300)) * 2, now)
|
||||
if self.dry_run:
|
||||
for row in self.state.expired(now):
|
||||
self.state.remove(row.ip, expired_before=now)
|
||||
return
|
||||
if (self.cfg.get('save_config') and self.state.router_dirty()) or self.state.unapplied() or self.state.expired(now) or now - self._last_sync >= float(self.cfg.get('sync_interval', 300)):
|
||||
self.sync_router_state()
|
||||
|
||||
def _router_loop(self):
|
||||
ready = False
|
||||
try:
|
||||
while not self.stop.is_set():
|
||||
try:
|
||||
if not ready:
|
||||
self.startup()
|
||||
ready = True
|
||||
else:
|
||||
self.housekeeping()
|
||||
except Exception as exc:
|
||||
log.error('router reconciliation failed; decisions retained for retry: %s', exc)
|
||||
self.wake.wait(float(self.cfg.get('router_interval', 2)))
|
||||
self.wake.clear()
|
||||
# A short batching window combines bursts into one commit.
|
||||
self.stop.wait(float(self.cfg.get('batch_delay', 0.2)))
|
||||
finally:
|
||||
self.state.close()
|
||||
|
||||
def _save_checkpoint(self, tailer):
|
||||
if self.dry_run or tailer._ino is None:
|
||||
return
|
||||
self.state.save_checkpoint(tailer._ino[0], tailer._ino[1], tailer._pos)
|
||||
|
||||
def run(self):
|
||||
if self._sock:
|
||||
# bind BEFORE suricata connects: the sensor unit's ordering
|
||||
# guarantees it (Wants/After leaf-controller). a bind failure is
|
||||
# fatal, not soft: a dead listener would mean silent alert loss.
|
||||
try:
|
||||
self._sock.bind()
|
||||
except OSError as exc:
|
||||
raise SystemExit('cannot bind alert socket %s: %s'
|
||||
% (self._sock.path, exc))
|
||||
self._sock.start()
|
||||
log.info('listening for sensor alerts on %s', self._sock.path)
|
||||
ckpt = self.state.get_checkpoint() if (not self.dry_run and self.cfg.get('eve.checkpoint', True)) else None
|
||||
tailer = EveTailer(self.cfg.get('eve.path'), float(self.cfg.get('poll_interval', 0.2)),
|
||||
from_start=self.dry_run, checkpoint=ckpt,
|
||||
idle_wait=self._sock.available.wait if self._sock else self.stop.wait)
|
||||
# Establish the live position before any slow API startup calls.
|
||||
tailer._open()
|
||||
self._save_checkpoint(tailer)
|
||||
notify = os.environ.get('NOTIFY_SOCKET')
|
||||
if notify:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as ready:
|
||||
ready.connect('\0' + notify[1:] if notify.startswith('@') else notify)
|
||||
ready.sendall(b'READY=1')
|
||||
if not self.dry_run:
|
||||
self._worker = threading.Thread(target=self._router_loop, name='vyos-worker')
|
||||
self._worker.start()
|
||||
log.info('watching %s (mode=%s)', self.cfg.get('eve.path'), self.cfg.get('policy.mode'))
|
||||
try:
|
||||
for event in tailer.events():
|
||||
if self.stop.is_set():
|
||||
break
|
||||
if self._sock:
|
||||
for rec in self._sock.drain():
|
||||
self._route(rec)
|
||||
if event is not None:
|
||||
# Socket mode has exactly one alert ingress; the file carries stats.
|
||||
if not self._sock or event.get('event_type') != 'alert':
|
||||
identity = hashlib.sha256(json.dumps(event, sort_keys=True).encode()).hexdigest()
|
||||
self.state._local.event_key = '%s:%s:%s:%s' % (*tailer._ino, tailer._pos, identity)
|
||||
try:
|
||||
self._route(event)
|
||||
finally:
|
||||
self.state._local.event_key = None
|
||||
self._emit_metrics(time.time())
|
||||
# Offset follows the delivered record, never the unread batch.
|
||||
# File mode is at-least-once across a crash before checkpoint.
|
||||
self._save_checkpoint(tailer)
|
||||
elif self.dry_run:
|
||||
self.housekeeping()
|
||||
finally:
|
||||
tailer.close()
|
||||
self.close()
|
||||
|
||||
def close(self):
|
||||
if self._closed:
|
||||
return
|
||||
self._closed = True
|
||||
if self._sock:
|
||||
self._sock.close()
|
||||
# Stop intake first, then persist every accepted queued alert.
|
||||
for rec in self._sock.drain(limit=self._sock.q.qsize()):
|
||||
self._route(rec)
|
||||
self.stop.set()
|
||||
self.wake.set()
|
||||
if self._worker:
|
||||
self._worker.join()
|
||||
self._worker = None
|
||||
self.state.close()
|
||||
self.audit.close()
|
||||
if self._capture_log:
|
||||
self._capture_log.close()
|
||||
self._lock.close()
|
||||
if self._scratch:
|
||||
self._scratch.cleanup()
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--config', default=os.path.join(os.path.dirname(__file__), '..', 'config.yaml'))
|
||||
ap.add_argument('--dry-run', action='store_true')
|
||||
ap.add_argument('-v', '--verbose', action='store_true')
|
||||
args = ap.parse_args(argv)
|
||||
logging.basicConfig(level=logging.DEBUG if args.verbose else logging.INFO,
|
||||
format='%(asctime)s %(levelname)s %(name)s: %(message)s')
|
||||
ctl = Controller(load_config(args.config), args.dry_run)
|
||||
def shutdown(*args):
|
||||
ctl.stop.set()
|
||||
ctl.wake.set()
|
||||
signal.signal(signal.SIGTERM, shutdown)
|
||||
signal.signal(signal.SIGINT, shutdown)
|
||||
ctl.run()
|
||||
return 0
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
105
controller/leaf_controller/metrics.py
Normal file
105
controller/leaf_controller/metrics.py
Normal file
@@ -0,0 +1,105 @@
|
||||
"""Counting-only metrics from Suricata stats records.
|
||||
|
||||
Stats records update counters without changing policy, SQLite state or
|
||||
router configuration. The caller controls when reports are logged.
|
||||
|
||||
Live eve.json (Suricata 7, threads: no + totals: yes) carries
|
||||
capture/decoder/detect sub-objects on each stats record; there is no
|
||||
'ifaces' object at totals level, so capture packets come from
|
||||
capture.kernel_packets (the real field on the wire).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
class Metrics:
|
||||
"""Cumulative Suricata counters; one delta report per full window.
|
||||
|
||||
The report gate exists because stats records arrive every
|
||||
metrics.reconcile_interval seconds; a backlog drain can deliver two
|
||||
stats records back-to-back, and the journal must still see at most one
|
||||
line per window.
|
||||
"""
|
||||
|
||||
def __init__(self, reconcile_interval: float = 30.0):
|
||||
self.interval = max(1.0, float(reconcile_interval))
|
||||
self.alerts = 0
|
||||
self.non_alert_events = 0
|
||||
self.stats_seen = 0
|
||||
self._totals: dict[str, int] = {}
|
||||
self._baseline: dict[str, int] | None = None
|
||||
self._last = 0.0
|
||||
|
||||
def note_alert(self) -> None:
|
||||
self.alerts += 1
|
||||
|
||||
def note_other(self) -> None:
|
||||
self.non_alert_events += 1
|
||||
|
||||
def note_stats(self, event: dict) -> None:
|
||||
"""Extract the cumulative counters; nothing else is logged or stored.
|
||||
|
||||
Live suricata emits both thread records (flush=False: per-thread
|
||||
counters, 'engine' key) and a totals record (uptime/memcap_pressure
|
||||
keys, 'detect' key). Totals are authoritative for rates; thread
|
||||
records are only accepted if no totals have ever been seen.
|
||||
"""
|
||||
stats = event.get("stats") or {}
|
||||
cap = stats.get("capture") or {}
|
||||
dec = stats.get("decoder") or {}
|
||||
det = stats.get("detect") or stats.get("engine") or {}
|
||||
totals = {
|
||||
"uptime": int(stats.get("uptime", 0)),
|
||||
"memcap_pressure": int(stats.get("memcap_pressure", 0)),
|
||||
"packets": int(cap.get("kernel_packets", 0)),
|
||||
"dropped": int(cap.get("kernel_drops", 0)),
|
||||
"capture_errors": int(cap.get("errors", 0)),
|
||||
"decoded": int(dec.get("pkts", 0)),
|
||||
"bytes": int(dec.get("bytes", 0)),
|
||||
"invalid": int(dec.get("invalid", 0)),
|
||||
"engine_alerts": int(det.get("alert", 0)),
|
||||
"suppressed": int(det.get("alerts_suppressed", 0)),
|
||||
"queue_overflow": int(det.get("alert_queue_overflow", 0)),
|
||||
}
|
||||
if "uptime" in stats or not self._totals:
|
||||
self._totals = totals
|
||||
self.stats_seen += 1
|
||||
|
||||
def report(self, now: float) -> dict | None:
|
||||
"""None unless a full flush window has passed since the last report.
|
||||
|
||||
First stats record only adopts the cumulative baseline: deltas are
|
||||
meaningless before the second stats tick (counters run since process
|
||||
start), so the first window must not pretend to be a measured one.
|
||||
Suricata's counters reset on restart: deltas never go negative, the
|
||||
reset is flagged instead.
|
||||
"""
|
||||
if not self._totals:
|
||||
return None
|
||||
if self._baseline is None:
|
||||
self._baseline = dict(self._totals)
|
||||
self._last = now
|
||||
return {"init": True, "alerts": self.alerts, "stats": dict(self._totals)}
|
||||
if now - self._last < self.interval * 0.75: # tolerate flush jitter
|
||||
return None
|
||||
window = now - self._last
|
||||
delta, restart = {}, False
|
||||
for key, value in self._totals.items():
|
||||
n = value - self._baseline.get(key, 0)
|
||||
if n < 0:
|
||||
restart = True
|
||||
n = value
|
||||
delta[key] = n
|
||||
self._baseline = dict(self._totals)
|
||||
self._last = now
|
||||
return {
|
||||
"init": False,
|
||||
"restart": restart,
|
||||
"window": round(window, 1),
|
||||
"alerts": self.alerts,
|
||||
"engine_alerts_delta": delta["engine_alerts"],
|
||||
"cps": round(delta["packets"] / window, 1), # captured pps
|
||||
"eps": round(delta["decoded"] / window, 1), # decoded pps
|
||||
"dropped": delta["dropped"],
|
||||
"delta": delta,
|
||||
"stats": dict(self._totals),
|
||||
}
|
||||
22
controller/leaf_controller/ops.py
Normal file
22
controller/leaf_controller/ops.py
Normal file
@@ -0,0 +1,22 @@
|
||||
"""Sensor-side operational helpers sharing the installed controller trust settings."""
|
||||
import os
|
||||
from .config import load_config, read_key_env
|
||||
from .vyos_client import VyosClient
|
||||
|
||||
|
||||
def configured_client():
|
||||
cfg=load_config('/etc/leaf-controller/config.yaml')
|
||||
name=cfg.get('vyos.key_env','LEAF_VYOS_KEY')
|
||||
key=os.environ.get(name) or read_key_env('/etc/leaf-controller/env', name)
|
||||
client=VyosClient(cfg.vyos_url(),key,cert_pin=cfg.get('vyos.tls_cert_pin') or None,
|
||||
ca_bundle=cfg.get('vyos.ca_bundle'),tls_verify=cfg.get('vyos.tls_verify',True),
|
||||
timeout=float(cfg.get('vyos.timeout',75)),retries=int(cfg.get('vyos.retries',2)))
|
||||
return cfg,client
|
||||
|
||||
|
||||
def redact(node):
|
||||
if isinstance(node,dict):
|
||||
return {k: '[REDACTED]' if any(s in k.lower() for s in ('key','password','secret','token','private'))
|
||||
else redact(v) for k,v in node.items()}
|
||||
if isinstance(node,list):return [redact(v) for v in node]
|
||||
return node
|
||||
91
controller/leaf_controller/policy.py
Normal file
91
controller/leaf_controller/policy.py
Normal file
@@ -0,0 +1,91 @@
|
||||
"""Alert -> decision. Rule metadata leaf_action drives intent, policy gates it."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
log = logging.getLogger("leaf.policy")
|
||||
|
||||
|
||||
@dataclass
|
||||
class Decision:
|
||||
ip: str
|
||||
kind: str # dest | src
|
||||
sid: int
|
||||
reason: str
|
||||
sig: str = ""
|
||||
hits: int = 1
|
||||
|
||||
|
||||
class Policy:
|
||||
def __init__(self, cfg, state):
|
||||
self.cfg = cfg
|
||||
self.state = state
|
||||
|
||||
@staticmethod
|
||||
def metadata_actions(event: dict) -> list[str]:
|
||||
md = event.get("alert", {}).get("metadata", {}) or {}
|
||||
vals = md.get("leaf_action", []) or []
|
||||
if isinstance(vals, str):
|
||||
vals = [vals]
|
||||
return [str(v).strip().lower() for v in vals]
|
||||
|
||||
def evaluate(self, event: dict, now: float | None = None) -> Decision | None:
|
||||
"""Return Decision to block, or None. `event` = eve alert record."""
|
||||
now = now or time.time()
|
||||
if event.get("event_type") != "alert":
|
||||
return None
|
||||
alert = event.get("alert", {})
|
||||
sid = int(alert.get("signature_id", -1))
|
||||
sig = str(alert.get("signature", ""))
|
||||
actions = self.metadata_actions(event)
|
||||
if not actions:
|
||||
return None
|
||||
|
||||
sids_only = [int(s) for s in self.cfg.get("policy.sids_only", [])]
|
||||
sids_deny = [int(s) for s in self.cfg.get("policy.sids_deny", [])]
|
||||
if sids_only and sid not in sids_only:
|
||||
return None
|
||||
if sid in sids_deny:
|
||||
return None
|
||||
|
||||
mode = self.cfg.get("policy.mode", "threshold")
|
||||
src_ip = str(event.get("src_ip", ""))
|
||||
dest_ip = str(event.get("dest_ip", ""))
|
||||
|
||||
candidates: list[tuple[str, str]] = []
|
||||
if "block" in actions:
|
||||
candidates.append((dest_ip, "dest"))
|
||||
if "block_src" in actions and self.cfg.get("policy.block_src", False):
|
||||
candidates.append((src_ip, "src"))
|
||||
|
||||
for ip, kind in candidates:
|
||||
dec = self._gate(ip, kind, sid, sig, now, mode)
|
||||
if dec:
|
||||
return dec
|
||||
return None
|
||||
|
||||
def _gate(self, ip: str, kind: str, sid: int, sig: str,
|
||||
now: float, mode: str) -> Decision | None:
|
||||
if not ip:
|
||||
return None
|
||||
import ipaddress
|
||||
try:
|
||||
if ipaddress.ip_address(ip).version != 4:
|
||||
return None
|
||||
except ValueError:
|
||||
return None
|
||||
if self.cfg.is_allowed_ip(ip):
|
||||
log.debug("allowlisted, not blocking: %s (sid=%s)", ip, sid)
|
||||
return None
|
||||
self.state.record_hit(sid, ip, now)
|
||||
min_hits = int(self.cfg.get("policy.min_hits", 1))
|
||||
window = float(self.cfg.get("policy.window_seconds", 300))
|
||||
hits = self.state.hit_count(sid, ip, window, now)
|
||||
if mode == "immediate" or hits >= min_hits:
|
||||
return Decision(ip=ip, kind=kind, sid=sid, sig=sig, hits=hits,
|
||||
reason=f"{mode}:{hits} hit(s) sid={sid} in {window:.0f}s")
|
||||
log.info("hit %d/%d for %s (sid=%s) — not yet blocking", hits, min_hits, ip, sid)
|
||||
return None
|
||||
176
controller/leaf_controller/state.py
Normal file
176
controller/leaf_controller/state.py
Normal file
@@ -0,0 +1,176 @@
|
||||
"""sqlite-backed state: block list w/ TTL, hit windowing, applied tracking."""
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
import time
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
|
||||
SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS controller_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS blocks (
|
||||
ip TEXT PRIMARY KEY, -- ip/cidr as pushed to VyOS (plain ip = /32 semantics)
|
||||
kind TEXT NOT NULL, -- dest|src
|
||||
sigs TEXT NOT NULL DEFAULT '',-- csv of triggering sids
|
||||
first_seen REAL NOT NULL,
|
||||
last_hit REAL NOT NULL,
|
||||
expires REAL NOT NULL,
|
||||
applied INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS hits (
|
||||
sid INTEGER NOT NULL,
|
||||
ip TEXT NOT NULL,
|
||||
ts REAL NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS hits_idx ON hits (sid, ip, ts);
|
||||
CREATE TABLE IF NOT EXISTS eve_ckpt (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
dev INTEGER NOT NULL, -- st_dev/st_ino of the EVE file last read
|
||||
ino INTEGER NOT NULL,
|
||||
pos INTEGER NOT NULL, -- byte offset of the next unread record
|
||||
updated_at REAL NOT NULL
|
||||
);
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class BlockRow:
|
||||
ip: str
|
||||
kind: str
|
||||
sigs: str
|
||||
first_seen: float
|
||||
last_hit: float
|
||||
expires: float
|
||||
applied: int
|
||||
|
||||
|
||||
class State:
|
||||
def __init__(self, path: str):
|
||||
self.path = path
|
||||
self._local = threading.local()
|
||||
self.con.executescript(SCHEMA)
|
||||
columns = {r[1] for r in self.con.execute("PRAGMA table_info(hits)")}
|
||||
if "event_key" not in columns:
|
||||
self.con.execute("ALTER TABLE hits ADD COLUMN event_key TEXT")
|
||||
self.con.execute("CREATE UNIQUE INDEX IF NOT EXISTS hits_event ON hits(event_key)")
|
||||
self.con.commit()
|
||||
|
||||
@property
|
||||
def con(self):
|
||||
# Alert processing and router I/O have independent connections.
|
||||
if not hasattr(self._local, "con"):
|
||||
con = sqlite3.connect(self.path, timeout=10)
|
||||
con.execute("PRAGMA journal_mode=WAL")
|
||||
con.execute("PRAGMA busy_timeout=10000")
|
||||
self._local.con = con
|
||||
return self._local.con
|
||||
|
||||
def close(self):
|
||||
if hasattr(self._local, "con"):
|
||||
self._local.con.close()
|
||||
del self._local.con
|
||||
|
||||
# ---- hit windowing (threshold mode) ----
|
||||
def record_hit(self, sid: int, ip: str, ts: float | None = None) -> None:
|
||||
event_key = getattr(self._local, "event_key", None)
|
||||
if event_key is not None:
|
||||
event_key = f'{event_key}:{sid}:{ip}'
|
||||
self.con.execute("INSERT OR IGNORE INTO hits (sid, ip, ts, event_key) VALUES (?,?,?,?)",
|
||||
(sid, ip, ts or time.time(), event_key))
|
||||
self.con.commit()
|
||||
|
||||
def hit_count(self, sid: int, ip: str, window: float, now: float | None = None) -> int:
|
||||
now = now or time.time()
|
||||
(n,) = self.con.execute(
|
||||
"SELECT count(*) FROM hits WHERE sid=? AND ip=? AND ts>=?",
|
||||
(sid, ip, now - window)).fetchone()
|
||||
return n
|
||||
|
||||
def prune_hits(self, max_age: float, now: float | None = None) -> None:
|
||||
now = now or time.time()
|
||||
self.con.execute("DELETE FROM hits WHERE ts < ?", (now - max_age,))
|
||||
self.con.commit()
|
||||
|
||||
# ---- blocks ----
|
||||
def _row(self, r: sqlite3.Row | tuple) -> BlockRow:
|
||||
return BlockRow(*r)
|
||||
|
||||
def get(self, ip: str) -> BlockRow | None:
|
||||
r = self.con.execute("SELECT * FROM blocks WHERE ip=?", (ip,)).fetchone()
|
||||
return self._row(r) if r else None
|
||||
|
||||
def upsert_hit(self, ip: str, kind: str, sid: int, ttl: float,
|
||||
now: float | None = None) -> BlockRow:
|
||||
"""Create block entry or refresh expiry/last_hit of existing one."""
|
||||
now = now or time.time()
|
||||
# Serialize the read/modify/write with expiry cleanup in the worker.
|
||||
with self.con:
|
||||
self.con.execute("BEGIN IMMEDIATE")
|
||||
row = self.get(ip)
|
||||
if row:
|
||||
sigs = row.sigs
|
||||
if str(sid) not in sigs.split(","):
|
||||
sigs = f"{sigs},{sid}" if sigs else str(sid)
|
||||
self.con.execute(
|
||||
"UPDATE blocks SET expires=?, last_hit=?, sigs=? WHERE ip=?",
|
||||
(now + ttl, now, sigs, ip))
|
||||
else:
|
||||
self.con.execute("INSERT INTO blocks VALUES (?,?,?,?,?,?,0)",
|
||||
(ip, kind, str(sid), now, now, now + ttl))
|
||||
result = self.get(ip)
|
||||
return result
|
||||
|
||||
def set_applied(self, ip: str, applied: int = 1) -> None:
|
||||
self.con.execute("UPDATE blocks SET applied=? WHERE ip=?", (applied, ip))
|
||||
self.con.commit()
|
||||
|
||||
def remove(self, ip: str, expired_before: float | None = None) -> bool:
|
||||
# A new alert may refresh a block while the router commit is in flight.
|
||||
with self.con:
|
||||
if expired_before is None:
|
||||
cur = self.con.execute("DELETE FROM blocks WHERE ip=?", (ip,))
|
||||
else:
|
||||
cur = self.con.execute("DELETE FROM blocks WHERE ip=? AND expires<=?",
|
||||
(ip, expired_before))
|
||||
if cur.rowcount:
|
||||
self.con.execute("DELETE FROM hits WHERE ip=?", (ip,))
|
||||
return bool(cur.rowcount)
|
||||
|
||||
def all_blocks(self) -> list[BlockRow]:
|
||||
return [self._row(r) for r in self.con.execute("SELECT * FROM blocks")]
|
||||
|
||||
def expired(self, now: float | None = None) -> list[BlockRow]:
|
||||
now = now or time.time()
|
||||
return [self._row(r) for r in
|
||||
self.con.execute("SELECT * FROM blocks WHERE expires<=?", (now,))]
|
||||
|
||||
def unapplied(self) -> list[BlockRow]:
|
||||
return [self._row(r) for r in
|
||||
self.con.execute("SELECT * FROM blocks WHERE applied=0")]
|
||||
|
||||
def count_active(self, now: float | None = None) -> int:
|
||||
now = now or time.time()
|
||||
(n,) = self.con.execute("SELECT count(*) FROM blocks WHERE expires>?", (now,)).fetchone()
|
||||
return n
|
||||
|
||||
# ---- EVE read checkpoint (survives controller restarts) ----
|
||||
def get_checkpoint(self) -> tuple | None:
|
||||
r = self.con.execute("SELECT dev, ino, pos FROM eve_ckpt WHERE id=1").fetchone()
|
||||
return tuple(r) if r else None
|
||||
|
||||
def save_checkpoint(self, dev: int, ino: int, pos: int, now: float | None = None) -> None:
|
||||
with self.con:
|
||||
self.con.execute(
|
||||
"INSERT INTO eve_ckpt VALUES (1,?,?,?,?) "
|
||||
"ON CONFLICT(id) DO UPDATE SET dev=excluded.dev, ino=excluded.ino, "
|
||||
"pos=excluded.pos, updated_at=excluded.updated_at",
|
||||
(dev, ino, pos, now or time.time()))
|
||||
|
||||
def router_dirty(self):
|
||||
r = self.con.execute("SELECT value FROM controller_meta WHERE key='router_dirty'").fetchone()
|
||||
return bool(r and r[0] == '1')
|
||||
|
||||
def set_router_dirty(self, dirty):
|
||||
with self.con:
|
||||
self.con.execute("INSERT OR REPLACE INTO controller_meta VALUES ('router_dirty', ?)",
|
||||
('1' if dirty else '0',))
|
||||
236
controller/leaf_controller/vyos_client.py
Normal file
236
controller/leaf_controller/vyos_client.py
Normal file
@@ -0,0 +1,236 @@
|
||||
"""Minimal VyOS HTTP API client (Stream 2026.03 / rolling: service https api rest).
|
||||
|
||||
Protocol (docs: automation/vyos-api.html):
|
||||
POST {base}/configure form fields: data=<json>, key=<key>
|
||||
POST {base}/retrieve {"op":"returnValues","path":[tokens]} -> {"returnValues":[...]}
|
||||
POST {base}/show {"op":"show","path":[...]}
|
||||
POST {base}/config-file {"op":"save"}
|
||||
GET {base}/info version/health, no auth
|
||||
Multipart/form-data encoding, exactly what curl -F sends.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import socket
|
||||
import ssl
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import requests
|
||||
from requests.adapters import HTTPAdapter
|
||||
from urllib3.connection import HTTPSConnection
|
||||
from urllib3.connectionpool import HTTPSConnectionPool
|
||||
|
||||
log = logging.getLogger("leaf.vyos")
|
||||
|
||||
|
||||
class VyosApiError(RuntimeError):
|
||||
def __init__(self, msg: str, status: int | None = None, body: Any = None):
|
||||
super().__init__(msg)
|
||||
self.status = status
|
||||
self.body = body
|
||||
|
||||
|
||||
def normalize_pin(pin: str) -> str:
|
||||
"""Accept openssl-style 'AA:BB:..' or plain hex; return lowercase hex digest."""
|
||||
p = pin.strip().lower().removeprefix("sha256:").replace(":", "")
|
||||
if len(p) != 64 or any(c not in "0123456789abcdef" for c in p):
|
||||
raise ValueError("tls_cert_pin must be a sha256 certificate fingerprint (hex)")
|
||||
return p
|
||||
|
||||
|
||||
class PinMismatchError(VyosApiError):
|
||||
pass
|
||||
|
||||
|
||||
class _PinnedHTTPSConnection(HTTPSConnection):
|
||||
"""TLS to an IP-hosted self-signed endpoint, anchored on a pinned cert.
|
||||
|
||||
Hostname verification is impossible for this router's cert (CN=localhost,
|
||||
no IP SAN), so trust comes from comparing the peer's SHA-256 fingerprint
|
||||
against the configured pin. CERT_NONE is safe *because* the pin is the
|
||||
trust anchor: a mismatch aborts the connection before any request.
|
||||
"""
|
||||
leaf_pin: str | None = None
|
||||
|
||||
def connect(self):
|
||||
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
sock = socket.create_connection((self.host, self.port), self.timeout,
|
||||
self.source_address)
|
||||
self.sock = ctx.wrap_socket(sock, server_hostname=self.host)
|
||||
der = self.sock.getpeercert(binary_form=True)
|
||||
if der is None:
|
||||
raise PinMismatchError("TLS handshake returned no peer certificate")
|
||||
digest = hashlib.sha256(der).hexdigest()
|
||||
if digest != self.leaf_pin:
|
||||
self.sock.close()
|
||||
raise PinMismatchError(
|
||||
f"TLS cert fingerprint mismatch for {self.host}; peer is "
|
||||
f"{digest}, pinned is {self.leaf_pin} (possible MiTM?)")
|
||||
# Pin matched: this connection IS authenticated to the pinned anchor.
|
||||
self.is_verified = True
|
||||
return None
|
||||
|
||||
|
||||
class VyosClient:
|
||||
def __init__(self, url: str, key: str, ca_bundle: str | None = None,
|
||||
tls_verify: bool = True, timeout: float = 10.0, retries: int = 3,
|
||||
cert_pin: str | None = None):
|
||||
self.url = url.rstrip("/")
|
||||
self.key = key
|
||||
self.timeout = timeout
|
||||
self.retries = retries
|
||||
self.sess = requests.Session()
|
||||
# Do not bypass the pinned adapter through an environment HTTP proxy.
|
||||
self.sess.trust_env = False
|
||||
if cert_pin:
|
||||
pin = normalize_pin(cert_pin)
|
||||
|
||||
class PinAdapter(HTTPAdapter):
|
||||
def init_poolmanager(self, *args, **kwargs):
|
||||
super().init_poolmanager(*args, **kwargs)
|
||||
conn_cls = type(
|
||||
"PinnedConn", (_PinnedHTTPSConnection,),
|
||||
{"leaf_pin": pin})
|
||||
self.poolmanager.pool_classes_by_scheme = dict(self.poolmanager.pool_classes_by_scheme)
|
||||
self.poolmanager.pool_classes_by_scheme["https"] = type(
|
||||
"PinnedPool", (HTTPSConnectionPool,),
|
||||
{"ConnectionCls": conn_cls})
|
||||
return self.poolmanager
|
||||
|
||||
self.sess.mount("https://", PinAdapter())
|
||||
elif ca_bundle:
|
||||
self.sess.verify = ca_bundle
|
||||
else:
|
||||
self.sess.verify = bool(tls_verify)
|
||||
if self.sess.verify is False:
|
||||
requests.packages.urllib3.disable_warnings() # explicit opt-in only
|
||||
|
||||
# ------------------------------------------------------------------ core
|
||||
def _post(self, endpoint: str, payload: dict, auth: bool = True) -> dict:
|
||||
# multipart/form-data exactly like `curl -F "data=..." -F "key=..."`:
|
||||
# part tuples (None, value) -> no filename -> plain form field
|
||||
fields = {"data": (None, json.dumps(payload))}
|
||||
if auth:
|
||||
fields["key"] = (None, self.key)
|
||||
resp = self.sess.post(f"{self.url}{endpoint}", files=fields, timeout=self.timeout)
|
||||
try:
|
||||
body = resp.json()
|
||||
except ValueError:
|
||||
body = resp.text
|
||||
if resp.status_code >= 400:
|
||||
raise VyosApiError(f"{endpoint} -> HTTP {resp.status_code}: {body!r}",
|
||||
status=resp.status_code, body=body)
|
||||
# VyOS returns success:false + error for semantic failures (often 200/400)
|
||||
if isinstance(body, dict) and body.get("success") is False:
|
||||
raise VyosApiError(f"{endpoint} rejected: {body.get('error')!r}",
|
||||
status=resp.status_code, body=body)
|
||||
if not isinstance(body, dict) or body.get("success") is not True:
|
||||
raise VyosApiError(f"{endpoint}: missing success acknowledgement")
|
||||
return body
|
||||
|
||||
# ------------------------------------------------------------- public API
|
||||
def info(self) -> dict:
|
||||
resp = self.sess.get(f"{self.url}/info", timeout=self.timeout)
|
||||
if resp.status_code >= 400:
|
||||
raise VyosApiError(f"/info -> HTTP {resp.status_code}", status=resp.status_code)
|
||||
body = resp.json()
|
||||
if isinstance(body, dict) and isinstance(body.get("data"), dict):
|
||||
return body["data"]
|
||||
return body
|
||||
|
||||
def configure(self, ops: list[dict] | dict) -> dict:
|
||||
"""One commit: a single op dict or a list of ops (atomic multi-op)."""
|
||||
payload = ops if isinstance(ops, list) else ops
|
||||
return self._post("/configure", payload)
|
||||
|
||||
def retrieve(self, op: str, path: list[str]) -> dict:
|
||||
return self._post("/retrieve", {"op": op, "path": path})
|
||||
|
||||
def group_members(self, group: str) -> list[str]:
|
||||
body = self.retrieve("returnValues",
|
||||
["firewall", "group", "address-group", group, "address"])
|
||||
# live 2026.03 returns {"success":true,"data":[...]}; older builds used
|
||||
# {"returnValues":[...]}. Accept both, never silently hide members.
|
||||
vals = body.get("data")
|
||||
if vals is None:
|
||||
if "returnValues" not in body and "data" not in body:
|
||||
raise VyosApiError("group member response contains no values field")
|
||||
vals = body.get("returnValues") or []
|
||||
if not isinstance(vals, list):
|
||||
vals = [vals] if vals else []
|
||||
return [str(v) for v in vals]
|
||||
|
||||
@staticmethod
|
||||
def member_op(op, group, ip):
|
||||
import ipaddress
|
||||
ipaddress.IPv4Address(ip) # This prototype manages IPv4 address groups only.
|
||||
return {"op": op, "path": ["firewall", "group", "address-group", group, "address", ip]}
|
||||
|
||||
def set_group_member(self, group: str, ip: str) -> dict:
|
||||
return self.configure(self.member_op("set", group, ip))
|
||||
|
||||
def del_group_member(self, group: str, ip: str) -> dict:
|
||||
return self.configure(self.member_op("delete", group, ip))
|
||||
|
||||
def verify_enforcement(self, group, require_src=False):
|
||||
fw = self.retrieve("showConfig", ["firewall"]).get("data", {})
|
||||
if not isinstance(fw, dict):
|
||||
raise VyosApiError("cannot inspect firewall enforcement hooks")
|
||||
directions = set()
|
||||
def walk(obj):
|
||||
if not isinstance(obj, dict):
|
||||
return
|
||||
if obj.get("action") == "drop":
|
||||
for side in ("source", "destination"):
|
||||
if obj.get(side, {}).get("group", {}).get("address-group") == group:
|
||||
directions.add(side)
|
||||
for value in obj.values():
|
||||
if isinstance(value, dict):
|
||||
walk(value)
|
||||
ipv4 = fw.get("ipv4", {})
|
||||
walk(ipv4.get("forward", {}))
|
||||
attached = set()
|
||||
def attachments(obj):
|
||||
if not isinstance(obj, dict):
|
||||
return
|
||||
hook = obj.get("firewall", {})
|
||||
if isinstance(hook, dict) and isinstance(hook.get("name"), str):
|
||||
attached.add(hook["name"])
|
||||
for value in obj.values():
|
||||
if isinstance(value, dict):
|
||||
attachments(value)
|
||||
attachments(fw.get("zone", {}))
|
||||
for name in attached:
|
||||
walk(ipv4.get("name", {}).get(name, {}))
|
||||
required = {"destination", "source"} if require_src else {"destination"}
|
||||
if not required <= directions:
|
||||
raise VyosApiError("missing drop hooks for " + ", ".join(sorted(required - directions)))
|
||||
|
||||
def show(self, command: list[str]) -> str:
|
||||
body = self._post("/show", {"op": "show", "path": command})
|
||||
out = body.get("output")
|
||||
if out is None:
|
||||
out = body.get("data", "")
|
||||
return str(out)
|
||||
|
||||
def save_config(self) -> dict:
|
||||
return self._post("/config-file", {"op": "save"})
|
||||
|
||||
# ----------------------------------------------------- retrying operations
|
||||
def op_with_retry(self, fn, *args, **kwargs) -> Any:
|
||||
last: Exception | None = None
|
||||
for attempt in range(1, self.retries + 1):
|
||||
try:
|
||||
return fn(*args, **kwargs)
|
||||
except (requests.RequestException, VyosApiError) as exc:
|
||||
last = exc
|
||||
wait = min(2 ** attempt, 30)
|
||||
log.warning("vyos op failed (attempt %d/%d): %s — retry in %ss",
|
||||
attempt, self.retries, exc, wait)
|
||||
time.sleep(wait)
|
||||
raise VyosApiError(f"vyos op failed after {self.retries} attempts: {last}")
|
||||
2
controller/requirements.txt
Normal file
2
controller/requirements.txt
Normal file
@@ -0,0 +1,2 @@
|
||||
requests>=2.31
|
||||
PyYAML>=6.0
|
||||
68
docs/ARCHITECTURE.md
Normal file
68
docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Controller architecture
|
||||
|
||||
## Alert ingestion
|
||||
|
||||
Suricata is the client of an AF_UNIX stream socket bound by `AlertSocket` in
|
||||
`alertsock.py`. The reader frames newline-delimited JSON, accepts alert records,
|
||||
and places them on a bounded queue. Oversized or malformed records and queue
|
||||
overflow are counted. A socket-path lock prevents competing controllers from
|
||||
replacing the listener, even when they use different state databases.
|
||||
|
||||
The queue wakes the main loop independently of file polling. The reader does
|
||||
not run policy or wait for router commits. During graceful shutdown, intake is
|
||||
stopped and accepted queued alerts are processed before SQLite closes.
|
||||
|
||||
The normal EVE file contains stats. `EveTailer` in `eve.py` follows complete
|
||||
records, preserves partial writes and drains the old inode during rotation.
|
||||
When socket mode is enabled, file alerts are ignored to avoid counting the same
|
||||
alert through two outputs.
|
||||
|
||||
For file-alert mode, set `alert_socket.path: ""` and configure Suricata to write
|
||||
alerts into the EVE file. Checkpoints advance per delivered record. File-hit
|
||||
identities include record content, inode, offset, SID and candidate address to
|
||||
prevent replay from inflating threshold counts. Processing is at-least-once;
|
||||
a restart cannot automatically recover unread records in an older rotated file.
|
||||
|
||||
## Policy and state
|
||||
|
||||
`policy.py` reads `leaf_action` metadata, applies SID restrictions, rejects
|
||||
protected addresses, and evaluates threshold or immediate mode. `state.py`
|
||||
stores hit windows, desired blocks, expiry times, applied flags, file checkpoints
|
||||
and pending router-save status in SQLite WAL mode.
|
||||
|
||||
The desired block row is persisted before API submission. Qualifying alerts
|
||||
refresh expiry. The active-address cap bounds the number of desired blocks.
|
||||
Source and destination actions share a router group; `kind` records the initial
|
||||
reason rather than choosing a separate group.
|
||||
|
||||
## Router reconciliation
|
||||
|
||||
A separate worker in `main.py` serializes router I/O using its own SQLite
|
||||
connection. Each pass reads actual membership, computes changes, commits a batch,
|
||||
and reads membership back before marking decisions applied or removals complete.
|
||||
|
||||
A timeout may still mean a commit succeeded. The following pass reads state
|
||||
before retrying. Expiry deletion uses a timestamp condition so that a fresh
|
||||
alert received during a slow request does not lose its refreshed decision.
|
||||
Failures retain desired state for retry.
|
||||
|
||||
`vyos_client.py` handles multipart API requests and response validation. TLS can
|
||||
use normal CA verification or a certificate fingerprint checked before sending
|
||||
credentials. Pinned pool classes are private to each client. Environment proxies
|
||||
are disabled for the API session.
|
||||
|
||||
## Metrics and persistence
|
||||
|
||||
Stats records update `metrics.py` counters without entering policy or changing
|
||||
router state. Reports include capture/decoder deltas and ingress counters.
|
||||
`decisions.jsonl` records decisions and verified enforcement changes; `capture.log`
|
||||
contains periodic metric windows. Both logs retain the newest size-bounded archives.
|
||||
|
||||
A zero controller queue-drop count covers that queue only. Suricata can lose
|
||||
alerts before delivery under send-buffer pressure or disconnection. The socket
|
||||
queue is volatile on crash. Durable decisions can be reconciled after restart,
|
||||
but alerts that never reached SQLite cannot be reconstructed from the stats file.
|
||||
|
||||
Dry-run creates temporary state and logs, disables socket ownership, avoids the
|
||||
API, and reads the configured file from its beginning. Use an alert fixture for
|
||||
policy replay; the normal stats-only file contains no alert history.
|
||||
144
docs/INSTALLATION.md
Normal file
144
docs/INSTALLATION.md
Normal file
@@ -0,0 +1,144 @@
|
||||
# Installation and operations
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Use a Linux sensor with systemd, Python 3, `requests`, `PyYAML`, Suricata 7,
|
||||
OpenSSH and tcpdump. Live tests also require Python 3 and ping on the test hosts.
|
||||
The supplied service uses `/usr/bin/python3`; install its dependencies for that
|
||||
interpreter. On Debian-based sensors, use the `python3-requests` and
|
||||
`python3-yaml` packages. A separate virtual environment can be used for local
|
||||
tests with `controller/requirements.txt`.
|
||||
|
||||
The supplied lab configuration uses sensor capture interface `eth1`, P2P
|
||||
address `10.0.249.1/31`, VyOS API `https://10.0.249.0:8443`, and address group
|
||||
`IDS-BLOCK`. Test hosts are `172.19.32.3` and `172.19.32.5`. These are example
|
||||
site settings; inspect and adapt them for another network.
|
||||
|
||||
Provision mirroring, API access and firewall hooks before starting the sensor.
|
||||
Both source and destination drop hooks must be reachable before accept rules
|
||||
that would bypass enforcement. The controller verifies hooks but does not
|
||||
provision routing or mirroring. The file in `vyos/` is an illustrative template,
|
||||
not a complete router configuration.
|
||||
|
||||
## Configuration and credentials
|
||||
|
||||
The installer preserves an existing `/etc/leaf-controller/config.yaml`. For a
|
||||
new installation, it seeds that file from `config/prototype-vyos.yaml`.
|
||||
`controller/config.yaml` is a generic example with different API/TLS settings;
|
||||
it is not the installer's lab default.
|
||||
|
||||
Review these settings before installation:
|
||||
|
||||
- API URL, timeout, retry count and TLS certificate fingerprint or CA bundle.
|
||||
- Protected networks, allowed SIDs, source actions, threshold, TTL and block cap.
|
||||
- Capture interface in the Suricata configuration and service unit.
|
||||
- Socket, EVE, SQLite and audit paths.
|
||||
|
||||
The lab API uses a certificate fingerprint because its self-signed certificate
|
||||
has no matching IP subject alternative name. Verify the fingerprint through a
|
||||
trusted channel before setting `vyos.tls_cert_pin`. For a CA-issued certificate,
|
||||
configure the appropriate CA bundle and hostname. Keep TLS verification enabled.
|
||||
|
||||
Create `/etc/leaf-controller/env`, owned by root with mode `0600`, containing:
|
||||
|
||||
```text
|
||||
LEAF_VYOS_KEY='replace-with-existing-api-key'
|
||||
```
|
||||
|
||||
Use a root editor or secret-management mechanism. Do not commit the real value
|
||||
or place it in command arguments. The credential file is read by systemd.
|
||||
|
||||
The router address group must be dedicated to this controller. With
|
||||
`adopt_foreign_members: false`, foreign members are removed during reconciliation.
|
||||
The optional adoption setting imports eligible foreign members only at startup.
|
||||
|
||||
## Log storage and installation
|
||||
|
||||
The installer requires `/var/log/suricata-leaf` to be on tmpfs. On a new sensor,
|
||||
review and run the provisioning script before installing:
|
||||
|
||||
```bash
|
||||
sudo env TMPFS_SIZE=8G bash scripts/provision-ids-tmpfs.sh
|
||||
sudo bash scripts/install-sensor.sh
|
||||
```
|
||||
|
||||
Allow enough memory and swap headroom for the configured tmpfs cap and Suricata.
|
||||
The provisioning script writes fstab and runs `mount -a`; review existing mounts
|
||||
first. Mounting over a nonempty directory hides its previous contents. EVE files
|
||||
on tmpfs are lost at reboot. SQLite and controller audit remain on persistent disk.
|
||||
|
||||
The installer stops the sensor/controller for upgrade, backs up existing files,
|
||||
units and SQLite under `/var/backups/leaf-proto/<timestamp>/`, installs the code
|
||||
and helpers, and enables these units:
|
||||
|
||||
- `leaf-controller.service`
|
||||
- `suricata-leaf.service`
|
||||
- `leaf-eve-rotate.timer`
|
||||
|
||||
It restores the previous deployment on installation failure. Backups can contain
|
||||
credentials and must stay private. The controller runs as `leafctrl`; Suricata
|
||||
runs as root with group `leafctrl`. The controller reports readiness after its
|
||||
socket is bound, allowing the sensor to start afterward.
|
||||
|
||||
## Status and diagnostics
|
||||
|
||||
```bash
|
||||
sudo systemctl status leaf-controller suricata-leaf leaf-eve-rotate.timer
|
||||
sudo journalctl -u leaf-controller -f
|
||||
sudo python3 /opt/leaf-controller/tools/vyos_smoke.py
|
||||
sudo python3 /opt/leaf-controller/tools/vyos_ctl.py members
|
||||
```
|
||||
|
||||
The smoke check reads API information, members and enforcement hooks.
|
||||
`vyos_show.py` reads scoped configuration and redacts sensitive fields.
|
||||
These diagnostic scripts do not write router configuration.
|
||||
|
||||
| Resource | Installed path |
|
||||
|---|---|
|
||||
| Controller | `/opt/leaf-controller/controller/leaf_controller/` |
|
||||
| Configuration | `/etc/leaf-controller/config.yaml` |
|
||||
| Credential | `/etc/leaf-controller/env` |
|
||||
| SQLite WAL state | `/var/lib/leaf-controller/state.sqlite` |
|
||||
| Decision audit | `/var/log/leaf-controller/decisions.jsonl` |
|
||||
| Capture metrics | `/var/log/leaf-controller/capture.log` |
|
||||
| EVE stats | `/var/log/suricata-leaf/eve.json` |
|
||||
| Alert socket | `/run/suricata/alerts.sock` |
|
||||
|
||||
The retention timer checks each minute after an initial two-minute delay. It
|
||||
renames EVE hourly or at 64 MiB, asks Suricata to reopen it, and retains six
|
||||
archives. A startup grace period prevents signalling a newly started engine.
|
||||
Retention can overshoot between timer runs; tmpfs capacity is the hard limit.
|
||||
Audit defaults to 16 MiB with three archives; capture metrics default to 10 MiB
|
||||
with six archives.
|
||||
|
||||
`save_config: false` leaves transient membership out of router boot configuration.
|
||||
SQLite supplies desired state when the router returns. If saving is enabled,
|
||||
failed saves remain dirty in SQLite and are retried.
|
||||
|
||||
## Credential rotation
|
||||
|
||||
Identify the controller's existing key ID before rotating:
|
||||
|
||||
```bash
|
||||
sudo python3 /opt/leaf-controller/tools/rotate-vyos-key.py --old-id <controller-key-id>
|
||||
```
|
||||
|
||||
The tool requires the selected ID to match the current credential. It creates
|
||||
and verifies a candidate, saves both keys, switches the local environment file,
|
||||
restarts the service, then retires only the selected old ID and saves again.
|
||||
Other API users' IDs are preserved.
|
||||
|
||||
Switch failures restore the local credential. A failure after retirement requires
|
||||
inspection before retrying; deleting arbitrary IDs is not a recovery procedure.
|
||||
No credential values are printed by the tool.
|
||||
|
||||
## Rollback
|
||||
|
||||
Stop Suricata, the controller and the rotation timer. Restore the selected file
|
||||
archive and its matching SQLite backup, reload systemd, and restart the previous
|
||||
units. Keep SQLite closed while replacing its files. When reverting a schema
|
||||
migration, restore the matching database as well as the code.
|
||||
|
||||
Do not clear the live database or shared router configuration as an upgrade or
|
||||
test shortcut. Live tests operate on existing routes and enforcement hooks;
|
||||
uplink, OSPF and static-route changes are outside their scope.
|
||||
11
docs/REFERENCES.md
Normal file
11
docs/REFERENCES.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# References
|
||||
|
||||
The lab uses Suricata 7 and VyOS 2026.03. Configuration and API schemas differ
|
||||
between releases; use documentation matching the installed software.
|
||||
|
||||
- [VyOS HTTP API](https://docs.vyos.io/en/rolling/automation/vyos-api.html)
|
||||
- [VyOS HTTPS service and API configuration](https://docs.vyos.io/en/rolling/configuration/service/https.html)
|
||||
- [VyOS Ethernet interfaces and mirroring](https://docs.vyos.io/en/rolling/configuration/interfaces/ethernet.html)
|
||||
- [VyOS firewall groups](https://docs.vyos.io/en/rolling/configuration/firewall/groups.html)
|
||||
- [Suricata EVE output](https://docs.suricata.io/en/suricata-7.0.10/output/eve/eve-json-output.html)
|
||||
- [Suricata EVE JSON format](https://docs.suricata.io/en/suricata-7.0.10/output/eve/eve-json-format.html)
|
||||
87
docs/TESTING.md
Normal file
87
docs/TESTING.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# Testing
|
||||
|
||||
## Unit and mock integration tests
|
||||
|
||||
Run from the repository root:
|
||||
|
||||
```bash
|
||||
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p 'test_*.py' -v
|
||||
bash tests/run_e2e.sh
|
||||
LEAF_E2E_SOCKET=/tmp/leaf-test.sock bash tests/run_e2e.sh
|
||||
python3 tests/checkpoint_e2e.py
|
||||
```
|
||||
|
||||
The unit suite covers socket ownership/framing, protected-address policy,
|
||||
thresholds, persistence, replay identities, reconciliation races, TLS pin
|
||||
isolation, retention, malformed telemetry and credential-switch failure paths.
|
||||
It uses local fixtures rather than the lab network.
|
||||
|
||||
The shell integration starts a localhost mock VyOS API and a controller. The
|
||||
first invocation uses file alerts; the second uses the Unix socket. They check
|
||||
thresholding, protected addresses, source blocking, rotation, adoption and TTL
|
||||
cleanup. They use port 18443 and disposable `tests/.e2e/` files: run them
|
||||
sequentially. Keep the socket path short enough for AF_UNIX pathname limits.
|
||||
|
||||
The checkpoint check verifies that file mode resumes at the next unread record.
|
||||
It does not assert socket-alert backlog during controller downtime.
|
||||
|
||||
## Offline Suricata replay
|
||||
|
||||
On a sensor with Suricata installed and its rules in `/etc/suricata-leaf/`:
|
||||
|
||||
```bash
|
||||
sudo python3 tests/replay_fixture.py
|
||||
```
|
||||
|
||||
The script generates a valid 24-SYN PCAP, replays it with isolated outputs and
|
||||
requires 24 decoded SYNs, two SID 1000030 alerts and zero fragment/checksum errors.
|
||||
It leaves the live capture configuration and services in place.
|
||||
|
||||
`tests/syn_packets.py` is the maintained TEST-NET SYN generator.
|
||||
`tests/drill_synsensor.py` can write ICMP/SYN PCAP fixtures or send bounded raw
|
||||
frames on a selected route. Capture visibility depends on the mirror topology;
|
||||
use the test hosts for the live enforcement checks.
|
||||
|
||||
## Live enforcement tests
|
||||
|
||||
Verify the sensor identity, SSH account/key, test hosts, routes and dedicated
|
||||
block group before running. Defaults describe the supplied lab. Override the
|
||||
connection arguments for another environment:
|
||||
|
||||
```bash
|
||||
python3 tests/live_e2e.py --help
|
||||
python3 tests/live_negative.py --help
|
||||
python3 tests/live_e2e.py --output evidence/live-e2e.json
|
||||
python3 tests/live_negative.py --output evidence/negative.json
|
||||
```
|
||||
|
||||
These tests use SSH to the sensor and LXCs, deploy temporary helpers, transmit
|
||||
bounded TEST-NET traffic and temporarily select a 180-second TTL. They require
|
||||
an empty block group and state for counter attribution. Run them sequentially.
|
||||
The original configuration is restored on exit; failure cleanup expires only
|
||||
the test's own targets. An interrupted connection can require checking the
|
||||
retained configuration backup before another run.
|
||||
|
||||
The positive test requires fresh capture, audited decisions, durable applied
|
||||
rows, verified router membership, source/destination drop-counter growth,
|
||||
natural TTL removal and clean post-expiry probes. It compares hashes of
|
||||
protected router configuration before and after.
|
||||
|
||||
The negative test sends benign traffic from another source while a block is
|
||||
active. It checks ingress observation, absence of the benign address from
|
||||
block membership/state, and selective IDS counter changes. Ingress capture
|
||||
is not a destination receipt or application-connectivity test.
|
||||
|
||||
Live results describe the tested rules, topology and traffic. They do not
|
||||
establish sustained throughput, production detection quality, lossless alert
|
||||
delivery, or successful full router/sensor reboot recovery.
|
||||
|
||||
## Recovery inspection
|
||||
|
||||
```bash
|
||||
VM=<sensor-user>@<sensor-address> LEAF_SSH_KEY=<key-path> bash scripts/reboot-check.sh
|
||||
```
|
||||
|
||||
Despite its filename, this script performs read-only recovery inspection. It
|
||||
checks service activity/enabling, tmpfs, socket presence and API/state status,
|
||||
and prints the current boot identity. It does not initiate or prove a reboot.
|
||||
87
scripts/install-sensor.sh
Normal file
87
scripts/install-sensor.sh
Normal file
@@ -0,0 +1,87 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
[ "$(id -u)" -eq 0 ] || { echo 'Run as root on the sensor' >&2; exit 1; }
|
||||
STAGE=$(cd "$(dirname "$0")/.." && pwd)
|
||||
python3 -c 'import yaml, requests'
|
||||
[ -s /etc/leaf-controller/env ] || { echo 'Create root-only /etc/leaf-controller/env with LEAF_VYOS_KEY first' >&2; exit 1; }
|
||||
[ "$(findmnt -n -o FSTYPE --target /var/log/suricata-leaf)" = tmpfs ] || { echo 'Provision the sensor log tmpfs first' >&2; exit 1; }
|
||||
if systemctl is-active --quiet suricata.service; then
|
||||
echo 'The stock Suricata service is active; stop it before installing a second sensor.' >&2
|
||||
exit 1
|
||||
fi
|
||||
systemctl stop leaf-eve-rotate.timer leaf-eve-rotate.service 2>/dev/null || true
|
||||
systemctl stop suricata-leaf.service leaf-controller.service 2>/dev/null || true
|
||||
BACKUP=/var/backups/leaf-proto/$(date +%Y%m%d-%H%M%S)
|
||||
install -d -m 700 "$BACKUP"
|
||||
items=()
|
||||
for item in etc/suricata-leaf etc/leaf-controller opt/leaf-controller home/yuru/leaf-proto etc/systemd/system/leaf-controller.service etc/systemd/system/suricata-leaf.service etc/systemd/system/leaf-eve-rotate.service etc/systemd/system/leaf-eve-rotate.timer etc/tmpfiles.d/leaf-suricata.conf; do
|
||||
if [ -e "/$item" ]; then items+=("$item"); fi
|
||||
done
|
||||
tar -czf "$BACKUP/sensor-files.tar.gz" -C / "${items[@]}"
|
||||
python3 - "$BACKUP" <<'PY'
|
||||
import sqlite3,sys,pathlib
|
||||
p=pathlib.Path('/var/lib/leaf-controller/state.sqlite')
|
||||
if p.exists():
|
||||
with sqlite3.connect(p) as source, sqlite3.connect(sys.argv[1]+'/state.sqlite') as target: source.backup(target)
|
||||
PY
|
||||
rollback() {
|
||||
rc=$?
|
||||
trap - ERR
|
||||
echo "Installation failed; restoring previous files from $BACKUP" >&2
|
||||
systemctl stop suricata-leaf leaf-controller leaf-eve-rotate.timer 2>/dev/null || true
|
||||
for unit in leaf-controller.service suricata-leaf.service leaf-eve-rotate.service leaf-eve-rotate.timer; do
|
||||
if ! tar -tzf "$BACKUP/sensor-files.tar.gz" | grep -qx "etc/systemd/system/$unit"; then
|
||||
systemctl disable "$unit" 2>/dev/null || true
|
||||
rm -f "/etc/systemd/system/$unit"
|
||||
fi
|
||||
done
|
||||
tar -xzf "$BACKUP/sensor-files.tar.gz" -C /
|
||||
if [ -f "$BACKUP/state.sqlite" ]; then
|
||||
install -o leafctrl -g leafctrl -m 640 "$BACKUP/state.sqlite" /var/lib/leaf-controller/state.sqlite
|
||||
rm -f /var/lib/leaf-controller/state.sqlite-wal /var/lib/leaf-controller/state.sqlite-shm
|
||||
fi
|
||||
systemctl daemon-reload
|
||||
systemctl start leaf-controller suricata-leaf || true
|
||||
exit "$rc"
|
||||
}
|
||||
trap rollback ERR
|
||||
getent group leafctrl >/dev/null || groupadd --system leafctrl
|
||||
id leafctrl >/dev/null 2>&1 || useradd --system --gid leafctrl --home-dir /nonexistent --shell /usr/sbin/nologin leafctrl
|
||||
install -d -o leafctrl -g leafctrl -m 750 /var/lib/leaf-controller /var/log/leaf-controller
|
||||
chown -R leafctrl:leafctrl /var/lib/leaf-controller /var/log/leaf-controller
|
||||
find /var/lib/leaf-controller /var/log/leaf-controller -type f -exec chmod 640 {} +
|
||||
install -d -m 755 /opt/leaf-controller/controller
|
||||
cp -a "$STAGE/controller/leaf_controller" /opt/leaf-controller/controller/
|
||||
chown -R root:root /opt/leaf-controller
|
||||
chmod -R a+rX /opt/leaf-controller
|
||||
install -d -o root -g leafctrl -m 750 /etc/leaf-controller
|
||||
# Preserve site policy and TLS settings on upgrades; seed only a new install.
|
||||
if [ ! -s /etc/leaf-controller/config.yaml ]; then
|
||||
install -m 644 "$STAGE/config/prototype-vyos.yaml" /etc/leaf-controller/config.yaml
|
||||
fi
|
||||
install -d -m 755 /opt/leaf-controller/tools
|
||||
install -m 755 "$STAGE/tests/live_helper.py" /opt/leaf-controller/tools/leaf-live-helper.py
|
||||
install -m 755 "$STAGE/scripts/rotate-eve.py" "$STAGE/scripts/rotate-vyos-key.py" "$STAGE/scripts/vyos_show.py" "$STAGE/scripts/vyos_smoke.py" "$STAGE/scripts/vyos_ctl.py" "$STAGE/scripts/probe-vyos-read.py" /opt/leaf-controller/tools/
|
||||
install -m 644 "$STAGE/suricata/leaf-eve-rotate.service" "$STAGE/suricata/leaf-eve-rotate.timer" /etc/systemd/system/
|
||||
install -m 644 "$STAGE/controller/leaf-controller.service" /etc/systemd/system/leaf-controller.service
|
||||
install -d -m 755 /etc/suricata-leaf/rules
|
||||
install -m 644 "$STAGE/suricata/suricata-leaf.yaml" /etc/suricata-leaf/suricata-leaf.yaml
|
||||
install -m 644 "$STAGE/suricata/rules/leaf-local.rules" /etc/suricata-leaf/rules/leaf-local.rules
|
||||
install -m 644 "$STAGE/suricata/suricata-leaf.service" /etc/systemd/system/suricata-leaf.service
|
||||
# tmpfs mount stays in place; recreate permissions after reboot via tmpfiles.
|
||||
printf '%s\n' 'd /var/log/suricata-leaf 0750 root leafctrl -' \
|
||||
'd /run/suricata 0770 root leafctrl -' > /etc/tmpfiles.d/leaf-suricata.conf
|
||||
systemd-tmpfiles --create /etc/tmpfiles.d/leaf-suricata.conf
|
||||
chown root:leafctrl /var/log/suricata-leaf
|
||||
chmod 750 /var/log/suricata-leaf
|
||||
find /var/log/suricata-leaf -maxdepth 1 -type f -exec chgrp leafctrl {} +
|
||||
find /var/log/suricata-leaf -maxdepth 1 -type f -exec chmod 640 {} +
|
||||
systemctl daemon-reload
|
||||
systemctl enable suricata-leaf.service leaf-controller.service leaf-eve-rotate.timer
|
||||
systemctl start suricata-leaf.service leaf-controller.service leaf-eve-rotate.timer
|
||||
sleep 2
|
||||
systemctl is-active --quiet suricata-leaf.service
|
||||
systemctl is-active --quiet leaf-controller.service
|
||||
trap - ERR
|
||||
printf 'Backup: %s\n' "$BACKUP"
|
||||
systemctl --no-pager --full status suricata-leaf.service leaf-controller.service
|
||||
5
scripts/probe-vyos-read.py
Normal file
5
scripts/probe-vyos-read.py
Normal file
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compatibility entry point for the scoped, redacted read-only inspector."""
|
||||
from pathlib import Path
|
||||
import runpy
|
||||
if __name__=='__main__':runpy.run_path(str(Path(__file__).with_name('vyos_show.py')),run_name='__main__')
|
||||
76
scripts/provision-ids-tmpfs.sh
Executable file
76
scripts/provision-ids-tmpfs.sh
Executable file
@@ -0,0 +1,76 @@
|
||||
#!/usr/bin/env bash
|
||||
# LEAF-PROTO sensor provisioning: RAM disk (tmpfs) for the suricata eve dir.
|
||||
#
|
||||
# eve.json under /var/log/suricata-leaf is rewritten constantly; on a
|
||||
# journal/flash disk that burns write cycles and adds latency. This puts the
|
||||
# directory on a size-capped tmpfs (RAM+swap backed), persisted via fstab.
|
||||
# Controller state stays on persistent storage in /var/lib/leaf-controller.
|
||||
# See docs/INSTALLATION.md for memory sizing and log retention.
|
||||
#
|
||||
# Usage: TMPFS_SIZE=6G sudo scripts/provision-ids-tmpfs.sh [mount_point]
|
||||
# (default mount point /var/log/suricata-leaf, default size 8G)
|
||||
set -euo pipefail
|
||||
|
||||
MOUNT_POINT="${1:-${MOUNT_POINT:-/var/log/suricata-leaf}}"
|
||||
SIZE="${TMPFS_SIZE:-${SIZE:-8G}}"
|
||||
|
||||
usage() {
|
||||
echo "usage: TMPFS_SIZE=8G sudo $0 [mount_point (default: /var/log/suricata-leaf)]"
|
||||
}
|
||||
|
||||
if [[ "${EUID:-$(id -u)}" -ne 0 ]]; then
|
||||
echo "error: run as root (needs mount/fstab writes)" >&2
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if findmnt -rn -T "$MOUNT_POINT" | grep -q 'tmpfs'; then
|
||||
echo "skip: $MOUNT_POINT is already on tmpfs:"
|
||||
findmnt -rn -T "$MOUNT_POINT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if ! [[ "$SIZE" =~ ^[0-9]+([MGK])?$ ]]; then
|
||||
echo "error: TMPFS_SIZE must like 6G/8G/512M (got: $SIZE)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "warning: tmpfs is RAM-backed — $SIZE cap counts against RAM+swap."
|
||||
free -h
|
||||
echo "note: eve.json on this mount is lost on reboot by design;"
|
||||
echo "note: controller block state lives on disk (/var/lib/leaf-controller), unaffected."
|
||||
|
||||
mkdir -p "$MOUNT_POINT"
|
||||
|
||||
# A pre-existing non-empty dir on another filesystem would be hidden under
|
||||
# the tmpfs after mounting — warn, since old eve files become unreachable.
|
||||
if [[ -n "$(findmnt -rn -T "$MOUNT_POINT" 2>/dev/null)" ]] &&
|
||||
! findmnt -rn -T "$MOUNT_POINT" | grep -q 'tmpfs' &&
|
||||
[[ -n "$(ls -A "$MOUNT_POINT" 2>/dev/null)" ]]; then
|
||||
echo "warning: $MOUNT_POINT already has content on another filesystem;"
|
||||
echo "warning: it will be hidden underneath the tmpfs — move it first if"
|
||||
echo "warning: you need the old eve.json:"
|
||||
ls -A "$MOUNT_POINT" | head -5
|
||||
fi
|
||||
|
||||
FSTAB_LINE="tmpfs ${MOUNT_POINT} tmpfs defaults,size=${SIZE},nosuid,noexec,nodev 0 0"
|
||||
if grep -qF " ${MOUNT_POINT} " /etc/fstab; then
|
||||
echo "fstab already has an entry for $MOUNT_POINT — not appending:"
|
||||
grep -F " ${MOUNT_POINT} " /etc/fstab
|
||||
else
|
||||
printf '\n# LEAF-PROTO: suricata eve on RAM disk (provision-ids-tmpfs.sh)\n%s\n' "$FSTAB_LINE" >> /etc/fstab
|
||||
echo "fstab entry appended."
|
||||
fi
|
||||
|
||||
systemctl daemon-reload
|
||||
mount -a
|
||||
|
||||
if findmnt -rn -T "$MOUNT_POINT" | grep -q 'tmpfs'; then
|
||||
echo "OK: mounted."
|
||||
findmnt -rn -T "$MOUNT_POINT"
|
||||
else
|
||||
echo "error: mount did not come up — check /etc/fstab" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
df -h "$MOUNT_POINT"
|
||||
17
scripts/reboot-check.sh
Normal file
17
scripts/reboot-check.sh
Normal file
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# Read-only recovery inspection. Never reboots either the sensor or VIPERA.
|
||||
set -euo pipefail
|
||||
VM="${VM:-yuru@192.168.100.84}"
|
||||
KEY="${LEAF_SSH_KEY:-$HOME/.ssh/id_ed25519_alice}"
|
||||
ssh -F /dev/null -o BatchMode=yes -o ConnectTimeout=8 -i "$KEY" "$VM" '
|
||||
set -eu
|
||||
[ "$(hostname)" = LEAF-PROTO ]
|
||||
systemctl is-active --quiet leaf-controller suricata-leaf leaf-eve-rotate.timer
|
||||
systemctl is-enabled --quiet leaf-controller suricata-leaf leaf-eve-rotate.timer
|
||||
[ "$(findmnt -n -o FSTYPE --target /var/log/suricata-leaf)" = tmpfs ]
|
||||
sudo -n test -S /run/suricata/alerts.sock
|
||||
printf "boot_id="; cat /proc/sys/kernel/random/boot_id
|
||||
uptime -s
|
||||
sudo -n python3 /opt/leaf-controller/tools/leaf-live-helper.py status
|
||||
printf "RECOVERY INSPECTION PASS (no reboot performed)\n"
|
||||
'
|
||||
60
scripts/rotate-eve.py
Normal file
60
scripts/rotate-eve.py
Normal file
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Rename EVE and ask Suricata to reopen it; invoked each minute by a timer.
|
||||
|
||||
Normal bound: seven 64-MiB/hour segments plus bytes written between timer
|
||||
runs. The tmpfs itself is the hard capacity limit. Not copytruncate.
|
||||
"""
|
||||
import argparse
|
||||
import fcntl
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
|
||||
def rotate(path, max_bytes=64*1024*1024, max_age=3600, keep=6, reopen=None, now=None):
|
||||
now = time.time() if now is None else now
|
||||
try:
|
||||
st = path.stat()
|
||||
except FileNotFoundError:
|
||||
return False
|
||||
# Both mtime and ctime change on writes. Keep an independent age clock.
|
||||
clock = path.with_name('.' + path.name + '.rotation-clock')
|
||||
try:
|
||||
last = float(clock.read_text())
|
||||
except (FileNotFoundError, ValueError):
|
||||
last = now
|
||||
clock.write_text(str(now))
|
||||
if st.st_size < max_bytes and now-last < max_age:
|
||||
return False
|
||||
dst = path.with_name(path.name + '.leaf-' + str(time.time_ns()).zfill(20))
|
||||
path.rename(dst)
|
||||
try:
|
||||
reopen()
|
||||
except BaseException:
|
||||
if not path.exists():
|
||||
dst.rename(path)
|
||||
raise
|
||||
clock.write_text(str(now))
|
||||
for f in sorted(path.parent.glob(path.name + '.leaf-*'))[:-keep] if keep else path.parent.glob(path.name + '.leaf-*'):
|
||||
f.unlink()
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('--path', type=Path, default=Path('/var/log/suricata-leaf/eve.json'))
|
||||
a=ap.parse_args()
|
||||
# Type=simple is active before Suricata installs signal handlers. Never
|
||||
# signal a just-started engine, including manual timer/service triggers.
|
||||
started = subprocess.run(['systemctl','show','--property=ActiveEnterTimestampMonotonic',
|
||||
'--value','suricata-leaf.service'],
|
||||
check=True,capture_output=True,text=True).stdout.strip()
|
||||
if not started or int(started) == 0 or time.monotonic() - int(started)/1e6 < 60:
|
||||
return
|
||||
with open('/run/leaf-eve-rotation.lock', 'a') as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
rotate(a.path, reopen=lambda: subprocess.run(
|
||||
['systemctl','kill','--kill-whom=main','--signal=HUP','suricata-leaf.service'], check=True))
|
||||
|
||||
if __name__ == '__main__': main()
|
||||
120
scripts/rotate-vyos-key.py
Normal file
120
scripts/rotate-vyos-key.py
Normal file
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Rotate exactly one explicitly selected LEAF key. Never prints API bodies/keys.
|
||||
|
||||
Run on sensor: sudo python3 .../rotate-vyos-key.py --old-id <LEAF-owned-id>
|
||||
Other applications' credentials are preserved. Retrying after retirement/save
|
||||
failure requires inspecting router key IDs first; no blind mass-delete mode.
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import secrets
|
||||
import string
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
sys.path.insert(0, '/opt/leaf-controller/controller')
|
||||
from leaf_controller.config import load_config, read_key_env
|
||||
from leaf_controller.vyos_client import VyosClient, VyosApiError
|
||||
|
||||
ENV_PATH = '/etc/leaf-controller/env'
|
||||
CFG_PATH = '/etc/leaf-controller/config.yaml'
|
||||
SERVICE = 'leaf-controller.service'
|
||||
KEYS_PATH = ['service', 'https', 'api', 'keys', 'id']
|
||||
|
||||
|
||||
def atomic_env(text):
|
||||
tmp=ENV_PATH+'.new'
|
||||
fd=os.open(tmp, os.O_WRONLY|os.O_CREAT|os.O_TRUNC, 0o600)
|
||||
os.fchmod(fd, 0o600)
|
||||
with os.fdopen(fd, 'w') as f:
|
||||
f.write(text); f.flush(); os.fsync(f.fileno())
|
||||
os.replace(tmp, ENV_PATH)
|
||||
fd=os.open(str(Path(ENV_PATH).parent), os.O_DIRECTORY)
|
||||
try: os.fsync(fd)
|
||||
finally: os.close(fd)
|
||||
|
||||
|
||||
def restart_service():
|
||||
subprocess.run(['systemctl','restart',SERVICE], check=True, capture_output=True)
|
||||
subprocess.run(['systemctl','is-active','--quiet',SERVICE], check=True, capture_output=True)
|
||||
|
||||
|
||||
def wait_auth(client, group, accepted=True, attempts=24):
|
||||
for i in range(attempts):
|
||||
try:
|
||||
client.group_members(group)
|
||||
if accepted: return
|
||||
except VyosApiError as exc:
|
||||
if not accepted and exc.status in (401,403): return
|
||||
except Exception:
|
||||
pass
|
||||
if i+1<attempts: time.sleep(5)
|
||||
raise RuntimeError('credential verification timed out')
|
||||
|
||||
|
||||
def rotate(old, new, old_id, new_id, old_key, new_key, env_text, group):
|
||||
body=old.retrieve('showConfig', KEYS_PATH[:-1])
|
||||
ids=(body.get('data') or {}).get('id', {})
|
||||
if old_id not in ids or ids[old_id].get('key') != old_key:
|
||||
raise RuntimeError('selected key ID does not match the current controller credential')
|
||||
if new_id in ids:
|
||||
raise RuntimeError('candidate ID already exists')
|
||||
swapped=False
|
||||
installed=False
|
||||
try:
|
||||
# A timeout can still commit; rollback attempts candidate deletion either way.
|
||||
installed=True
|
||||
old.configure({'op':'set','path':KEYS_PATH+[new_id,'key',new_key]})
|
||||
wait_auth(new, group)
|
||||
# Persist both keys before the local switch; old credential remains a recovery path.
|
||||
new.save_config()
|
||||
out,n=re.subn(r'(?m)^LEAF_VYOS_KEY=.*$', "LEAF_VYOS_KEY='"+new_key+"'", env_text)
|
||||
if n!=1: raise RuntimeError('expected exactly one credential assignment')
|
||||
swapped=True
|
||||
atomic_env(out)
|
||||
restart_service()
|
||||
wait_auth(new, group, attempts=2)
|
||||
except BaseException:
|
||||
# Restore the credential before restarting. If recovery fails, retain candidate.
|
||||
if swapped:
|
||||
atomic_env(env_text)
|
||||
restart_service()
|
||||
if installed:
|
||||
old.configure({'op':'delete','path':KEYS_PATH+[new_id]})
|
||||
old.save_config()
|
||||
raise
|
||||
# Retirement is a separate phase: rollback is unsafe after this commit.
|
||||
new.configure({'op':'delete','path':KEYS_PATH+[old_id]})
|
||||
new.save_config() # Failure is fatal; never claim persistence succeeded.
|
||||
wait_auth(old, group, accepted=False)
|
||||
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('--old-id', required=True)
|
||||
a=ap.parse_args()
|
||||
if os.geteuid()!=0: ap.error('run as root on the sensor')
|
||||
cfg=load_config(CFG_PATH)
|
||||
if cfg.get('vyos.key_env','LEAF_VYOS_KEY')!='LEAF_VYOS_KEY':
|
||||
ap.error('this tool requires the standard LEAF_VYOS_KEY assignment')
|
||||
text=Path(ENV_PATH).read_text()
|
||||
old_key=read_key_env(ENV_PATH)
|
||||
new_key=''.join(secrets.choice(string.ascii_letters+string.digits) for _ in range(32))
|
||||
new_id='leaf-'+str(time.time_ns())
|
||||
def client(key):
|
||||
return VyosClient(cfg.vyos_url(), key, cert_pin=cfg.get('vyos.tls_cert_pin') or None,
|
||||
ca_bundle=cfg.get('vyos.ca_bundle'), tls_verify=cfg.get('vyos.tls_verify',True),
|
||||
timeout=float(cfg.get('vyos.timeout',75)), retries=1)
|
||||
try:
|
||||
rotate(client(old_key),client(new_key),a.old_id,new_id,old_key,new_key,text,cfg.get('block_group'))
|
||||
except BaseException:
|
||||
# Deliberately omit exception details: router responses can contain secrets.
|
||||
print('ROTATION FAILED. Inspect service and router key IDs before retrying. Candidate ID: '+new_id,
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
print('ROTATION COMPLETE: selected old ID retired, config persisted, new ID='+new_id)
|
||||
return 0
|
||||
|
||||
if __name__=='__main__': raise SystemExit(main())
|
||||
19
scripts/vyos_ctl.py
Normal file
19
scripts/vyos_ctl.py
Normal file
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read-only lab client. Membership writes belong to the durable controller.
|
||||
|
||||
Legacy add/del/drop-chain commands have been retired; they bypassed ownership.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
sys.path[:0]=[str(Path(__file__).resolve().parents[1]/'controller'),'/opt/leaf-controller/controller']
|
||||
from leaf_controller.ops import configured_client,redact
|
||||
|
||||
def main():
|
||||
p=argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument('command',choices=['members','forward-rule'],default='members',nargs='?')
|
||||
a=p.parse_args();cfg,c=configured_client()
|
||||
if a.command=='members':print(json.dumps(c.group_members(cfg.get('block_group'))))
|
||||
else:print(json.dumps(redact(c.retrieve('showConfig',['firewall','ipv4','forward']))))
|
||||
if __name__=='__main__':main()
|
||||
13
scripts/vyos_show.py
Normal file
13
scripts/vyos_show.py
Normal file
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read-only firewall, mirror and protected-path inspection. No credential dump."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
sys.path[:0]=[str(Path(__file__).resolve().parents[1]/'controller'),'/opt/leaf-controller/controller']
|
||||
from leaf_controller.ops import configured_client, redact
|
||||
|
||||
def main():
|
||||
_,c=configured_client()
|
||||
for path in (['firewall'],['interfaces'],['protocols','ospf'],['protocols','static']):
|
||||
print(json.dumps({'path':path,'data':redact(c.retrieve('showConfig',path).get('data'))}))
|
||||
if __name__=='__main__':main()
|
||||
15
scripts/vyos_smoke.py
Normal file
15
scripts/vyos_smoke.py
Normal file
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read-only pinned API smoke check. Does not add members or save router config."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
sys.path[:0]=[str(Path(__file__).resolve().parents[1]/'controller'),'/opt/leaf-controller/controller']
|
||||
from leaf_controller.ops import configured_client
|
||||
|
||||
def main():
|
||||
cfg,c=configured_client()
|
||||
info=c.info();members=c.group_members(cfg.get('block_group'))
|
||||
c.verify_enforcement(cfg.get('block_group'),cfg.get('policy.block_src',False))
|
||||
print(json.dumps({'version':info.get('version'),'members':members,'enforcement_hooks':True}))
|
||||
print('READ-ONLY SMOKE PASS')
|
||||
if __name__=='__main__':main()
|
||||
8
suricata/leaf-eve-rotate.service
Normal file
8
suricata/leaf-eve-rotate.service
Normal file
@@ -0,0 +1,8 @@
|
||||
[Unit]
|
||||
Description=Rotate LEAF EVE stats on tmpfs
|
||||
Requisite=suricata-leaf.service
|
||||
After=suricata-leaf.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/bin/python3 /opt/leaf-controller/tools/rotate-eve.py
|
||||
UMask=0027
|
||||
8
suricata/leaf-eve-rotate.timer
Normal file
8
suricata/leaf-eve-rotate.timer
Normal file
@@ -0,0 +1,8 @@
|
||||
[Unit]
|
||||
Description=Check LEAF EVE retention every minute
|
||||
[Timer]
|
||||
OnActiveSec=2min
|
||||
OnUnitActiveSec=1min
|
||||
AccuracySec=5s
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
34
suricata/rules/leaf-local.rules
Normal file
34
suricata/rules/leaf-local.rules
Normal file
@@ -0,0 +1,34 @@
|
||||
# LEAF-PROTO local rules. Only rules with `metadata: leaf_action block;` are
|
||||
# acted on by leaf-controller (override via config policy). SIDs reserved: 1000000-1099999.
|
||||
#
|
||||
# Inbound flows (attacker->victim): default policy blocks nothing (dest is your LAN).
|
||||
# Outbound flows (victim->external C2 / exfil / scan-out): dest_ip gets blocked on VyOS.
|
||||
|
||||
# Suricata 7: HOME_NET/EXTERNAL_NET are defined in suricata-leaf.yaml
|
||||
# (vars: address-groups). 'var' statements in rule files are not supported.
|
||||
|
||||
# --- lab test rule: any outbound ICMP to TEST-NET (trigger demo) ---
|
||||
alert icmp any any -> 192.0.2.0/24 any (msg:"LEAF TEST ICMP to TEST-NET"; sid:1000001; rev:1; metadata: leaf_action block;)
|
||||
|
||||
# --- outbound scan: many TCP SYNs from one host to many hosts in 10s ---
|
||||
alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"LEAF SCAN-OUT tcp syn flood"; flags:S; threshold: type threshold, track by_src, count 25, seconds 10; sid:1000002; rev:1; metadata: leaf_action block;)
|
||||
# NOTE: the threshold is engine-side noisy for lab use; controller also windows by (sid,dest)
|
||||
|
||||
# --- obvious C2 patterns (examples; extend with your own) ---
|
||||
alert tcp any any -> any any (msg:"LEAF possible meterpreter reverse"; flow:to_server; content:"|0a 16 1a 05|"; depth:4; sid:1000010; rev:1; metadata: leaf_action block;)
|
||||
|
||||
# --- DNS tunneling heuristic: prefer et open ruleset + thresholding for this ---
|
||||
|
||||
# --- inbound (src-side block candidates; block_src must be enabled in controller) ---
|
||||
# SYN+FIN is an abnormal flag combination, not an SSH authentication failure.
|
||||
# Encrypted SSH login failures cannot be inferred from TCP flags.
|
||||
alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg:"LEAF abnormal SYN FIN inbound"; flow:to_server; flags:SF; threshold: type threshold, track by_src, count 20, seconds 30; sid:1000020; rev:1; metadata: leaf_action block_src;)
|
||||
|
||||
# --- lateral/brute observable on the VEX-mirror posture: SYN-only toward sensor ---
|
||||
alert tcp 172.19.32.0/24 any -> 10.0.249.1 22 (msg:"LEAF LATERAL-BRUTE ssh sweep from VEX-LAN at sensor"; flags:S; threshold: type threshold, track by_src, count 15, seconds 30; sid:1000021; rev:1; metadata: leaf_action block_src;)
|
||||
|
||||
# SRC-MODE drill: mirror-visible transit wave with out-of-floor spoofed src.
|
||||
# dst 192.0.2.x is transit-only dst reachable through the mirror (locally-dst
|
||||
# traffic toward 10.0.249.1 never appears on the mirror, so 1000020/1000021 cannot
|
||||
# fire there). Spoofed src is 203.0.113.0/24 (RFC5737) -> never a real peer.
|
||||
alert tcp 203.0.113.0/24 any -> 192.0.2.0/24 22 (msg:"LEAF SRC-MODE spoofed-src at TEST-NET dst"; flags:S; threshold: type threshold, track by_src, count 12, seconds 30; sid:1000030; rev:1; metadata: leaf_action block_src;)
|
||||
25
suricata/suricata-leaf.service
Normal file
25
suricata/suricata-leaf.service
Normal file
@@ -0,0 +1,25 @@
|
||||
[Unit]
|
||||
Description=LEAF passive Suricata sensor on eth1
|
||||
# Start after the controller has bound its alert socket and reported ready.
|
||||
# Runtime reconnect occurs on a later alert write; alerts during downtime
|
||||
# are not stored for replay.
|
||||
After=network-online.target leaf-controller.service
|
||||
Wants=network-online.target leaf-controller.service
|
||||
RequiresMountsFor=/var/log/suricata-leaf
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStartPre=/usr/bin/suricata -T -c /etc/suricata-leaf/suricata-leaf.yaml -l /var/log/suricata-leaf
|
||||
ExecStart=/usr/bin/suricata --af-packet=eth1 -c /etc/suricata-leaf/suricata-leaf.yaml -l /var/log/suricata-leaf
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
KillSignal=SIGINT
|
||||
TimeoutStopSec=60
|
||||
User=root
|
||||
Group=leafctrl
|
||||
UMask=0027
|
||||
ProtectSystem=full
|
||||
ProtectHome=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
89
suricata/suricata-leaf.yaml
Normal file
89
suricata/suricata-leaf.yaml
Normal file
@@ -0,0 +1,89 @@
|
||||
%YAML 1.1
|
||||
---
|
||||
# LEAF-PROTO Suricata IDS config (IDS VM). Self-contained drop-in.
|
||||
# Run: suricata -c /etc/suricata/suricata-leaf.yaml -i MIRROR_IFACE
|
||||
# (Debian: apt install suricata suricata-update; mirror NIC must be promisc/untagged-capable)
|
||||
|
||||
vars:
|
||||
main: /var/lib/suricata-leaf
|
||||
address-groups:
|
||||
HOME_NET: "[192.168.100.0/24,10.0.249.0/31,172.19.32.0/24]"
|
||||
EXTERNAL_NET: "!$HOME_NET"
|
||||
|
||||
default-rule-path: /etc/suricata-leaf/rules
|
||||
rule-files:
|
||||
- leaf-local.rules
|
||||
|
||||
# --- log output location --------------------------------------------------
|
||||
# eve.json is written below as an absolute path. Provision
|
||||
# /var/log/suricata-leaf on a 6-8G tmpfs (RAM disk) to keep eve writes off
|
||||
# the physical disk. leaf-eve-rotate.timer renames the file hourly or at
|
||||
# 64 MiB, signals HUP to reopen it, and keeps six backups. The controller
|
||||
# drains the old inode before opening the new one.
|
||||
logging:
|
||||
default-log-level: info
|
||||
outputs:
|
||||
- console:
|
||||
enabled: no
|
||||
- file:
|
||||
enabled: yes
|
||||
level: info
|
||||
filename: suricata.log
|
||||
|
||||
# --- capture: mirror NIC -------------------------------------------------
|
||||
af-packet:
|
||||
- interface: eth1 # mirror NIC on LEAF-PROTO VM (10.0.249.1/31)
|
||||
cluster-id: 98
|
||||
cluster-type: cluster_flow
|
||||
defrag: yes
|
||||
use-mmap: yes
|
||||
tpacket-v3: no
|
||||
bpf-filter: "not ip host 255.255.255.255 and not udp port 5353"
|
||||
|
||||
# --- output split: alerts over the controller socket, stats on file ------
|
||||
# The controller binds /run/suricata/alerts.sock (AF_UNIX listener) and
|
||||
# suricata connects as the CLIENT; ordering is contract-critical: the
|
||||
# leaf-controller unit binds at startup, the sensor unit Wants+After it.
|
||||
# The controller reports readiness after binding the socket.
|
||||
# Stats stay in eve.json on tmpfs. Alerts use the socket and can be lost
|
||||
# during disconnection or if the sender cannot write without blocking.
|
||||
outputs:
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
filetype: unix_stream
|
||||
filename: /run/suricata/alerts.sock # the socket path, not a log file
|
||||
types:
|
||||
- alert:
|
||||
metadata: yes
|
||||
payload: no # keep EVE small; set packet-based extras later if needed
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
filetype: regular
|
||||
filename: /var/log/suricata-leaf/eve.json
|
||||
types:
|
||||
- stats:
|
||||
totals: yes
|
||||
threads: no
|
||||
|
||||
# --- engine: fine for 1-2 Gbps mirror ------------------------------------
|
||||
threading:
|
||||
pfring:
|
||||
enabled: no
|
||||
|
||||
pcap:
|
||||
buffer-size: 16777216
|
||||
|
||||
stream:
|
||||
memcap: 128mb
|
||||
|
||||
flow:
|
||||
memcap: 64mb
|
||||
|
||||
stats:
|
||||
enabled: yes
|
||||
interval: 30
|
||||
|
||||
host-mode: auto
|
||||
|
||||
# reference: ET open ruleset optional via `suricata-update --sources` but do NOT
|
||||
# tag whole ET for leaf_action; only our local rules carry metadata: leaf_action.
|
||||
27
tests/checkpoint_e2e.py
Normal file
27
tests/checkpoint_e2e.py
Normal file
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Offline file-mode checkpoint proof; never stops the live sensor.
|
||||
|
||||
Socket mode has no alert backlog while disconnected. File mode checkpoint
|
||||
behavior is verified here; live socket reconnect is a separate drill.
|
||||
"""
|
||||
import json
|
||||
import tempfile
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'controller'))
|
||||
from leaf_controller.eve import EveTailer
|
||||
|
||||
def main():
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path=Path(tmp)/'eve.json'
|
||||
path.write_text('')
|
||||
t=EveTailer(str(path)); t._open(); ck=(*t._ino,t._pos); t.close()
|
||||
path.write_text('{"event_type":"alert","n":1}\n{"event_type":"alert","n":2}\n')
|
||||
t=EveTailer(str(path),checkpoint=ck); events=t.events()
|
||||
assert next(events)['n']==1
|
||||
ck=(*t._ino,t._pos); events.close()
|
||||
t=EveTailer(str(path),checkpoint=ck); events=t.events()
|
||||
assert next(events)['n']==2
|
||||
events.close()
|
||||
print('CHECKPOINT PASS: restart resumes at the next unread record')
|
||||
if __name__=='__main__': main()
|
||||
189
tests/drill_synsensor.py
Normal file
189
tests/drill_synsensor.py
Normal file
@@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Sensor-side synthetic injector for the live socket drills.
|
||||
|
||||
Uses AF_PACKET raw sockets and requires root. Frames are addressed to
|
||||
the selected gateway MAC. Traffic is visible to the sensor only when its
|
||||
path crosses the configured mirror; select the interface and route accordingly.
|
||||
|
||||
Bounded by design: source and destination must be TEST-NET
|
||||
(192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24) unless --allow-real-src
|
||||
is passed (then src stays the sensor's real address). count is capped.
|
||||
--pcap writes a fixture without sending traffic.
|
||||
|
||||
Modes:
|
||||
icmp echo-request -> sid 1000001 (leaf_action block, any src)
|
||||
syn TCP SYN ->port -> sid 1000030 when src is 203.0.113.0/24
|
||||
|
||||
--no-arp: do not ping the gateway first; use the existing ARP entry only.
|
||||
--src SRC: forge src (TEST-NET enforced); ARP is still only ever
|
||||
for the gateway MAC, the forged src never touches ARP.
|
||||
"""
|
||||
import argparse
|
||||
import ipaddress
|
||||
import re
|
||||
import socket
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
TESTNET = ('192.0.2.0/24', '198.51.100.0/24', '203.0.113.0/24')
|
||||
_NETS = tuple(ipaddress.ip_network(n) for n in TESTNET)
|
||||
|
||||
|
||||
def in_testnet(addr):
|
||||
ip = ipaddress.ip_address(addr)
|
||||
return any(ip in net for net in _NETS)
|
||||
|
||||
|
||||
def checksum(data):
|
||||
if len(data) % 2:
|
||||
data += b'\x00'
|
||||
total = sum(struct.unpack('!%dH' % (len(data) // 2), data))
|
||||
while total >> 16:
|
||||
total = (total & 0xffff) + (total >> 16)
|
||||
return (~total) & 0xffff
|
||||
|
||||
|
||||
def syn_packet(src, dst, dport, index, smac, dmac, flags=2):
|
||||
a, b = socket.inet_aton(src), socket.inet_aton(dst)
|
||||
tcp = struct.pack('!HHIIBBHHH', 40000 + index % 20000, dport,
|
||||
7000 + index, 0, 0x50, flags, 1024, 0, 0)
|
||||
pseudo = a + b + struct.pack('!BBH', 0, 6, len(tcp))
|
||||
tcp = tcp[:16] + struct.pack('!H', checksum(pseudo + tcp)) + tcp[18:]
|
||||
ip = struct.pack('!BBHHHBBH4s4s', 0x45, 0, 40, index % 65536, 0, 64, 6,
|
||||
0, a, b)
|
||||
ip = ip[:10] + struct.pack('!H', checksum(ip)) + ip[12:]
|
||||
return dmac + smac + b'\x08\x00' + ip + tcp
|
||||
|
||||
|
||||
def icmp_packet(src, dst, index, smac, dmac):
|
||||
payload = b'LEAFDRILL' + bytes((index % 100,) * 8)
|
||||
icmp = struct.pack('!BBHHH', 8, 0, 0, 0x4C42, index % 65536) + payload
|
||||
ck = checksum(icmp)
|
||||
icmp = icmp[:2] + struct.pack('!H', ck) + icmp[4:]
|
||||
total = 20 + len(icmp)
|
||||
a, b = socket.inet_aton(src), socket.inet_aton(dst)
|
||||
ip = struct.pack('!BBHHHBBH4s4s', 0x45, 0, total, index % 65536, 0, 64,
|
||||
1, 0, a, b)
|
||||
ip = ip[:10] + struct.pack('!H', checksum(ip)) + ip[12:]
|
||||
return dmac + smac + b'\x08\x00' + ip + icmp
|
||||
|
||||
|
||||
def pcap_write(path, packets):
|
||||
with path.open('wb') as f:
|
||||
f.write(struct.pack('<IHHIIII', 0xa1b2c3d4, 2, 4, 0, 0, 65535, 1))
|
||||
t = time.time()
|
||||
for i, data in enumerate(packets):
|
||||
stamp = t + i * 0.001
|
||||
f.write(struct.pack('<IIII', int(stamp), int(stamp % 1 * 1e6),
|
||||
len(data), len(data)) + data)
|
||||
|
||||
|
||||
def resolve_gateway_mac(gateway, interface):
|
||||
text = subprocess.run(['ip', 'neigh', 'show', gateway, 'dev', interface],
|
||||
capture_output=True, text=True).stdout
|
||||
match = re.search(r'lladdr ([0-9a-f:]{17})', text)
|
||||
return bytes.fromhex(match.group(1).replace(':', '')) if match else None
|
||||
|
||||
|
||||
def route_via(dst):
|
||||
"""Resolve (gw, dev) for a destination via the kernel routing table."""
|
||||
out = subprocess.run(['ip', 'route', 'get', dst],
|
||||
capture_output=True, text=True).stdout
|
||||
m = re.search(r'via (\S+) dev (\S+)', out)
|
||||
return m.groups() if m else (None, None)
|
||||
|
||||
|
||||
def mac(value):
|
||||
return bytes.fromhex(value.replace(':', ''))
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('mode', choices=['icmp', 'syn'])
|
||||
ap.add_argument('--dst', required=True,
|
||||
help='comma-separated TEST-NET destinations (one-shot '
|
||||
'flows per dst in (a)); single dst for bursts in (b)')
|
||||
ap.add_argument('--count', type=int, default=1, help='packets per dst')
|
||||
ap.add_argument('--port', type=int, default=22)
|
||||
ap.add_argument('--src', default='', help='spoofed TEST-NET source')
|
||||
ap.add_argument('--allow-real-src', action='store_true',
|
||||
help='use the interface address as src (no spoofing)')
|
||||
ap.add_argument('--interface', default='eth0')
|
||||
ap.add_argument('--gateway', default='', help='override next-hop (default: '
|
||||
'kernel route for first --dst)')
|
||||
ap.add_argument('--no-arp', action='store_true',
|
||||
help='skip gateway ping probe, use cached ARP entry only')
|
||||
ap.add_argument('--interval', type=float, default=0.25)
|
||||
ap.add_argument('--pcap', type=Path,
|
||||
help='write fixture pcap instead of sending (never on live)')
|
||||
a = ap.parse_args()
|
||||
|
||||
nets = list(_NETS)
|
||||
dsts = [d.strip() for d in a.dst.split(',') if d.strip()]
|
||||
if not dsts or not all(in_testnet(d) for d in dsts):
|
||||
ap.error('--dst must be TEST-NET address(es) only')
|
||||
if a.src and not in_testnet(a.src):
|
||||
ap.error('--src must be TEST-NET')
|
||||
if a.src and a.allow_real_src:
|
||||
ap.error('--src and --allow-real-src are mutually exclusive')
|
||||
if not 1 <= a.count <= 500 or not 1 <= a.port <= 65535:
|
||||
ap.error('count must be 1..500 and port 1..65535')
|
||||
|
||||
smac = dmac = b'\x00' * 6 # PCAP fixtures need no real Ethernet endpoint
|
||||
def build(src, dst, i):
|
||||
if a.mode == 'icmp':
|
||||
return icmp_packet(src, dst, i, smac, dmac)
|
||||
return syn_packet(src, dst, a.port, i, smac, dmac)
|
||||
|
||||
if a.pcap:
|
||||
packets = []
|
||||
for dst in dsts:
|
||||
for i in range(a.count):
|
||||
packets.append(build(a.src or '198.51.100.7', dst, i))
|
||||
pcap_write(a.pcap, packets)
|
||||
print('wrote %d packets to %s' % (len(packets), a.pcap))
|
||||
return
|
||||
|
||||
gw, dev = route_via(dsts[0])
|
||||
if a.gateway:
|
||||
gw = a.gateway
|
||||
iface = dev or a.interface
|
||||
if not gw:
|
||||
ap.error('no via-route for %s on this sensor' % dsts[0])
|
||||
smac = mac(Path('/sys/class/net', iface, 'address').read_text().strip())
|
||||
dmac = resolve_gateway_mac(gw, iface)
|
||||
if dmac is None:
|
||||
if a.no_arp:
|
||||
ap.error('gateway MAC not cached and --no-arp set')
|
||||
subprocess.run(['ping', '-c1', '-W1', gw], capture_output=True)
|
||||
dmac = resolve_gateway_mac(gw, iface)
|
||||
if dmac is None:
|
||||
ap.error('gateway MAC unresolved for %s' % gw)
|
||||
if not a.src and not a.allow_real_src:
|
||||
ap.error('src defaults to real interface address: pass --src '
|
||||
'or --allow-real-src')
|
||||
if a.allow_real_src:
|
||||
out = subprocess.run(['ip', '-4', 'addr', 'show', iface],
|
||||
capture_output=True, text=True).stdout
|
||||
m = re.search(r'inet (\d+\.\d+\.\d+\.\d+)/', out)
|
||||
if not m:
|
||||
ap.error('no inet address on interface')
|
||||
real = m.group(1)
|
||||
|
||||
sent = 0
|
||||
with socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(0x0800)) as s:
|
||||
s.bind((iface, 0))
|
||||
for dst in dsts:
|
||||
src = a.src or real
|
||||
for i in range(a.count):
|
||||
s.send(build(src, dst, i))
|
||||
sent += 1
|
||||
time.sleep(a.interval)
|
||||
print('sent %d %s packets -> %s' % (sent, a.mode, ','.join(dsts)))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
117
tests/live_e2e.py
Normal file
117
tests/live_e2e.py
Normal file
@@ -0,0 +1,117 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Bounded lab proof: real packets, fresh alerts, router drop counters, TTL cleanup.
|
||||
Run on workstation: python3 tests/live_e2e.py --output /path/to/evidence.json
|
||||
Requires deployed LEAF services. Temporarily uses TTL=180; restores config on exit.
|
||||
"""
|
||||
import argparse,json,subprocess,time
|
||||
from pathlib import Path
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('--sensor',default='yuru@192.168.100.84') # lease drifts; verify by hostname LEAF-PROTO
|
||||
ap.add_argument('--lxc',default='root@172.19.32.3')
|
||||
ap.add_argument('--second-lxc',default='root@172.19.32.5')
|
||||
ap.add_argument('--ssh-key',default=str(Path.home()/'.ssh/id_ed25519_alice'))
|
||||
ap.add_argument('--output',type=Path,required=True)
|
||||
a=ap.parse_args(); root=Path(__file__).resolve().parents[1]
|
||||
ssh=['ssh','-F','/dev/null','-i',a.ssh_key,'-o','BatchMode=yes','-o','ConnectTimeout=8']
|
||||
scp=['scp','-F','/dev/null','-i',a.ssh_key,'-o','BatchMode=yes']
|
||||
def run(host,cmd):
|
||||
return subprocess.run(ssh+[host,cmd],capture_output=True,text=True,check=True,timeout=150).stdout
|
||||
def status(since=0.0):
|
||||
return json.loads(run(a.sensor,f'sudo -n python3 /tmp/leaf-live-helper.py status --since {since}'))
|
||||
def syn(n):
|
||||
print(run(a.lxc,f'python3 /tmp/leaf-send-syn.py 203.0.113.12 192.0.2.201 22 {n}').strip(),flush=True)
|
||||
def ping(n):
|
||||
# Different source hosts guarantee distinct Suricata ICMP flows.
|
||||
for i in range(n):
|
||||
host=a.lxc if i%2==0 else a.second_lxc
|
||||
run(host,'ping -c 1 -W 1 192.0.2.202 >/dev/null 2>&1 || test $? -eq 1')
|
||||
evidence={'started':time.time(),'passed':False}
|
||||
changed=False
|
||||
try:
|
||||
subprocess.run(scp+[str(root/'tests/live_helper.py'),a.sensor+':/tmp/leaf-live-helper.py'],check=True)
|
||||
subprocess.run(scp+[str(root/'tests/syn_packets.py'),a.lxc+':/tmp/leaf-send-syn.py'],check=True)
|
||||
baseline=status(); evidence['baseline']=baseline
|
||||
targets={'203.0.113.12','192.0.2.202'}
|
||||
if baseline['members'] or baseline['blocks']: raise RuntimeError('live proof requires an idle empty block group/state for counter attribution')
|
||||
run(a.sensor,'sudo -n python3 /tmp/leaf-live-helper.py short-ttl')
|
||||
changed=True
|
||||
run(a.sensor,'sudo -n systemctl restart leaf-controller')
|
||||
t0=time.time();evidence['traffic_started']=t0
|
||||
capture=subprocess.Popen(ssh+[a.sensor,f"sudo -n timeout 15 tcpdump -ni eth1 -c 24 -w /tmp/leaf-live-wave-{int(t0)}.pcap 'tcp and src host 203.0.113.12'"],stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True)
|
||||
time.sleep(1);syn(24);ping(2)
|
||||
out,err=capture.communicate(timeout=20)
|
||||
evidence['capture']={'exit':capture.returncode,'summary':err.strip()}
|
||||
if capture.returncode!=0 or '24 packets captured' not in err: raise RuntimeError('capture did not prove 24 packets')
|
||||
deadline=time.time()+180
|
||||
while time.time()<deadline:
|
||||
current=status(t0)
|
||||
applied={row['ip'] for row in current['blocks'] if row['applied']}
|
||||
verified={row['ip'] for row in current['audit'] if row.get('action')=='block' and row.get('verified')}
|
||||
if targets <= set(current['members']) & applied & verified: break
|
||||
print('waiting for router commits',flush=True);time.sleep(3)
|
||||
else: raise RuntimeError('both blocks were not verified')
|
||||
evidence['blocked']=current
|
||||
# Detection evidence comes from the controller's own audit trail
|
||||
# (decisions.jsonl, surfaced by the helper as audit rows), NOT from
|
||||
# /var/log/suricata-leaf/eve.json: since the unix_stream migration,
|
||||
# the eve FILE only carries stats; alerts flow over the socket and
|
||||
# land as 'decision' rows with sid. Both sids must be represented.
|
||||
ids=[r['sid'] for r in current['audit'] if r.get('action')=='decision']
|
||||
if not {1000030,1000001} <= set(ids):
|
||||
# 'decision' rows are written the moment the alert lands on the
|
||||
# socket; the block row lands up to ~60 s later (VyOS commit). If
|
||||
# the reads disagree, re-poll briefly before declaring failure.
|
||||
grace=time.time()+30
|
||||
while not {1000030,1000001} <= set(ids):
|
||||
if time.time()>=grace: raise RuntimeError('fresh detection evidence missing')
|
||||
time.sleep(5)
|
||||
current=status(t0); evidence['blocked']=current
|
||||
ids=[r['sid'] for r in current['audit'] if r.get('action')=='decision']
|
||||
print('Both blocks verified; testing real drop counters',flush=True)
|
||||
before=current['drop_counters'];syn(3);ping(1)
|
||||
current=status(t0); evidence['enforced']=current
|
||||
for rule in ('8','9'):
|
||||
if current['drop_counters'].get(rule,0)<=before.get(rule,0): raise RuntimeError('drop counter did not increment for rule '+rule)
|
||||
print('Source and destination drop counters increased; waiting for TTL cleanup',flush=True)
|
||||
deadline=time.time()+300
|
||||
while time.time()<deadline:
|
||||
current=status(t0)
|
||||
tracked={row['ip'] for row in current['blocks']}
|
||||
if not targets & (set(current['members']) | tracked): break
|
||||
time.sleep(5)
|
||||
else: raise RuntimeError('TTL cleanup did not remove test members')
|
||||
evidence['expired']=current
|
||||
# With no remaining IDS-BLOCK members, post-expiry probes must not
|
||||
# increase either IDS drop counter.
|
||||
before=current['drop_counters'].get('8',0); before9=current['drop_counters'].get('9',0)
|
||||
syn(3);ping(1)
|
||||
time.sleep(2)
|
||||
current=status(t0);evidence['after_expiry_probes']=current
|
||||
if current['drop_counters'].get('8',0)>before or current['drop_counters'].get('9',0)>before9:
|
||||
raise RuntimeError('test drop counters still increment after expiry')
|
||||
unblocked={e['ip'] for e in current['audit'] if e.get('action')=='unblock' and e.get('verified')}
|
||||
if not targets<=unblocked: raise RuntimeError('verified unblock audit missing')
|
||||
final=status();evidence['final']=final
|
||||
if final['protected_hashes']!=baseline['protected_hashes']: raise RuntimeError('protected router configuration changed')
|
||||
evidence['passed']=True
|
||||
print('LIVE E2E PASS: capture -> alert -> durable decisions -> verified router blocks -> drop counters -> TTL cleanup',flush=True)
|
||||
finally:
|
||||
try:
|
||||
if changed:
|
||||
try:
|
||||
if not evidence['passed']:
|
||||
run(a.sensor,f'sudo -n python3 /tmp/leaf-live-helper.py cleanup --since {evidence.get("traffic_started",evidence["started"])}')
|
||||
finally:
|
||||
print(run(a.sensor,'sudo -n python3 /tmp/leaf-live-helper.py restore && sudo -n systemctl restart leaf-controller').strip(),flush=True)
|
||||
except Exception as exc:
|
||||
evidence['restoration_error']=str(exc)
|
||||
evidence['passed']=False
|
||||
raise
|
||||
finally:
|
||||
evidence['finished']=time.time()
|
||||
a.output.parent.mkdir(parents=True,exist_ok=True)
|
||||
a.output.write_text(json.dumps(evidence,indent=2)+'\n')
|
||||
|
||||
if __name__=='__main__':main()
|
||||
60
tests/live_helper.py
Normal file
60
tests/live_helper.py
Normal file
@@ -0,0 +1,60 @@
|
||||
"""Sensor-side helper for the bounded live proof. Never emits credentials."""
|
||||
import argparse,json,sys,time,sqlite3,re,hashlib
|
||||
from pathlib import Path
|
||||
import yaml
|
||||
sys.path.insert(0,'/opt/leaf-controller/controller')
|
||||
from leaf_controller.vyos_client import VyosClient
|
||||
p=argparse.ArgumentParser();p.add_argument('command',choices=['status','short-ttl','restore','cleanup']);p.add_argument('--since',type=float,default=0)
|
||||
a=p.parse_args()
|
||||
config=Path('/etc/leaf-controller/config.yaml'); backup=Path('/run/leaf-e2e-config.original.yaml')
|
||||
if a.command=='short-ttl':
|
||||
if backup.exists(): raise SystemExit('Previous live test backup exists; restore it first')
|
||||
backup.write_bytes(config.read_bytes());backup.chmod(0o600)
|
||||
cfg=yaml.safe_load(config.read_text());cfg['policy']['ttl_seconds']=180; cfg['batch_delay']=3
|
||||
config.write_text(yaml.safe_dump(cfg,sort_keys=False));print('temporary TTL=180, batching=3s');raise SystemExit
|
||||
if a.command=='restore':
|
||||
if backup.exists(): config.write_bytes(backup.read_bytes());backup.unlink()
|
||||
print('original controller configuration restored');raise SystemExit
|
||||
if a.command=='cleanup':
|
||||
con=sqlite3.connect('/var/lib/leaf-controller/state.sqlite',timeout=10)
|
||||
with con:
|
||||
con.execute("UPDATE blocks SET expires=? WHERE ip IN (?,?) AND first_seen>=?",
|
||||
(time.time(), '203.0.113.12', '192.0.2.202', a.since))
|
||||
print('Requested expiry for this run only'); raise SystemExit
|
||||
from leaf_controller.config import load_config, read_key_env
|
||||
cfg=load_config(str(config))
|
||||
key=read_key_env('/etc/leaf-controller/env', cfg.get('vyos.key_env', 'LEAF_VYOS_KEY'))
|
||||
c=VyosClient(cfg.vyos_url(),key,cert_pin=cfg.get('vyos.tls_cert_pin') or None,
|
||||
ca_bundle=cfg.get('vyos.ca_bundle'),tls_verify=cfg.get('vyos.tls_verify',True),
|
||||
timeout=75,retries=1)
|
||||
result={'now':time.time(),'members':c.group_members('IDS-BLOCK')}
|
||||
text=c._post('/show',{'op':'show','path':['firewall','ipv4','name','VEXLAN-MAOSPF-v4']}).get('data','')
|
||||
result['drop_counters']={}
|
||||
for line in text.splitlines():
|
||||
match=re.match(r'^\s*([89])\s+drop\s+\S+\s+(\d+)\s+(\d+)',line)
|
||||
if match: result['drop_counters'][match[1]]=int(match[2])
|
||||
text=c._post('/show',{'op':'show','path':['firewall','ipv4','forward','filter']}).get('data','')
|
||||
result['forward_destination_drops']=0
|
||||
for line in text.splitlines():
|
||||
match=re.match(r'^\s*10\s+drop\s+\S+\s+(\d+)\s+(\d+)',line)
|
||||
if match: result['forward_destination_drops']=int(match[1])
|
||||
result['named_drop_counters']=dict(result['drop_counters'])
|
||||
result['drop_counters']['8']=result['drop_counters'].get('8',0)+result['forward_destination_drops']
|
||||
con=sqlite3.connect('/var/lib/leaf-controller/state.sqlite')
|
||||
result['blocks']=[dict(zip(['ip','kind','expires','applied'],r)) for r in con.execute('select ip,kind,expires,applied from blocks')]
|
||||
result['audit']=[]
|
||||
for line in Path('/var/log/leaf-controller/decisions.jsonl').read_text().splitlines():
|
||||
try: row=json.loads(line)
|
||||
except ValueError: continue
|
||||
if row.get('ts',0)>=a.since and row.get('ip') in ('203.0.113.12','192.0.2.202'): result['audit'].append(row)
|
||||
# The live EVE file is stats-only and can temporarily be absent during rename.
|
||||
# Detection evidence is the audit trail above; do not interpret no file alerts
|
||||
# as proof of no detections, or depend on stats-file availability for API status.
|
||||
result['alerts']=None
|
||||
result['alert_evidence']='controller audit; socket alerts are not journaled to EVE'
|
||||
result['protected_hashes']={}
|
||||
if a.since==0:
|
||||
for path in [['interfaces','ethernet','eth0'],['protocols','ospf'],['protocols','static']]:
|
||||
data=c.retrieve('showConfig',path).get('data')
|
||||
result['protected_hashes']['/'.join(path)]=hashlib.sha256(json.dumps(data,sort_keys=True).encode()).hexdigest()
|
||||
print(json.dumps(result))
|
||||
144
tests/live_negative.py
Normal file
144
tests/live_negative.py
Normal file
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Negative IDS counter drill: benign traffic does not match the IDS drop while a blocked
|
||||
src sits in IDS-BLOCK. Run on workstation: python3 tests/live_negative.py \
|
||||
--output /path/to/evidence.json
|
||||
|
||||
Flow: trigger a real block (sid 1000030 injector, spoofed src 203.0.113.12),
|
||||
then from a DIFFERENT host (second LXC) with a clean TEST-NET src (192.0.2.97)
|
||||
push benign TCP SYN
|
||||
flows (count below the 12-packet sid threshold) at the existing transit dst on
|
||||
a port the drill never touches. Proof while the block is active:
|
||||
- benign SYN packets are captured on the ingress mirror (does not establish destination receipt);
|
||||
- the IDS-BLOCK member for the blocked src is still present and verified;
|
||||
- rule-9 drop counter grows ONLY for the re-probed blocked src, not benign;
|
||||
- the benign source is absent from IDS-BLOCK and the state database.
|
||||
Then TTL expiry cleans up like live_e2e does. TTL=180 for the run; config
|
||||
restored on exit. Never adds listeners/services on the router; uses existing
|
||||
transit destinations. Health and capture metrics are included in the results.
|
||||
"""
|
||||
import argparse,json,subprocess,time,ipaddress
|
||||
from pathlib import Path
|
||||
|
||||
BLOCKED='203.0.113.12'
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('--sensor',default='yuru@192.168.100.84')
|
||||
ap.add_argument('--lxc',default='root@172.19.32.3')
|
||||
ap.add_argument('--benign-host',default='root@172.19.32.5')
|
||||
ap.add_argument('--benign-src',default='192.0.2.97')
|
||||
ap.add_argument('--ssh-key',default=str(Path.home()/'.ssh/id_ed25519_alice'))
|
||||
ap.add_argument('--output',type=Path,required=True)
|
||||
a=ap.parse_args(); root=Path(__file__).resolve().parents[1]
|
||||
try:
|
||||
addr=ipaddress.IPv4Address(a.benign_src)
|
||||
except ValueError:
|
||||
ap.error('--benign-src must be a TEST-NET IPv4 address')
|
||||
if not any(addr in ipaddress.ip_network(n) for n in ('192.0.2.0/24','198.51.100.0/24','203.0.113.0/24')) or a.benign_src==BLOCKED:
|
||||
ap.error('--benign-src must be a distinct TEST-NET address')
|
||||
ssh=['ssh','-F','/dev/null','-i',a.ssh_key,'-o','BatchMode=yes','-o','ConnectTimeout=8']
|
||||
scp=['scp','-F','/dev/null','-i',a.ssh_key,'-o','BatchMode=yes']
|
||||
def run(host,cmd):
|
||||
return subprocess.run(ssh+[host,cmd],capture_output=True,text=True,check=True,timeout=150).stdout
|
||||
def status(since=0):
|
||||
return json.loads(run(a.sensor,f'sudo -n python3 /tmp/leaf-live-helper.py status --since {since}'))
|
||||
def syn(host,src,dst,port,n):
|
||||
print(run(host,f'python3 /tmp/leaf-send-syn.py {src} {dst} {port} {n}').strip(),flush=True)
|
||||
def metrics_tail():
|
||||
lines=run(a.sensor,'sudo -n journalctl -u leaf-controller --since "-4 min" --no-pager | grep "leaf.metrics" | tail -4')
|
||||
capture=run(a.sensor,'sudo -n tail -n 2 /var/log/leaf-controller/capture.log 2>/dev/null || true')
|
||||
return {'health':[l.split('leaf.metrics: ',1)[1] for l in lines.splitlines() if 'leaf.metrics'],
|
||||
'capture_windows':[json.loads(l) for l in capture.splitlines() if l]}
|
||||
evidence={'started':time.time(),'passed':False}
|
||||
changed=False
|
||||
try:
|
||||
subprocess.run(scp+[str(root/'tests/live_helper.py'),a.sensor+':/tmp/leaf-live-helper.py'],check=True)
|
||||
subprocess.run(scp+[str(root/'tests/syn_packets.py'),a.lxc+':/tmp/leaf-send-syn.py'],check=True)
|
||||
subprocess.run(scp+[str(root/'tests/syn_packets.py'),a.benign_host+':/tmp/leaf-send-syn.py'],check=True)
|
||||
baseline=status(); evidence['baseline']=baseline
|
||||
if baseline['members'] or baseline['blocks']: raise RuntimeError('negative proof requires idle empty group/state for counter attribution')
|
||||
if BLOCKED in set(baseline['members']): raise RuntimeError('drill address already blocked; refusing to alter existing ownership')
|
||||
if a.benign_src in set(baseline['members']): raise RuntimeError('benign src already blocked; run clean of it first')
|
||||
# An empty baseline allows rule-9 changes to be attributed to the
|
||||
# test source. Ingress capture establishes observation, not delivery.
|
||||
clean_member_state = not baseline['members']
|
||||
run(a.sensor,'sudo -n python3 /tmp/leaf-live-helper.py short-ttl')
|
||||
changed=True
|
||||
run(a.sensor,'sudo -n systemctl restart leaf-controller')
|
||||
t0=time.time(); evidence['traffic_started']=t0
|
||||
capture=subprocess.Popen(ssh+[a.sensor,f"sudo -n timeout 15 tcpdump -ni eth1 -c 24 -w /tmp/leaf-neg-block-{int(t0)}.pcap 'tcp and src host 203.0.113.12 and dst port 22'"],stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True)
|
||||
time.sleep(1); syn(a.lxc,BLOCKED,'192.0.2.201',22,24)
|
||||
out,err=capture.communicate(timeout=20)
|
||||
evidence['block_trigger']={'exit':capture.returncode,'summary':err.strip()}
|
||||
if capture.returncode!=0 or '24 packets captured' not in err: raise RuntimeError('block-trigger capture did not prove 24 packets')
|
||||
deadline=time.time()+180
|
||||
while time.time()<deadline:
|
||||
current=status(t0)
|
||||
applied={row['ip'] for row in current['blocks'] if row['applied']}
|
||||
verified={row['ip'] for row in current['audit'] if row.get('action')=='block' and row.get('verified')}
|
||||
if BLOCKED in set(current['members']) and BLOCKED in applied and BLOCKED in verified: break
|
||||
print('waiting for router commit',flush=True);time.sleep(3)
|
||||
else: raise RuntimeError('block was not verified')
|
||||
evidence['blocked']=current
|
||||
print(f'block verified; benign flows from {a.benign_src} under active block',flush=True)
|
||||
before_neg=current['drop_counters']
|
||||
neg=subprocess.Popen(ssh+[a.sensor,f"sudo -n timeout 15 tcpdump -ni eth1 -c 12 -w /tmp/leaf-neg-benign-{int(t0)}.pcap 'tcp and src host {a.benign_src} and dst port 80'"],stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True)
|
||||
time.sleep(1); syn(a.benign_host,a.benign_src,'192.0.2.201',80,6)
|
||||
time.sleep(2); syn(a.benign_host,a.benign_src,'192.0.2.201',80,6)
|
||||
out,err=neg.communicate(timeout=20)
|
||||
evidence['benign_capture']={'exit':neg.returncode,'summary':err.strip()}
|
||||
if neg.returncode!=0 or '12 packets captured' not in err: raise RuntimeError('benign flow was not observed on the ingress mirror under active block')
|
||||
current=status(t0); evidence['during_benign']=current
|
||||
if BLOCKED not in set(current['members']): raise RuntimeError('blocked member vanished during benign window')
|
||||
if a.benign_src in set(current['members']): raise RuntimeError('benign src entered IDS-BLOCK')
|
||||
if a.benign_src in {row['ip'] for row in current['blocks']}: raise RuntimeError('benign src entered state db')
|
||||
if clean_member_state and current['drop_counters'].get('9',0)!=before_neg.get('9',0):
|
||||
raise RuntimeError('rule-9 drop counter grew for benign src')
|
||||
print('benign flows observed without IDS drop-counter growth; re-probing blocked src',flush=True)
|
||||
before=current['drop_counters']; syn(a.lxc,BLOCKED,'192.0.2.201',22,3)
|
||||
current=status(t0); evidence['enforced']=current
|
||||
if current['drop_counters'].get('9',0)<=before.get('9',0): raise RuntimeError('rule-9 drop counter did not grow for blocked src')
|
||||
if clean_member_state:
|
||||
again=status(t0); before2=again['drop_counters']; syn(a.benign_host,a.benign_src,'192.0.2.201',80,3)
|
||||
current=status(t0); evidence['benign_reprobe']=current
|
||||
if current['drop_counters'].get('9',0)!=before2.get('9',0): raise RuntimeError('rule-9 counter grew for benign re-probe')
|
||||
# Socket mode has no file alerts. State/member absence above and counters
|
||||
# are the evidence; ingress capture cannot prove destination delivery.
|
||||
print('drop counter grew only for blocked src; waiting for TTL cleanup',flush=True)
|
||||
deadline=time.time()+300
|
||||
while time.time()<deadline:
|
||||
current=status(t0)
|
||||
tracked={row['ip'] for row in current['blocks']}
|
||||
if BLOCKED not in set(current['members'])|tracked: break
|
||||
time.sleep(5)
|
||||
else: raise RuntimeError('TTL cleanup did not remove drill member')
|
||||
evidence['expired']=current
|
||||
counters=current['drop_counters']; syn(a.lxc,BLOCKED,'192.0.2.201',22,3)
|
||||
current=status(t0); evidence['after_expiry_probes']=current
|
||||
if current['drop_counters']!=counters: raise RuntimeError('drill drop counters still increment after expiry')
|
||||
unblocked={row['ip'] for row in current['audit'] if row.get('action')=='unblock' and row.get('verified')}
|
||||
if BLOCKED not in unblocked: raise RuntimeError('verified unblock audit missing')
|
||||
final=status(); evidence['final']=final
|
||||
if final['protected_hashes']!=baseline['protected_hashes']: raise RuntimeError('protected router configuration changed')
|
||||
evidence['metrics']=metrics_tail()
|
||||
evidence['passed']=True
|
||||
print('NEGATIVE DRILL PASS: block verified -> benign flows observed on mirror -> '
|
||||
'rule-9 grows only for blocked src -> benign never entered IDS-BLOCK -> TTL cleanup',flush=True)
|
||||
finally:
|
||||
try:
|
||||
if changed:
|
||||
try:
|
||||
if not evidence['passed']:
|
||||
run(a.sensor,f'sudo -n python3 /tmp/leaf-live-helper.py cleanup --since {evidence.get("traffic_started",evidence["started"])}')
|
||||
finally:
|
||||
print(run(a.sensor,'sudo -n python3 /tmp/leaf-live-helper.py restore && sudo -n systemctl restart leaf-controller').strip(),flush=True)
|
||||
except Exception as exc:
|
||||
evidence['restoration_error']=str(exc)
|
||||
evidence['passed']=False
|
||||
raise
|
||||
finally:
|
||||
evidence['finished']=time.time()
|
||||
a.output.parent.mkdir(parents=True,exist_ok=True)
|
||||
a.output.write_text(json.dumps(evidence,indent=2)+'\n')
|
||||
|
||||
if __name__=='__main__':main()
|
||||
176
tests/mock_vyos_server.py
Normal file
176
tests/mock_vyos_server.py
Normal file
@@ -0,0 +1,176 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Mock VyOS HTTP API (2026.03 subset) for offline E2E tests.
|
||||
|
||||
Endpoints: GET /info ; POST /configure|/retrieve|/show|/config-file (multipart form: data, key)
|
||||
State: in-memory address-group; member list mirrored to --state-file on every mutation.
|
||||
Semantics mirrored from docs: bad key -> 401; semantic errors -> success:false + error.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import email.parser
|
||||
import json
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
STATE = {"members": set(), "config_comments": {}}
|
||||
LOCK = threading.Lock()
|
||||
|
||||
|
||||
def dump_state(path: str) -> None:
|
||||
with open(path, "w") as fh:
|
||||
json.dump({"members": sorted(STATE["members"])}, fh)
|
||||
|
||||
|
||||
def parse_form(handler) -> dict:
|
||||
ctype = handler.headers.get("Content-Type", "")
|
||||
length = int(handler.headers.get("Content-Length", "0"))
|
||||
body = handler.rfile.read(length)
|
||||
if "multipart/form-data" in ctype:
|
||||
import email
|
||||
import email.policy
|
||||
msg = email.parser.BytesParser(policy=email.policy.compat32).parsebytes(
|
||||
b"Content-Type: " + ctype.encode() + b"\r\nMIME-Version: 1.0\r\n\r\n" + body)
|
||||
out = {}
|
||||
for part in msg.walk():
|
||||
if part.get_content_type() == "multipart/form-data":
|
||||
continue
|
||||
disp = part.get("Content-Disposition", "")
|
||||
if "name=" not in disp:
|
||||
continue
|
||||
name = disp.split('name="', 1)[1].split('"', 1)[0]
|
||||
out[name] = part.get_payload(decode=True).decode()
|
||||
return out
|
||||
# urlencoded fallback
|
||||
from urllib.parse import parse_qs
|
||||
return {k: v[0] for k, v in parse_qs(body.decode()).items()}
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
key_expected = "testkey"
|
||||
state_file = "mock_state.json"
|
||||
log_lines = []
|
||||
|
||||
def log_message(self, fmt, *args): # quiet
|
||||
Handler.log_lines.append(fmt % args)
|
||||
|
||||
def _send(self, code: int, obj) -> None:
|
||||
body = json.dumps(obj).encode()
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self):
|
||||
if self.path == "/info":
|
||||
self._send(200, {"version": "VYOS mock (22.04 mock for 2026.03 semantics)",
|
||||
"api": {"status": "healthy"}})
|
||||
else:
|
||||
self._send(404, {"success": False, "error": "not found"})
|
||||
|
||||
def do_POST(self):
|
||||
form = parse_form(self)
|
||||
if form.get("key") != self.key_expected:
|
||||
self._send(401, {"success": False, "error": "bad auth token"})
|
||||
return
|
||||
try:
|
||||
payload = json.loads(form.get("data", "{}"))
|
||||
except json.JSONDecodeError:
|
||||
self._send(400, {"success": False, "error": "data is not valid json"})
|
||||
return
|
||||
path = self.path.rstrip("/")
|
||||
try:
|
||||
if path == "/configure":
|
||||
self.handle_configure(payload)
|
||||
elif path == "/retrieve":
|
||||
self.handle_retrieve(payload)
|
||||
elif path == "/show":
|
||||
self._send(200, {"success": True, "data": "\n".join(
|
||||
f"[{k}] {v}" for k, v in STATE["config_comments"].items())})
|
||||
elif path == "/config-file":
|
||||
if payload.get("op") == "save":
|
||||
self._send(200, {"success": True})
|
||||
else:
|
||||
self._send(400, {"success": False, "error": "unknown op"})
|
||||
else:
|
||||
self._send(404, {"success": False, "error": "no endpoint"})
|
||||
except Exception as exc: # mock: keep simple
|
||||
self._send(500, {"success": False, "error": f"mock internal: {exc}"})
|
||||
|
||||
# ------------------------------------------------------------ configure
|
||||
def _group_path(self, tokens) -> bool:
|
||||
return (len(tokens) >= 6 and tokens[:4] ==
|
||||
["firewall", "group", "address-group", "IDS-BLOCK"] and
|
||||
tokens[4] == "address")
|
||||
|
||||
@staticmethod
|
||||
def _member(tokens) -> str:
|
||||
return tokens[5]
|
||||
|
||||
def handle_configure(self, payload) -> None:
|
||||
ops = payload if isinstance(payload, list) else [payload]
|
||||
with LOCK:
|
||||
for op in ops:
|
||||
if not isinstance(op, dict):
|
||||
self._send(400, {"success": False, "error": "op must be object"})
|
||||
return
|
||||
kind, tokens = op.get("op"), op.get("path", [])
|
||||
if kind not in ("set", "delete", "comment") or not self._group_path(tokens):
|
||||
self._send(400, {"success": False,
|
||||
"error": f"mock: unhandled op/path {kind} {tokens}"})
|
||||
return
|
||||
member = self._member(tokens)
|
||||
if kind == "comment":
|
||||
STATE["config_comments"][member] = op.get("comment", "")
|
||||
continue
|
||||
if kind == "set":
|
||||
STATE["members"].add(member)
|
||||
else:
|
||||
STATE["members"].discard(member)
|
||||
dump_state(self.state_file)
|
||||
self._send(200, {"success": True,
|
||||
"comments": [STATE["config_comments"].get(self._member(o["path"]), "")
|
||||
for o in ops
|
||||
if isinstance(o.get("path"), list) and self._group_path(o["path"])]})
|
||||
|
||||
# ------------------------------------------------------------ retrieve
|
||||
def handle_retrieve(self, payload) -> None:
|
||||
if payload.get("op") == "showConfig" and payload.get("path") == ["firewall"]:
|
||||
self._send(200, {"success": True, "data": {"ipv4": {"forward": {"filter": {"rule": {
|
||||
"8": {"action": "drop", "destination": {"group": {"address-group": "IDS-BLOCK"}}},
|
||||
"9": {"action": "drop", "source": {"group": {"address-group": "IDS-BLOCK"}}}
|
||||
}}}}}})
|
||||
return
|
||||
if payload.get("op") != "returnValues":
|
||||
self._send(400, {"success": False, "error": "mock: only returnValues supported"})
|
||||
return
|
||||
tokens = payload.get("path", [])
|
||||
if (len(tokens) == 5 and tokens[:4] ==
|
||||
["firewall", "group", "address-group", "IDS-BLOCK"] and tokens[4] == "address"):
|
||||
with LOCK:
|
||||
self._send(200, {"success": True, "data": sorted(STATE["members"])})
|
||||
else:
|
||||
self._send(400, {"success": False, "error": f"mock: no values for {tokens}"})
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--port", type=int, default=18443)
|
||||
ap.add_argument("--state-file", default="mock_state.json")
|
||||
ap.add_argument("--key", default="testkey")
|
||||
args = ap.parse_args()
|
||||
Handler.state_file = args.state_file
|
||||
Handler.key_expected = args.key
|
||||
try: # allow pre-seeding foreign members
|
||||
with open(args.state_file) as fh:
|
||||
STATE["members"] = set(json.load(fh).get("members", []))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
srv = ThreadingHTTPServer(("127.0.0.1", args.port), Handler)
|
||||
print(f"mock vyos api on 127.0.0.1:{args.port} state={args.state_file}", flush=True)
|
||||
srv.serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
31
tests/replay_fixture.py
Normal file
31
tests/replay_fixture.py
Normal file
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run on the sensor as root: offline packet/rule proof with isolated outputs."""
|
||||
import json,subprocess,tempfile
|
||||
from pathlib import Path
|
||||
import yaml
|
||||
from syn_packets import packet
|
||||
import struct,time
|
||||
|
||||
def main():
|
||||
root=Path(tempfile.mkdtemp(prefix='leaf-replay-'))
|
||||
pcap=root/'fixture.pcap'
|
||||
with pcap.open('wb') as out:
|
||||
out.write(struct.pack('<IHHIIII',0xa1b2c3d4,2,4,0,0,65535,1))
|
||||
for i in range(24):
|
||||
frame=packet('203.0.113.12','192.0.2.201',22,i)
|
||||
out.write(struct.pack('<IIII',int(time.time()),i*1000,len(frame),len(frame))+frame)
|
||||
cfg=yaml.safe_load(Path('/etc/suricata-leaf/suricata-leaf.yaml').read_text())
|
||||
cfg['outputs']=[{'eve-log': {'enabled': True, 'filetype': 'regular',
|
||||
'filename': str(root/'eve.json'), 'types': [{'alert': {'metadata': True}}, 'stats']}}]
|
||||
conf=root/'config.yaml';conf.write_text('%YAML 1.1\n---\n'+yaml.safe_dump(cfg))
|
||||
subprocess.run(['suricata','-r',str(pcap),'-c',str(conf),'-l',str(root),'--runmode','single'],check=True)
|
||||
events=[json.loads(line) for line in (root/'eve.json').read_text().splitlines()]
|
||||
alerts=[e for e in events if e.get('event_type')=='alert']
|
||||
stats=[e['stats'] for e in events if e.get('event_type')=='stats'][-1]
|
||||
assert [e['alert']['signature_id'] for e in alerts]==[1000030,1000030],alerts
|
||||
assert stats['decoder']['pkts']==24 and stats['decoder']['invalid']==0
|
||||
assert stats['tcp']['syn']==24 and stats['tcp']['invalid_checksum']==0
|
||||
assert stats['defrag']['ipv4']['fragments']==0
|
||||
print('OFFLINE REPLAY PASS: 24 valid SYNs, zero fragments/checksum errors, two sid 1000030 alerts')
|
||||
print('Artifacts:',root)
|
||||
if __name__=='__main__':main()
|
||||
166
tests/run_e2e.sh
Normal file
166
tests/run_e2e.sh
Normal file
@@ -0,0 +1,166 @@
|
||||
#!/usr/bin/env bash
|
||||
# E2E prototype test: mock VyOS API + fake Suricata EVE -> controller decisions.
|
||||
# Run from repo root: bash tests/run_e2e.sh
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
T="$ROOT/tests/.e2e"
|
||||
PORT=18443
|
||||
VAULT="$(python3 -c 'import secrets; print(secrets.token_urlsafe(18))')" # ephemeral test credential
|
||||
|
||||
rm -rf "$T"; mkdir -p "$T"
|
||||
trap 'kill $(cat "$T/pids" 2>/dev/null) 2>/dev/null' EXIT
|
||||
|
||||
# seed a foreign member directly into mock state (controller should adopt+expire it)
|
||||
python3 - "$T/mock_state.json" <<'PY'
|
||||
import json,sys
|
||||
json.dump({"members": ["198.51.100.7"]}, open(sys.argv[1],"w"))
|
||||
PY
|
||||
|
||||
# --- 1. mock vyos api ---------------------------------------------------
|
||||
python3 "$ROOT/tests/mock_vyos_server.py" --port $PORT --key "$VAULT" \
|
||||
--state-file "$T/mock_state.json" > "$T/mock.log" 2>&1 &
|
||||
echo $! >> "$T/pids"
|
||||
sleep 1
|
||||
grep -q "mock vyos api" "$T/mock.log" || { echo "FAIL: mock api not started"; cat "$T/mock.log"; exit 1; }
|
||||
|
||||
# --- 2. controller config ------------------------------------------------
|
||||
cat > "$T/config.yaml" <<YAML
|
||||
vyos:
|
||||
url: http://127.0.0.1:$PORT
|
||||
key_env: LEAF_VAULT
|
||||
tls_verify: true
|
||||
block_group: IDS-BLOCK
|
||||
adopt_foreign_members: true
|
||||
eve:
|
||||
path: $T/eve.json
|
||||
alert_socket:
|
||||
path: "${LEAF_E2E_SOCKET:-}"
|
||||
queue_size: 1000
|
||||
policy:
|
||||
mode: threshold
|
||||
min_hits: 2
|
||||
window_seconds: 300
|
||||
ttl_seconds: 6
|
||||
max_blocks: 10
|
||||
block_src: true
|
||||
state_db: $T/state.sqlite
|
||||
audit_log: $T/decisions.jsonl
|
||||
poll_interval: 0.5
|
||||
sync_interval: 4.0
|
||||
YAML
|
||||
touch "$T/eve.json"
|
||||
send_event() {
|
||||
if [ -n "${LEAF_E2E_SOCKET:-}" ]; then
|
||||
python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX); s.connect(sys.argv[1]); s.sendall(sys.stdin.buffer.read()); s.close()' "$LEAF_E2E_SOCKET"
|
||||
else
|
||||
cat >> "$T/eve.json"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- 3. run controller ---------------------------------------------------
|
||||
( cd "$ROOT/controller" && LEAF_VAULT=$VAULT exec python3 -m leaf_controller.main \
|
||||
--config "$T/config.yaml" ) > "$T/controller.log" 2>&1 &
|
||||
echo $! >> "$T/pids"
|
||||
sleep 2
|
||||
|
||||
alert() { # sid dest_ip action
|
||||
python3 -c '
|
||||
import json,sys
|
||||
print(json.dumps({"timestamp":"2026-11-03T12:00:00.000000Z","event_type":"alert",
|
||||
"src_ip":"203.0.113.7","dest_ip":sys.argv[2],"src_port":41000,"dest_port":80,
|
||||
"proto":"TCP","alert":{"signature_id":int(sys.argv[1]),"signature":"e2e test",
|
||||
"severity":2,"action":"allowed",
|
||||
"metadata":{"leaf_action":[sys.argv[3]]}}}))' "$1" "$2" "$3" | send_event; }
|
||||
|
||||
# --- 4. threshold: first hit no-op, second hit must block 192.0.2.5 ------
|
||||
alert 1000001 192.0.2.5 block; sleep 1.5
|
||||
alert 1000001 192.0.2.5 block; sleep 2
|
||||
grep -q "192.0.2.5" "$T/mock_state.json" || { echo "FAIL: expected block of 192.0.2.5"; cat "$T/controller.log"; exit 1; }
|
||||
|
||||
# --- 5. allow-list: LAN dest must never be blockable ----------------------
|
||||
alert 1000001 10.66.0.9 block; alert 1000001 10.66.0.9 block; sleep 2
|
||||
grep -q "10.66.0.9" "$T/mock_state.json" && { echo "FAIL: allowlisted IP was blocked!"; exit 1; }
|
||||
|
||||
# --- 6. log rotation: rotate eve.json, new events must still be seen -----
|
||||
mv "$T/eve.json" "$T/eve.json.1"; touch "$T/eve.json"
|
||||
send_event() {
|
||||
if [ -n "${LEAF_E2E_SOCKET:-}" ]; then
|
||||
python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX); s.connect(sys.argv[1]); s.sendall(sys.stdin.buffer.read()); s.close()' "$LEAF_E2E_SOCKET"
|
||||
else
|
||||
cat >> "$T/eve.json"
|
||||
fi
|
||||
}
|
||||
alert 1000002 192.0.2.9 block; sleep 1
|
||||
alert 1000002 192.0.2.9 block; sleep 2
|
||||
grep -q "192.0.2.9" "$T/mock_state.json" || { echo "FAIL: rotation: expected block of 192.0.2.9"; cat "$T/controller.log"; exit 1; }
|
||||
|
||||
# --- 7. foreign member adoption: 198.51.100.7 was adopted, then expires --
|
||||
grep -q "adopted foreign member 198.51.100.7" "$T/controller.log" || { echo "FAIL: foreign member not adopted"; grep -i adopt "$T/controller.log"; exit 1; }
|
||||
|
||||
# --- 8. TTL expiry: both blocked IPs disappear within ~10s ---------------
|
||||
end=$((SECONDS+15))
|
||||
while [ $SECONDS -lt $end ]; do
|
||||
if ! grep -q "192.0.2.5" "$T/mock_state.json" && ! grep -q "192.0.2.9" "$T/mock_state.json"; then break; fi
|
||||
sleep 1
|
||||
done
|
||||
grep -q "192.0.2.5" "$T/mock_state.json" && { echo "FAIL: TTL expiry didn't unblock 192.0.2.5"; cat "$T/mock_state.json"; exit 1; }
|
||||
grep -q "192.0.2.9" "$T/mock_state.json" && { echo "FAIL: TTL expiry didn't unblock 192.0.2.9"; cat "$T/mock_state.json"; exit 1; }
|
||||
|
||||
# --- 9b. src-mode: block_src metadata + policy.block_src=true ------------
|
||||
# attacker src from TEST-NET, victim on allowlisted LAN -> src must be blocked,
|
||||
# victim dest must NOT be blocked, and allowlisted src must be ignored.
|
||||
# allowlisted (RFC1918-floor) src must never be blocked, even in src mode
|
||||
cat > "$T/fakeeve_floor.py" <<'PY'
|
||||
import json,sys
|
||||
print(json.dumps({"timestamp":"2026-11-03T12:04:00.000000Z","event_type":"alert",
|
||||
"src_ip":"10.130.0.5","dest_ip":"10.66.0.9","src_port":41000,"dest_port":22,
|
||||
"proto":"TCP","alert":{"signature_id":1000020,"signature":"e2e floor test","severity":2,
|
||||
"action":"allowed","metadata":{"leaf_action":["block_src"]}}}))
|
||||
PY
|
||||
python3 "$T/fakeeve_floor.py" | send_event; sleep 1
|
||||
python3 "$T/fakeeve_floor.py" | send_event; sleep 2
|
||||
grep -q "10.130.0.5" "$T/mock_state.json" && { echo "FAIL: allowlisted src was blocked!"; exit 1; }
|
||||
|
||||
cat > "$T/fakeeve_src.py" <<'PY'
|
||||
import json,sys
|
||||
# src from TEST-NET (blockable) hitting allowlisted dest
|
||||
line = json.dumps({"timestamp":"2026-11-03T12:05:00.000000Z","event_type":"alert",
|
||||
"src_ip":"198.51.100.42","dest_ip":"10.66.0.9","src_port":41001,"dest_port":22,
|
||||
"proto":"TCP","alert":{"signature_id":1000020,"signature":"e2e src test","severity":2,
|
||||
"action":"allowed","metadata":{"leaf_action":["block_src"]}}})
|
||||
print(line)
|
||||
PY
|
||||
python3 "$T/fakeeve_src.py" | send_event; sleep 1
|
||||
python3 "$T/fakeeve_src.py" | send_event; sleep 2
|
||||
grep -q "198.51.100.42" "$T/mock_state.json" || { echo "FAIL: src-mode block of 198.51.100.42 missing"; cat "$T/controller.log"; exit 1; }
|
||||
|
||||
# --- 9c. unix_stream socket replay: fake suricata client -> controller ----
|
||||
# exercises the real AF_UNIX path end-to-end: the controller listens, a
|
||||
# client connects and streams newline-delimited JSON; the same threshold
|
||||
# pair must block 192.0.2.42 through the socket path alone
|
||||
if [ -n "${LEAF_E2E_SOCKET:-}" ]; then
|
||||
python3 - "$LEAF_E2E_SOCKET" <<'PY'
|
||||
import json, socket, sys
|
||||
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
s.connect(sys.argv[1])
|
||||
rec = {"timestamp": "2026-11-03T12:06:00.000000Z", "event_type": "alert",
|
||||
"src_ip": "203.0.113.77", "dest_ip": "192.0.2.42",
|
||||
"src_port": 41002, "dest_port": 80, "proto": "TCP",
|
||||
"alert": {"signature_id": 1000001, "signature": "socket e2e",
|
||||
"severity": 2, "action": "allowed",
|
||||
"metadata": {"leaf_action": ["block"]}}}
|
||||
for _ in range(2):
|
||||
s.sendall((json.dumps(rec) + "\n").encode())
|
||||
s.close()
|
||||
PY
|
||||
sleep 2
|
||||
grep -q "192.0.2.42" "$T/mock_state.json" || { echo "FAIL: socket replay: expected block of 192.0.2.42"; cat "$T/controller.log"; exit 1; }
|
||||
|
||||
fi
|
||||
|
||||
# --- 10. audit log sanity --------------------------------------------------
|
||||
grep -q '"action": "block"' "$T/decisions.jsonl" || { echo "FAIL: audit missing block entries"; exit 1; }
|
||||
grep -q '"action": "unblock"' "$T/decisions.jsonl" || { echo "FAIL: audit missing unblock entries"; exit 1; }
|
||||
|
||||
echo "E2E PASS"
|
||||
59
tests/syn_packets.py
Normal file
59
tests/syn_packets.py
Normal file
@@ -0,0 +1,59 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Bounded TEST-NET SYN generator; --pcap writes a fixture without sending traffic."""
|
||||
import argparse
|
||||
import ipaddress
|
||||
import re
|
||||
import socket
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
def checksum(data):
|
||||
if len(data)%2: data += b'\0'
|
||||
total=sum(struct.unpack('!%dH'%(len(data)//2),data))
|
||||
while total>>16: total=(total&0xffff)+(total>>16)
|
||||
return (~total)&0xffff
|
||||
|
||||
def packet(src,dst,dport,index,smac=b'\x02\0\0\0\0\1',dmac=b'\x02\0\0\0\0\2',flags=2):
|
||||
a,b=socket.inet_aton(src),socket.inet_aton(dst)
|
||||
tcp=struct.pack('!HHIIBBHHH',40000+index%20000,dport,7000+index,0,0x50,flags,1024,0,0)
|
||||
pseudo=a+b+struct.pack('!BBH',0,6,len(tcp))
|
||||
tcp=tcp[:16]+struct.pack('!H',checksum(pseudo+tcp))+tcp[18:]
|
||||
ip=struct.pack('!BBHHHBBH4s4s',0x45,0,40,index%65536,0,64,6,0,a,b)
|
||||
ip=ip[:10]+struct.pack('!H',checksum(ip))+ip[12:]
|
||||
return dmac+smac+b'\x08\0'+ip+tcp
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('src'); ap.add_argument('dst'); ap.add_argument('port',type=int); ap.add_argument('count',type=int)
|
||||
ap.add_argument('--pcap',type=Path); ap.add_argument('--interface',default='eth0'); ap.add_argument('--gateway',default='172.19.32.1')
|
||||
a=ap.parse_args()
|
||||
nets=[ipaddress.ip_network(n) for n in ('192.0.2.0/24','198.51.100.0/24','203.0.113.0/24')]
|
||||
if not all(any(ipaddress.ip_address(ip) in n for n in nets) for ip in (a.src,a.dst)):
|
||||
ap.error('source and destination must be TEST-NET addresses')
|
||||
if not 1<=a.count<=500 or not 1<=a.port<=65535: ap.error('count must be 1..500 and port 1..65535')
|
||||
if a.pcap:
|
||||
with a.pcap.open('wb') as f:
|
||||
f.write(struct.pack('<IHHIIII',0xa1b2c3d4,2,4,0,0,65535,1))
|
||||
t=time.time()
|
||||
for i in range(a.count):
|
||||
data=packet(a.src,a.dst,a.port,i)
|
||||
stamp=t+i*.001
|
||||
f.write(struct.pack('<IIII',int(stamp),int(stamp%1*1e6),len(data),len(data))+data)
|
||||
print('wrote',a.count,'packets to',a.pcap)
|
||||
return
|
||||
def mac(value): return bytes.fromhex(value.replace(':',''))
|
||||
smac=mac(Path('/sys/class/net',a.interface,'address').read_text().strip())
|
||||
subprocess.run(['ping','-c1','-W1',a.gateway],capture_output=True)
|
||||
text=subprocess.run(['ip','neigh','show',a.gateway,'dev',a.interface],capture_output=True,text=True,check=True).stdout
|
||||
match=re.search(r'lladdr ([0-9a-f:]{17})',text)
|
||||
if not match: ap.error('gateway MAC unresolved')
|
||||
with socket.socket(socket.AF_PACKET,socket.SOCK_RAW,socket.htons(0x0800)) as s:
|
||||
s.bind((a.interface,0))
|
||||
for i in range(a.count):
|
||||
s.send(packet(a.src,a.dst,a.port,i,smac,mac(match.group(1))))
|
||||
time.sleep(.01)
|
||||
print('sent',a.count,'SYNs',a.src,'->',a.dst+':'+str(a.port))
|
||||
|
||||
if __name__=='__main__': main()
|
||||
135
tests/test_alertsock.py
Normal file
135
tests/test_alertsock.py
Normal file
@@ -0,0 +1,135 @@
|
||||
"""Framing tests for the suricata unix_stream listener (suricata is the
|
||||
client; the controller owns the listening socket). Uses real AF_UNIX
|
||||
connections, no suricata binary needed."""
|
||||
import os
|
||||
import socket
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'controller'))
|
||||
from leaf_controller.alertsock import AlertSocket
|
||||
|
||||
|
||||
class AlertSocketTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.path = os.path.join(self.tmp.name, 'alerts.sock')
|
||||
self.sock = AlertSocket(self.path, 100)
|
||||
self.sock.bind()
|
||||
self.sock.start()
|
||||
|
||||
def tearDown(self):
|
||||
self.sock.close()
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _client(self):
|
||||
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
s.connect(self.path)
|
||||
return s
|
||||
|
||||
def _wait(self, n=1, pred=None, timeout=5.0):
|
||||
"""wait until drain() has >= n records, or pred() is true"""
|
||||
deadline = time.time() + timeout
|
||||
recs = []
|
||||
while time.time() < deadline:
|
||||
recs = self.sock.drain()
|
||||
if len(recs) >= n and (pred is None or pred()):
|
||||
return recs
|
||||
time.sleep(0.05)
|
||||
return recs
|
||||
|
||||
def _rec(self, n):
|
||||
return ('{"event_type": "alert", "n": %d}\n' % n).encode()
|
||||
|
||||
def test_single_connection_streams_records(self):
|
||||
s = self._client()
|
||||
s.sendall(self._rec(1) + self._rec(2))
|
||||
s.close()
|
||||
recs = self._wait(2)
|
||||
self.assertEqual([r['n'] for r in recs], [1, 2])
|
||||
|
||||
def test_partial_line_frames_only_on_newline(self):
|
||||
s = self._client()
|
||||
s.sendall(b'{"event_type": "alert", "n"')
|
||||
time.sleep(0.3)
|
||||
self.assertEqual(self.sock.drain(), [])
|
||||
s.sendall(b': 7}\n')
|
||||
recs = self._wait(1)
|
||||
self.assertEqual([r['n'] for r in recs], [7])
|
||||
s.close()
|
||||
|
||||
def test_non_alert_records_counted_other_never_queued(self):
|
||||
# ingress gate: the socket output block carries alerts only; a
|
||||
# stray stats record must never reach policy/state
|
||||
s = self._client()
|
||||
s.sendall(b'{"event_type": "stats", "x": 1}\n')
|
||||
s.close()
|
||||
self._wait(0, pred=lambda: self.sock.counts['other'] >= 1)
|
||||
self.assertEqual(self.sock.drain(), [])
|
||||
self.assertGreaterEqual(self.sock.counts['other'], 1)
|
||||
|
||||
def test_reconnect_yields_continuous_stream(self):
|
||||
self._client().close()
|
||||
s = self._client()
|
||||
s.sendall(self._rec(1) + self._rec(2))
|
||||
s.close()
|
||||
recs = self._wait(2)
|
||||
self.assertEqual([r['n'] for r in recs], [1, 2])
|
||||
self.assertGreaterEqual(self.sock.counts['connections'], 2)
|
||||
|
||||
def test_garbage_and_non_object_lines_counted_not_hung_controller(self):
|
||||
s = self._client()
|
||||
s.sendall(b'not-json\n[1,2,3]\n' + self._rec(1))
|
||||
s.close()
|
||||
# unparseable line -> parse_errors; valid JSON but not an alert
|
||||
# object -> other gate; the good record must still flow through
|
||||
recs = self._wait(1, pred=lambda: self.sock.counts['parse_errors'] >= 1
|
||||
and self.sock.counts['other'] >= 1)
|
||||
self.assertEqual([r['n'] for r in recs], [1])
|
||||
self.assertGreaterEqual(self.sock.counts['parse_errors'], 1)
|
||||
self.assertGreaterEqual(self.sock.counts['other'], 1)
|
||||
|
||||
def test_queue_overflow_drops_are_counted(self):
|
||||
# bounded queue: a stalled consumer forces the eager-drain semantics
|
||||
# and losses must be visible in counts, never silently dropped
|
||||
self.sock.close()
|
||||
self.sock = AlertSocket(self.path, 1)
|
||||
self.sock.bind()
|
||||
self.sock.start()
|
||||
s = self._client()
|
||||
s.sendall(self._rec(1) + self._rec(2) + self._rec(3))
|
||||
s.close()
|
||||
deadline = time.time() + 5.0
|
||||
while time.time() < deadline and self.sock.counts['queue_drops'] < 1:
|
||||
time.sleep(0.05)
|
||||
self.assertGreaterEqual(self.sock.counts['queue_drops'], 1)
|
||||
recs = self.sock.drain()
|
||||
self.assertEqual(len(recs), 1) # exactly the survivor of the burst
|
||||
|
||||
def test_close_removes_socket_file(self):
|
||||
self.assertTrue(os.path.exists(self.path))
|
||||
self.sock.close()
|
||||
self.assertFalse(os.path.exists(self.path))
|
||||
|
||||
def test_bind_replaces_stale_socket_path(self):
|
||||
# An unowned socket whose listener has exited can be reclaimed.
|
||||
# Active listeners and separately locked socket paths are protected.
|
||||
self.sock.close()
|
||||
stale = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
stale.bind(self.path)
|
||||
stale.close()
|
||||
self.sock = AlertSocket(self.path, 10)
|
||||
self.sock.bind()
|
||||
self.sock.start()
|
||||
s = self._client()
|
||||
s.sendall(self._rec(1))
|
||||
s.close()
|
||||
recs = self._wait(1)
|
||||
self.assertEqual([r['n'] for r in recs], [1])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
448
tests/test_controller.py
Normal file
448
tests/test_controller.py
Normal file
@@ -0,0 +1,448 @@
|
||||
"""Offline regression tests. No lab access or router writes."""
|
||||
import copy
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'controller'))
|
||||
from leaf_controller.config import Config, DEFAULTS
|
||||
from leaf_controller.eve import EveTailer
|
||||
from leaf_controller.main import Controller
|
||||
from leaf_controller.vyos_client import (
|
||||
PinMismatchError, VyosClient, VyosApiError, normalize_pin)
|
||||
|
||||
class Router:
|
||||
member_op = staticmethod(VyosClient.member_op)
|
||||
def __init__(self, members=()):
|
||||
self.members = set(members)
|
||||
self.commits = []
|
||||
self.fail = False
|
||||
self.after_commit = None
|
||||
def op_with_retry(self, fn, *args): return fn(*args)
|
||||
def group_members(self, group): return sorted(self.members)
|
||||
def configure(self, ops):
|
||||
self.commits.append(ops)
|
||||
for op in ops:
|
||||
ip = op['path'][-1]
|
||||
if op['op'] == 'set': self.members.add(ip)
|
||||
else: self.members.discard(ip)
|
||||
if self.after_commit: self.after_commit()
|
||||
if self.fail: raise TimeoutError('commit acknowledged too late')
|
||||
return {'success': True}
|
||||
def save_config(self): pass
|
||||
|
||||
class Tests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.tmp.name)
|
||||
raw = copy.deepcopy(DEFAULTS)
|
||||
raw.update(state_db=str(self.root/'state.sqlite'), audit_log=str(self.root/'audit.jsonl'))
|
||||
raw['policy'].update(mode='immediate', block_src=True)
|
||||
with patch.object(Config, 'vyos_key', return_value='offline-test'):
|
||||
self.ctl = Controller(Config(raw), dry_run=False)
|
||||
self.router = self.ctl.client = Router()
|
||||
def tearDown(self):
|
||||
self.ctl.close()
|
||||
self.tmp.cleanup()
|
||||
def alert(self, ip='192.0.2.77', action='block', src='203.0.113.11'):
|
||||
return {'event_type':'alert','src_ip':src,'dest_ip':ip,
|
||||
'alert':{'signature_id':1000001,'metadata':{'leaf_action':[action]}}}
|
||||
def test_decision_is_durable_without_api(self):
|
||||
self.ctl.handle_event(self.alert())
|
||||
self.assertIsNotNone(self.ctl.state.get('192.0.2.77'))
|
||||
self.assertFalse(self.router.commits)
|
||||
def test_batch_dedup_readback_and_expiry(self):
|
||||
for ip in ('192.0.2.77','192.0.2.78','192.0.2.77'):
|
||||
self.ctl.handle_event(self.alert(ip))
|
||||
self.ctl.sync_router_state()
|
||||
self.assertEqual(len(self.router.commits),1)
|
||||
self.assertEqual(len(self.router.commits[0]),2)
|
||||
self.assertTrue(self.ctl.state.get('192.0.2.77').applied)
|
||||
self.ctl.state.con.execute('UPDATE blocks SET expires=?',(time.time()-1,))
|
||||
self.ctl.state.con.commit()
|
||||
self.ctl.sync_router_state()
|
||||
self.assertEqual(self.router.members,set())
|
||||
self.assertEqual(self.ctl.state.all_blocks(),[])
|
||||
def test_expired_startup_not_adopted(self):
|
||||
self.ctl.state.upsert_hit('192.0.2.77','dest',1,1,now=time.time()-10)
|
||||
self.router.members.add('192.0.2.77')
|
||||
self.ctl.sync_router_state()
|
||||
self.assertIsNone(self.ctl.state.get('192.0.2.77'))
|
||||
self.assertFalse(self.router.members)
|
||||
def test_state_owned_group_removes_stale_router_members(self):
|
||||
self.ctl.cfg.raw['adopt_foreign_members']=False
|
||||
self.router.members.add('192.0.2.77')
|
||||
self.ctl.sync_router_state()
|
||||
self.assertFalse(self.router.members)
|
||||
self.assertEqual(self.ctl.state.all_blocks(),[])
|
||||
|
||||
def test_timeout_verified_before_resubmit(self):
|
||||
self.ctl.handle_event(self.alert())
|
||||
self.router.fail=True
|
||||
with self.assertRaises(TimeoutError): self.ctl.sync_router_state()
|
||||
self.router.fail=False
|
||||
self.ctl.sync_router_state()
|
||||
self.assertEqual(len(self.router.commits),1)
|
||||
self.assertTrue(self.ctl.state.get('192.0.2.77').applied)
|
||||
def test_refresh_during_delete_is_retained(self):
|
||||
self.ctl.state.upsert_hit('192.0.2.77','dest',1,1,now=time.time()-10)
|
||||
self.router.members.add('192.0.2.77')
|
||||
self.router.after_commit=lambda: self.ctl.handle_event(self.alert())
|
||||
self.ctl.sync_router_state()
|
||||
self.assertGreater(self.ctl.state.get('192.0.2.77').expires,time.time())
|
||||
self.assertFalse(self.ctl.state.get('192.0.2.77').applied)
|
||||
self.router.after_commit=None
|
||||
self.ctl.sync_router_state()
|
||||
self.assertIn('192.0.2.77',self.router.members)
|
||||
def test_slow_commit_does_not_block_decisions(self):
|
||||
self.ctl.handle_event(self.alert())
|
||||
started, release = threading.Event(), threading.Event()
|
||||
self.router.after_commit=lambda: (started.set(),release.wait(3))
|
||||
def worker():
|
||||
try: self.ctl.sync_router_state()
|
||||
finally: self.ctl.state.close()
|
||||
t=threading.Thread(target=worker); t.start()
|
||||
self.assertTrue(started.wait(2))
|
||||
t0=time.monotonic()
|
||||
for n in range(10): self.ctl.handle_event(self.alert(f'192.0.2.{n+100}'))
|
||||
elapsed=time.monotonic()-t0
|
||||
release.set(); t.join(3)
|
||||
self.assertLess(elapsed,1)
|
||||
self.assertEqual(self.ctl.state.count_active(),11)
|
||||
def test_floor_src_ipv6_and_bad_events(self):
|
||||
for ip in ('10.0.249.0','172.19.32.3','192.168.100.83','127.0.0.1','0.0.0.0','255.255.255.255','::1','bogus'):
|
||||
self.ctl.handle_event(self.alert(ip))
|
||||
for e in ({'event_type':'alert','alert':None}, {'event_type':'alert','alert':{'signature_id':'bad'}}, {'event_type':'tls'}):
|
||||
self.ctl.handle_event(e)
|
||||
self.assertEqual(self.ctl.state.count_active(),0)
|
||||
self.ctl.handle_event(self.alert(action='block_src'))
|
||||
self.assertIsNotNone(self.ctl.state.get('203.0.113.11'))
|
||||
def test_dry_run_cannot_seed_live_blocks(self):
|
||||
dry = Controller(self.ctl.cfg, dry_run=True)
|
||||
try:
|
||||
dry.handle_event(self.alert())
|
||||
self.assertIsNotNone(dry.state.get('192.0.2.77'))
|
||||
self.assertIsNone(self.ctl.state.get('192.0.2.77'))
|
||||
finally: dry.close()
|
||||
|
||||
def test_cap(self):
|
||||
self.ctl.cfg.raw['policy']['max_blocks']=1
|
||||
self.ctl.handle_event(self.alert())
|
||||
self.ctl.handle_event(self.alert('192.0.2.78'))
|
||||
self.assertEqual(self.ctl.state.count_active(),1)
|
||||
def test_single_instance_and_wal(self):
|
||||
with self.assertRaises(SystemExit): Controller(self.ctl.cfg,dry_run=False)
|
||||
self.assertEqual(self.ctl.state.con.execute('pragma journal_mode').fetchone()[0],'wal')
|
||||
def test_drift_repaired(self):
|
||||
self.ctl.handle_event(self.alert()); self.ctl.sync_router_state()
|
||||
self.router.members.clear(); self.ctl.sync_router_state()
|
||||
self.assertIn('192.0.2.77',self.router.members)
|
||||
def test_partial_write_rotation_truncation(self):
|
||||
f=self.root/'eve.json'; f.write_bytes(b'')
|
||||
tail=EveTailer(str(f),from_start=True); fh=tail._open()
|
||||
f.write_bytes(b'{"event_type":')
|
||||
self.assertEqual(tail._read_ready(fh),[])
|
||||
with f.open('ab') as w: w.write(b'"alert"}\n')
|
||||
lines=tail._read_ready(fh)
|
||||
self.assertEqual(tail._parse(lines[0]),{'event_type':'alert'})
|
||||
f.write_bytes(b'')
|
||||
self.assertTrue(tail._needs_reopen())
|
||||
f.rename(self.root/'eve.old'); f.write_text('{}\n')
|
||||
self.assertTrue(tail._needs_reopen())
|
||||
tail.close()
|
||||
def test_rotation_drains_more_than_one_batch(self):
|
||||
f=self.root/'eve.json'; f.write_text('{}\n'*1100)
|
||||
tail=EveTailer(str(f),poll_interval=.001,from_start=True)
|
||||
tail._open()
|
||||
f.rename(self.root/'eve.old'); f.write_text('{"new":true}\n')
|
||||
events=tail.events()
|
||||
received=[next(events) for _ in range(1101)]
|
||||
self.assertEqual(received[-1],{'new':True})
|
||||
self.assertEqual(received[:-1],[{}]*1100)
|
||||
events.close()
|
||||
|
||||
def test_checkpoint_resumes_alerts_written_during_shutdown(self):
|
||||
f=self.root/'eve.json'; f.write_text('')
|
||||
tail=EveTailer(str(f)); tail._open()
|
||||
self.ctl.state.save_checkpoint(*tail._ino, tail._pos)
|
||||
tail.close()
|
||||
# controller is down; suricata keeps appending
|
||||
with f.open('ab') as w: w.write(b'{"event_type":"alert","missed":true}\n')
|
||||
ck=self.ctl.state.get_checkpoint()
|
||||
resumed=EveTailer(str(f), checkpoint=ck); fh=resumed._open()
|
||||
self.assertEqual([resumed._parse(l) for l in resumed._read_ready(fh)],
|
||||
[{'event_type':'alert','missed':True}])
|
||||
resumed.close()
|
||||
def test_checkpoint_ignored_on_first_run_never_replays_history(self):
|
||||
f=self.root/'eve.json'; f.write_text('{"ancient":true}\n')
|
||||
t=EveTailer(str(f)); fh=t._open()
|
||||
self.assertEqual(t._read_ready(fh),[])
|
||||
self.assertIsNone(self.ctl.state.get_checkpoint())
|
||||
t.close()
|
||||
def test_checkpoint_across_rotation_reads_new_file_from_start(self):
|
||||
f=self.root/'eve.json'; f.write_text('{"old":true}\n')
|
||||
t=EveTailer(str(f)); t._open()
|
||||
self.ctl.state.save_checkpoint(*t._ino, t._pos); t.close()
|
||||
f.rename(self.root/'eve.old'); f.write_text('{"fresh":true}\n')
|
||||
r=EveTailer(str(f), checkpoint=self.ctl.state.get_checkpoint()); fh=r._open()
|
||||
self.assertEqual([r._parse(l) for l in r._read_ready(fh)],[{'fresh':True}])
|
||||
r.close()
|
||||
def test_audit_log_rotation_bounded(self):
|
||||
from leaf_controller.main import Audit
|
||||
p=str(self.root/'dec.jsonl')
|
||||
a=Audit(p, max_bytes=50, keep=2)
|
||||
for i in range(20): a.write({'i':i,'pad':'x'*80})
|
||||
a.close()
|
||||
rotated=list(self.root.glob('dec.jsonl.*'))
|
||||
self.assertEqual(len(rotated),2) # exactly 'keep' retained
|
||||
for old in rotated:
|
||||
self.assertTrue(old.stat().st_size>0)
|
||||
# every rotated line is a valid decision record
|
||||
recs=[]
|
||||
for f in rotated:
|
||||
with open(f) as fh:
|
||||
recs += [json.loads(l) for l in fh]
|
||||
self.assertTrue(all(r['pad']=='x'*80 for r in recs))
|
||||
|
||||
def test_new_file_after_startup_is_not_skipped(self):
|
||||
f=self.root/'missing-eve.json'
|
||||
tail=EveTailer(str(f))
|
||||
self.assertIsNone(tail._open())
|
||||
f.write_text('{"event_type":"alert"}\n')
|
||||
fh=tail._open()
|
||||
self.assertEqual(len(tail._read_ready(fh)),1)
|
||||
tail.close()
|
||||
|
||||
def test_idle_reader_waits(self):
|
||||
f=self.root/'eve.json'; f.touch()
|
||||
tail=EveTailer(str(f),poll_interval=.05)
|
||||
events=tail.events(); start=time.monotonic()
|
||||
self.assertIsNone(next(events))
|
||||
self.assertGreaterEqual(time.monotonic()-start,.04)
|
||||
events.close()
|
||||
def test_packet_headers_and_checksums(self):
|
||||
import pathlib, sys
|
||||
from syn_packets import packet, checksum
|
||||
import socket, struct
|
||||
frame=packet('203.0.113.12','192.0.2.201',22,0)
|
||||
ip,tcp=frame[14:34],frame[34:]
|
||||
self.assertEqual(len(frame),54)
|
||||
self.assertEqual(struct.unpack('!H',ip[6:8])[0],0)
|
||||
self.assertEqual(ip[8:10],bytes([64,6]))
|
||||
self.assertEqual(tcp[13],2)
|
||||
self.assertEqual(checksum(ip),0)
|
||||
pseudo=socket.inet_aton('203.0.113.12')+socket.inet_aton('192.0.2.201')+struct.pack('!BBH',0,6,20)
|
||||
self.assertEqual(checksum(pseudo+tcp),0)
|
||||
def test_unattached_firewall_hook_rejected(self):
|
||||
client=VyosClient('http://example.invalid','test')
|
||||
hook={'action':'drop','source':{'group':{'address-group':'IDS-BLOCK'}}}
|
||||
fw={'ipv4':{'forward':{'filter':{'rule':{'10':{'action':'drop','destination':{'group':{'address-group':'IDS-BLOCK'}}}}}},'name':{'unused':{'rule':{'9':hook}}}}}
|
||||
with patch.object(client,'retrieve',return_value={'data':fw}):
|
||||
with self.assertRaises(VyosApiError): client.verify_enforcement('IDS-BLOCK',True)
|
||||
fw['zone']={'zone':{'MAOSPF':{'from':{'VEXLAN':{'firewall':{'name':'unused'}}}}}}
|
||||
with patch.object(client,'retrieve',return_value={'data':fw}): client.verify_enforcement('IDS-BLOCK',True)
|
||||
|
||||
def test_show_uses_path(self):
|
||||
client=VyosClient('http://example.invalid','test')
|
||||
with patch.object(client,'_post',return_value={'success':True,'data':'counters'}) as post:
|
||||
self.assertEqual(client.show(['firewall']), 'counters')
|
||||
post.assert_called_once_with('/show',{'op':'show','path':['firewall']})
|
||||
|
||||
def test_ambiguous_api_success_rejected(self):
|
||||
client=VyosClient('http://example.invalid','test')
|
||||
response=type('Response',(),{'status_code':200,'json':lambda _: {}})()
|
||||
with patch.object(client.sess,'post',return_value=response):
|
||||
with self.assertRaises(VyosApiError): client.configure([])
|
||||
def test_readback_failure_keeps_pending(self):
|
||||
self.ctl.handle_event(self.alert())
|
||||
self.router.group_members=lambda group: []
|
||||
with self.assertRaises(VyosApiError): self.ctl.sync_router_state()
|
||||
self.assertFalse(self.ctl.state.get('192.0.2.77').applied)
|
||||
|
||||
|
||||
class MetricsTests(unittest.TestCase):
|
||||
"""Stats records update metrics without entering the policy path."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.tmp.name)
|
||||
raw = copy.deepcopy(DEFAULTS)
|
||||
raw.update(state_db=str(self.root/'state.sqlite'),
|
||||
audit_log=str(self.root/'audit.jsonl'))
|
||||
raw['metrics'].update(capture_interval=0, capture_max_bytes=200,
|
||||
capture_log_keep=2)
|
||||
with patch.object(Config, 'vyos_key', return_value='offline-test'):
|
||||
self.ctl = Controller(Config(raw), dry_run=False)
|
||||
self.router = self.ctl.client = Router()
|
||||
|
||||
def tearDown(self):
|
||||
self.ctl.close()
|
||||
self.tmp.cleanup()
|
||||
|
||||
def stats(self, packets=1000, dropped=0, engine_alerts=3, uptime=120):
|
||||
"""Mirrors the live eve.json shape: capture/decoder/detect sub-objects,
|
||||
no 'ifaces' at totals level (threads: no + totals: yes)."""
|
||||
return {'event_type': 'stats', 'stats': {
|
||||
'uptime': uptime, 'memcap_pressure': 0,
|
||||
'capture': {'kernel_packets': packets, 'kernel_drops': dropped,
|
||||
'errors': 0},
|
||||
'decoder': {'pkts': packets, 'bytes': packets * 200, 'invalid': 0},
|
||||
'detect': {'alert': engine_alerts, 'alerts_suppressed': 0,
|
||||
'alert_queue_overflow': 0}}}
|
||||
|
||||
def thread_stats(self, packets=500):
|
||||
"""Live thread records (flush=False): per-thread counters, no uptime."""
|
||||
return {'event_type': 'stats', 'stats': {
|
||||
'capture': {'kernel_packets': packets, 'kernel_drops': 0,
|
||||
'errors': 0},
|
||||
'decoder': {'pkts': packets, 'bytes': packets * 100, 'invalid': 0},
|
||||
'engine': {'alert': 1, 'alerts_suppressed': 0,
|
||||
'alert_queue_overflow': 0}}}
|
||||
|
||||
def test_stats_never_touch_state_router_or_audit(self):
|
||||
# Verify that stats cannot produce policy decisions:
|
||||
# 1) routed stats can only ever be counted
|
||||
for i in range(20):
|
||||
self.ctl._route(self.stats(packets=100 * i))
|
||||
self.ctl._route({'event_type': 'dns', 'src_ip': '192.0.2.77'})
|
||||
self.ctl._route({'event_type': 'http', 'src_ip': '192.0.2.78'})
|
||||
# 2) even a stats record handed straight to handle_event cannot act
|
||||
self.ctl.handle_event(self.stats())
|
||||
self.assertEqual(self.ctl.state.all_blocks(), [])
|
||||
self.assertEqual(self.ctl.state.count_active(), 0)
|
||||
self.assertFalse(self.router.commits)
|
||||
self.assertFalse(self.router.members)
|
||||
self.assertEqual((self.root/'audit.jsonl').read_text(), '')
|
||||
self.assertFalse(list(self.root.glob('capture.log*')))
|
||||
|
||||
def test_health_line_once_per_full_window(self):
|
||||
self.ctl._route(self.stats(packets=1000))
|
||||
self.ctl._emit_metrics(1000.0) # baseline adoption only
|
||||
self.ctl._route(self.stats(packets=2000, dropped=1))
|
||||
with self.assertLogs('leaf.metrics', level='INFO') as logs:
|
||||
self.ctl._emit_metrics(1030.0) # a full window: one line
|
||||
self.ctl._route(self.stats(packets=2100))
|
||||
self.ctl._emit_metrics(1040.0) # partial window: stays silent
|
||||
health = [l for l in logs.output if 'health:' in l]
|
||||
self.assertEqual(len(health), 1)
|
||||
self.assertIn('cps=33.3', health[0]) # 1000 pkts / 30 s window
|
||||
self.assertIn('dropped+=1', health[0])
|
||||
|
||||
def test_counter_restart_never_goes_negative(self):
|
||||
self.ctl._route(self.stats(packets=5000))
|
||||
self.ctl._emit_metrics(1000.0)
|
||||
self.ctl._route(self.stats(packets=100, uptime=30)) # suricata restart
|
||||
rep = self.ctl.metrics.report(1030.0)
|
||||
self.assertTrue(rep['restart'])
|
||||
self.assertEqual(rep['delta']['packets'], 100)
|
||||
self.assertGreaterEqual(rep['cps'], 0)
|
||||
|
||||
def test_totals_authoritative_over_thread_records(self):
|
||||
# live emits alternating thread(flush=False)+totals; mixing the two
|
||||
# counter sets would produce bogus rates
|
||||
self.ctl._route(self.thread_stats(packets=500)) # only kind seen: adopted
|
||||
self.ctl._emit_metrics(1000.0) # baseline
|
||||
self.ctl._route(self.stats(packets=2000)) # totals authoritative
|
||||
self.ctl._route(self.thread_stats(packets=100)) # must not override
|
||||
rep = self.ctl.metrics.report(1030.0)
|
||||
self.assertFalse(rep['restart'])
|
||||
self.assertEqual(rep['delta']['packets'], 1500)
|
||||
|
||||
def test_capture_log_bounded_by_size(self):
|
||||
t = 10000.0
|
||||
for i in range(1, 40):
|
||||
self.ctl._route(self.stats(packets=1000 * i, dropped=i))
|
||||
self.ctl._emit_metrics(t)
|
||||
t += 40
|
||||
current = self.root/'capture.log'
|
||||
rotated = sorted(self.root.glob('capture.log.*'))
|
||||
# bounded growth is the point: current + at most keep rotated files.
|
||||
# Audit rotates write-then-check, so the newest line may itself sit in
|
||||
# the most recent rotated file; current can legitimately be empty.
|
||||
self.assertLessEqual(len(rotated), 2)
|
||||
self.assertTrue(current.exists())
|
||||
recs = []
|
||||
for p in [current] + rotated:
|
||||
for l in p.read_text().splitlines():
|
||||
rec = json.loads(l)
|
||||
self.assertIn('dropped', rec)
|
||||
self.assertIn('delta', rec)
|
||||
recs.append(rec)
|
||||
assert recs
|
||||
# Each retained line is a complete window snapshot.
|
||||
# This check covers the size bound; retention order is tested separately.
|
||||
self.assertTrue(any(r['dropped'] > 0 for r in recs))
|
||||
# audit rotation is: append, size-check, rotate, prune-to-keep — so an
|
||||
# over-cap write can land on disk briefly before the prune settles:
|
||||
# worst settled state is current (~cap) + keep rotated (~cap each)
|
||||
disk = sum(x.stat().st_size for x in [current] + rotated)
|
||||
self.assertLess(disk, 200 * 4 + 400) # (keep+2) x cap + slop
|
||||
|
||||
|
||||
class TLSPinTests(unittest.TestCase):
|
||||
"""Cert-pin trust against a local TLS server with a throwaway cert."""
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
import shutil, ssl, subprocess
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
if shutil.which('openssl') is None:
|
||||
raise unittest.SkipTest('openssl required for TLS pin tests')
|
||||
cls.tmp = tempfile.TemporaryDirectory()
|
||||
key, crt = cls.tmp.name + '/k.pem', cls.tmp.name + '/c.pem'
|
||||
subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes',
|
||||
'-keyout', key, '-out', crt, '-days', '1',
|
||||
'-subj', '/CN=localhost'],
|
||||
check=True, capture_output=True)
|
||||
out = subprocess.run(['openssl', 'x509', '-in', crt, '-noout',
|
||||
'-fingerprint', '-sha256'],
|
||||
check=True, capture_output=True, text=True).stdout
|
||||
cls.pin = out.split('=')[1].strip()
|
||||
|
||||
class H(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
body = b'{"success": true, "data": {"proto": "pin-test"}}'
|
||||
self.send_response(200)
|
||||
self.send_header('Content-Type', 'application/json')
|
||||
self.send_header('Content-Length', str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
def log_message(self, *a):
|
||||
pass
|
||||
|
||||
cls.srv = HTTPServer(('127.0.0.1', 0), H)
|
||||
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||
ctx.load_cert_chain(crt, key)
|
||||
cls.srv.socket = ctx.wrap_socket(cls.srv.socket, server_side=True)
|
||||
cls.port = cls.srv.socket.getsockname()[1]
|
||||
threading.Thread(target=cls.srv.serve_forever, daemon=True).start()
|
||||
cls.url = f'https://127.0.0.1:{cls.port}'
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.srv.shutdown(); cls.srv.server_close(); cls.tmp.cleanup()
|
||||
|
||||
def test_pin_normalizes_colon_hex(self):
|
||||
self.assertEqual(normalize_pin(self.pin), self.pin.lower().replace(':', ''))
|
||||
self.assertEqual(normalize_pin('sha256:' + self.pin.upper()),
|
||||
self.pin.lower().replace(':', ''))
|
||||
with self.assertRaises(ValueError):
|
||||
normalize_pin('not-a-pin')
|
||||
|
||||
def test_pinned_client_connects(self):
|
||||
c = VyosClient(self.url, 'k', cert_pin=self.pin, timeout=10, retries=1)
|
||||
self.assertEqual(c.info().get('proto'), 'pin-test')
|
||||
|
||||
def test_wrong_pin_aborts_before_request(self):
|
||||
wrong = 'ab' * 32
|
||||
c = VyosClient(self.url, 'k', cert_pin=wrong, timeout=10, retries=1)
|
||||
with self.assertRaises(PinMismatchError):
|
||||
c.info()
|
||||
|
||||
|
||||
if __name__=='__main__': unittest.main()
|
||||
58
tests/test_drill_synsensor.py
Normal file
58
tests/test_drill_synsensor.py
Normal file
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Offline unit tests for the drill injector builders (pure local)."""
|
||||
import importlib.util
|
||||
import struct
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
'drill_synsensor', Path(__file__).resolve().parent.parent
|
||||
/ 'tests' / 'drill_synsensor.py')
|
||||
ds = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(ds)
|
||||
|
||||
|
||||
def onesc(buf):
|
||||
if len(buf) % 2:
|
||||
buf += b'\x00'
|
||||
t = sum(struct.unpack('!%dH' % (len(buf) // 2), buf))
|
||||
t = (t & 0xFFFF) + (t >> 16)
|
||||
t = (t & 0xFFFF) + (t >> 16)
|
||||
return t
|
||||
|
||||
|
||||
class Builders(unittest.TestCase):
|
||||
def test_icmp_checksums(self):
|
||||
p = ds.icmp_packet('192.168.100.84', '192.0.2.44', 7,
|
||||
b'\x11\x22\x33\x44\x55\x66', b'\x66\x55\x44\x33\x22\x11')
|
||||
self.assertEqual(p[12:14], b'\x08\x00')
|
||||
ip = p[14:34]
|
||||
self.assertEqual(onesc(ip) & 0xFFFF, 0xFFFF) # hdr sum incl cksum
|
||||
icmp = p[34:]
|
||||
self.assertEqual(icmp[0], 8)
|
||||
z = icmp[:2] + b'\x00\x00' + icmp[4:]
|
||||
t = onesc(z)
|
||||
self.assertEqual((~t) & 0xFFFF, struct.unpack('!H', icmp[2:4])[0])
|
||||
|
||||
def test_syn_pseudo_checksum(self):
|
||||
p = ds.syn_packet('203.0.113.5', '192.0.2.5', 22, 3,
|
||||
b'\x00' * 6, b'\x00' * 6)
|
||||
ip = p[14:34]
|
||||
self.assertEqual(onesc(ip) & 0xFFFF, 0xFFFF)
|
||||
tcp = p[34:]
|
||||
self.assertEqual(tcp[13] & 0x02, 0x02) # SYN flag
|
||||
a, b = ip[12:16], ip[16:20]
|
||||
ph = a + b + struct.pack('!BBH', 0, 6, len(tcp))
|
||||
z = tcp[:16] + b'\x00\x00' + tcp[18:]
|
||||
t = onesc(ph + z)
|
||||
self.assertEqual((~t) & 0xFFFF, struct.unpack('!H', tcp[16:18])[0])
|
||||
|
||||
def test_testnet_guard(self):
|
||||
self.assertTrue(ds.in_testnet('192.0.2.5'))
|
||||
self.assertTrue(ds.in_testnet('203.0.113.12'))
|
||||
self.assertFalse(ds.in_testnet('172.16.0.1'))
|
||||
self.assertFalse(ds.in_testnet('192.168.100.84'))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
211
tests/test_repairs.py
Normal file
211
tests/test_repairs.py
Normal file
@@ -0,0 +1,211 @@
|
||||
"""Regression tests for ingestion, replay, persistence and credential handling."""
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest.mock import patch, Mock
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]/'controller'))
|
||||
from leaf_controller.alertsock import AlertSocket, MAX_RECORD
|
||||
from leaf_controller.config import Config, DEFAULTS, load_config, read_key_env
|
||||
from leaf_controller.eve import EveTailer
|
||||
from leaf_controller.main import Audit, Controller
|
||||
from leaf_controller.vyos_client import VyosClient
|
||||
from test_controller import Router
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[1]
|
||||
def module(name, path):
|
||||
spec=importlib.util.spec_from_file_location(name, ROOT/path)
|
||||
m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m);return m
|
||||
rotation=module('key_rotation', 'scripts/rotate-vyos-key.py')
|
||||
retention=module('eve_rotation', 'scripts/rotate-eve.py')
|
||||
injector=module('injector','tests/drill_synsensor.py')
|
||||
|
||||
class Repairs(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp=tempfile.TemporaryDirectory();self.root=Path(self.tmp.name)
|
||||
def tearDown(self): self.tmp.cleanup()
|
||||
def controller(self):
|
||||
raw=copy.deepcopy(DEFAULTS)
|
||||
raw.update(state_db=str(self.root/'state'),audit_log=str(self.root/'audit'))
|
||||
raw['alert_socket']['path']=str(self.root/'alerts.sock')
|
||||
raw['policy'].update(mode='immediate',block_src=True)
|
||||
with patch.object(Config,'vyos_key',return_value='fake'):
|
||||
c=Controller(Config(raw))
|
||||
c.client=Router();self.addCleanup(c.close);return c
|
||||
def event(self):
|
||||
return dict(event_type='alert',src_ip='203.0.113.12',dest_ip='192.0.2.202',
|
||||
alert=dict(signature_id=1000001,metadata=dict(leaf_action=['block'])))
|
||||
def test_dry_run_preserves_live_socket_and_audit(self):
|
||||
c=self.controller();c._sock.bind()
|
||||
before=os.stat(c._sock.path).st_ino
|
||||
dry=Controller(c.cfg,dry_run=True)
|
||||
try:
|
||||
dry.handle_event(self.event())
|
||||
self.assertIsNone(dry._sock)
|
||||
finally: dry.close()
|
||||
self.assertEqual(os.stat(c._sock.path).st_ino,before)
|
||||
self.assertEqual((self.root/'audit').read_text(),'')
|
||||
def test_distinct_state_cannot_steal_socket(self):
|
||||
a=AlertSocket(str(self.root/'sock'));a.bind();self.addCleanup(a.close)
|
||||
b=AlertSocket(a.path)
|
||||
with self.assertRaises(OSError):b.bind()
|
||||
self.assertTrue(Path(a.path).exists())
|
||||
def test_unmanaged_active_listener_preserved(self):
|
||||
path=str(self.root/'sock')
|
||||
with socket.socket(socket.AF_UNIX) as s:
|
||||
s.bind(path);s.listen()
|
||||
with self.assertRaises(OSError):AlertSocket(path).bind()
|
||||
self.assertTrue(Path(path).exists())
|
||||
def test_non_socket_path_preserved(self):
|
||||
p=self.root/'sock';p.write_text('preserve')
|
||||
with self.assertRaises(OSError):AlertSocket(str(p)).bind()
|
||||
self.assertEqual(p.read_text(),'preserve')
|
||||
def test_shutdown_persists_accepted_queue(self):
|
||||
c=self.controller();c._sock._enqueue(json.dumps(self.event()).encode())
|
||||
c.close()
|
||||
from leaf_controller.state import State
|
||||
state=State(str(self.root/'state'))
|
||||
try:self.assertIsNotNone(state.get('192.0.2.202'))
|
||||
finally:state.close()
|
||||
def test_oversized_and_recursive_frames_do_not_kill_reader(self):
|
||||
a=AlertSocket(str(self.root/'sock'));a.bind();a.start();self.addCleanup(a.close)
|
||||
with socket.socket(socket.AF_UNIX) as s:
|
||||
s.connect(a.path)
|
||||
s.sendall(b'x'*(MAX_RECORD+65536)+b'\n'+b'['*2000+b']'*2000+b'\n'+
|
||||
json.dumps(self.event()).encode()+b'\n')
|
||||
self.assertTrue(a.available.wait(3))
|
||||
self.assertGreaterEqual(a.counts['parse_errors'],1)
|
||||
self.assertEqual(a.counts['parse_errors']+a.counts['other'],2)
|
||||
self.assertEqual(len(a.drain()),1)
|
||||
self.assertTrue(a._thread.is_alive())
|
||||
def test_partial_eof_counted(self):
|
||||
a=AlertSocket(str(self.root/'sock'));a.bind();a.start();self.addCleanup(a.close)
|
||||
with socket.socket(socket.AF_UNIX) as s:s.connect(a.path);s.sendall(b'{')
|
||||
end=time.monotonic()+2
|
||||
while a.counts['parse_errors']==0 and time.monotonic()<end:time.sleep(.01)
|
||||
self.assertEqual(a.counts['parse_errors'],1)
|
||||
def test_checkpoint_does_not_skip_unread_batch(self):
|
||||
p=self.root/'eve';p.write_text('{"n":1}\n{"n":2}\n{"n":3}\n')
|
||||
t=EveTailer(str(p),from_start=True);g=t.events()
|
||||
self.assertEqual(next(g),{'n':1});ck=(*t._ino,t._pos);g.close()
|
||||
self.assertEqual(ck[2],len('{"n":1}\n'))
|
||||
t=EveTailer(str(p),checkpoint=ck);g=t.events()
|
||||
self.assertEqual(next(g),{'n':2});g.close()
|
||||
def test_replayed_file_hit_does_not_inflate_threshold(self):
|
||||
c=self.controller();c.cfg.raw['policy'].update(mode='threshold',min_hits=2)
|
||||
c.state._local.event_key='1:2:10'
|
||||
c.handle_event(self.event());c.handle_event(self.event())
|
||||
self.assertIsNone(c.state.get('192.0.2.202'))
|
||||
c.state._local.event_key='1:2:20';c.handle_event(self.event())
|
||||
self.assertIsNotNone(c.state.get('192.0.2.202'))
|
||||
def test_failed_save_retried_after_membership_already_matches(self):
|
||||
c=self.controller();c.cfg.raw['save_config']=True;c.handle_event(self.event())
|
||||
c.client.save_config=Mock(side_effect=[RuntimeError('save failed'),None])
|
||||
with self.assertRaises(RuntimeError):c.sync_router_state()
|
||||
self.assertTrue(c.state.router_dirty())
|
||||
c.sync_router_state()
|
||||
self.assertEqual(c.client.save_config.call_count,2)
|
||||
self.assertEqual(len(c.client.commits),1)
|
||||
self.assertFalse(c.state.router_dirty())
|
||||
def test_audit_keeps_latest_in_rapid_rotation(self):
|
||||
p=self.root/'log';a=Audit(str(p),max_bytes=1,keep=3)
|
||||
for i in range(20):a.write({'i':i})
|
||||
a.close()
|
||||
rows=[json.loads(f.read_text())['i'] for f in self.root.glob('log.*')]
|
||||
self.assertEqual(sorted(rows),[17,18,19])
|
||||
def test_tls_pins_do_not_mutate_global_or_other_clients(self):
|
||||
from urllib3.poolmanager import pool_classes_by_scheme
|
||||
before=dict(pool_classes_by_scheme)
|
||||
a=VyosClient('https://localhost','x',cert_pin='a'*64)
|
||||
b=VyosClient('https://localhost','x',cert_pin='b'*64)
|
||||
self.assertEqual(pool_classes_by_scheme,before)
|
||||
self.assertFalse(a.sess.trust_env)
|
||||
self.assertEqual(a.sess.adapters['https://'].poolmanager.pool_classes_by_scheme['https'].ConnectionCls.leaf_pin,'a'*64)
|
||||
self.assertEqual(b.sess.adapters['https://'].poolmanager.pool_classes_by_scheme['https'].ConnectionCls.leaf_pin,'b'*64)
|
||||
def test_rotation_reopens_and_keeps_newest(self):
|
||||
p=self.root/'eve';p.write_text('first\n');calls=[]
|
||||
for i in range(9):
|
||||
p.write_text(str(i))
|
||||
retention.rotate(p,max_bytes=1,keep=3,reopen=lambda:(calls.append(1),p.write_text('')))
|
||||
self.assertEqual(len(calls),9)
|
||||
self.assertEqual(sorted(int(f.read_text()) for f in self.root.glob('eve.leaf-*')),[6,7,8])
|
||||
def test_rotation_reopen_failure_restores_file(self):
|
||||
p=self.root/'eve';p.write_text('original')
|
||||
with self.assertRaises(RuntimeError):
|
||||
retention.rotate(p,max_bytes=1,reopen=Mock(side_effect=RuntimeError()))
|
||||
self.assertEqual(p.read_text(),'original')
|
||||
def test_env_quotes_are_parsed_without_evaluation(self):
|
||||
p=self.root/'env';p.write_text("LEAF_VYOS_KEY='abc123'\n")
|
||||
self.assertEqual(read_key_env(p),'abc123')
|
||||
def test_invalid_queue_and_ttl_rejected(self):
|
||||
p=self.root/'config'
|
||||
for value in ('alert_socket: {queue_size: 0}', 'policy: {ttl_seconds: -1}', 'policy: {min_hits: 1.5}'):
|
||||
p.write_text(value)
|
||||
with self.assertRaises(ValueError):load_config(p)
|
||||
def test_key_rotation_preserves_other_app_ids(self):
|
||||
old=Mock();new=Mock();old.retrieve.return_value={'data':{'id':{'leaf':{'key':'old'},'other':{'key':'unrelated'}}}}
|
||||
with patch.object(rotation,'wait_auth'),patch.object(rotation,'atomic_env'),patch.object(rotation,'restart_service'):
|
||||
rotation.rotate(old,new,'leaf','leaf-new','old','new','LEAF_VYOS_KEY=old\n','IDS-BLOCK')
|
||||
self.assertEqual(new.configure.call_args.args[0]['path'],rotation.KEYS_PATH+['leaf'])
|
||||
self.assertEqual(new.configure.call_count,1)
|
||||
def test_key_rotation_restart_failure_rolls_back(self):
|
||||
old=Mock();new=Mock();old.retrieve.return_value={'data':{'id':{'leaf':{'key':'old'}}}}
|
||||
with patch.object(rotation,'wait_auth'),patch.object(rotation,'atomic_env') as write,patch.object(rotation,'restart_service',side_effect=[RuntimeError('failed'),None]):
|
||||
with self.assertRaises(RuntimeError):rotation.rotate(old,new,'leaf','new-id','old','new','LEAF_VYOS_KEY=old\n','IDS-BLOCK')
|
||||
self.assertEqual(write.call_args.args[0],'LEAF_VYOS_KEY=old\n')
|
||||
self.assertEqual(old.configure.call_args.args[0]['path'],rotation.KEYS_PATH+['new-id'])
|
||||
new.configure.assert_not_called()
|
||||
def test_key_rotation_refuses_nonmatching_id(self):
|
||||
old=Mock();old.retrieve.return_value={'data':{'id':{'other':{'key':'different'}}}}
|
||||
with self.assertRaises(RuntimeError):rotation.rotate(old,Mock(),'other','new','old','new','text','group')
|
||||
old.configure.assert_not_called()
|
||||
def test_live_injector_uses_resolved_ethernet_addresses(self):
|
||||
fake=Mock();fake.__enter__=Mock(return_value=fake);fake.__exit__=Mock(return_value=False)
|
||||
with patch.object(sys,'argv',['injector','syn','--src','203.0.113.12','--dst','192.0.2.201','--count','1','--interval','0']),patch.object(injector,'route_via',return_value=('10.0.249.0','eth1')),patch.object(Path,'read_text',return_value='11:22:33:44:55:66'),patch.object(injector,'resolve_gateway_mac',return_value=bytes.fromhex('665544332211')),patch.object(injector.socket,'socket',return_value=fake):
|
||||
injector.main()
|
||||
self.assertEqual(fake.send.call_args.args[0][:12],bytes.fromhex('665544332211112233445566'))
|
||||
|
||||
def test_hourly_rotation_clock_survives_continuous_writes(self):
|
||||
p=self.root/'eve';p.write_text('one')
|
||||
self.assertFalse(retention.rotate(p,max_bytes=999999,now=100,reopen=lambda:None))
|
||||
p.write_text('two')
|
||||
self.assertTrue(retention.rotate(p,max_bytes=999999,now=3701,reopen=lambda:p.write_text('')))
|
||||
|
||||
def test_rotation_skips_engine_startup(self):
|
||||
with patch.object(sys,'argv',['rotate-eve']),patch.object(retention.subprocess,'run',return_value=Mock(stdout='100000000')),patch.object(retention.time,'monotonic',return_value=101),patch.object(retention,'rotate') as r:
|
||||
retention.main()
|
||||
r.assert_not_called()
|
||||
|
||||
def test_diagnostics_redact_nested_credentials(self):
|
||||
from leaf_controller.ops import redact
|
||||
result=redact({'service':{'api':{'keys':{'id':{'leaf':{'key':'secret'}}}}},
|
||||
'password':'secret', 'interfaces':{'eth1':{'address':['10.0.249.1/31']}}})
|
||||
self.assertNotIn('secret',json.dumps(result))
|
||||
self.assertEqual(result['interfaces']['eth1']['address'],['10.0.249.1/31'])
|
||||
|
||||
def test_file_identity_can_count_both_candidate_targets(self):
|
||||
c=self.controller();c.cfg.raw['policy'].update(mode='threshold',min_hits=2)
|
||||
c.state._local.event_key='one-file-record'
|
||||
event=self.event();event['alert']['metadata']['leaf_action']=['block','block_src']
|
||||
c.handle_event(event)
|
||||
self.assertEqual(c.state.hit_count(1000001,'192.0.2.202',300),1)
|
||||
self.assertEqual(c.state.hit_count(1000001,'203.0.113.12',300),1)
|
||||
|
||||
def test_malformed_stats_does_not_poison_following_alerts(self):
|
||||
c=self.controller()
|
||||
for stats in ({'capture':[1]}, {'uptime':'bad'}, {'uptime':float('inf')}):
|
||||
c._route({'event_type':'stats','stats':stats})
|
||||
c._route(self.event())
|
||||
self.assertIsNotNone(c.state.get('192.0.2.202'))
|
||||
|
||||
def test_nonfinite_signature_does_not_stop_processing(self):
|
||||
c=self.controller();bad=self.event();bad['alert']['signature_id']=float('inf')
|
||||
c.handle_event(bad);c.handle_event(self.event())
|
||||
self.assertIsNotNone(c.state.get('192.0.2.202'))
|
||||
15
vyos/router-leaf.snippet.txt
Normal file
15
vyos/router-leaf.snippet.txt
Normal file
@@ -0,0 +1,15 @@
|
||||
#### Illustrative new-install template only. NOT the live VIPERA configuration.
|
||||
#### Do not apply this file to VIPERA. Existing OSPF, uplink, mirror and routes
|
||||
#### are outside LEAF maintenance scope. Verify live hooks through the client.
|
||||
#### Router OS variants differ: inspect the target schema before configuring.
|
||||
|
||||
# One dedicated group, and source/destination drop hooks ahead of accept rules.
|
||||
set firewall group address-group IDS-BLOCK
|
||||
set firewall ipv4 forward filter rule 10 action drop
|
||||
set firewall ipv4 forward filter rule 10 destination group address-group IDS-BLOCK
|
||||
set firewall ipv4 forward filter rule 11 action drop
|
||||
set firewall ipv4 forward filter rule 11 source group address-group IDS-BLOCK
|
||||
|
||||
# API service, TLS trust and mirroring are provisioned by the administrator.
|
||||
# No embedded key, interface changes, broad accept rules or commit/save here.
|
||||
# Controller save_config=false keeps transient membership out of boot config.
|
||||
Reference in New Issue
Block a user