docs: BGP network map + CHIMERA architecture plan
Ground truth read from CiscoConfigs: 3 RR clusters (HQ 64572/6.7.6.7, NEKOS 64600/172.19.67.0, VyOS transit 64580/67.67.21.37), RR-PEERS+CLIENTS peer-group scheme, config quirks (nested AF stanzas, NYATER 4th RR, commented-out RCC on VyOS transit RRs, shared .186 router-id). Architecture: declarative YAML inventory, normalized bgpd parser, vtysh plan/diff/apply engine with canary rollout and rollback, CLI verbs, docker FRR e2e lab, roadmap M0-M5.
This commit is contained in:
221
docs/ARCHITECTURE.md
Normal file
221
docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,221 @@
|
||||
# CHIMERA architecture & codebase plan
|
||||
|
||||
Status: design document. No code yet. This file defines the repo layout,
|
||||
the data model, and the engine behavior we build toward.
|
||||
|
||||
## Design principles
|
||||
|
||||
1. **Desired state lives in git, never in the router.** A small YAML
|
||||
inventory describes each RR *cluster* (cluster-id, routers, peer-groups,
|
||||
peers). CHIMERA compiles it into per-RR deltas. Archived configs in
|
||||
`CiscoConfigs/` are only an *import source*, not the runtime store.
|
||||
2. **Never wholesale-replace `frr.conf`.** RRs carry hand-tuned OSPF, interface
|
||||
and system stanzas. CHIMERA only emits targeted `configure terminal`
|
||||
commands (vtysh), then `write memory` per box. Whole-file templating is a
|
||||
v3 maybe, gated by a `file-mode` flag, not the default.
|
||||
3. **Peer-groups are first-class.** All bulk operations (add/modify a group
|
||||
across *all* RRs of a cluster) are peer-group-scoped; FRR propagates
|
||||
group attributes to members, and per-peer overrides win over group
|
||||
settings. Single-peer edits only when explicitly asked.
|
||||
4. **Mesh-aware generation.** The RR-PEERS group is derived per-RR by
|
||||
subtracting self; the inventory lists the cluster's RR loopbacks once and
|
||||
each RR's neighbor list is generated (this is why the four MUYU/FOX/WOLF/…
|
||||
files are near-duplicates with self-exclusion).
|
||||
5. **Plan → diff → apply, always.** Every mutating command renders a delta
|
||||
first (`--diff` is default output). `apply` takes the same plan object.
|
||||
6. **Drift detection is a first-class verb** (`status`), not an aftercare:
|
||||
parse live `vtysh -c "show running-config bgpd"` against inventory.
|
||||
7. **Serial, per-cluster rollout with a canary.** Apply to one RR first,
|
||||
verify, then the rest. Most peer-group attribute changes don't tear
|
||||
sessions down; anything that *would* reset sessions (remote-as change,
|
||||
password, shutdown, un-resourced peer-group deletion) gets flagged in the
|
||||
plan with a loud warning.
|
||||
|
||||
## Execution model (FRR on Debian/LXC)
|
||||
|
||||
Transport: SSH (paramiko or plain `ssh` subprocess) →
|
||||
`vtysh -c "show running-config bgpd"` for reads (integrated-vtysh-config is
|
||||
turned ON on every RR, so this is literally `/etc/frr/frr.conf` in bgpd view).
|
||||
|
||||
Apply = one batch of `vtysh -c` commands per box:
|
||||
|
||||
```
|
||||
vtysh -c "configure terminal" -c "router bgp 64572" -c "address-family l2vpn evpn" -c "neighbor CLIENTS send-community both"
|
||||
vtysh -c "configure terminal" ... # FRR's integrated config applies atomically per invocation
|
||||
vtysh -c "write memory"
|
||||
```
|
||||
|
||||
Command sequencing rules the compiler must respect (from the FRR 10.x bgpd
|
||||
docs):
|
||||
|
||||
- `neighbor <G> peer-group` and `neighbor <G> remote-as` before attaching any
|
||||
member to the group.
|
||||
- Global BGP params (`bgp cluster-id`, `bgp router-id`) live under
|
||||
`router bgp <AS>`, not under AF sub-modes.
|
||||
- AF-scoped ops (`route-reflector-client`, `activate`, `send-community`)
|
||||
require entering the right `address-family` block first.
|
||||
- Deleting an AF block with `no ...` implicitly kills its neighbor/AF lines —
|
||||
plan must expand that into explicit removals when diffing, so dry-run output
|
||||
is truthful.
|
||||
- Configs contain duplicated/merged AF stanzas (see BGP-NETWORK.md quirks):
|
||||
the parser normalizes them, the writer emits canonical single-block form.
|
||||
First apply of a normalize-only diff is a safe no-op-semantics change we
|
||||
can land quietly.
|
||||
|
||||
Rollback: before touching a box, `vtysh -c "show running-config bgpd"` →
|
||||
`/var/backups/chimera/<host>/<ts>.conf` plus `cp /etc/frr/frr.conf` (root or
|
||||
`sudo -n`); `chimera rollback <host> [--from <ts>]` re-applies the snapshot
|
||||
through the same vtysh path.
|
||||
|
||||
Optional per-box pre/post hooks (`hooks:` in inventory) to run `systemctl
|
||||
reload frr`-style recovery, though normally unnecessary.
|
||||
|
||||
## Data model
|
||||
|
||||
Inventory YAML (one file per cluster, e.g. `inventory/hq-evpn.yml`):
|
||||
|
||||
```yaml
|
||||
cluster:
|
||||
name: hq-evpn
|
||||
as: 64572
|
||||
cluster_id: 6.7.6.7 # keep notation as configured
|
||||
address_family_default: [ipv4_unicast, l2vpn_evpn]
|
||||
routers:
|
||||
- { name: MUYU-RR, ssh: muyu-rr.hq.panchain.org, loopback: 10.255.254.246 }
|
||||
- { name: FOX-RR, ssh: fox-rr.hq.panchain.org, loopback: 10.255.254.247 }
|
||||
- { name: WOLF-RR, ssh: wolf-rr.hq.panchain.org, loopback: 10.255.254.245 }
|
||||
- { name: NYATER, ssh: nyater.hq.panchain.org, loopback: 10.255.254.255, role: rr+switch }
|
||||
peer_groups:
|
||||
RR-PEERS:
|
||||
remote_as: 64572
|
||||
update_source: lo
|
||||
af:
|
||||
ipv4_unicast: { activate: true, send_community: both }
|
||||
l2vpn_evpn: { activate: true, send_community: both }
|
||||
CLIENTS:
|
||||
remote_as: 64572
|
||||
update_source: lo
|
||||
af:
|
||||
ipv4_unicast: { activate: true, send_community: both, route_reflector_client: true }
|
||||
l2vpn_evpn: { activate: true, send_community: both, route_reflector_client: true }
|
||||
peers:
|
||||
# RR-PEERS members: declared once; generated per-RR minus self
|
||||
- { group: RR-PEERS, loopback_ref: [MUYU-RR, FOX-RR, WOLF-RR, NYATER], mesh: true }
|
||||
# clients:
|
||||
- { group: CLIENTS, ip: 10.255.254.250, desc: URAKAZE }
|
||||
- { group: CLIENTS, ip: 10.255.254.234, desc: VICTORIA }
|
||||
...
|
||||
```
|
||||
|
||||
Core objects (Python, `attrs`/`dataclasses`):
|
||||
|
||||
- `RouterReflector` — host, ssh target, loopback, role (rr | rr-switch),
|
||||
parsed snapshot of its running bgp config.
|
||||
- `BgpCluster` — AS, cluster_id, list of RRs, peer-groups, peers; owns the
|
||||
`validate()` rules (cluster_id must be identical on every RR; RRs never in
|
||||
an RCC group; AS consistency).
|
||||
- `PeerGroup` — name, attrs (remote_as, update_source, password, timers…),
|
||||
per-AF settings (`activate`, `send_community`, `route_reflector_client`,
|
||||
`allowas_in`, route-maps, … as encountered).
|
||||
- `Peer` — ip, group membership, per-peer overrides dict.
|
||||
- `RunningBgpConfig` — normalized parse of one RR's bgpd view
|
||||
(`router bgp` → globals → peer-groups → peers → AF blocks).
|
||||
- `Delta` — ordered list of `BgpCommand`s with `resets_session: bool` flag.
|
||||
|
||||
`chimera.cli` and the future web layer are **thin callers of the core**; the
|
||||
core must not import anything from cli/web (so a `chimera.core` import in a
|
||||
notebook works too).
|
||||
|
||||
## Repo layout
|
||||
|
||||
```
|
||||
CHIMERA/
|
||||
pyproject.toml # package = chimera; console_script = chimera
|
||||
src/chimera/
|
||||
core/
|
||||
models.py # dataclasses above
|
||||
inventory.py # YAML load/validate (pydantic or jsonschema)
|
||||
normalize.py # parse quirks → canonical model
|
||||
planner.py # desired × live → Delta; sequencing rules
|
||||
commands.py # BgpCommand rendering (vtysh arg list)
|
||||
transport/
|
||||
ssh.py # paramiko/openssh subprocess runner
|
||||
frr.py # vtysh read/apply/write/rollback
|
||||
vyos.py # (later) set-syntax provider for VyOS transit
|
||||
parse/
|
||||
bgp_grammar.py # line-based bgpd config parser
|
||||
cli/
|
||||
main.py # typer app: the verbs below
|
||||
web/
|
||||
app.py # FastAPI, same core; read-only pages first
|
||||
templates/ # Jinja + htmx; no SPA needed
|
||||
tests/
|
||||
unit/ e2e/
|
||||
inventory/ # git-tracked desired state
|
||||
hq-evpn.yml nekos-evpn.yml hq-transit.yml
|
||||
docs/ # this file, BGP-NETWORK.md, …
|
||||
docker/
|
||||
rr-lab/ # frr:10.3 containers, 4-RR full mesh: e2e lab
|
||||
```
|
||||
|
||||
## CLI verbs (target surface)
|
||||
|
||||
```
|
||||
chimera init # generate inventory stubs
|
||||
chimera import --live # bootstrap inventory from running RRs
|
||||
chimera status [cluster] # per-RR drift report: group attrs, RCC, cluster_id
|
||||
chimera plan peergroup edit <g> --cluster <c> --af l2vpn-evpn --send-community both
|
||||
chimera peer add 10.255.254.243 --group CLIENTS --cluster hq-evpn
|
||||
chimera peergroup add <g> --cluster <c> --remote-as 64572 ...
|
||||
chimera cluster set-id <c> <id> # all RRs; refuses until all agree
|
||||
chimera apply <plan-id> [--canary-first] [--only <rr>] [--yes]
|
||||
chimera rollback <rr> [--from <ts>]
|
||||
```
|
||||
|
||||
Mutating verbs print the Delta by default and exit; `--yes` applies.
|
||||
`chimera cluster set-id` refuses to emit until `status` shows every RR
|
||||
reachable — changing cluster-id on a subset restarts the TCP-BFD-less RR
|
||||
cluster's ORF/count logic mid-reflection; we want one plan for all boxes.
|
||||
|
||||
## Web UI plan (phase 2, after CLI core is solid)
|
||||
|
||||
- Pages: cluster overview (4 RRs × sessions table), drift report,
|
||||
desired-state editor (validated YAML), plan preview with rendered vtysh
|
||||
commands, apply progress log.
|
||||
- Auth: local/single-user token first; Tailscale-gated in practice.
|
||||
- Same core as CLI — no duplicated business logic.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit: parser golden files — the actual MUYU/YUUKA/REIMU configs copied into
|
||||
`tests/parse/golden/` (they contain the weird nested-AF stanzas — regression
|
||||
suite for the normalizer).
|
||||
- Property test: model → render → re-parse == normalize(parse) (round-trip).
|
||||
- E2e lab: `docker/rr-lab` = 4× `frr:10.3` containers, loopback on dummy0,
|
||||
full-mesh RR + 2 fake clients; CHIMERA apply against them in CI.
|
||||
- Fake transport mode (`--transport fake`) that shells out to a stub — keeps
|
||||
e2e cheap without docker.
|
||||
|
||||
## Roadmap
|
||||
|
||||
1. **M0** parser + normalizer + golden tests (no SSH yet; parse the archived
|
||||
`.ios` files). ~ This unlocks everything else.
|
||||
2. **M1** inventory + `status` (read-only drift report over SSH).
|
||||
3. **M2** planner + commands + `apply`/`rollback`, canary rollout.
|
||||
4. **M3** verbs: peergroup add/edit, peer add, cluster set-id.
|
||||
5. **M4** web read-only dashboard; apply via UI with plan-preview gate.
|
||||
6. **M5** VyOS provider (HQ transit NOZOMI/SHUN) after FRR flow is proven —
|
||||
the transit RR quirk (commented-out RCC) is a `status` warning fixture.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Which FRR version is on the live RRs exactly? Repo headers say `frr version
|
||||
10.3`; docs target 10.7.1 — verify with `vtysh -c "show version"` during
|
||||
M1, pin grammar to what runs.
|
||||
- HQ's 4 RR members are 3 LXC + NYATER switch: does orchestration treat
|
||||
NYATER's bgpd (switch platform, multicast AF) identically? Assumed yes —
|
||||
it speaks the same bgpd grammar.
|
||||
- Is there a management network/VRF for SSH to the RRs (`vrf` flag on ssh
|
||||
runner may be needed)?
|
||||
- Do we want route-map/route-reflector-set-extra-community knobs in the data
|
||||
model now, or keep minimal until encountered in the configs?
|
||||
Reference in New Issue
Block a user