diff --git a/README.md b/README.md index 7cddc8e..7e2fadf 100644 --- a/README.md +++ b/README.md @@ -2,4 +2,18 @@ ## BGP Route Reflector synchronizer -![Unrelated pic](https://data.shupogaki.org/assets/1/HOQhOGVaoAA_2t3.jpeg) \ No newline at end of file +![Unrelated pic](https://data.shupogaki.org/assets/1/HOQhOGVaoAA_2t3.jpeg) + +## Docs + +- [docs/BGP-NETWORK.md](docs/BGP-NETWORK.md) — the real BGP network: AS plan, + RR clusters (HQ AS 64572 cluster-id 6.7.6.7, NEKOS AS 64600 cluster-id + 172.19.67.0, transit AS 64580), peer-groups (RR-PEERS / CLIENTS), addressing, + and every config quirk CHIMERA must handle. +- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — codebase layout, data model, + vtysh plan/apply engine, CLI verbs, test lab, roadmap. + +Source configs orchestrated by CHIMERA: +`CiscoConfigs/MP-BGP_EVPN_VXLAN_V2/HQ/FRR/RouteReflectorLXC/` (HQ FRR RRs on +Debian LXC), `.../NEKOS/FRR/RR/` + `NEKOS/REIMU` (NEKOS RRs), VyOS transit in +`CiscoConfigs/VyOS/HQ/`. FRR docs: https://docs.frrouting.org/en/latest/ . \ No newline at end of file diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..e41ba01 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,221 @@ +# CHIMERA architecture & codebase plan + +Status: design document. No code yet. This file defines the repo layout, +the data model, and the engine behavior we build toward. + +## Design principles + +1. **Desired state lives in git, never in the router.** A small YAML + inventory describes each RR *cluster* (cluster-id, routers, peer-groups, + peers). CHIMERA compiles it into per-RR deltas. Archived configs in + `CiscoConfigs/` are only an *import source*, not the runtime store. +2. **Never wholesale-replace `frr.conf`.** RRs carry hand-tuned OSPF, interface + and system stanzas. CHIMERA only emits targeted `configure terminal` + commands (vtysh), then `write memory` per box. Whole-file templating is a + v3 maybe, gated by a `file-mode` flag, not the default. +3. **Peer-groups are first-class.** All bulk operations (add/modify a group + across *all* RRs of a cluster) are peer-group-scoped; FRR propagates + group attributes to members, and per-peer overrides win over group + settings. Single-peer edits only when explicitly asked. +4. **Mesh-aware generation.** The RR-PEERS group is derived per-RR by + subtracting self; the inventory lists the cluster's RR loopbacks once and + each RR's neighbor list is generated (this is why the four MUYU/FOX/WOLF/… + files are near-duplicates with self-exclusion). +5. **Plan → diff → apply, always.** Every mutating command renders a delta + first (`--diff` is default output). `apply` takes the same plan object. +6. **Drift detection is a first-class verb** (`status`), not an aftercare: + parse live `vtysh -c "show running-config bgpd"` against inventory. +7. **Serial, per-cluster rollout with a canary.** Apply to one RR first, + verify, then the rest. Most peer-group attribute changes don't tear + sessions down; anything that *would* reset sessions (remote-as change, + password, shutdown, un-resourced peer-group deletion) gets flagged in the + plan with a loud warning. + +## Execution model (FRR on Debian/LXC) + +Transport: SSH (paramiko or plain `ssh` subprocess) → +`vtysh -c "show running-config bgpd"` for reads (integrated-vtysh-config is +turned ON on every RR, so this is literally `/etc/frr/frr.conf` in bgpd view). + +Apply = one batch of `vtysh -c` commands per box: + +``` +vtysh -c "configure terminal" -c "router bgp 64572" -c "address-family l2vpn evpn" -c "neighbor CLIENTS send-community both" +vtysh -c "configure terminal" ... # FRR's integrated config applies atomically per invocation +vtysh -c "write memory" +``` + +Command sequencing rules the compiler must respect (from the FRR 10.x bgpd +docs): + +- `neighbor peer-group` and `neighbor remote-as` before attaching any + member to the group. +- Global BGP params (`bgp cluster-id`, `bgp router-id`) live under + `router bgp `, not under AF sub-modes. +- AF-scoped ops (`route-reflector-client`, `activate`, `send-community`) + require entering the right `address-family` block first. +- Deleting an AF block with `no ...` implicitly kills its neighbor/AF lines — + plan must expand that into explicit removals when diffing, so dry-run output + is truthful. +- Configs contain duplicated/merged AF stanzas (see BGP-NETWORK.md quirks): + the parser normalizes them, the writer emits canonical single-block form. + First apply of a normalize-only diff is a safe no-op-semantics change we + can land quietly. + +Rollback: before touching a box, `vtysh -c "show running-config bgpd"` → +`/var/backups/chimera//.conf` plus `cp /etc/frr/frr.conf` (root or +`sudo -n`); `chimera rollback [--from ]` re-applies the snapshot +through the same vtysh path. + +Optional per-box pre/post hooks (`hooks:` in inventory) to run `systemctl +reload frr`-style recovery, though normally unnecessary. + +## Data model + +Inventory YAML (one file per cluster, e.g. `inventory/hq-evpn.yml`): + +```yaml +cluster: + name: hq-evpn + as: 64572 + cluster_id: 6.7.6.7 # keep notation as configured + address_family_default: [ipv4_unicast, l2vpn_evpn] + routers: + - { name: MUYU-RR, ssh: muyu-rr.hq.panchain.org, loopback: 10.255.254.246 } + - { name: FOX-RR, ssh: fox-rr.hq.panchain.org, loopback: 10.255.254.247 } + - { name: WOLF-RR, ssh: wolf-rr.hq.panchain.org, loopback: 10.255.254.245 } + - { name: NYATER, ssh: nyater.hq.panchain.org, loopback: 10.255.254.255, role: rr+switch } + peer_groups: + RR-PEERS: + remote_as: 64572 + update_source: lo + af: + ipv4_unicast: { activate: true, send_community: both } + l2vpn_evpn: { activate: true, send_community: both } + CLIENTS: + remote_as: 64572 + update_source: lo + af: + ipv4_unicast: { activate: true, send_community: both, route_reflector_client: true } + l2vpn_evpn: { activate: true, send_community: both, route_reflector_client: true } + peers: + # RR-PEERS members: declared once; generated per-RR minus self + - { group: RR-PEERS, loopback_ref: [MUYU-RR, FOX-RR, WOLF-RR, NYATER], mesh: true } + # clients: + - { group: CLIENTS, ip: 10.255.254.250, desc: URAKAZE } + - { group: CLIENTS, ip: 10.255.254.234, desc: VICTORIA } + ... +``` + +Core objects (Python, `attrs`/`dataclasses`): + +- `RouterReflector` — host, ssh target, loopback, role (rr | rr-switch), + parsed snapshot of its running bgp config. +- `BgpCluster` — AS, cluster_id, list of RRs, peer-groups, peers; owns the + `validate()` rules (cluster_id must be identical on every RR; RRs never in + an RCC group; AS consistency). +- `PeerGroup` — name, attrs (remote_as, update_source, password, timers…), + per-AF settings (`activate`, `send_community`, `route_reflector_client`, + `allowas_in`, route-maps, … as encountered). +- `Peer` — ip, group membership, per-peer overrides dict. +- `RunningBgpConfig` — normalized parse of one RR's bgpd view + (`router bgp` → globals → peer-groups → peers → AF blocks). +- `Delta` — ordered list of `BgpCommand`s with `resets_session: bool` flag. + +`chimera.cli` and the future web layer are **thin callers of the core**; the +core must not import anything from cli/web (so a `chimera.core` import in a +notebook works too). + +## Repo layout + +``` +CHIMERA/ + pyproject.toml # package = chimera; console_script = chimera + src/chimera/ + core/ + models.py # dataclasses above + inventory.py # YAML load/validate (pydantic or jsonschema) + normalize.py # parse quirks → canonical model + planner.py # desired × live → Delta; sequencing rules + commands.py # BgpCommand rendering (vtysh arg list) + transport/ + ssh.py # paramiko/openssh subprocess runner + frr.py # vtysh read/apply/write/rollback + vyos.py # (later) set-syntax provider for VyOS transit + parse/ + bgp_grammar.py # line-based bgpd config parser + cli/ + main.py # typer app: the verbs below + web/ + app.py # FastAPI, same core; read-only pages first + templates/ # Jinja + htmx; no SPA needed + tests/ + unit/ e2e/ + inventory/ # git-tracked desired state + hq-evpn.yml nekos-evpn.yml hq-transit.yml + docs/ # this file, BGP-NETWORK.md, … + docker/ + rr-lab/ # frr:10.3 containers, 4-RR full mesh: e2e lab +``` + +## CLI verbs (target surface) + +``` +chimera init # generate inventory stubs +chimera import --live # bootstrap inventory from running RRs +chimera status [cluster] # per-RR drift report: group attrs, RCC, cluster_id +chimera plan peergroup edit --cluster --af l2vpn-evpn --send-community both +chimera peer add 10.255.254.243 --group CLIENTS --cluster hq-evpn +chimera peergroup add --cluster --remote-as 64572 ... +chimera cluster set-id # all RRs; refuses until all agree +chimera apply [--canary-first] [--only ] [--yes] +chimera rollback [--from ] +``` + +Mutating verbs print the Delta by default and exit; `--yes` applies. +`chimera cluster set-id` refuses to emit until `status` shows every RR +reachable — changing cluster-id on a subset restarts the TCP-BFD-less RR +cluster's ORF/count logic mid-reflection; we want one plan for all boxes. + +## Web UI plan (phase 2, after CLI core is solid) + +- Pages: cluster overview (4 RRs × sessions table), drift report, + desired-state editor (validated YAML), plan preview with rendered vtysh + commands, apply progress log. +- Auth: local/single-user token first; Tailscale-gated in practice. +- Same core as CLI — no duplicated business logic. + +## Testing + +- Unit: parser golden files — the actual MUYU/YUUKA/REIMU configs copied into + `tests/parse/golden/` (they contain the weird nested-AF stanzas — regression + suite for the normalizer). +- Property test: model → render → re-parse == normalize(parse) (round-trip). +- E2e lab: `docker/rr-lab` = 4× `frr:10.3` containers, loopback on dummy0, + full-mesh RR + 2 fake clients; CHIMERA apply against them in CI. +- Fake transport mode (`--transport fake`) that shells out to a stub — keeps + e2e cheap without docker. + +## Roadmap + +1. **M0** parser + normalizer + golden tests (no SSH yet; parse the archived + `.ios` files). ~ This unlocks everything else. +2. **M1** inventory + `status` (read-only drift report over SSH). +3. **M2** planner + commands + `apply`/`rollback`, canary rollout. +4. **M3** verbs: peergroup add/edit, peer add, cluster set-id. +5. **M4** web read-only dashboard; apply via UI with plan-preview gate. +6. **M5** VyOS provider (HQ transit NOZOMI/SHUN) after FRR flow is proven — + the transit RR quirk (commented-out RCC) is a `status` warning fixture. + +## Open questions + +- Which FRR version is on the live RRs exactly? Repo headers say `frr version + 10.3`; docs target 10.7.1 — verify with `vtysh -c "show version"` during + M1, pin grammar to what runs. +- HQ's 4 RR members are 3 LXC + NYATER switch: does orchestration treat + NYATER's bgpd (switch platform, multicast AF) identically? Assumed yes — + it speaks the same bgpd grammar. +- Is there a management network/VRF for SSH to the RRs (`vrf` flag on ssh + runner may be needed)? +- Do we want route-map/route-reflector-set-extra-community knobs in the data + model now, or keep minimal until encountered in the configs? diff --git a/docs/BGP-NETWORK.md b/docs/BGP-NETWORK.md new file mode 100644 index 0000000..b33aea6 --- /dev/null +++ b/docs/BGP-NETWORK.md @@ -0,0 +1,134 @@ +# BGP network map (ground truth from CiscoConfigs repo) + +Source of truth: the archived configs in +`/hdd/GiTea-REPO/CiscoConfigs/MP-BGP_EVPN_VXLAN_V2/` and +`/hdd/GiTea-REPO/CiscoConfigs/VyOS/`. Everything below was read out of those +files; live state may drift (CHIMERA's job is to detect exactly that). + +## 1. AS plan + +| AS | Role | +|---------|-------------------------------------------------------------| +| `64572` | HQ EVPN/VXLAN **underlay** (FRR + switch RRs, iBGP full mesh w/ RRs) | +| `64580` | HQ **transit** routers (VyOS: NOZOMI, SHUN, VIPERA, REISA, EDGE-2, EDGE-9, EDGE-MIKU, LXC-1) | +| `64581` | MU3S / `mu3.panchain.org` (KIMIKONO, EDGE) | +| `64590` | SHUPO (cloud: Hetzner VMs + `shupo.panchain.org`) — also hosts FUMOS VRRP pair (HIKARI/MIYU/MEGU) | +| `64600` | NEKOS site (VRRP pair SOF/OHR, router EIN, edge, RRs) | +| `64610` | ORC site (KIKI) | + +## 2. Route Reflector clusters + +### HQ underlay — AS 64572, cluster-id `6.7.6.7` + +Four RRs, full mesh between them, loopbacks in `10.255.254.0/24` +(advertised via OSPF): + +| RR | Loopback / router-id | Form | Config | +|-------------|----------------------|--------------------|--------| +| MUYU-RR | `10.255.254.246` | Debian LXC (`frr version 10.3`, `service integrated-vtysh-config`) | `HQ/FRR/RouteReflectorLXC/MUYU/MUYU.ios` | +| FOX-RR | `10.255.254.247` | Debian LXC | `.../FOX/FOX.ios` | +| WOLF-RR | `10.255.254.245` | Debian LXC | `.../WOLF/WOLF.ios` | +| NYATER | `10.255.254.255` (bgp router-id `2.1.3.7`) | HQ hardware switch (also runs multicast AF) | `HQ/NYATER.ios` — NOT in RouteReflectorLXC/ | + +Peer-groups (identical on all four, `update-source lo`): + +- `RR-PEERS` — iBGP mesh members (the *other* RRs). **Never** + route-reflector-client. Each RR file lists the others minus itself — that + self-exclusion is why the four files are near-duplicates. +- `CLIENTS` — every underlay router; `route-reflector-client` in both + `address-family ipv4 unicast` and `address-family l2vpn evpn`. + +RR-PEERS on e.g. MUYU: `.255`, `.247`, `.245`. CLIENTS (from MUYU.ios): +`.254 .253 .252 .251 .249 .248 .244 .242 .240 .250 .234`. +Known client identities: `.250` = URAKAZE (Debian), `.234` = VICTORIA +(OpenWRT AP). Others map to HQ hosts/hypervisors — resolve by importing live +state. + +Address families activated on RRs: `l2vpn evpn` + `ipv4 unicast`, both with +`send-community both` on RR-PEERS and CLIENTS. + +### NEKOS EVPN — AS 64600, cluster-id `172.19.67.0` + +Four RRs, full mesh, loopbacks `10.255.254.0/24` (NEKOS OSPF domain): + +| RR | Loopback / router-id | Form | +|-------------|----------------------|------| +| KOKONUT-RR | `10.255.254.183` | `NEKOS/FRR/RR/KOKONUT-RR.ios` | +| LH-RR | `10.255.254.184` | `NEKOS/FRR/RR/LH-RR.ios` | +| YUUKA-RR | `10.255.254.185` | `NEKOS/FRR/RR/YUUKA-RR.ios` | +| REIMU | `10.255.254.187` | bare metal, `NEKOS/REIMU/REIMU.ios` | + +Same peer-group scheme: `RR-PEERS` (the other 3) + `CLIENTS` +(route-reflector-client in `ipv4 unicast` **and** `l2vpn evpn`, +`send-community both`, `update-source lo`). + +NEKOS CLIENTS: `.200 .199 .198 .197 .196 .195` (LXC/VyOS class), +`.191 .190 .189 .188 .186` (hardware: YUUKA=`.190`, LOLI-HUNTER=`.189`, +KOKONUT=`.188`, SENKMX and PANKO both config'd with `.186` — see quirk list). + +VyOS clients (SOF, OHR, EIN, Edge NIHA/EROHA/AOBA) peer with **all four RRs** +via their own peer-group `NEKOS` (`remote-as 64600`, +`update-source <172.19.x loopback>`). + +### HQ transit — AS 64580, cluster-id `67.67.21.37` (VyOS) + +- NOZOMI (bgp router-id `192.168.1.2`, iBGP `update-source 10.255.254.230`) and SHUN (`update-source 10.255.254.228`) are both configured with `bgp parameters cluster-id 67.67.21.37`. +- iBGP peer-group `HQ` toward the other transit RR, AFs: `ipv4-unicast`, + `ipv6-unicast`, `l2vpn-evpn` (plus multicast in comments). +- E-BGP peer-groups by site: `SHUPO` (64590), `NEKOS` (64600), `ORC` (64610), + `MU3S`/`MU3` (64581), `HOME` (64600, on YUZU). +- ⚠ **In the archived configs every `route-reflector-client` line on + NOZOMI/SHUN is commented out.** cluster-id with no RCC = ineffective + reflection. Either the live boxes have RCC enabled out-of-band, or this is + dead config. First thing CHIMERA drift-check should confirm. + +## 3. Client side patterns (what CHIMERA manages *about*, but doesn't *is*) + +- HQ FRR clients (URAKAZE, MOMI, IBUKI-M-ATOR, OPS, WOLF-HUNTER, APs…): + plain `router bgp 64572`, one neighbor per RR (`.255 .245 .246 .247`), + no client-side peer-group, `neighbor ... route-reflector-client` on *their* + side is absent (correct — RCC belongs on the RR), `timers 30 60`, + `bgp graceful-restart`, `no bgp default ipv4-unicast`. +- HQ VyOS routers (YUZU, IZUNA): transit iBGP peer-group `HQ` + eBGP + peer-groups per site AS. +- Underlay is OSPF (`auto-cost reference-bandwidth 40000` on HQ FRR, + `1000` on NEKOS RR files — mismatch is intentional per-site). + +## 4. Quirks & gotchas (write these into the parser's expectations) + +1. **Duplicated/nested AF stanzas.** `MUYU/FOX/WOLF`, `YUUKA-RR`, `REIMU` + contain a second `address-family l2vpn evpn` block (in REIMU it's even + *nested* inside the first, with no closing `exit-address-family`). FRR's + parser accepts and merges these, but `write memory`-generated config would + never look like this — hand-edit residue. CHIMERA must normalize to one + block per AF when writing. +2. **NYATER is an RR that lives outside the LXC folder** and uses non-loopback + router-id `2.1.3.7`; it also carries ipv4/ipv6 **multicast** address-family + neighbors (`HQ/NYATER.ios`). Grammar must cover multicast AF. +3. **PANKO and SENKMX share router-id `10.255.254.186`** in their files + (looks like one active-at-a-time config pair). Inventory must treat `.186` + as a single BGP peer identity. +4. **MOMI** (`10.255.254.243`, `HQ/FRR/MOMI/frr.ios`) has the whole `router bgp` + block commented out and is absent from the RR client lists — parked node. +5. VyOS `set protocols bgp neighbor remote-as` **duplicates** the + peer-group `remote-as` for every member — redundant but harmless; parser + can drop it. +6. All RRs: `frr version 10.3`, `service integrated-vtysh-config` → the live + store is `/etc/frr/frr.conf` managed through `vtysh -c "write memory"`. + Docs reference latest (10.7.1); no behavior change expected for + peer-group/route-reflector-client semantics between 10.3 and 10.7. +7. FRR hard rule respected everywhere: **RRs of the same cluster are meshed, + never RCC'd** (`RR-PEERS` never carries route-reflector-client). +8. `bgp cluster-id 172.19.67.0` uses IP-form notation; FRR also accepts + `A:B:C` and decimal forms. Treat as an arbitrary 32-bit value, keep the + exact notation in desired-state files. + +## 5. FRR reference + +Docs: https://docs.frrouting.org/en/latest/ (`bgpd` chapter = route +reflector: `bgp cluster-id`, per-AF `neighbor X route-reflector-client`, +peer-group inheritance; attributes set on a peer-group propagate to members, +per-peer overrides win). Order-of-commands matters in `configure terminal`: +define `neighbor G peer-group` + `remote-as` **before** attaching members to +it, and AF sub-mode changes require entering the right +`address-family` block first.