docs: BGP network map + CHIMERA architecture plan
Ground truth read from CiscoConfigs: 3 RR clusters (HQ 64572/6.7.6.7, NEKOS 64600/172.19.67.0, VyOS transit 64580/67.67.21.37), RR-PEERS+CLIENTS peer-group scheme, config quirks (nested AF stanzas, NYATER 4th RR, commented-out RCC on VyOS transit RRs, shared .186 router-id). Architecture: declarative YAML inventory, normalized bgpd parser, vtysh plan/diff/apply engine with canary rollout and rollback, CLI verbs, docker FRR e2e lab, roadmap M0-M5.
This commit is contained in:
134
docs/BGP-NETWORK.md
Normal file
134
docs/BGP-NETWORK.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# BGP network map (ground truth from CiscoConfigs repo)
|
||||
|
||||
Source of truth: the archived configs in
|
||||
`/hdd/GiTea-REPO/CiscoConfigs/MP-BGP_EVPN_VXLAN_V2/` and
|
||||
`/hdd/GiTea-REPO/CiscoConfigs/VyOS/`. Everything below was read out of those
|
||||
files; live state may drift (CHIMERA's job is to detect exactly that).
|
||||
|
||||
## 1. AS plan
|
||||
|
||||
| AS | Role |
|
||||
|---------|-------------------------------------------------------------|
|
||||
| `64572` | HQ EVPN/VXLAN **underlay** (FRR + switch RRs, iBGP full mesh w/ RRs) |
|
||||
| `64580` | HQ **transit** routers (VyOS: NOZOMI, SHUN, VIPERA, REISA, EDGE-2, EDGE-9, EDGE-MIKU, LXC-1) |
|
||||
| `64581` | MU3S / `mu3.panchain.org` (KIMIKONO, EDGE) |
|
||||
| `64590` | SHUPO (cloud: Hetzner VMs + `shupo.panchain.org`) — also hosts FUMOS VRRP pair (HIKARI/MIYU/MEGU) |
|
||||
| `64600` | NEKOS site (VRRP pair SOF/OHR, router EIN, edge, RRs) |
|
||||
| `64610` | ORC site (KIKI) |
|
||||
|
||||
## 2. Route Reflector clusters
|
||||
|
||||
### HQ underlay — AS 64572, cluster-id `6.7.6.7`
|
||||
|
||||
Four RRs, full mesh between them, loopbacks in `10.255.254.0/24`
|
||||
(advertised via OSPF):
|
||||
|
||||
| RR | Loopback / router-id | Form | Config |
|
||||
|-------------|----------------------|--------------------|--------|
|
||||
| MUYU-RR | `10.255.254.246` | Debian LXC (`frr version 10.3`, `service integrated-vtysh-config`) | `HQ/FRR/RouteReflectorLXC/MUYU/MUYU.ios` |
|
||||
| FOX-RR | `10.255.254.247` | Debian LXC | `.../FOX/FOX.ios` |
|
||||
| WOLF-RR | `10.255.254.245` | Debian LXC | `.../WOLF/WOLF.ios` |
|
||||
| NYATER | `10.255.254.255` (bgp router-id `2.1.3.7`) | HQ hardware switch (also runs multicast AF) | `HQ/NYATER.ios` — NOT in RouteReflectorLXC/ |
|
||||
|
||||
Peer-groups (identical on all four, `update-source lo`):
|
||||
|
||||
- `RR-PEERS` — iBGP mesh members (the *other* RRs). **Never**
|
||||
route-reflector-client. Each RR file lists the others minus itself — that
|
||||
self-exclusion is why the four files are near-duplicates.
|
||||
- `CLIENTS` — every underlay router; `route-reflector-client` in both
|
||||
`address-family ipv4 unicast` and `address-family l2vpn evpn`.
|
||||
|
||||
RR-PEERS on e.g. MUYU: `.255`, `.247`, `.245`. CLIENTS (from MUYU.ios):
|
||||
`.254 .253 .252 .251 .249 .248 .244 .242 .240 .250 .234`.
|
||||
Known client identities: `.250` = URAKAZE (Debian), `.234` = VICTORIA
|
||||
(OpenWRT AP). Others map to HQ hosts/hypervisors — resolve by importing live
|
||||
state.
|
||||
|
||||
Address families activated on RRs: `l2vpn evpn` + `ipv4 unicast`, both with
|
||||
`send-community both` on RR-PEERS and CLIENTS.
|
||||
|
||||
### NEKOS EVPN — AS 64600, cluster-id `172.19.67.0`
|
||||
|
||||
Four RRs, full mesh, loopbacks `10.255.254.0/24` (NEKOS OSPF domain):
|
||||
|
||||
| RR | Loopback / router-id | Form |
|
||||
|-------------|----------------------|------|
|
||||
| KOKONUT-RR | `10.255.254.183` | `NEKOS/FRR/RR/KOKONUT-RR.ios` |
|
||||
| LH-RR | `10.255.254.184` | `NEKOS/FRR/RR/LH-RR.ios` |
|
||||
| YUUKA-RR | `10.255.254.185` | `NEKOS/FRR/RR/YUUKA-RR.ios` |
|
||||
| REIMU | `10.255.254.187` | bare metal, `NEKOS/REIMU/REIMU.ios` |
|
||||
|
||||
Same peer-group scheme: `RR-PEERS` (the other 3) + `CLIENTS`
|
||||
(route-reflector-client in `ipv4 unicast` **and** `l2vpn evpn`,
|
||||
`send-community both`, `update-source lo`).
|
||||
|
||||
NEKOS CLIENTS: `.200 .199 .198 .197 .196 .195` (LXC/VyOS class),
|
||||
`.191 .190 .189 .188 .186` (hardware: YUUKA=`.190`, LOLI-HUNTER=`.189`,
|
||||
KOKONUT=`.188`, SENKMX and PANKO both config'd with `.186` — see quirk list).
|
||||
|
||||
VyOS clients (SOF, OHR, EIN, Edge NIHA/EROHA/AOBA) peer with **all four RRs**
|
||||
via their own peer-group `NEKOS` (`remote-as 64600`,
|
||||
`update-source <172.19.x loopback>`).
|
||||
|
||||
### HQ transit — AS 64580, cluster-id `67.67.21.37` (VyOS)
|
||||
|
||||
- NOZOMI (bgp router-id `192.168.1.2`, iBGP `update-source 10.255.254.230`) and SHUN (`update-source 10.255.254.228`) are both configured with `bgp parameters cluster-id 67.67.21.37`.
|
||||
- iBGP peer-group `HQ` toward the other transit RR, AFs: `ipv4-unicast`,
|
||||
`ipv6-unicast`, `l2vpn-evpn` (plus multicast in comments).
|
||||
- E-BGP peer-groups by site: `SHUPO` (64590), `NEKOS` (64600), `ORC` (64610),
|
||||
`MU3S`/`MU3` (64581), `HOME` (64600, on YUZU).
|
||||
- ⚠ **In the archived configs every `route-reflector-client` line on
|
||||
NOZOMI/SHUN is commented out.** cluster-id with no RCC = ineffective
|
||||
reflection. Either the live boxes have RCC enabled out-of-band, or this is
|
||||
dead config. First thing CHIMERA drift-check should confirm.
|
||||
|
||||
## 3. Client side patterns (what CHIMERA manages *about*, but doesn't *is*)
|
||||
|
||||
- HQ FRR clients (URAKAZE, MOMI, IBUKI-M-ATOR, OPS, WOLF-HUNTER, APs…):
|
||||
plain `router bgp 64572`, one neighbor per RR (`.255 .245 .246 .247`),
|
||||
no client-side peer-group, `neighbor ... route-reflector-client` on *their*
|
||||
side is absent (correct — RCC belongs on the RR), `timers 30 60`,
|
||||
`bgp graceful-restart`, `no bgp default ipv4-unicast`.
|
||||
- HQ VyOS routers (YUZU, IZUNA): transit iBGP peer-group `HQ` + eBGP
|
||||
peer-groups per site AS.
|
||||
- Underlay is OSPF (`auto-cost reference-bandwidth 40000` on HQ FRR,
|
||||
`1000` on NEKOS RR files — mismatch is intentional per-site).
|
||||
|
||||
## 4. Quirks & gotchas (write these into the parser's expectations)
|
||||
|
||||
1. **Duplicated/nested AF stanzas.** `MUYU/FOX/WOLF`, `YUUKA-RR`, `REIMU`
|
||||
contain a second `address-family l2vpn evpn` block (in REIMU it's even
|
||||
*nested* inside the first, with no closing `exit-address-family`). FRR's
|
||||
parser accepts and merges these, but `write memory`-generated config would
|
||||
never look like this — hand-edit residue. CHIMERA must normalize to one
|
||||
block per AF when writing.
|
||||
2. **NYATER is an RR that lives outside the LXC folder** and uses non-loopback
|
||||
router-id `2.1.3.7`; it also carries ipv4/ipv6 **multicast** address-family
|
||||
neighbors (`HQ/NYATER.ios`). Grammar must cover multicast AF.
|
||||
3. **PANKO and SENKMX share router-id `10.255.254.186`** in their files
|
||||
(looks like one active-at-a-time config pair). Inventory must treat `.186`
|
||||
as a single BGP peer identity.
|
||||
4. **MOMI** (`10.255.254.243`, `HQ/FRR/MOMI/frr.ios`) has the whole `router bgp`
|
||||
block commented out and is absent from the RR client lists — parked node.
|
||||
5. VyOS `set protocols bgp neighbor <ip> remote-as` **duplicates** the
|
||||
peer-group `remote-as` for every member — redundant but harmless; parser
|
||||
can drop it.
|
||||
6. All RRs: `frr version 10.3`, `service integrated-vtysh-config` → the live
|
||||
store is `/etc/frr/frr.conf` managed through `vtysh -c "write memory"`.
|
||||
Docs reference latest (10.7.1); no behavior change expected for
|
||||
peer-group/route-reflector-client semantics between 10.3 and 10.7.
|
||||
7. FRR hard rule respected everywhere: **RRs of the same cluster are meshed,
|
||||
never RCC'd** (`RR-PEERS` never carries route-reflector-client).
|
||||
8. `bgp cluster-id 172.19.67.0` uses IP-form notation; FRR also accepts
|
||||
`A:B:C` and decimal forms. Treat as an arbitrary 32-bit value, keep the
|
||||
exact notation in desired-state files.
|
||||
|
||||
## 5. FRR reference
|
||||
|
||||
Docs: https://docs.frrouting.org/en/latest/ (`bgpd` chapter = route
|
||||
reflector: `bgp cluster-id`, per-AF `neighbor X route-reflector-client`,
|
||||
peer-group inheritance; attributes set on a peer-group propagate to members,
|
||||
per-peer overrides win). Order-of-commands matters in `configure terminal`:
|
||||
define `neighbor G peer-group` + `remote-as` **before** attaching members to
|
||||
it, and AF sub-mode changes require entering the right
|
||||
`address-family` block first.
|
||||
Reference in New Issue
Block a user