CHIMERA/docs/ARCHITECTURE.md

# CHIMERA architecture & codebase plan

Status: design document. No code yet. This file defines the repo layout,
the data model, and the engine behavior we build toward.

## Design principles

1. **Desired state lives in git, never in the router.** A small YAML
   inventory describes each RR *cluster* (cluster-id, routers, peer-groups,
   peers). CHIMERA compiles it into per-RR deltas. Archived configs in
   `CiscoConfigs/` are only an *import source*, not the runtime store.
2. **Never wholesale-replace `frr.conf`.** RRs carry hand-tuned OSPF, interface
   and system stanzas. CHIMERA only emits targeted `configure terminal`
   commands (vtysh), then `write memory` per box. Whole-file templating is a
   v3 maybe, gated by a `file-mode` flag, not the default.
3. **Peer-groups are first-class.** All bulk operations (add/modify a group
   across *all* RRs of a cluster) are peer-group-scoped; FRR propagates
   group attributes to members, and per-peer overrides win over group
   settings. Single-peer edits only when explicitly asked.
4. **Mesh-aware generation.** The RR-PEERS group is derived per-RR by
   subtracting self; the inventory lists the cluster's RR loopbacks once and
   each RR's neighbor list is generated (this is why the four MUYU/FOX/WOLF/…
   files are near-duplicates with self-exclusion).
5. **Plan → diff → apply, always.** Every mutating command renders a delta
   first (`--diff` is default output). `apply` takes the same plan object.
6. **Drift detection is a first-class verb** (`status`), not an aftercare:
   parse live `vtysh -c "show running-config bgpd"` against inventory.
7. **Serial, per-cluster rollout with a canary.** Apply to one RR first,
   verify, then the rest. Most peer-group attribute changes don't tear
   sessions down; anything that *would* reset sessions (remote-as change,
   password, shutdown, un-resourced peer-group deletion) gets flagged in the
   plan with a loud warning.

## Execution model (FRR on Debian/LXC)

Transport: SSH (paramiko or plain `ssh` subprocess) →
`vtysh -c "show running-config bgpd"` for reads (integrated-vtysh-config is
turned ON on every RR, so this is literally `/etc/frr/frr.conf` in bgpd view).

Apply = one batch of `vtysh -c` commands per box:

```
vtysh -c "configure terminal" -c "router bgp 64572" -c "address-family l2vpn evpn" -c "neighbor CLIENTS send-community both"
vtysh -c "configure terminal" ...   # FRR's integrated config applies atomically per invocation
vtysh -c "write memory"
```

Command sequencing rules the compiler must respect (from the FRR 10.x bgpd
docs):

- `neighbor <G> peer-group` and `neighbor <G> remote-as` before attaching any
  member to the group.
- Global BGP params (`bgp cluster-id`, `bgp router-id`) live under
  `router bgp <AS>`, not under AF sub-modes.
- AF-scoped ops (`route-reflector-client`, `activate`, `send-community`)
  require entering the right `address-family` block first.
- Deleting an AF block with `no ...` implicitly kills its neighbor/AF lines —
  plan must expand that into explicit removals when diffing, so dry-run output
  is truthful.
- Configs contain duplicated/merged AF stanzas (see BGP-NETWORK.md quirks):
  the parser normalizes them, the writer emits canonical single-block form.
  First apply of a normalize-only diff is a safe no-op-semantics change we
  can land quietly.

Rollback: before touching a box, `vtysh -c "show running-config bgpd"` →
`/var/backups/chimera/<host>/<ts>.conf` plus `cp /etc/frr/frr.conf` (root or
`sudo -n`); `chimera rollback <host> [--from <ts>]` re-applies the snapshot
through the same vtysh path.

Optional per-box pre/post hooks (`hooks:` in inventory) to run `systemctl
reload frr`-style recovery, though normally unnecessary.

## Data model

Inventory YAML (one file per cluster, e.g. `inventory/hq-evpn.yml`):

```yaml
cluster:
  name: hq-evpn
  as: 64572
  cluster_id: 6.7.6.7            # keep notation as configured
  address_family_default: [ipv4_unicast, l2vpn_evpn]
  routers:
    - { name: MUYU-RR,  ssh: muyu-rr.hq.panchain.org, loopback: 10.255.254.246 }
    - { name: FOX-RR,   ssh: fox-rr.hq.panchain.org,  loopback: 10.255.254.247 }
    - { name: WOLF-RR,  ssh: wolf-rr.hq.panchain.org, loopback: 10.255.254.245 }
    - { name: NYATER,   ssh: nyater.hq.panchain.org,  loopback: 10.255.254.255, role: rr+switch }
  peer_groups:
    RR-PEERS:
      remote_as: 64572
      update_source: lo
      af:
        ipv4_unicast: { activate: true, send_community: both }
        l2vpn_evpn:   { activate: true, send_community: both }
    CLIENTS:
      remote_as: 64572
      update_source: lo
      af:
        ipv4_unicast: { activate: true, send_community: both, route_reflector_client: true }
        l2vpn_evpn:   { activate: true, send_community: both, route_reflector_client: true }
  peers:
    # RR-PEERS members: declared once; generated per-RR minus self
    - { group: RR-PEERS, loopback_ref: [MUYU-RR, FOX-RR, WOLF-RR, NYATER], mesh: true }
    # clients:
    - { group: CLIENTS, ip: 10.255.254.250, desc: URAKAZE }
    - { group: CLIENTS, ip: 10.255.254.234, desc: VICTORIA }
    ...
```

Core objects (Python, `attrs`/`dataclasses`):

- `RouterReflector` — host, ssh target, loopback, role (rr | rr-switch),
  parsed snapshot of its running bgp config.
- `BgpCluster` — AS, cluster_id, list of RRs, peer-groups, peers; owns the
  `validate()` rules (cluster_id must be identical on every RR; RRs never in
  an RCC group; AS consistency).
- `PeerGroup` — name, attrs (remote_as, update_source, password, timers…),
  per-AF settings (`activate`, `send_community`, `route_reflector_client`,
  `allowas_in`, route-maps, … as encountered).
- `Peer` — ip, group membership, per-peer overrides dict.
- `RunningBgpConfig` — normalized parse of one RR's bgpd view
  (`router bgp` → globals → peer-groups → peers → AF blocks).
- `Delta` — ordered list of `BgpCommand`s with `resets_session: bool` flag.

`chimera.cli` and the future web layer are **thin callers of the core**; the
core must not import anything from cli/web (so a `chimera.core` import in a
notebook works too).

## Repo layout

```
CHIMERA/
  pyproject.toml               # package = chimera; console_script = chimera
  src/chimera/
    core/
      models.py                # dataclasses above
      inventory.py             # YAML load/validate (pydantic or jsonschema)
      normalize.py             # parse quirks → canonical model
      planner.py               # desired × live → Delta; sequencing rules
      commands.py              # BgpCommand rendering (vtysh arg list)
    transport/
      ssh.py                   # paramiko/openssh subprocess runner
      frr.py                   # vtysh read/apply/write/rollback
      vyos.py                  # (later) set-syntax provider for VyOS transit
    parse/
      bgp_grammar.py           # line-based bgpd config parser
    cli/
      main.py                  # typer app: the verbs below
    web/
      app.py                   # FastAPI, same core; read-only pages first
      templates/               # Jinja + htmx; no SPA needed
    tests/
      unit/  e2e/
  inventory/                   # git-tracked desired state
    hq-evpn.yml  nekos-evpn.yml  hq-transit.yml
  docs/                        # this file, BGP-NETWORK.md, …
  docker/
    rr-lab/                    # frr:10.3 containers, 4-RR full mesh: e2e lab
```

## CLI verbs (target surface)

```
chimera init                      # generate inventory stubs
chimera import --live             # bootstrap inventory from running RRs
chimera status [cluster]          # per-RR drift report: group attrs, RCC, cluster_id
chimera plan peergroup edit <g> --cluster <c> --af l2vpn-evpn --send-community both
chimera peer add 10.255.254.243 --group CLIENTS --cluster hq-evpn
chimera peergroup add <g> --cluster <c> --remote-as 64572 ...
chimera cluster set-id <c> <id>  # all RRs; refuses until all agree
chimera apply <plan-id> [--canary-first] [--only <rr>] [--yes]
chimera rollback <rr> [--from <ts>]
```

Mutating verbs print the Delta by default and exit; `--yes` applies.
`chimera cluster set-id` refuses to emit until `status` shows every RR
reachable — changing cluster-id on a subset restarts the TCP-BFD-less RR
cluster's ORF/count logic mid-reflection; we want one plan for all boxes.

## Web UI plan (phase 2, after CLI core is solid)

- Pages: cluster overview (4 RRs × sessions table), drift report,
  desired-state editor (validated YAML), plan preview with rendered vtysh
  commands, apply progress log.
- Auth: local/single-user token first; Tailscale-gated in practice.
- Same core as CLI — no duplicated business logic.

## Testing

- Unit: parser golden files — the actual MUYU/YUUKA/REIMU configs copied into
  `tests/parse/golden/` (they contain the weird nested-AF stanzas — regression
  suite for the normalizer).
- Property test: model → render → re-parse == normalize(parse) (round-trip).
- E2e lab: `docker/rr-lab` = 4× `frr:10.3` containers, loopback on dummy0,
  full-mesh RR + 2 fake clients; CHIMERA apply against them in CI.
- Fake transport mode (`--transport fake`) that shells out to a stub — keeps
  e2e cheap without docker.

## Roadmap

1. **M0** parser + normalizer + golden tests (no SSH yet; parse the archived
   `.ios` files). ~ This unlocks everything else.
2. **M1** inventory + `status` (read-only drift report over SSH).
3. **M2** planner + commands + `apply`/`rollback`, canary rollout.
4. **M3** verbs: peergroup add/edit, peer add, cluster set-id.
5. **M4** web read-only dashboard; apply via UI with plan-preview gate.
6. **M5** VyOS provider (HQ transit NOZOMI/SHUN) after FRR flow is proven —
   the transit RR quirk (commented-out RCC) is a `status` warning fixture.

## Open questions

- Which FRR version is on the live RRs exactly? Repo headers say `frr version
  10.3`; docs target 10.7.1 — verify with `vtysh -c "show version"` during
  M1, pin grammar to what runs.
- HQ's 4 RR members are 3 LXC + NYATER switch: does orchestration treat
  NYATER's bgpd (switch platform, multicast AF) identically? Assumed yes —
  it speaks the same bgpd grammar.
- Is there a management network/VRF for SSH to the RRs (`vrf` flag on ssh
  runner may be needed)?
- Do we want route-map/route-reflector-set-extra-community knobs in the data
  model now, or keep minimal until encountered in the configs?