configs for a boundary clock

BoundaryClock/README.md

@@ -0,0 +1,235 @@
+# Linux PTP Boundary Clock
+
+In this case a server bridges the Raspberry Pi 5 grandmaster's RJ45 connection into the two QSFP+ links that feed NEXUS (the C93180LC-EX).
+
+The physical PTP topology is:
+
+```text
+RPi5 Grandmaster
+      │
+      │ RJ45
+      \/
+    eno4
+ Linux boundary clock
+      │
+      ├─ enp4s0   ── QSFP+ ──> NEXUS Eth1/5
+      └─ enp5s0d1 ─ QSFP+ ──> NEXUS Eth1/6
+```
+
+`eno4` is one of the servers's onboard RJ45 NICs. `enp4s0` and `enp5s0d1` are the two ports on the Mellanox ConnectX-3 Pro card.
+
+It runs one multi-port `ptp4l` process as a real boundary clock. `phc2sys -a -r` follows the `ptp4l` port states and synchronizes the separate PHCs plus the Linux system clock.
+
+## Files in this directory
+
+| File | Install as | Purpose |
+|---|---|---|
+| `ptp4l-boundary.conf` | `/etc/linuxptp/ptp4l-boundary.conf` | Three-port boundary-clock configuration |
+| `ptp4l-boundary.service` | `/etc/systemd/system/ptp4l-boundary.service` | Starts `ptp4l` on all three interfaces |
+| `phc2sys-boundary.service` | `/etc/systemd/system/phc2sys-boundary.service` | Synchronizes the separate PHCs and `CLOCK_REALTIME` |
+
+## 1. Install the required packages
+
+```bash
+sudo apt update
+sudo apt install linuxptp ethtool
+```
+
+## 2. Verify interface names and hardware timestamping
+
+The bundled configuration expects exactly these interfaces:
+
+```text
+eno4       upstream to RPi5
+enp4s0     downstream to NEXUS
+enp5s0d1   downstream to NEXUS
+```
+
+Verify them:
+
+```bash
+ip -br link show eno4 enp4s0 enp5s0d1
+```
+
+Check hardware timestamping on all three ports:
+
+```bash
+ethtool -T eno4
+ethtool -T enp4s0
+ethtool -T enp5s0d1
+```
+
+Each PTP port needs hardware TX/RX/raw timestamp support and a PHC.
+
+Check which PHC belongs to each interface:
+
+```bash
+for i in eno4 enp4s0 enp5s0d1; do
+    echo "=== $i ==="
+    ethtool -T "$i" | grep -E 'PTP Hardware Clock|SOF_TIMESTAMPING_(TX|RX|RAW)_HARDWARE'
+    readlink -f /sys/class/net/"$i"/device/ptp/ptp* 2>/dev/null || true
+done
+```
+
+The expected layout is normally:
+
+```text
+eno4                  -> onboard NIC PHC
+enp4s0 + enp5s0d1     -> Mellanox PHC
+```
+
+Because the upstream onboard NIC and Mellanox card use different PHCs, the configuration intentionally contains:
+
+```text
+boundary_clock_jbod 1
+```
+
+Do not remove it unless you have verified that every active `ptp4l` port is backed by the same PHC.
+
+## 3. Install the boundary-clock files
+
+From this directory:
+
+```bash
+sudo install -D -m 0644 ptp4l-boundary.conf /etc/linuxptp/ptp4l-boundary.conf
+sudo install -D -m 0644 ptp4l-boundary.service /etc/systemd/system/ptp4l-boundary.service
+sudo install -D -m 0644 phc2sys-boundary.service /etc/systemd/system/phc2sys-boundary.service
+```
+
+The PTP config is intentionally asymmetric:
+
+```text
+[eno4]
+    BMCA may select this as CLIENT/SLAVE to the RPi5
+
+[enp4s0]
+    serverOnly 1
+
+[enp5s0d1]
+    serverOnly 1
+```
+
+Both Mellanox links therefore advertise the server toward NEXUS, while only `eno4` can become the upstream client port.
+
+## 4. Disable competing Linux clock synchronization
+
+`phc2sys-boundary.service` uses `-r`, so the server's `CLOCK_REALTIME` is also disciplined from PTP. Do not leave chrony, ntpd, or systemd-timesyncd simultaneously controlling the system clock.
+
+For example:
+
+```bash
+sudo systemctl disable --now chrony 2>/dev/null || true
+sudo systemctl disable --now systemd-timesyncd 2>/dev/null || true
+sudo systemctl disable --now ntp 2>/dev/null || true
+sudo systemctl disable --now ntpd 2>/dev/null || true
+```
+
+## 5. Enable the services
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable --now ptp4l-boundary.service
+sudo systemctl enable --now phc2sys-boundary.service
+```
+
+The bundled `phc2sys` command is:
+
+```bash
+phc2sys -a -r -m
+```
+
+Here:
+
+```text
+-a   follow ptp4l port states and automatically choose PHC source/sinks
+-r   also synchronize CLOCK_REALTIME as a sink
+```
+
+Use one `-r`, not `-rr`, for this design. System's system clock should not become an alternate PTP source.
+
+## 6. Expected steady-state port roles
+
+With the RPi5 grandmaster reachable on `eno4`, the server should converge to:
+
+```text
+eno4       CLIENT / SLAVE
+enp4s0     SERVER / MASTER
+enp5s0d1   SERVER / MASTER
+```
+
+The grandmaster identity should still be the RPi5 identity, and the server should report approximately:
+
+```text
+stepsRemoved = 1
+```
+
+The two Mellanox-facing ports are separate redundant paths toward the same Nexus. On NEXUS, one of those Nexus ports should become `Slave` and the other `Passive`.
+
+## 7. Verification
+
+Service state:
+
+```bash
+systemctl status ptp4l-boundary
+systemctl status phc2sys-boundary
+```
+
+Live logs:
+
+```bash
+journalctl -fu ptp4l-boundary
+journalctl -fu phc2sys-boundary
+```
+
+Query the PTP datasets:
+
+```bash
+sudo pmc -u -b 0 'GET PORT_DATA_SET'
+sudo pmc -u -b 0 'GET CURRENT_DATA_SET'
+sudo pmc -u -b 0 'GET PARENT_DATA_SET'
+sudo pmc -u -b 0 'GET TIME_STATUS_NP'
+```
+
+Things to check:
+
+```text
+- eno4 has selected the RPi5 as its upstream master/server
+- both Mellanox ports are master/server ports
+- grandmasterIdentity is the RPi5 identity
+- stepsRemoved is 1
+- phc2sys offsets settle near zero
+```
+
+If you want to inspect each interface's PTP hardware again:
+
+```bash
+ethtool -T eno4
+ethtool -T enp4s0
+ethtool -T enp5s0d1
+```
+
+## 8. Failure behavior
+
+The two NEXUS links are redundant from the Nexus BMCA perspective.
+
+Normal state on NEXUS:
+
+```text
+one link   Slave
+other link Passive
+```
+
+If the selected link fails, the passive link should be eligible to become the new `Slave` port without changing the RPi5 grandmaster identity.
+
+If `eno4` loses the RPi5 completely, the server no longer has its intended upstream reference. Investigate `ptp4l-boundary` and `phc2sys-boundary` before trusting time downstream.
+
+## 9. Important filename/interface consistency
+
+The actual second Mellanox interface is `enp5s0d1`. The `ptp4l-boundary.service` already uses that name, and the configuration in this bundle should also contain:
+
+```text
+[enp5s0d1]
+serverOnly 1
+```
+
+If the OS renames any NIC after a hardware/firmware change, update both `ptp4l-boundary.conf` and `ptp4l-boundary.service` before restarting PTP.

BoundaryClock/phc2sys-boundary.service

@@ -0,0 +1,15 @@
+# /etc/systemd/system/phc2sys-boundary.service
+
+[Unit]
+Description=Synchronize PTP Boundary Clock PHCs
+After=ptp4l-boundary.service
+Requires=ptp4l-boundary.service
+
+[Service]
+Type=simple
+ExecStart=/usr/sbin/phc2sys -a -r -m
+Restart=always
+RestartSec=3
+
+[Install]
+WantedBy=multi-user.target

BoundaryClock/ptp4l-boundary.conf

@@ -0,0 +1,38 @@
+# /etc/linuxptp/ptp4l-boundary.conf
+
+[global]
+
+time_stamping           hardware
+
+domainNumber            0
+network_transport       UDPv4
+delay_mechanism         E2E
+
+# Needed when the two NICs have different PHCs.
+boundary_clock_jbod     1
+
+# If both interfaces somehow share the same PHC, remove:
+# boundary_clock_jbod     1
+
+# We want the RPi GM to beat this clock in BMCA.
+priority1               200
+priority2               128
+
+summary_interval        1
+
+
+# Upstream toward RPi5.
+# This is allowed to become the PTP client according to BMCA.
+[eno4]
+
+
+# Never accept another master from this direction.
+# Downstream link 1 to Nexus
+[enp4s0]
+serverOnly              1
+
+
+# Downstream link 2 to Nexus
+[enp5s0d1]
+serverOnly              1
+

BoundaryClock/ptp4l-boundary.service

@@ -0,0 +1,20 @@
+# /etc/systemd/system/ptp4l-boundary.service
+
+[Unit]
+Description=PTP Boundary Clock
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+ExecStart=/usr/sbin/ptp4l \
+    -i eno4 \
+    -i enp4s0 \
+    -i enp5s0d1 \
+    -f /etc/linuxptp/ptp4l-boundary.conf \
+    -m
+Restart=always
+RestartSec=3
+
+[Install]
+WantedBy=multi-user.target