diff --git a/CLIENTS/README.md b/CLIENTS/README.md new file mode 100644 index 0000000..af87f1f --- /dev/null +++ b/CLIENTS/README.md @@ -0,0 +1,269 @@ +# Downstream Linux PTP Clients + +This directory contains the reusable Linux configuration for servers downstream from NEXUS switch. + +The intended path is: + +```text +RPi5 Grandmaster + │ + \/ +Linux Boundary Clock + │ + \/ +Nexus Boundary Clock + │ + \/ +Linux client NIC PHC + │ + phc2sys + │ + \/ +CLOCK_REALTIME (UTC) +``` + +Each direct client normally needs only: + +1. `ptp` enabled on its physical NEXUS switch port; +2. `ptp4l` to synchronize the NIC PHC from NEXUS; and +3. `phc2sys` to synchronize Linux `CLOCK_REALTIME` from that PHC. + +## Files in this directory + +| File | Install as | Purpose | +|---|---|---| +| `ptp4l-client.conf` | `/etc/linuxptp/ptp4l-client.conf` | Ordinary-clock/client-only PTP configuration | +| `ptp4l-client.service` | `/etc/systemd/system/ptp4l-client.service` | Runs `ptp4l` on the client NIC | +| `phc2sys-client.service` | `/etc/systemd/system/phc2sys-client.service` | Synchronizes `CLOCK_REALTIME` from the client NIC PHC | + +The bundled files currently use `enp4s0` as the client interface. Change that name everywhere if the target server uses another interface. + +## 1. Enable PTP on the corresponding Nexus port + +On NEXUS: + +```text +configure terminal + +interface Ethernet1/ + ptp + +end +copy running-config startup-config +``` + +Example: + +```text +interface Ethernet1/10 + ptp +``` + +Once the client is online, the Nexus port should normally become `Master` in: + +```text +show ptp brief +``` + +## 2. Install LinuxPTP + +On the client: + +```bash +sudo apt update +sudo apt install linuxptp ethtool +``` + +## 3. Identify the PTP NIC + +Find the interface connected to NEXUS: + +```bash +ip -br link +``` + +The examples below assume: + +```text +enp4s0 +``` + +If your interface has a different name, replace `enp4s0` in all three files before installing them. + +For example, search the bundled files with: + +```bash +grep -Rni 'enp4s0' . +``` + +## 4. Verify hardware timestamping + +```bash +ethtool -T enp4s0 +``` + +The NIC should expose hardware transmit, receive, and raw hardware timestamping plus a PTP Hardware Clock. + +You can map it to the PHC device with: + +```bash +readlink -f /sys/class/net/enp4s0/device/ptp/ptp* +``` + +If the NIC does not support hardware timestamping, do not use the bundled `time_stamping hardware` configuration unchanged. + +## 5. Disable competing system-clock synchronization + +`phc2sys-client.service` will discipline `CLOCK_REALTIME`. Disable services that would compete for the same system clock: + +```bash +sudo systemctl disable --now chrony 2>/dev/null || true +sudo systemctl disable --now systemd-timesyncd 2>/dev/null || true +sudo systemctl disable --now ntp 2>/dev/null || true +sudo systemctl disable --now ntpd 2>/dev/null || true +``` + +## 6. Install the client files + +If the interface really is `enp4s0`, install the files directly: + +```bash +sudo install -D -m 0644 ptp4l-client.conf /etc/linuxptp/ptp4l-client.conf +sudo install -D -m 0644 ptp4l-client.service /etc/systemd/system/ptp4l-client.service +sudo install -D -m 0644 phc2sys-client.service /etc/systemd/system/phc2sys-client.service +``` + +If the interface name is different, edit the local copies first and replace `enp4s0` in: + +```text +ptp4l-client.conf +ptp4l-client.service +phc2sys-client.service +``` + +The PTP client config intentionally uses: + +```text +domainNumber 0 +network_transport UDPv4 +delay_mechanism E2E +clientOnly 1 +``` + +`clientOnly 1` prevents a downstream server from ever trying to become the grandmaster. + +## 7. Enable the services + +```bash +sudo systemctl daemon-reload +sudo systemctl enable --now ptp4l-client.service +sudo systemctl enable --now phc2sys-client.service +``` + +The data flow is: + +```text +ptp4l: NEXUS -> NIC PHC +phc2sys: NIC PHC -> CLOCK_REALTIME +``` + +The bundled `phc2sys` service uses: + +```bash +phc2sys -s enp4s0 -c CLOCK_REALTIME -w -m +``` + +`-w` waits for `ptp4l` and obtains the PTP/UTC offset from it, so the client does not need a hard-coded `-37` offset. + +## 8. Verification + +Check service state: + +```bash +systemctl status ptp4l-client +systemctl status phc2sys-client +``` + +Follow the logs: + +```bash +journalctl -fu ptp4l-client +journalctl -fu phc2sys-client +``` + +`ptp4l` should progress into the synchronized client/slave state and its master offsets should settle near zero. + +Query PTP datasets: + +```bash +sudo pmc -u -b 0 'GET PORT_DATA_SET' +sudo pmc -u -b 0 'GET CURRENT_DATA_SET' +sudo pmc -u -b 0 'GET PARENT_DATA_SET' +sudo pmc -u -b 0 'GET TIME_STATUS_NP' +``` + +For a client directly downstream from NEXUS, expect approximately: + +```text +portState CLIENT/SLAVE +stepsRemoved 3 +grandmasterIdentity +``` + +In the current topology the RPi5 grandmaster identity observed on NEXUS is: + +```text +88a29efffe826ce6 +``` + +If the RPi5 NIC or grandmaster identity changes later, use the currently advertised identity instead of treating that value as permanent. + +Also verify the Nexus side: + +```text +show ptp brief +``` + +The client-facing Nexus port should normally be `Master`. + +## 9. Quick deployment checklist for another server + +```text +NEXUS: + interface Ethernet1/X + ptp + +Linux server: + 1. install linuxptp + ethtool + 2. verify `ethtool -T ` + 3. replace `enp4s0` in the three bundled files if needed + 4. install the config and two systemd units + 5. disable chrony/timesyncd/ntpd if they discipline CLOCK_REALTIME + 6. enable ptp4l-client and phc2sys-client + 7. verify logs + pmc output +``` + +## 10. Troubleshooting + +If `ptp4l` stays in LISTENING: + +```bash +sudo tcpdump -ni enp4s0 'udp port 319 or udp port 320' +journalctl -u ptp4l-client -n 100 --no-pager +``` + +Then check NEXUS: + +```text +show ptp brief +show ptp packet +``` + +If the NIC PHC synchronizes but Linux system time does not, inspect: + +```bash +journalctl -u phc2sys-client -n 100 --no-pager +systemctl status chrony systemd-timesyncd ntp ntpd +``` + +Only one service should be responsible for disciplining `CLOCK_REALTIME` on the client. diff --git a/CLIENTS/phc2sys-client.service b/CLIENTS/phc2sys-client.service new file mode 100644 index 0000000..dfeaea9 --- /dev/null +++ b/CLIENTS/phc2sys-client.service @@ -0,0 +1,20 @@ +# /etc/systemd/system/phc2sys-client.service + + +[Unit] +Description=Synchronize Linux system clock from PTP PHC +After=ptp4l-client.service +Requires=ptp4l-client.service + +[Service] +Type=simple +ExecStart=/usr/sbin/phc2sys \ + -s enp4s0 \ + -c CLOCK_REALTIME \ + -w \ + -m +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target diff --git a/CLIENTS/ptp4l-client.conf b/CLIENTS/ptp4l-client.conf new file mode 100644 index 0000000..a67710c --- /dev/null +++ b/CLIENTS/ptp4l-client.conf @@ -0,0 +1,21 @@ +# /etc/linuxptp/ptp4l-client.conf + + +[global] + +# Use NIC hardware timestamping +time_stamping hardware + +# Match the rest of your PTP domain +domainNumber 0 +network_transport UDPv4 +delay_mechanism E2E + +# This machine must never try to become GM +clientOnly 1 + +# Useful logging while setting it up +summary_interval 1 + + +[enp4s0] diff --git a/CLIENTS/ptp4l-client.service b/CLIENTS/ptp4l-client.service new file mode 100644 index 0000000..3fcf02f --- /dev/null +++ b/CLIENTS/ptp4l-client.service @@ -0,0 +1,19 @@ +# /etc/systemd/system/ptp4l-client.service + + +[Unit] +Description=PTP client on enp4s0 +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +ExecStart=/usr/sbin/ptp4l \ + -i enp4s0 \ + -f /etc/linuxptp/ptp4l-client.conf \ + -m +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target