RPi5-PTP-server/BoundaryClock/README.md

Linux PTP Boundary Clock

In this case a server bridges the Raspberry Pi 5 grandmaster's RJ45 connection into the two QSFP+ links that feed NEXUS (the C93180LC-EX).

The physical PTP topology is:

RPi5 Grandmaster
      │
      │ RJ45
      \/
    eno4
 Linux boundary clock
      │
      ├─ enp4s0   ── QSFP+ ──> NEXUS Eth1/5
      └─ enp5s0d1 ─ QSFP+ ──> NEXUS Eth1/6

eno4 is one of the servers's onboard RJ45 NICs. enp4s0 and enp5s0d1 are the two ports on the Mellanox ConnectX-3 Pro card.

It runs one multi-port ptp4l process as a real boundary clock. phc2sys -a -r follows the ptp4l port states and synchronizes the separate PHCs plus the Linux system clock.

Files in this directory

File	Install as	Purpose
ptp4l-boundary.conf	/etc/linuxptp/ptp4l-boundary.conf	Three-port boundary-clock configuration
ptp4l-boundary.service	/etc/systemd/system/ptp4l-boundary.service	Starts ptp4l on all three interfaces
phc2sys-boundary.service	/etc/systemd/system/phc2sys-boundary.service	Synchronizes the separate PHCs and CLOCK_REALTIME

1. Install the required packages

sudo apt update
sudo apt install linuxptp ethtool

2. Verify interface names and hardware timestamping

The bundled configuration expects exactly these interfaces:

eno4       upstream to RPi5
enp4s0     downstream to NEXUS
enp5s0d1   downstream to NEXUS

Verify them:

ip -br link show eno4 enp4s0 enp5s0d1

Check hardware timestamping on all three ports:

ethtool -T eno4
ethtool -T enp4s0
ethtool -T enp5s0d1

Each PTP port needs hardware TX/RX/raw timestamp support and a PHC.

Check which PHC belongs to each interface:

for i in eno4 enp4s0 enp5s0d1; do
    echo "=== $i ==="
    ethtool -T "$i" | grep -E 'PTP Hardware Clock|SOF_TIMESTAMPING_(TX|RX|RAW)_HARDWARE'
    readlink -f /sys/class/net/"$i"/device/ptp/ptp* 2>/dev/null || true
done

The expected layout is normally:

eno4                  -> onboard NIC PHC
enp4s0 + enp5s0d1     -> Mellanox PHC

Because the upstream onboard NIC and Mellanox card use different PHCs, the configuration intentionally contains:

boundary_clock_jbod 1

Do not remove it unless you have verified that every active ptp4l port is backed by the same PHC.

3. Install the boundary-clock files

From this directory:

sudo install -D -m 0644 ptp4l-boundary.conf /etc/linuxptp/ptp4l-boundary.conf
sudo install -D -m 0644 ptp4l-boundary.service /etc/systemd/system/ptp4l-boundary.service
sudo install -D -m 0644 phc2sys-boundary.service /etc/systemd/system/phc2sys-boundary.service

The PTP config is intentionally asymmetric:

[eno4]
    BMCA may select this as CLIENT/SLAVE to the RPi5

[enp4s0]
    serverOnly 1

[enp5s0d1]
    serverOnly 1

Both Mellanox links therefore advertise the server toward NEXUS, while only eno4 can become the upstream client port.

4. Disable competing Linux clock synchronization

phc2sys-boundary.service uses -r, so the server's CLOCK_REALTIME is also disciplined from PTP. Do not leave chrony, ntpd, or systemd-timesyncd simultaneously controlling the system clock.

For example:

sudo systemctl disable --now chrony 2>/dev/null || true
sudo systemctl disable --now systemd-timesyncd 2>/dev/null || true
sudo systemctl disable --now ntp 2>/dev/null || true
sudo systemctl disable --now ntpd 2>/dev/null || true

5. Enable the services

sudo systemctl daemon-reload
sudo systemctl enable --now ptp4l-boundary.service
sudo systemctl enable --now phc2sys-boundary.service

The bundled phc2sys command is:

phc2sys -a -r -m

Here:

-a   follow ptp4l port states and automatically choose PHC source/sinks
-r   also synchronize CLOCK_REALTIME as a sink

Use one -r, not -rr, for this design. System's system clock should not become an alternate PTP source.

6. Expected steady-state port roles

With the RPi5 grandmaster reachable on eno4, the server should converge to:

eno4       CLIENT / SLAVE
enp4s0     SERVER / MASTER
enp5s0d1   SERVER / MASTER

The grandmaster identity should still be the RPi5 identity, and the server should report approximately:

stepsRemoved = 1

The two Mellanox-facing ports are separate redundant paths toward the same Nexus. On NEXUS, one of those Nexus ports should become Slave and the other Passive.

7. Verification

Service state:

systemctl status ptp4l-boundary
systemctl status phc2sys-boundary

Live logs:

journalctl -fu ptp4l-boundary
journalctl -fu phc2sys-boundary

Query the PTP datasets:

sudo pmc -u -b 0 'GET PORT_DATA_SET'
sudo pmc -u -b 0 'GET CURRENT_DATA_SET'
sudo pmc -u -b 0 'GET PARENT_DATA_SET'
sudo pmc -u -b 0 'GET TIME_STATUS_NP'

Things to check:

- eno4 has selected the RPi5 as its upstream master/server
- both Mellanox ports are master/server ports
- grandmasterIdentity is the RPi5 identity
- stepsRemoved is 1
- phc2sys offsets settle near zero

If you want to inspect each interface's PTP hardware again:

ethtool -T eno4
ethtool -T enp4s0
ethtool -T enp5s0d1

8. Failure behavior

The two NEXUS links are redundant from the Nexus BMCA perspective.

Normal state on NEXUS:

one link   Slave
other link Passive

If the selected link fails, the passive link should be eligible to become the new Slave port without changing the RPi5 grandmaster identity.

If eno4 loses the RPi5 completely, the server no longer has its intended upstream reference. Investigate ptp4l-boundary and phc2sys-boundary before trusting time downstream.

9. Important filename/interface consistency

The actual second Mellanox interface is enp5s0d1. The ptp4l-boundary.service already uses that name, and the configuration in this bundle should also contain:

[enp5s0d1]
serverOnly 1

If the OS renames any NIC after a hardware/firmware change, update both ptp4l-boundary.conf and ptp4l-boundary.service before restarting PTP.