Added exporter for Grandmaster, borderClocl and client clock
This commit is contained in:
370
exporters/README.md
Normal file
370
exporters/README.md
Normal file
@@ -0,0 +1,370 @@
|
|||||||
|
# PTP Prometheus Exporters
|
||||||
|
|
||||||
|
Python Prometheus exporters for the PTP topology:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GT-U7
|
||||||
|
│
|
||||||
|
\/
|
||||||
|
RPi5 Grandmaster
|
||||||
|
Grandmaster
|
||||||
|
│
|
||||||
|
\/
|
||||||
|
Linux boundary clock
|
||||||
|
│ │
|
||||||
|
\/ \/
|
||||||
|
NEXUS / C93180LC-EX
|
||||||
|
Nexus boundary clock
|
||||||
|
│
|
||||||
|
\/
|
||||||
|
Linux downstream clients
|
||||||
|
```
|
||||||
|
|
||||||
|
There are two exporter programs:
|
||||||
|
|
||||||
|
- `linux_ptp_exporter.py` — runs locally on Linux grandmasters, Linux boundary clocks, and Linux clients.
|
||||||
|
- `nexus_ptp_exporter.py` — runs on a Linux monitoring/Prometheus host and polls the Nexus over SSH.
|
||||||
|
|
||||||
|
The Linux exporter uses the **read-only** `ptp4l` management socket (`/var/run/ptp4lro`) and `pmc`. It does not alter PTP state.
|
||||||
|
|
||||||
|
## Metrics worth graphing
|
||||||
|
|
||||||
|
For Linux PTP nodes:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ptp_master_offset_nanoseconds
|
||||||
|
ptp_mean_path_delay_nanoseconds
|
||||||
|
ptp_steps_removed
|
||||||
|
ptp_gm_present
|
||||||
|
ptp_port_state
|
||||||
|
ptp_port_messages_total
|
||||||
|
ptp_port_service_events_total
|
||||||
|
ptp_phc2sys_offset_nanoseconds
|
||||||
|
ptp_service_up
|
||||||
|
```
|
||||||
|
|
||||||
|
On the RPi5 grandmaster, additionally:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ptp_gm_chrony_system_time_offset_seconds
|
||||||
|
ptp_gm_chrony_last_offset_seconds
|
||||||
|
ptp_gm_chrony_rms_offset_seconds
|
||||||
|
ptp_gm_chrony_root_dispersion_seconds
|
||||||
|
ptp_gm_chrony_source
|
||||||
|
```
|
||||||
|
|
||||||
|
For NYATER:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ptp_nexus_clock_locked
|
||||||
|
ptp_nexus_offset_from_master_nanoseconds
|
||||||
|
ptp_nexus_mean_path_delay_nanoseconds
|
||||||
|
ptp_nexus_steps_removed
|
||||||
|
ptp_nexus_port_state
|
||||||
|
ptp_nexus_port_messages_total
|
||||||
|
ptp_nexus_parent_info
|
||||||
|
```
|
||||||
|
|
||||||
|
## 1. Install on a Linux PTP node
|
||||||
|
|
||||||
|
Debian/Raspberry Pi OS/Proxmox example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install python3-venv linuxptp
|
||||||
|
|
||||||
|
sudo useradd --system --home /opt/ptp-exporter --shell /usr/sbin/nologin ptp-exporter
|
||||||
|
sudo mkdir -p /opt/ptp-exporter
|
||||||
|
sudo chown ptp-exporter:ptp-exporter /opt/ptp-exporter
|
||||||
|
|
||||||
|
sudo -u ptp-exporter python3 -m venv /opt/ptp-exporter/venv
|
||||||
|
sudo -u ptp-exporter /opt/ptp-exporter/venv/bin/pip install prometheus-client
|
||||||
|
|
||||||
|
sudo install -m 0755 linux_ptp_exporter.py /opt/ptp-exporter/linux_ptp_exporter.py
|
||||||
|
```
|
||||||
|
|
||||||
|
linuxptp's default `/var/run/ptp4lro` is a read-only management socket intended for monitoring and is normally mode `0666`, so the exporter does not need write access to `/var/run/ptp4l`.
|
||||||
|
|
||||||
|
If `ptp4l` on your distro uses `/var/run/ptp/ptp4lro` instead, pass that with `--uds`.
|
||||||
|
|
||||||
|
### RPi5 grandmaster
|
||||||
|
|
||||||
|
Install:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo install -m 0644 systemd/ptp-exporter-grandmaster.service \
|
||||||
|
/etc/systemd/system/ptp-exporter.service
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now ptp-exporter
|
||||||
|
```
|
||||||
|
|
||||||
|
The unit runs:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
linux_ptp_exporter.py \
|
||||||
|
--role grandmaster \
|
||||||
|
--chrony \
|
||||||
|
--ptp4l-service ptp4l-gm.service \
|
||||||
|
--phc2sys-service phc2sys-gm.service
|
||||||
|
```
|
||||||
|
|
||||||
|
This exports both the PTP side and the GNSS/PPS -> chrony side.
|
||||||
|
|
||||||
|
### Linux boundary clock
|
||||||
|
|
||||||
|
Install:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo install -m 0644 systemd/ptp-exporter-boundary.service \
|
||||||
|
/etc/systemd/system/ptp-exporter.service
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now ptp-exporter
|
||||||
|
```
|
||||||
|
|
||||||
|
<!-- This is especially useful on WOLF because it exports both:
|
||||||
|
|
||||||
|
```text
|
||||||
|
RPi5 -> eno4 PHC via ptp4l/pmc
|
||||||
|
eno4 PHC -> Mellanox PHC via recent phc2sys journal samples
|
||||||
|
``` -->
|
||||||
|
|
||||||
|
<!-- It also exports the per-port linuxptp packet counters from `PORT_STATS_NP`. -->
|
||||||
|
|
||||||
|
### Downstream Linux client
|
||||||
|
|
||||||
|
Install:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo install -m 0644 systemd/ptp-exporter-client.service \
|
||||||
|
/etc/systemd/system/ptp-exporter.service
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now ptp-exporter
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. Journal permissions
|
||||||
|
|
||||||
|
The exact `pmc` metrics do not require journal access.
|
||||||
|
|
||||||
|
`ptp_phc2sys_*` metrics are parsed from the latest `phc2sys -m` systemd journal lines. The included services use:
|
||||||
|
|
||||||
|
```text
|
||||||
|
SupplementaryGroups=systemd-journal
|
||||||
|
```
|
||||||
|
|
||||||
|
so the exporter can read those lines without running as root.
|
||||||
|
|
||||||
|
If your distribution does not have the `systemd-journal` group, remove that line. The exporter will continue working; only the `ptp_phc2sys_*` metrics will be absent.
|
||||||
|
|
||||||
|
## 3. Test a Linux exporter
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:9559/metrics | grep '^ptp_'
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful direct checks:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pmc -u -s /var/run/ptp4lro -b 0 'GET CURRENT_DATA_SET'
|
||||||
|
pmc -u -s /var/run/ptp4lro -b 0 'GET TIME_STATUS_NP'
|
||||||
|
pmc -u -s /var/run/ptp4lro -b 0 'GET PORT_PROPERTIES_NP'
|
||||||
|
pmc -u -s /var/run/ptp4lro -b 0 'GET PORT_STATS_NP'
|
||||||
|
pmc -u -s /var/run/ptp4lro -b 0 'GET PORT_SERVICE_STATS_NP'
|
||||||
|
```
|
||||||
|
|
||||||
|
`PORT_STATS_NP` is particularly useful because it exposes real per-port Sync, Follow_Up, Delay_Req, Delay_Resp, Announce, Management, etc. RX/TX counters.
|
||||||
|
|
||||||
|
## 4. Nexus exporter
|
||||||
|
|
||||||
|
Python cannot run directly on NX-OS, so `nexus_ptp_exporter.py` should run on a Linux monitoring host.
|
||||||
|
|
||||||
|
Install:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo useradd --system --home /opt/ptp-exporter --shell /usr/sbin/nologin ptp-exporter
|
||||||
|
sudo mkdir -p /opt/ptp-exporter
|
||||||
|
sudo chown ptp-exporter:ptp-exporter /opt/ptp-exporter
|
||||||
|
|
||||||
|
sudo -u ptp-exporter python3 -m venv /opt/ptp-exporter/venv
|
||||||
|
sudo -u ptp-exporter /opt/ptp-exporter/venv/bin/pip install prometheus-client paramiko
|
||||||
|
|
||||||
|
sudo install -m 0755 nexus_ptp_exporter.py /opt/ptp-exporter/nexus_ptp_exporter.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Use SSH key authentication if possible. The account only needs permission to run:
|
||||||
|
|
||||||
|
```text
|
||||||
|
show ptp clock
|
||||||
|
show ptp parent
|
||||||
|
show ptp brief
|
||||||
|
show ptp counters all
|
||||||
|
```
|
||||||
|
|
||||||
|
Example manual start:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
/opt/ptp-exporter/venv/bin/python \
|
||||||
|
/opt/ptp-exporter/nexus_ptp_exporter.py \
|
||||||
|
--host 10.255.254.255 \
|
||||||
|
--target-name NYATER \
|
||||||
|
--username prometheus \
|
||||||
|
--key-file /opt/ptp-exporter/.ssh/id_ed25519 \
|
||||||
|
--port 9560
|
||||||
|
```
|
||||||
|
|
||||||
|
`10.255.254.255` above is only an example based on the PTP source/loopback shown in the current topology. Use whichever management-reachable address you actually want SSH to connect to.
|
||||||
|
|
||||||
|
The exporter checks the system SSH known-hosts database by default. Do not use `--accept-new-host-key` for a permanent deployment unless you explicitly want trust-on-first-use behavior.
|
||||||
|
|
||||||
|
### Password authentication
|
||||||
|
|
||||||
|
If required:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export NEXUS_PTP_PASSWORD='...'
|
||||||
|
|
||||||
|
nexus_ptp_exporter.py \
|
||||||
|
--host <NYATER-address> \
|
||||||
|
--username <user> \
|
||||||
|
--password-env NEXUS_PTP_PASSWORD
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not put the password directly in `ExecStart=`.
|
||||||
|
|
||||||
|
## 5. Prometheus scrape configuration
|
||||||
|
|
||||||
|
See `examples/prometheus.yml`.
|
||||||
|
|
||||||
|
Typical jobs:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
scrape_configs:
|
||||||
|
- job_name: ptp-linux
|
||||||
|
static_configs:
|
||||||
|
- targets:
|
||||||
|
- GrandMaster:9559
|
||||||
|
- BoundaryClock:9559
|
||||||
|
- server-a:9559
|
||||||
|
- server-b:9559
|
||||||
|
|
||||||
|
- job_name: ptp-nexus
|
||||||
|
static_configs:
|
||||||
|
- targets:
|
||||||
|
- prometheus-host:9560
|
||||||
|
```
|
||||||
|
|
||||||
|
The Nexus target is the **exporter host**, not NEXUS itself.
|
||||||
|
|
||||||
|
## 6. Suggested PromQL
|
||||||
|
|
||||||
|
### Offset from immediate master
|
||||||
|
|
||||||
|
```promql
|
||||||
|
ptp_master_offset_nanoseconds
|
||||||
|
```
|
||||||
|
|
||||||
|
or Nexus:
|
||||||
|
|
||||||
|
```promql
|
||||||
|
ptp_nexus_offset_from_master_nanoseconds
|
||||||
|
```
|
||||||
|
|
||||||
|
### Absolute value of offset
|
||||||
|
|
||||||
|
```promql
|
||||||
|
abs(ptp_master_offset_nanoseconds)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5-minute worst-case Linux offset
|
||||||
|
|
||||||
|
```promql
|
||||||
|
max_over_time(abs(ptp_master_offset_nanoseconds)[5m])
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5-minute RMS-like standard deviation
|
||||||
|
|
||||||
|
```promql
|
||||||
|
stddev_over_time(ptp_master_offset_nanoseconds[5m])
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ensure GM is present
|
||||||
|
|
||||||
|
```promql
|
||||||
|
ptp_gm_present == 1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ensure Nexus is locked
|
||||||
|
|
||||||
|
```promql
|
||||||
|
ptp_nexus_clock_locked == 1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check WOLF / client port states
|
||||||
|
|
||||||
|
```promql
|
||||||
|
ptp_port_state{state="client"} == 1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check Nexus upstream redundancy
|
||||||
|
|
||||||
|
```promql
|
||||||
|
ptp_nexus_port_state{state=~"slave|passive"}
|
||||||
|
```
|
||||||
|
|
||||||
|
### PTP packet rate
|
||||||
|
|
||||||
|
```promql
|
||||||
|
rate(ptp_port_messages_total[5m])
|
||||||
|
```
|
||||||
|
|
||||||
|
For NEXUS:
|
||||||
|
|
||||||
|
```promql
|
||||||
|
rate(ptp_nexus_port_messages_total[5m])
|
||||||
|
```
|
||||||
|
<!--
|
||||||
|
## 7. Suggested alerts
|
||||||
|
|
||||||
|
Examples are in `examples/alerts.yml`.
|
||||||
|
|
||||||
|
The useful first alerts are:
|
||||||
|
|
||||||
|
- exporter scrape failed;
|
||||||
|
- grandmaster disappeared;
|
||||||
|
- Nexus unlocked;
|
||||||
|
- master offset exceeds a threshold for several minutes;
|
||||||
|
- a client leaves CLIENT/SLAVE state;
|
||||||
|
- WOLF upstream port leaves CLIENT/SLAVE state;
|
||||||
|
- PTP packet counters stop increasing unexpectedly.
|
||||||
|
|
||||||
|
Do not start with extremely tight offset alerts such as 100 ns until you have collected a few days of normal behavior. First establish the actual distribution of your own hardware. -->
|
||||||
|
|
||||||
|
## 8. Security / firewall
|
||||||
|
|
||||||
|
The exporters listen on all addresses by default. Restrict TCP/9559 and TCP/9560 to the Prometheus server with the host firewall, or start them with a specific management address:
|
||||||
|
|
||||||
|
```text
|
||||||
|
--listen <management-IP>
|
||||||
|
```
|
||||||
|
|
||||||
|
The HTTP endpoint has no authentication; network-level restriction is recommended.
|
||||||
|
|
||||||
|
## 9. Notes about PTP semantics
|
||||||
|
|
||||||
|
`ptp_master_offset_nanoseconds` is the local clock's offset from its **immediate PTP master**, not a proof of absolute UTC accuracy.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
MOMI GNSS/PPS
|
||||||
|
-> RPi CLOCK_REALTIME
|
||||||
|
-> RPi PHC
|
||||||
|
-> WOLF eno4 PHC
|
||||||
|
-> WOLF Mellanox PHC
|
||||||
|
-> NYATER
|
||||||
|
-> final client
|
||||||
|
```
|
||||||
|
|
||||||
|
Each stage can contribute error. Grafana should therefore show both the per-hop PTP offsets and the RPi chrony/GNSS health.
|
||||||
5
exporters/examples/nexus-ptp-exporter.env
Normal file
5
exporters/examples/nexus-ptp-exporter.env
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
# /etc/default/nexus-ptp-exporter
|
||||||
|
NEXUS_HOST=10.255.255.255
|
||||||
|
NEXUS_NAME=NYATER
|
||||||
|
NEXUS_USER=prometheus
|
||||||
|
NEXUS_KEY_FILE=/opt/ptp-exporter/.ssh/id_ed25519
|
||||||
18
exporters/examples/prometheus.yml
Normal file
18
exporters/examples/prometheus.yml
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
scrape_configs:
|
||||||
|
- job_name: "ptp-linux"
|
||||||
|
scrape_interval: 10s
|
||||||
|
static_configs:
|
||||||
|
- targets:
|
||||||
|
- GrandMaster:9559
|
||||||
|
- BoundaryClock:9559
|
||||||
|
- server-a:9559
|
||||||
|
- server-b:9559
|
||||||
|
|
||||||
|
# nexus_ptp_exporter.py runs on a Linux monitoring host and polls NYATER via SSH.
|
||||||
|
- job_name: "ptp-nexus"
|
||||||
|
scrape_interval: 15s
|
||||||
|
static_configs:
|
||||||
|
- targets:
|
||||||
|
- prometheus-host:9560
|
||||||
|
|
||||||
|
|
||||||
817
exporters/linux_ptp_exporter.py
Normal file
817
exporters/linux_ptp_exporter.py
Normal file
@@ -0,0 +1,817 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Prometheus exporter for linuxptp nodes.
|
||||||
|
|
||||||
|
Designed for:
|
||||||
|
* GNSS/PPS grandmasters (ptp4l + phc2sys + optional chrony)
|
||||||
|
* Linux boundary clocks (including boundary_clock_jbod)
|
||||||
|
* Ordinary-clock / downstream Linux clients
|
||||||
|
|
||||||
|
The exporter reads ptp4l through its read-only Unix-domain management socket
|
||||||
|
(/var/run/ptp4lro by default) using pmc. It does not modify PTP state.
|
||||||
|
|
||||||
|
Optional data sources:
|
||||||
|
* chronyc tracking / sources - useful on a GNSS-backed grandmaster
|
||||||
|
* systemd service state
|
||||||
|
* recent phc2sys journal output, to expose the PHC<->system/PHC servo offset
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from typing import Dict, Iterable, List, Optional, Tuple
|
||||||
|
|
||||||
|
from prometheus_client import REGISTRY, start_http_server
|
||||||
|
from prometheus_client.core import (
|
||||||
|
CounterMetricFamily,
|
||||||
|
GaugeMetricFamily,
|
||||||
|
InfoMetricFamily,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
PMC_DATASETS = [
|
||||||
|
"DEFAULT_DATA_SET",
|
||||||
|
"CURRENT_DATA_SET",
|
||||||
|
"PARENT_DATA_SET",
|
||||||
|
"TIME_PROPERTIES_DATA_SET",
|
||||||
|
"TIME_STATUS_NP",
|
||||||
|
"PORT_PROPERTIES_NP",
|
||||||
|
"PORT_STATS_NP",
|
||||||
|
"PORT_SERVICE_STATS_NP",
|
||||||
|
]
|
||||||
|
|
||||||
|
PMC_HEADER_RE = re.compile(
|
||||||
|
r"^\s*(?P<source>\S+)\s+seq\s+\d+\s+RESPONSE\s+MANAGEMENT\s+(?P<dataset>[A-Z0-9_]+)\s*$"
|
||||||
|
)
|
||||||
|
PMC_KV_RE = re.compile(r"^\s*(?P<key>[A-Za-z0-9_.-]+)\s+(?P<value>.+?)\s*$")
|
||||||
|
|
||||||
|
PHC2SYS_RE = re.compile(
|
||||||
|
r"(?P<clock>[A-Za-z0-9_.:/-]+)\s+phc\s+offset\s+"
|
||||||
|
r"(?P<offset>[+-]?\d+)\s+\S+\s+freq\s+(?P<freq>[+-]?\d+)"
|
||||||
|
r"(?:\s+delay\s+(?P<delay>\d+))?"
|
||||||
|
)
|
||||||
|
|
||||||
|
CHRONY_SOURCE_RE = re.compile(
|
||||||
|
r"^\s*(?P<mode>[\^=#])(?P<state>[*+\-?x~])\s+(?P<source>\S+)"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def run_command(argv: List[str], timeout: float = 3.0) -> str:
|
||||||
|
proc = subprocess.run(
|
||||||
|
argv,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
text=True,
|
||||||
|
timeout=timeout,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"command failed ({proc.returncode}): {' '.join(argv)}\n{proc.stdout.strip()}"
|
||||||
|
)
|
||||||
|
return proc.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def parse_scalar(value: str):
|
||||||
|
value = value.strip()
|
||||||
|
low = value.lower()
|
||||||
|
if low == "true":
|
||||||
|
return 1.0
|
||||||
|
if low == "false":
|
||||||
|
return 0.0
|
||||||
|
try:
|
||||||
|
if low.startswith("0x"):
|
||||||
|
return float(int(low, 16))
|
||||||
|
return float(value)
|
||||||
|
except ValueError:
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def parse_pmc(text: str) -> List[Dict[str, object]]:
|
||||||
|
"""Parse pmc's multi-response textual output into response dictionaries."""
|
||||||
|
blocks: List[Dict[str, object]] = []
|
||||||
|
current: Optional[Dict[str, object]] = None
|
||||||
|
|
||||||
|
for line in text.splitlines():
|
||||||
|
if line.startswith("sending:"):
|
||||||
|
continue
|
||||||
|
|
||||||
|
match = PMC_HEADER_RE.match(line)
|
||||||
|
if match:
|
||||||
|
current = {
|
||||||
|
"_source": match.group("source"),
|
||||||
|
"_dataset": match.group("dataset"),
|
||||||
|
}
|
||||||
|
blocks.append(current)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if current is None:
|
||||||
|
continue
|
||||||
|
|
||||||
|
match = PMC_KV_RE.match(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
|
||||||
|
current[match.group("key")] = parse_scalar(match.group("value"))
|
||||||
|
|
||||||
|
return blocks
|
||||||
|
|
||||||
|
|
||||||
|
def first_block(blocks: Iterable[Dict[str, object]], dataset: str) -> Optional[Dict[str, object]]:
|
||||||
|
for block in blocks:
|
||||||
|
if block.get("_dataset") == dataset:
|
||||||
|
return block
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def blocks_for(blocks: Iterable[Dict[str, object]], dataset: str) -> List[Dict[str, object]]:
|
||||||
|
return [b for b in blocks if b.get("_dataset") == dataset]
|
||||||
|
|
||||||
|
|
||||||
|
def as_float(value, default: float = math.nan) -> float:
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
return float(value)
|
||||||
|
if isinstance(value, str):
|
||||||
|
try:
|
||||||
|
if value.lower().startswith("0x"):
|
||||||
|
return float(int(value, 16))
|
||||||
|
return float(value)
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def as_str(value, default: str = "") -> str:
|
||||||
|
if value is None:
|
||||||
|
return default
|
||||||
|
return str(value)
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_port_state(state: str) -> str:
|
||||||
|
# linuxptp releases may use SLAVE/MASTER or CLIENT/SERVER terminology.
|
||||||
|
state = state.strip().upper()
|
||||||
|
aliases = {
|
||||||
|
"SLAVE": "client",
|
||||||
|
"CLIENT": "client",
|
||||||
|
"MASTER": "server",
|
||||||
|
"SERVER": "server",
|
||||||
|
"GRAND_MASTER": "server",
|
||||||
|
"PASSIVE": "passive",
|
||||||
|
"LISTENING": "listening",
|
||||||
|
"UNCALIBRATED": "uncalibrated",
|
||||||
|
"FAULTY": "faulty",
|
||||||
|
"DISABLED": "disabled",
|
||||||
|
"INITIALIZING": "initializing",
|
||||||
|
"PRE_MASTER": "pre_master",
|
||||||
|
}
|
||||||
|
return aliases.get(state, state.lower())
|
||||||
|
|
||||||
|
|
||||||
|
def parse_chrony_tracking(text: str) -> Dict[str, object]:
|
||||||
|
result: Dict[str, object] = {}
|
||||||
|
|
||||||
|
for line in text.splitlines():
|
||||||
|
if ":" not in line:
|
||||||
|
continue
|
||||||
|
key, raw = [part.strip() for part in line.split(":", 1)]
|
||||||
|
result[key] = raw
|
||||||
|
|
||||||
|
def first_number(key: str) -> Optional[float]:
|
||||||
|
raw = result.get(key)
|
||||||
|
if raw is None:
|
||||||
|
return None
|
||||||
|
m = re.search(r"[+-]?(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?", str(raw))
|
||||||
|
return float(m.group(0)) if m else None
|
||||||
|
|
||||||
|
parsed: Dict[str, object] = {}
|
||||||
|
numeric_map = {
|
||||||
|
"Stratum": "stratum",
|
||||||
|
"Last offset": "last_offset_seconds",
|
||||||
|
"RMS offset": "rms_offset_seconds",
|
||||||
|
"Residual freq": "residual_frequency_ppm",
|
||||||
|
"Skew": "skew_ppm",
|
||||||
|
"Root delay": "root_delay_seconds",
|
||||||
|
"Root dispersion": "root_dispersion_seconds",
|
||||||
|
"Update interval": "update_interval_seconds",
|
||||||
|
}
|
||||||
|
for chrony_key, out_key in numeric_map.items():
|
||||||
|
value = first_number(chrony_key)
|
||||||
|
if value is not None:
|
||||||
|
parsed[out_key] = value
|
||||||
|
|
||||||
|
# Frequency has a direction word: "12.3 ppm fast" or "slow".
|
||||||
|
freq = first_number("Frequency")
|
||||||
|
if freq is not None:
|
||||||
|
raw = str(result.get("Frequency", "")).lower()
|
||||||
|
if "slow" in raw:
|
||||||
|
freq = -abs(freq)
|
||||||
|
elif "fast" in raw:
|
||||||
|
freq = abs(freq)
|
||||||
|
parsed["frequency_ppm"] = freq
|
||||||
|
|
||||||
|
# "System time : 0.000001 seconds fast/slow of NTP time"
|
||||||
|
system_time = first_number("System time")
|
||||||
|
if system_time is not None:
|
||||||
|
raw = str(result.get("System time", "")).lower()
|
||||||
|
# Positive = local system clock is ahead/fast, negative = behind/slow.
|
||||||
|
if "slow" in raw:
|
||||||
|
system_time = -abs(system_time)
|
||||||
|
elif "fast" in raw:
|
||||||
|
system_time = abs(system_time)
|
||||||
|
parsed["system_time_offset_seconds"] = system_time
|
||||||
|
|
||||||
|
if "Leap status" in result:
|
||||||
|
parsed["leap_status"] = str(result["Leap status"])
|
||||||
|
if "Reference ID" in result:
|
||||||
|
parsed["reference_id"] = str(result["Reference ID"])
|
||||||
|
|
||||||
|
return parsed
|
||||||
|
|
||||||
|
|
||||||
|
def parse_chrony_sources(text: str) -> List[Dict[str, str]]:
|
||||||
|
rows: List[Dict[str, str]] = []
|
||||||
|
for line in text.splitlines():
|
||||||
|
m = CHRONY_SOURCE_RE.match(line)
|
||||||
|
if m:
|
||||||
|
rows.append(m.groupdict())
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def systemd_active(service: str) -> float:
|
||||||
|
try:
|
||||||
|
out = run_command(["systemctl", "is-active", service], timeout=2.0).strip()
|
||||||
|
return 1.0 if out == "active" else 0.0
|
||||||
|
except Exception:
|
||||||
|
return 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def read_phc2sys_journal(service: str) -> Dict[str, Dict[str, float]]:
|
||||||
|
"""
|
||||||
|
Return the latest phc2sys servo sample per destination clock found in the
|
||||||
|
last journal lines. Values are nanoseconds / ppb as printed by phc2sys.
|
||||||
|
"""
|
||||||
|
text = run_command(
|
||||||
|
[
|
||||||
|
"journalctl",
|
||||||
|
"-u",
|
||||||
|
service,
|
||||||
|
"-n",
|
||||||
|
"250",
|
||||||
|
"--no-pager",
|
||||||
|
"-o",
|
||||||
|
"cat",
|
||||||
|
],
|
||||||
|
timeout=3.0,
|
||||||
|
)
|
||||||
|
latest: Dict[str, Dict[str, float]] = {}
|
||||||
|
for line in text.splitlines():
|
||||||
|
m = PHC2SYS_RE.search(line)
|
||||||
|
if not m:
|
||||||
|
continue
|
||||||
|
row = {
|
||||||
|
"offset_ns": float(m.group("offset")),
|
||||||
|
"frequency_ppb": float(m.group("freq")),
|
||||||
|
}
|
||||||
|
if m.group("delay") is not None:
|
||||||
|
row["delay_ns"] = float(m.group("delay"))
|
||||||
|
latest[m.group("clock")] = row
|
||||||
|
return latest
|
||||||
|
|
||||||
|
|
||||||
|
class LinuxPTPCollector:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
role: str,
|
||||||
|
uds: str,
|
||||||
|
pmc_binary: str,
|
||||||
|
command_timeout: float,
|
||||||
|
chrony: bool,
|
||||||
|
ptp4l_service: Optional[str],
|
||||||
|
phc2sys_service: Optional[str],
|
||||||
|
pmc_runtime_dir: str,
|
||||||
|
):
|
||||||
|
self.role = role
|
||||||
|
self.uds = uds
|
||||||
|
self.pmc_binary = pmc_binary
|
||||||
|
self.command_timeout = command_timeout
|
||||||
|
self.chrony = chrony
|
||||||
|
self.ptp4l_service = ptp4l_service
|
||||||
|
self.phc2sys_service = phc2sys_service
|
||||||
|
self.pmc_runtime_dir = pmc_runtime_dir
|
||||||
|
self.hostname = socket.gethostname()
|
||||||
|
|
||||||
|
def _pmc(self):
|
||||||
|
"""
|
||||||
|
Query each management dataset independently.
|
||||||
|
|
||||||
|
linuxptp versions differ in which *_NP management IDs they expose, and
|
||||||
|
the read-only UDS may intentionally not return some port-scoped data.
|
||||||
|
One failed optional query must therefore not discard valid clock data.
|
||||||
|
"""
|
||||||
|
blocks: List[Dict[str, object]] = []
|
||||||
|
status: Dict[str, float] = {}
|
||||||
|
errors: Dict[str, str] = {}
|
||||||
|
|
||||||
|
for dataset in PMC_DATASETS:
|
||||||
|
# `-s` is the ptp4l SERVER socket. `pmc` also needs its own
|
||||||
|
# CLIENT-side Unix socket. Without `-i`, pmc defaults to
|
||||||
|
# /var/run/pmc.$pid, which an unprivileged exporter cannot create.
|
||||||
|
# Use our systemd-owned RuntimeDirectory instead.
|
||||||
|
local_uds = os.path.join(
|
||||||
|
self.pmc_runtime_dir,
|
||||||
|
f"pmc.{os.getpid()}.{threading.get_ident()}.{time.time_ns()}",
|
||||||
|
)
|
||||||
|
argv = [
|
||||||
|
self.pmc_binary,
|
||||||
|
"-u",
|
||||||
|
"-i",
|
||||||
|
local_uds,
|
||||||
|
"-s",
|
||||||
|
self.uds,
|
||||||
|
"-b",
|
||||||
|
"0",
|
||||||
|
f"GET {dataset}",
|
||||||
|
]
|
||||||
|
try:
|
||||||
|
output = run_command(argv, timeout=self.command_timeout)
|
||||||
|
parsed = parse_pmc(output)
|
||||||
|
if parsed:
|
||||||
|
blocks.extend(parsed)
|
||||||
|
status[dataset] = 1.0
|
||||||
|
else:
|
||||||
|
status[dataset] = 0.0
|
||||||
|
errors[dataset] = "no_response"
|
||||||
|
except Exception as exc:
|
||||||
|
status[dataset] = 0.0
|
||||||
|
errors[dataset] = type(exc).__name__
|
||||||
|
|
||||||
|
return blocks, status, errors
|
||||||
|
|
||||||
|
def collect(self):
|
||||||
|
start = time.monotonic()
|
||||||
|
scrape_success = 1.0
|
||||||
|
error_stage = ""
|
||||||
|
|
||||||
|
try:
|
||||||
|
blocks, dataset_status, dataset_errors = self._pmc()
|
||||||
|
core_datasets = {
|
||||||
|
"DEFAULT_DATA_SET",
|
||||||
|
"CURRENT_DATA_SET",
|
||||||
|
"PARENT_DATA_SET",
|
||||||
|
"TIME_PROPERTIES_DATA_SET",
|
||||||
|
"TIME_STATUS_NP",
|
||||||
|
}
|
||||||
|
core_ok = any(
|
||||||
|
dataset_status.get(dataset, 0.0) == 1.0
|
||||||
|
for dataset in core_datasets
|
||||||
|
)
|
||||||
|
if not core_ok:
|
||||||
|
scrape_success = 0.0
|
||||||
|
error_stage = "pmc"
|
||||||
|
except Exception:
|
||||||
|
blocks = []
|
||||||
|
dataset_status = {}
|
||||||
|
dataset_errors = {}
|
||||||
|
scrape_success = 0.0
|
||||||
|
error_stage = "pmc"
|
||||||
|
|
||||||
|
role_info = InfoMetricFamily(
|
||||||
|
"ptp_linux_node",
|
||||||
|
"Linux PTP exporter node information",
|
||||||
|
labels=["hostname", "role"],
|
||||||
|
)
|
||||||
|
role_info.add_metric([self.hostname, self.role], {})
|
||||||
|
yield role_info
|
||||||
|
|
||||||
|
# -------- Core datasets --------
|
||||||
|
current = first_block(blocks, "CURRENT_DATA_SET")
|
||||||
|
if current:
|
||||||
|
metric = GaugeMetricFamily(
|
||||||
|
"ptp_master_offset_nanoseconds",
|
||||||
|
"Current PTP offset from the selected immediate master",
|
||||||
|
)
|
||||||
|
metric.add_metric([], as_float(current.get("offsetFromMaster")))
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
metric = GaugeMetricFamily(
|
||||||
|
"ptp_mean_path_delay_nanoseconds",
|
||||||
|
"Current estimated mean path delay to the immediate master",
|
||||||
|
)
|
||||||
|
metric.add_metric([], as_float(current.get("meanPathDelay")))
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
metric = GaugeMetricFamily(
|
||||||
|
"ptp_steps_removed",
|
||||||
|
"Number of boundary-clock steps from the grandmaster",
|
||||||
|
)
|
||||||
|
metric.add_metric([], as_float(current.get("stepsRemoved")))
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
time_status = first_block(blocks, "TIME_STATUS_NP")
|
||||||
|
if time_status:
|
||||||
|
metric = GaugeMetricFamily(
|
||||||
|
"ptp_gm_present",
|
||||||
|
"Whether linuxptp reports a grandmaster as present",
|
||||||
|
)
|
||||||
|
metric.add_metric([], as_float(time_status.get("gmPresent"), 0.0))
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
# TIME_STATUS_NP master_offset is useful too, and often an integer.
|
||||||
|
metric = GaugeMetricFamily(
|
||||||
|
"ptp_time_status_master_offset_nanoseconds",
|
||||||
|
"Master offset from linuxptp TIME_STATUS_NP",
|
||||||
|
)
|
||||||
|
metric.add_metric([], as_float(time_status.get("master_offset")))
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
gm = as_str(time_status.get("gmIdentity"))
|
||||||
|
if gm:
|
||||||
|
info = InfoMetricFamily(
|
||||||
|
"ptp_grandmaster",
|
||||||
|
"Currently selected PTP grandmaster identity",
|
||||||
|
labels=["identity"],
|
||||||
|
)
|
||||||
|
info.add_metric([gm], {})
|
||||||
|
yield info
|
||||||
|
|
||||||
|
parent = first_block(blocks, "PARENT_DATA_SET")
|
||||||
|
if parent:
|
||||||
|
info = InfoMetricFamily(
|
||||||
|
"ptp_parent",
|
||||||
|
"Immediate parent and grandmaster identity",
|
||||||
|
labels=["parent_port_identity", "grandmaster_identity"],
|
||||||
|
)
|
||||||
|
info.add_metric(
|
||||||
|
[
|
||||||
|
as_str(parent.get("parentPortIdentity")),
|
||||||
|
as_str(parent.get("grandmasterIdentity")),
|
||||||
|
],
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
yield info
|
||||||
|
|
||||||
|
gauges = {
|
||||||
|
"ptp_grandmaster_priority1": ("grandmasterPriority1", "Grandmaster priority1"),
|
||||||
|
"ptp_grandmaster_priority2": ("grandmasterPriority2", "Grandmaster priority2"),
|
||||||
|
"ptp_grandmaster_clock_class": ("gm.ClockClass", "Grandmaster clockClass"),
|
||||||
|
"ptp_grandmaster_clock_accuracy_code": (
|
||||||
|
"gm.ClockAccuracy",
|
||||||
|
"Grandmaster IEEE 1588 clockAccuracy code",
|
||||||
|
),
|
||||||
|
"ptp_grandmaster_offset_scaled_log_variance": (
|
||||||
|
"gm.OffsetScaledLogVariance",
|
||||||
|
"Grandmaster offsetScaledLogVariance",
|
||||||
|
),
|
||||||
|
}
|
||||||
|
for metric_name, (field, help_text) in gauges.items():
|
||||||
|
if field in parent:
|
||||||
|
g = GaugeMetricFamily(metric_name, help_text)
|
||||||
|
g.add_metric([], as_float(parent.get(field)))
|
||||||
|
yield g
|
||||||
|
|
||||||
|
default = first_block(blocks, "DEFAULT_DATA_SET")
|
||||||
|
if default:
|
||||||
|
info = InfoMetricFamily(
|
||||||
|
"ptp_local_clock",
|
||||||
|
"Local linuxptp clock identity",
|
||||||
|
labels=["identity"],
|
||||||
|
)
|
||||||
|
info.add_metric([as_str(default.get("clockIdentity"))], {})
|
||||||
|
yield info
|
||||||
|
|
||||||
|
for metric_name, field, help_text in [
|
||||||
|
("ptp_domain_number", "domainNumber", "PTP domain number"),
|
||||||
|
("ptp_local_clock_class", "clockClass", "Local clockClass"),
|
||||||
|
("ptp_local_clock_accuracy_code", "clockAccuracy", "Local clockAccuracy code"),
|
||||||
|
("ptp_local_priority1", "priority1", "Local priority1"),
|
||||||
|
("ptp_local_priority2", "priority2", "Local priority2"),
|
||||||
|
("ptp_local_number_ports", "numberPorts", "Number of PTP ports"),
|
||||||
|
]:
|
||||||
|
if field in default:
|
||||||
|
g = GaugeMetricFamily(metric_name, help_text)
|
||||||
|
g.add_metric([], as_float(default.get(field)))
|
||||||
|
yield g
|
||||||
|
|
||||||
|
time_props = first_block(blocks, "TIME_PROPERTIES_DATA_SET")
|
||||||
|
if time_props:
|
||||||
|
fields = [
|
||||||
|
("ptp_current_utc_offset_seconds", "currentUtcOffset", "PTP TAI-UTC offset"),
|
||||||
|
("ptp_current_utc_offset_valid", "currentUtcOffsetValid", "UTC offset valid flag"),
|
||||||
|
("ptp_timescale", "ptpTimescale", "PTP timescale flag"),
|
||||||
|
("ptp_time_traceable", "timeTraceable", "PTP time traceable flag"),
|
||||||
|
("ptp_frequency_traceable", "frequencyTraceable", "PTP frequency traceable flag"),
|
||||||
|
("ptp_leap61", "leap61", "Positive leap-second flag"),
|
||||||
|
("ptp_leap59", "leap59", "Negative leap-second flag"),
|
||||||
|
("ptp_time_source_code", "timeSource", "IEEE 1588 timeSource code"),
|
||||||
|
]
|
||||||
|
for metric_name, field, help_text in fields:
|
||||||
|
if field in time_props:
|
||||||
|
g = GaugeMetricFamily(metric_name, help_text)
|
||||||
|
g.add_metric([], as_float(time_props.get(field)))
|
||||||
|
yield g
|
||||||
|
|
||||||
|
# -------- Port properties and counters --------
|
||||||
|
port_properties = blocks_for(blocks, "PORT_PROPERTIES_NP")
|
||||||
|
port_to_interface: Dict[str, str] = {}
|
||||||
|
|
||||||
|
port_state = GaugeMetricFamily(
|
||||||
|
"ptp_port_state",
|
||||||
|
"Current PTP port state; one sample with value 1 is emitted per port",
|
||||||
|
labels=["interface", "port_identity", "state"],
|
||||||
|
)
|
||||||
|
timestamping = GaugeMetricFamily(
|
||||||
|
"ptp_port_timestamping",
|
||||||
|
"Timestamping mode reported by linuxptp",
|
||||||
|
labels=["interface", "port_identity", "mode"],
|
||||||
|
)
|
||||||
|
|
||||||
|
for port in port_properties:
|
||||||
|
identity = as_str(port.get("portIdentity"))
|
||||||
|
interface = as_str(port.get("interface"), identity)
|
||||||
|
port_to_interface[identity] = interface
|
||||||
|
state = normalize_port_state(as_str(port.get("portState")))
|
||||||
|
port_state.add_metric([interface, identity, state], 1.0)
|
||||||
|
timestamping.add_metric(
|
||||||
|
[interface, identity, as_str(port.get("timestamping")).lower()],
|
||||||
|
1.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
if port_properties:
|
||||||
|
yield port_state
|
||||||
|
yield timestamping
|
||||||
|
|
||||||
|
packet_counter = CounterMetricFamily(
|
||||||
|
"ptp_port_messages",
|
||||||
|
"PTP messages received/transmitted by linuxptp",
|
||||||
|
labels=["interface", "port_identity", "direction", "message_type"],
|
||||||
|
)
|
||||||
|
have_packet_counters = False
|
||||||
|
|
||||||
|
for block in blocks_for(blocks, "PORT_STATS_NP"):
|
||||||
|
identity = as_str(block.get("portIdentity"))
|
||||||
|
interface = port_to_interface.get(identity, identity)
|
||||||
|
for key, value in block.items():
|
||||||
|
if not (key.startswith("rx_") or key.startswith("tx_")):
|
||||||
|
continue
|
||||||
|
direction, message_type = key.split("_", 1)
|
||||||
|
packet_counter.add_metric(
|
||||||
|
[interface, identity, direction, message_type.lower()],
|
||||||
|
as_float(value, 0.0),
|
||||||
|
)
|
||||||
|
have_packet_counters = True
|
||||||
|
|
||||||
|
if have_packet_counters:
|
||||||
|
yield packet_counter
|
||||||
|
|
||||||
|
service_counter = CounterMetricFamily(
|
||||||
|
"ptp_port_service_events",
|
||||||
|
"linuxptp port service timeout/mismatch counters",
|
||||||
|
labels=["interface", "port_identity", "event"],
|
||||||
|
)
|
||||||
|
have_service_counters = False
|
||||||
|
|
||||||
|
for block in blocks_for(blocks, "PORT_SERVICE_STATS_NP"):
|
||||||
|
identity = as_str(block.get("portIdentity"))
|
||||||
|
interface = port_to_interface.get(identity, identity)
|
||||||
|
for key, value in block.items():
|
||||||
|
if key.startswith("_") or key == "portIdentity":
|
||||||
|
continue
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
service_counter.add_metric(
|
||||||
|
[interface, identity, key.lower()],
|
||||||
|
as_float(value, 0.0),
|
||||||
|
)
|
||||||
|
have_service_counters = True
|
||||||
|
|
||||||
|
if have_service_counters:
|
||||||
|
yield service_counter
|
||||||
|
|
||||||
|
# -------- systemd service health --------
|
||||||
|
services: List[Tuple[str, Optional[str]]] = [
|
||||||
|
("ptp4l", self.ptp4l_service),
|
||||||
|
("phc2sys", self.phc2sys_service),
|
||||||
|
]
|
||||||
|
service_up = GaugeMetricFamily(
|
||||||
|
"ptp_service_up",
|
||||||
|
"Whether the configured local PTP-related systemd service is active",
|
||||||
|
labels=["component", "service"],
|
||||||
|
)
|
||||||
|
have_services = False
|
||||||
|
for component, service in services:
|
||||||
|
if service:
|
||||||
|
service_up.add_metric([component, service], systemd_active(service))
|
||||||
|
have_services = True
|
||||||
|
if have_services:
|
||||||
|
yield service_up
|
||||||
|
|
||||||
|
# -------- phc2sys servo data from journal --------
|
||||||
|
if self.phc2sys_service:
|
||||||
|
try:
|
||||||
|
rows = read_phc2sys_journal(self.phc2sys_service)
|
||||||
|
offset = GaugeMetricFamily(
|
||||||
|
"ptp_phc2sys_offset_nanoseconds",
|
||||||
|
"Latest phc2sys servo offset from journal output",
|
||||||
|
labels=["clock"],
|
||||||
|
)
|
||||||
|
frequency = GaugeMetricFamily(
|
||||||
|
"ptp_phc2sys_frequency_ppb",
|
||||||
|
"Latest phc2sys frequency correction from journal output",
|
||||||
|
labels=["clock"],
|
||||||
|
)
|
||||||
|
delay = GaugeMetricFamily(
|
||||||
|
"ptp_phc2sys_delay_nanoseconds",
|
||||||
|
"Latest phc2sys clock-read delay from journal output",
|
||||||
|
labels=["clock"],
|
||||||
|
)
|
||||||
|
for clock, row in rows.items():
|
||||||
|
offset.add_metric([clock], row["offset_ns"])
|
||||||
|
frequency.add_metric([clock], row["frequency_ppb"])
|
||||||
|
if "delay_ns" in row:
|
||||||
|
delay.add_metric([clock], row["delay_ns"])
|
||||||
|
if rows:
|
||||||
|
yield offset
|
||||||
|
yield frequency
|
||||||
|
yield delay
|
||||||
|
except Exception:
|
||||||
|
# Do not fail the PTP scrape merely because journal access is unavailable.
|
||||||
|
pass
|
||||||
|
|
||||||
|
# -------- chrony data on the grandmaster --------
|
||||||
|
if self.chrony:
|
||||||
|
try:
|
||||||
|
tracking = parse_chrony_tracking(
|
||||||
|
run_command(["chronyc", "tracking"], timeout=self.command_timeout)
|
||||||
|
)
|
||||||
|
chrony_fields = [
|
||||||
|
("ptp_gm_chrony_stratum", "stratum", "Chrony stratum"),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_system_time_offset_seconds",
|
||||||
|
"system_time_offset_seconds",
|
||||||
|
"CLOCK_REALTIME offset from chrony's reference; positive means local clock ahead",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_last_offset_seconds",
|
||||||
|
"last_offset_seconds",
|
||||||
|
"Chrony last measured offset",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_rms_offset_seconds",
|
||||||
|
"rms_offset_seconds",
|
||||||
|
"Chrony RMS offset",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_frequency_ppm",
|
||||||
|
"frequency_ppm",
|
||||||
|
"Chrony frequency correction in ppm",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_residual_frequency_ppm",
|
||||||
|
"residual_frequency_ppm",
|
||||||
|
"Chrony residual frequency in ppm",
|
||||||
|
),
|
||||||
|
("ptp_gm_chrony_skew_ppm", "skew_ppm", "Chrony frequency skew in ppm"),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_root_delay_seconds",
|
||||||
|
"root_delay_seconds",
|
||||||
|
"Chrony root delay",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ptp_gm_chrony_root_dispersion_seconds",
|
||||||
|
"root_dispersion_seconds",
|
||||||
|
"Chrony root dispersion",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
for metric_name, field, help_text in chrony_fields:
|
||||||
|
if field in tracking:
|
||||||
|
g = GaugeMetricFamily(metric_name, help_text)
|
||||||
|
g.add_metric([], float(tracking[field]))
|
||||||
|
yield g
|
||||||
|
|
||||||
|
leap = tracking.get("leap_status")
|
||||||
|
if leap is not None:
|
||||||
|
info = InfoMetricFamily(
|
||||||
|
"ptp_gm_chrony_tracking",
|
||||||
|
"Chrony tracking metadata",
|
||||||
|
labels=["reference_id", "leap_status"],
|
||||||
|
)
|
||||||
|
info.add_metric(
|
||||||
|
[
|
||||||
|
str(tracking.get("reference_id", "")),
|
||||||
|
str(leap),
|
||||||
|
],
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
yield info
|
||||||
|
|
||||||
|
sources = parse_chrony_sources(
|
||||||
|
run_command(["chronyc", "sources", "-n"], timeout=self.command_timeout)
|
||||||
|
)
|
||||||
|
source_metric = GaugeMetricFamily(
|
||||||
|
"ptp_gm_chrony_source",
|
||||||
|
"Chrony source state. State '*' is selected, '+' is combined, '?' unreachable, etc.",
|
||||||
|
labels=["source", "mode", "state"],
|
||||||
|
)
|
||||||
|
for source in sources:
|
||||||
|
source_metric.add_metric(
|
||||||
|
[source["source"], source["mode"], source["state"]],
|
||||||
|
1.0,
|
||||||
|
)
|
||||||
|
if sources:
|
||||||
|
yield source_metric
|
||||||
|
except Exception:
|
||||||
|
if scrape_success:
|
||||||
|
# Core PTP data is still valid; expose chrony failure separately.
|
||||||
|
pass
|
||||||
|
|
||||||
|
# -------- PMC dataset health --------
|
||||||
|
dataset_metric = GaugeMetricFamily(
|
||||||
|
"ptp_exporter_pmc_dataset_success",
|
||||||
|
"1 if this linuxptp management dataset returned at least one response",
|
||||||
|
labels=["dataset", "error"],
|
||||||
|
)
|
||||||
|
for dataset in PMC_DATASETS:
|
||||||
|
ok = dataset_status.get(dataset, 0.0)
|
||||||
|
error = "none" if ok else dataset_errors.get(dataset, "unknown")
|
||||||
|
dataset_metric.add_metric([dataset, error], ok)
|
||||||
|
yield dataset_metric
|
||||||
|
|
||||||
|
# -------- exporter self-health --------
|
||||||
|
success = GaugeMetricFamily(
|
||||||
|
"ptp_exporter_scrape_success",
|
||||||
|
"1 if the core linuxptp PMC scrape succeeded",
|
||||||
|
labels=["stage"],
|
||||||
|
)
|
||||||
|
success.add_metric([error_stage or "ok"], scrape_success)
|
||||||
|
yield success
|
||||||
|
|
||||||
|
duration = GaugeMetricFamily(
|
||||||
|
"ptp_exporter_scrape_duration_seconds",
|
||||||
|
"Time spent collecting PTP metrics",
|
||||||
|
)
|
||||||
|
duration.add_metric([], time.monotonic() - start)
|
||||||
|
yield duration
|
||||||
|
|
||||||
|
|
||||||
|
def build_arg_parser() -> argparse.ArgumentParser:
|
||||||
|
p = argparse.ArgumentParser(description="Prometheus exporter for linuxptp")
|
||||||
|
p.add_argument(
|
||||||
|
"--role",
|
||||||
|
choices=["grandmaster", "boundary", "client"],
|
||||||
|
required=True,
|
||||||
|
help="Logical role of this Linux node",
|
||||||
|
)
|
||||||
|
p.add_argument("--listen", default="0.0.0.0")
|
||||||
|
p.add_argument("--port", type=int, default=9559)
|
||||||
|
p.add_argument("--uds", default="/var/run/ptp4lro", help="ptp4l read-only UDS")
|
||||||
|
p.add_argument("--pmc-binary", default="/usr/sbin/pmc")
|
||||||
|
p.add_argument(
|
||||||
|
"--pmc-runtime-dir",
|
||||||
|
default="/run/ptp-exporter",
|
||||||
|
help="Writable directory used for pmc's client-side Unix sockets",
|
||||||
|
)
|
||||||
|
p.add_argument("--timeout", type=float, default=3.0)
|
||||||
|
p.add_argument(
|
||||||
|
"--chrony",
|
||||||
|
action="store_true",
|
||||||
|
help="Also export chrony tracking/source metrics; intended for the GNSS grandmaster",
|
||||||
|
)
|
||||||
|
p.add_argument("--ptp4l-service", default=None)
|
||||||
|
p.add_argument("--phc2sys-service", default=None)
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
args = build_arg_parser().parse_args()
|
||||||
|
|
||||||
|
collector = LinuxPTPCollector(
|
||||||
|
role=args.role,
|
||||||
|
uds=args.uds,
|
||||||
|
pmc_binary=args.pmc_binary,
|
||||||
|
command_timeout=args.timeout,
|
||||||
|
chrony=args.chrony,
|
||||||
|
ptp4l_service=args.ptp4l_service,
|
||||||
|
phc2sys_service=args.phc2sys_service,
|
||||||
|
pmc_runtime_dir=args.pmc_runtime_dir,
|
||||||
|
)
|
||||||
|
REGISTRY.register(collector)
|
||||||
|
|
||||||
|
start_http_server(args.port, addr=args.listen)
|
||||||
|
print(
|
||||||
|
f"linux PTP exporter listening on {args.listen}:{args.port} "
|
||||||
|
f"(role={args.role}, uds={args.uds})",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
while True:
|
||||||
|
time.sleep(3600)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
450
exporters/nexus_ptp_exporter.py
Normal file
450
exporters/nexus_ptp_exporter.py
Normal file
@@ -0,0 +1,450 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Prometheus exporter for Cisco Nexus NX-OS PTP data.
|
||||||
|
|
||||||
|
This exporter runs on a Linux monitoring host, NOT on the Nexus itself.
|
||||||
|
It connects to NX-OS over SSH and collects:
|
||||||
|
|
||||||
|
show ptp clock
|
||||||
|
show ptp parent
|
||||||
|
show ptp brief
|
||||||
|
show ptp counters all
|
||||||
|
|
||||||
|
SSH keys are preferred. Password authentication can be supplied via an
|
||||||
|
environment variable if required.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import socket
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from typing import Dict, List, Optional, Tuple
|
||||||
|
|
||||||
|
import paramiko
|
||||||
|
from prometheus_client import REGISTRY, start_http_server
|
||||||
|
from prometheus_client.core import CounterMetricFamily, GaugeMetricFamily, InfoMetricFamily
|
||||||
|
|
||||||
|
|
||||||
|
KV_RE = re.compile(r"^\s*([^:]+?)\s*:\s*(.*?)\s*$")
|
||||||
|
BRIEF_RE = re.compile(r"^\s*(Eth\S+)\s+([A-Za-z_-]+)\s*$")
|
||||||
|
COUNTER_HEADER_RE = re.compile(r"PTP Packet Counters of Interface\s+(\S+):", re.I)
|
||||||
|
COUNTER_ROW_RE = re.compile(
|
||||||
|
r"^\s*(Announce|Sync|Follow\s*Up|FollowUp|Delay Request|Delay Response|"
|
||||||
|
r"PDelay Request|PDelay Response|PDelay Follow\s*Up|PDelay FollowUp|"
|
||||||
|
r"Management)\s+(\d+)\s+(\d+)\s*$",
|
||||||
|
re.I,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def number(value: str, default: float = math.nan) -> float:
|
||||||
|
value = value.strip()
|
||||||
|
try:
|
||||||
|
if value.lower().startswith("0x"):
|
||||||
|
return float(int(value, 16))
|
||||||
|
return float(value)
|
||||||
|
except ValueError:
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def parse_clock(text: str) -> Dict[str, str]:
|
||||||
|
result: Dict[str, str] = {}
|
||||||
|
in_quality = False
|
||||||
|
|
||||||
|
for line in text.splitlines():
|
||||||
|
stripped = line.strip()
|
||||||
|
|
||||||
|
if stripped == "Clock Quality:":
|
||||||
|
in_quality = True
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = KV_RE.match(line)
|
||||||
|
if not m:
|
||||||
|
continue
|
||||||
|
|
||||||
|
key = m.group(1).strip()
|
||||||
|
value = m.group(2).strip()
|
||||||
|
|
||||||
|
if in_quality and key in {"Class", "Accuracy", "Offset (log variance)"}:
|
||||||
|
result[f"Clock Quality {key}"] = value
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Any normal top-level line ends the Clock Quality sub-section.
|
||||||
|
in_quality = False
|
||||||
|
result[key] = value
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def parse_parent(text: str) -> Dict[str, str]:
|
||||||
|
result: Dict[str, str] = {}
|
||||||
|
in_gm_quality = False
|
||||||
|
|
||||||
|
for line in text.splitlines():
|
||||||
|
stripped = line.strip()
|
||||||
|
|
||||||
|
if stripped == "Grandmaster Clock Quality:":
|
||||||
|
in_gm_quality = True
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = KV_RE.match(line)
|
||||||
|
if not m:
|
||||||
|
continue
|
||||||
|
|
||||||
|
key = m.group(1).strip()
|
||||||
|
value = m.group(2).strip()
|
||||||
|
|
||||||
|
if in_gm_quality and key in {
|
||||||
|
"Class",
|
||||||
|
"Accuracy",
|
||||||
|
"Offset (log variance)",
|
||||||
|
"Priority1",
|
||||||
|
"Priority2",
|
||||||
|
}:
|
||||||
|
result[f"GM {key}"] = value
|
||||||
|
continue
|
||||||
|
|
||||||
|
if stripped in {"Parent Clock:", "Grandmaster Clock:"}:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Keep the quality context only while parsing its fields.
|
||||||
|
if key not in {
|
||||||
|
"Class",
|
||||||
|
"Accuracy",
|
||||||
|
"Offset (log variance)",
|
||||||
|
"Priority1",
|
||||||
|
"Priority2",
|
||||||
|
}:
|
||||||
|
in_gm_quality = False
|
||||||
|
|
||||||
|
result[key] = value
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def parse_brief(text: str) -> List[Tuple[str, str]]:
|
||||||
|
rows: List[Tuple[str, str]] = []
|
||||||
|
for line in text.splitlines():
|
||||||
|
m = BRIEF_RE.match(line)
|
||||||
|
if m:
|
||||||
|
rows.append((m.group(1), m.group(2).lower()))
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_message_type(name: str) -> str:
|
||||||
|
return re.sub(r"[^a-z0-9]+", "_", name.lower()).strip("_")
|
||||||
|
|
||||||
|
|
||||||
|
def parse_counters(text: str) -> List[Tuple[str, str, float, float]]:
|
||||||
|
"""
|
||||||
|
Returns (interface, message_type, tx, rx) tuples.
|
||||||
|
"""
|
||||||
|
rows: List[Tuple[str, str, float, float]] = []
|
||||||
|
current_interface: Optional[str] = None
|
||||||
|
|
||||||
|
for line in text.splitlines():
|
||||||
|
m = COUNTER_HEADER_RE.search(line)
|
||||||
|
if m:
|
||||||
|
current_interface = m.group(1).rstrip(":")
|
||||||
|
continue
|
||||||
|
|
||||||
|
if not current_interface:
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = COUNTER_ROW_RE.match(line)
|
||||||
|
if m:
|
||||||
|
rows.append(
|
||||||
|
(
|
||||||
|
current_interface,
|
||||||
|
normalize_message_type(m.group(1)),
|
||||||
|
float(m.group(2)),
|
||||||
|
float(m.group(3)),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
class NXOSClient:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
host: str,
|
||||||
|
port: int,
|
||||||
|
username: str,
|
||||||
|
password: Optional[str],
|
||||||
|
key_file: Optional[str],
|
||||||
|
timeout: float,
|
||||||
|
accept_new_host_key: bool,
|
||||||
|
):
|
||||||
|
self.host = host
|
||||||
|
self.port = port
|
||||||
|
self.username = username
|
||||||
|
self.password = password
|
||||||
|
self.key_file = key_file
|
||||||
|
self.timeout = timeout
|
||||||
|
self.accept_new_host_key = accept_new_host_key
|
||||||
|
|
||||||
|
def run_commands(self, commands: List[str]) -> Dict[str, str]:
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.load_system_host_keys()
|
||||||
|
if self.accept_new_host_key:
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
else:
|
||||||
|
client.set_missing_host_key_policy(paramiko.RejectPolicy())
|
||||||
|
|
||||||
|
client.connect(
|
||||||
|
hostname=self.host,
|
||||||
|
port=self.port,
|
||||||
|
username=self.username,
|
||||||
|
password=self.password,
|
||||||
|
key_filename=self.key_file,
|
||||||
|
timeout=self.timeout,
|
||||||
|
banner_timeout=self.timeout,
|
||||||
|
auth_timeout=self.timeout,
|
||||||
|
look_for_keys=self.key_file is None and self.password is None,
|
||||||
|
allow_agent=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
results: Dict[str, str] = {}
|
||||||
|
try:
|
||||||
|
for command in commands:
|
||||||
|
stdin, stdout, stderr = client.exec_command(command, timeout=self.timeout)
|
||||||
|
output = stdout.read().decode(errors="replace")
|
||||||
|
error = stderr.read().decode(errors="replace").strip()
|
||||||
|
status = stdout.channel.recv_exit_status()
|
||||||
|
if status != 0:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"{command!r} failed with status {status}: {error or output}"
|
||||||
|
)
|
||||||
|
results[command] = output
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
class NexusPTPCollector:
|
||||||
|
COMMANDS = [
|
||||||
|
"show ptp clock",
|
||||||
|
"show ptp parent",
|
||||||
|
"show ptp brief",
|
||||||
|
"show ptp counters all",
|
||||||
|
]
|
||||||
|
|
||||||
|
def __init__(self, nxos: NXOSClient, target_name: str, cache_seconds: float):
|
||||||
|
self.nxos = nxos
|
||||||
|
self.target_name = target_name
|
||||||
|
self.cache_seconds = cache_seconds
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
self.last_fetch = 0.0
|
||||||
|
self.cached: Optional[Dict[str, str]] = None
|
||||||
|
self.last_fetch_success = 0.0
|
||||||
|
self.last_error = ""
|
||||||
|
|
||||||
|
def _fetch(self) -> Dict[str, str]:
|
||||||
|
now = time.monotonic()
|
||||||
|
with self.lock:
|
||||||
|
if self.cached is not None and (now - self.last_fetch) < self.cache_seconds:
|
||||||
|
return self.cached
|
||||||
|
try:
|
||||||
|
self.cached = self.nxos.run_commands(self.COMMANDS)
|
||||||
|
self.last_fetch_success = 1.0
|
||||||
|
self.last_error = ""
|
||||||
|
except Exception as exc:
|
||||||
|
self.last_fetch_success = 0.0
|
||||||
|
self.last_error = type(exc).__name__
|
||||||
|
if self.cached is None:
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
self.last_fetch = time.monotonic()
|
||||||
|
return self.cached or {}
|
||||||
|
|
||||||
|
def collect(self):
|
||||||
|
start = time.monotonic()
|
||||||
|
|
||||||
|
try:
|
||||||
|
outputs = self._fetch()
|
||||||
|
success = self.last_fetch_success
|
||||||
|
except Exception as exc:
|
||||||
|
outputs = {}
|
||||||
|
success = 0.0
|
||||||
|
self.last_error = type(exc).__name__
|
||||||
|
|
||||||
|
clock = parse_clock(outputs.get("show ptp clock", ""))
|
||||||
|
parent = parse_parent(outputs.get("show ptp parent", ""))
|
||||||
|
ports = parse_brief(outputs.get("show ptp brief", ""))
|
||||||
|
counters = parse_counters(outputs.get("show ptp counters all", ""))
|
||||||
|
|
||||||
|
target_info = InfoMetricFamily(
|
||||||
|
"ptp_nexus_target",
|
||||||
|
"Nexus PTP exporter target",
|
||||||
|
labels=["target", "host"],
|
||||||
|
)
|
||||||
|
target_info.add_metric([self.target_name, self.nxos.host], {})
|
||||||
|
yield target_info
|
||||||
|
|
||||||
|
if clock:
|
||||||
|
info = InfoMetricFamily(
|
||||||
|
"ptp_nexus_local_clock",
|
||||||
|
"Nexus local PTP clock identity and source address",
|
||||||
|
labels=["clock_identity", "source_ip", "device_type"],
|
||||||
|
)
|
||||||
|
info.add_metric(
|
||||||
|
[
|
||||||
|
clock.get("Clock Identity", ""),
|
||||||
|
clock.get("PTP Source IP Address", ""),
|
||||||
|
clock.get("PTP Device Type", ""),
|
||||||
|
],
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
yield info
|
||||||
|
|
||||||
|
locked = GaugeMetricFamily(
|
||||||
|
"ptp_nexus_clock_locked",
|
||||||
|
"1 when the Nexus PTP clock state is Locked",
|
||||||
|
)
|
||||||
|
locked.add_metric([], 1.0 if clock.get("PTP Clock state", "").lower() == "locked" else 0.0)
|
||||||
|
yield locked
|
||||||
|
|
||||||
|
fields = [
|
||||||
|
("ptp_nexus_offset_from_master_nanoseconds", "Offset From Master", "Nexus offset from immediate PTP master"),
|
||||||
|
("ptp_nexus_mean_path_delay_nanoseconds", "Mean Path Delay", "Nexus mean path delay"),
|
||||||
|
("ptp_nexus_steps_removed", "Steps removed", "Boundary-clock steps from grandmaster"),
|
||||||
|
("ptp_nexus_domain_number", "Clock Domain", "PTP domain"),
|
||||||
|
("ptp_nexus_priority1", "Priority1", "Nexus PTP priority1"),
|
||||||
|
("ptp_nexus_priority2", "Priority2", "Nexus PTP priority2"),
|
||||||
|
("ptp_nexus_clock_class", "Clock Quality Class", "Nexus local clockClass"),
|
||||||
|
("ptp_nexus_clock_accuracy_code", "Clock Quality Accuracy", "Nexus clockAccuracy code"),
|
||||||
|
("ptp_nexus_offset_scaled_log_variance", "Clock Quality Offset (log variance)", "Nexus offsetScaledLogVariance"),
|
||||||
|
("ptp_nexus_port_count", "Number of PTP ports", "Number of PTP-enabled ports"),
|
||||||
|
]
|
||||||
|
for metric_name, field, help_text in fields:
|
||||||
|
if field in clock:
|
||||||
|
g = GaugeMetricFamily(metric_name, help_text)
|
||||||
|
g.add_metric([], number(clock[field]))
|
||||||
|
yield g
|
||||||
|
|
||||||
|
if parent:
|
||||||
|
info = InfoMetricFamily(
|
||||||
|
"ptp_nexus_parent",
|
||||||
|
"Nexus immediate PTP parent and grandmaster",
|
||||||
|
labels=[
|
||||||
|
"parent_clock_identity",
|
||||||
|
"parent_port_number",
|
||||||
|
"parent_ip",
|
||||||
|
"grandmaster_clock_identity",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
info.add_metric(
|
||||||
|
[
|
||||||
|
parent.get("Parent Clock Identity", ""),
|
||||||
|
parent.get("Parent Port Number", ""),
|
||||||
|
parent.get("Parent IP", ""),
|
||||||
|
parent.get("Grandmaster Clock Identity", ""),
|
||||||
|
],
|
||||||
|
{},
|
||||||
|
)
|
||||||
|
yield info
|
||||||
|
|
||||||
|
fields = [
|
||||||
|
("ptp_nexus_grandmaster_clock_class", "GM Class", "Selected grandmaster clockClass"),
|
||||||
|
("ptp_nexus_grandmaster_clock_accuracy_code", "GM Accuracy", "Selected grandmaster clockAccuracy code"),
|
||||||
|
("ptp_nexus_grandmaster_offset_scaled_log_variance", "GM Offset (log variance)", "Selected grandmaster offsetScaledLogVariance"),
|
||||||
|
("ptp_nexus_grandmaster_priority1", "GM Priority1", "Selected grandmaster priority1"),
|
||||||
|
("ptp_nexus_grandmaster_priority2", "GM Priority2", "Selected grandmaster priority2"),
|
||||||
|
]
|
||||||
|
for metric_name, field, help_text in fields:
|
||||||
|
if field in parent:
|
||||||
|
g = GaugeMetricFamily(metric_name, help_text)
|
||||||
|
g.add_metric([], number(parent[field]))
|
||||||
|
yield g
|
||||||
|
|
||||||
|
if ports:
|
||||||
|
metric = GaugeMetricFamily(
|
||||||
|
"ptp_nexus_port_state",
|
||||||
|
"Current Nexus PTP port state; one sample with value 1 per port",
|
||||||
|
labels=["interface", "state"],
|
||||||
|
)
|
||||||
|
for interface, state in ports:
|
||||||
|
metric.add_metric([interface, state], 1.0)
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
if counters:
|
||||||
|
metric = CounterMetricFamily(
|
||||||
|
"ptp_nexus_port_messages",
|
||||||
|
"PTP packet counters reported by NX-OS",
|
||||||
|
labels=["interface", "direction", "message_type"],
|
||||||
|
)
|
||||||
|
for interface, message_type, tx, rx in counters:
|
||||||
|
metric.add_metric([interface, "tx", message_type], tx)
|
||||||
|
metric.add_metric([interface, "rx", message_type], rx)
|
||||||
|
yield metric
|
||||||
|
|
||||||
|
scrape = GaugeMetricFamily(
|
||||||
|
"ptp_nexus_exporter_scrape_success",
|
||||||
|
"1 if the most recent NX-OS SSH collection succeeded",
|
||||||
|
labels=["target", "error"],
|
||||||
|
)
|
||||||
|
scrape.add_metric([self.target_name, self.last_error or "none"], success)
|
||||||
|
yield scrape
|
||||||
|
|
||||||
|
duration = GaugeMetricFamily(
|
||||||
|
"ptp_nexus_exporter_scrape_duration_seconds",
|
||||||
|
"Time spent serving this exporter collection",
|
||||||
|
labels=["target"],
|
||||||
|
)
|
||||||
|
duration.add_metric([self.target_name], time.monotonic() - start)
|
||||||
|
yield duration
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
p = argparse.ArgumentParser(description="Prometheus exporter for Cisco Nexus PTP")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--target-name", default=None)
|
||||||
|
p.add_argument("--ssh-port", type=int, default=22)
|
||||||
|
p.add_argument("--username", required=True)
|
||||||
|
p.add_argument("--key-file", default=None)
|
||||||
|
p.add_argument(
|
||||||
|
"--password-env",
|
||||||
|
default=None,
|
||||||
|
help="Read SSH password from this environment variable",
|
||||||
|
)
|
||||||
|
p.add_argument("--ssh-timeout", type=float, default=5.0)
|
||||||
|
p.add_argument("--cache-seconds", type=float, default=5.0)
|
||||||
|
p.add_argument("--accept-new-host-key", action="store_true")
|
||||||
|
p.add_argument("--listen", default="0.0.0.0")
|
||||||
|
p.add_argument("--port", type=int, default=9560)
|
||||||
|
args = p.parse_args()
|
||||||
|
|
||||||
|
password = os.environ.get(args.password_env) if args.password_env else None
|
||||||
|
|
||||||
|
nxos = NXOSClient(
|
||||||
|
host=args.host,
|
||||||
|
port=args.ssh_port,
|
||||||
|
username=args.username,
|
||||||
|
password=password,
|
||||||
|
key_file=args.key_file,
|
||||||
|
timeout=args.ssh_timeout,
|
||||||
|
accept_new_host_key=args.accept_new_host_key,
|
||||||
|
)
|
||||||
|
collector = NexusPTPCollector(
|
||||||
|
nxos=nxos,
|
||||||
|
target_name=args.target_name or args.host,
|
||||||
|
cache_seconds=args.cache_seconds,
|
||||||
|
)
|
||||||
|
REGISTRY.register(collector)
|
||||||
|
|
||||||
|
start_http_server(args.port, addr=args.listen)
|
||||||
|
print(
|
||||||
|
f"Nexus PTP exporter listening on {args.listen}:{args.port}, target={args.host}",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
while True:
|
||||||
|
time.sleep(3600)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
2
exporters/requirements.txt
Normal file
2
exporters/requirements.txt
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
prometheus-client>=0.20,<1
|
||||||
|
paramiko>=3.4,<5
|
||||||
22
exporters/systemd/nexus-ptp-exporter.service
Normal file
22
exporters/systemd/nexus-ptp-exporter.service
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Prometheus exporter for NYATER Nexus PTP
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=ptp-exporter
|
||||||
|
Group=ptp-exporter
|
||||||
|
EnvironmentFile=-/etc/default/nexus-ptp-exporter
|
||||||
|
ExecStart=/opt/ptp-exporter/venv/bin/python /opt/ptp-exporter/nexus_ptp_exporter.py \
|
||||||
|
--host ${NEXUS_HOST} \
|
||||||
|
--target-name ${NEXUS_NAME} \
|
||||||
|
--username ${NEXUS_USER} \
|
||||||
|
--key-file ${NEXUS_KEY_FILE} \
|
||||||
|
--listen 0.0.0.0 \
|
||||||
|
--port 9560
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
24
exporters/systemd/ptp-exporter-boundary.service
Normal file
24
exporters/systemd/ptp-exporter-boundary.service
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Prometheus exporter for Linux PTP boundary clock
|
||||||
|
After=network-online.target ptp4l-boundary.service phc2sys-boundary.service
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=ptp-exporter
|
||||||
|
Group=ptp-exporter
|
||||||
|
SupplementaryGroups=systemd-journal
|
||||||
|
RuntimeDirectory=ptp-exporter
|
||||||
|
RuntimeDirectoryMode=0750
|
||||||
|
ExecStart=/opt/ptp-exporter/venv/bin/python /opt/ptp-exporter/linux_ptp_exporter.py \
|
||||||
|
--role boundary \
|
||||||
|
--listen 0.0.0.0 \
|
||||||
|
--port 9559 \
|
||||||
|
--uds /var/run/ptp4lro \
|
||||||
|
--ptp4l-service ptp4l-boundary.service \
|
||||||
|
--phc2sys-service phc2sys-boundary.service
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=3
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
24
exporters/systemd/ptp-exporter-client.service
Normal file
24
exporters/systemd/ptp-exporter-client.service
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Prometheus exporter for Linux PTP client
|
||||||
|
After=network-online.target ptp4l-client.service phc2sys-client.service
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=ptp-exporter
|
||||||
|
Group=ptp-exporter
|
||||||
|
SupplementaryGroups=systemd-journal
|
||||||
|
RuntimeDirectory=ptp-exporter
|
||||||
|
RuntimeDirectoryMode=0750
|
||||||
|
ExecStart=/opt/ptp-exporter/venv/bin/python /opt/ptp-exporter/linux_ptp_exporter.py \
|
||||||
|
--role client \
|
||||||
|
--listen 0.0.0.0 \
|
||||||
|
--port 9559 \
|
||||||
|
--uds /var/run/ptp4lro \
|
||||||
|
--ptp4l-service ptp4l-client.service \
|
||||||
|
--phc2sys-service phc2sys-client.service
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=3
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
25
exporters/systemd/ptp-exporter-grandmaster.service
Normal file
25
exporters/systemd/ptp-exporter-grandmaster.service
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Prometheus exporter for PTP grandmaster
|
||||||
|
After=network-online.target ptp4l-gm.service phc2sys-gm.service chrony.service
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=ptp-exporter
|
||||||
|
Group=ptp-exporter
|
||||||
|
SupplementaryGroups=systemd-journal
|
||||||
|
RuntimeDirectory=ptp-exporter
|
||||||
|
RuntimeDirectoryMode=0750
|
||||||
|
ExecStart=/opt/ptp-exporter/venv/bin/python /opt/ptp-exporter/linux_ptp_exporter.py \
|
||||||
|
--role grandmaster \
|
||||||
|
--listen 0.0.0.0 \
|
||||||
|
--port 9559 \
|
||||||
|
--uds /var/run/ptp4lro \
|
||||||
|
--chrony \
|
||||||
|
--ptp4l-service ptp4l-gm.service \
|
||||||
|
--phc2sys-service phc2sys-gm.service
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=3
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
78
exporters/test_parsers.py
Normal file
78
exporters/test_parsers.py
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Small parser smoke tests. Does not connect to any device."""
|
||||||
|
|
||||||
|
from linux_ptp_exporter import parse_pmc
|
||||||
|
from nexus_ptp_exporter import parse_clock, parse_parent, parse_brief
|
||||||
|
|
||||||
|
PMC_SAMPLE = """
|
||||||
|
sending: GET TIME_STATUS_NP
|
||||||
|
141877.fffe.3b4c2c-0 seq 0 RESPONSE MANAGEMENT TIME_STATUS_NP
|
||||||
|
master_offset -13
|
||||||
|
ingress_time 1788117645030216721
|
||||||
|
cumulativeScaledRateOffset +0.000000000
|
||||||
|
scaledLastGmPhaseChange 0
|
||||||
|
gmTimeBaseIndicator 0
|
||||||
|
lastGmPhaseChange 0x0000'0000000000000000.0000
|
||||||
|
gmPresent true
|
||||||
|
gmIdentity 88a29e.fffe.826ce6
|
||||||
|
"""
|
||||||
|
|
||||||
|
NEXUS_CLOCK = """
|
||||||
|
PTP Device Type : boundary-clock
|
||||||
|
PTP Source IP Address : 10.255.254.255
|
||||||
|
Clock Identity : 28:6f:7f:ff:fe:21:0f:8d
|
||||||
|
Clock Domain: 0
|
||||||
|
Number of PTP ports: 2
|
||||||
|
Priority1 : 255
|
||||||
|
Priority2 : 255
|
||||||
|
Clock Quality:
|
||||||
|
Class : 248
|
||||||
|
Accuracy : 254
|
||||||
|
Offset (log variance) : 65535
|
||||||
|
Offset From Master : 40
|
||||||
|
Mean Path Delay : 226
|
||||||
|
Steps removed : 2
|
||||||
|
PTP Clock state : Locked
|
||||||
|
"""
|
||||||
|
|
||||||
|
NEXUS_PARENT = """
|
||||||
|
Parent Clock:
|
||||||
|
Parent Clock Identity: 14:18:77:ff:fe:3b:4c:2c
|
||||||
|
Parent Port Number: 4
|
||||||
|
Parent IP: 10.255.255.8
|
||||||
|
Grandmaster Clock:
|
||||||
|
Grandmaster Clock Identity: 88:a2:9e:ff:fe:82:6c:e6
|
||||||
|
Grandmaster Clock Quality:
|
||||||
|
Class: 6
|
||||||
|
Accuracy: 254
|
||||||
|
Offset (log variance): 65535
|
||||||
|
Priority1: 128
|
||||||
|
Priority2: 128
|
||||||
|
"""
|
||||||
|
|
||||||
|
NEXUS_BRIEF = """
|
||||||
|
Port State
|
||||||
|
--------------------- ------------
|
||||||
|
Eth1/5 Slave
|
||||||
|
Eth1/6 Passive
|
||||||
|
"""
|
||||||
|
|
||||||
|
blocks = parse_pmc(PMC_SAMPLE)
|
||||||
|
assert blocks[0]["master_offset"] == -13.0
|
||||||
|
assert blocks[0]["gmPresent"] == 1.0
|
||||||
|
assert blocks[0]["gmIdentity"] == "88a29e.fffe.826ce6"
|
||||||
|
|
||||||
|
clock = parse_clock(NEXUS_CLOCK)
|
||||||
|
assert clock["Offset From Master"] == "40"
|
||||||
|
assert clock["Clock Quality Class"] == "248"
|
||||||
|
assert clock["PTP Clock state"] == "Locked"
|
||||||
|
|
||||||
|
parent = parse_parent(NEXUS_PARENT)
|
||||||
|
assert parent["Parent IP"] == "10.255.255.8"
|
||||||
|
assert parent["Grandmaster Clock Identity"] == "88:a2:9e:ff:fe:82:6c:e6"
|
||||||
|
assert parent["GM Priority1"] == "128"
|
||||||
|
|
||||||
|
brief = parse_brief(NEXUS_BRIEF)
|
||||||
|
assert brief == [("Eth1/5", "slave"), ("Eth1/6", "passive")]
|
||||||
|
|
||||||
|
print("parser smoke tests: OK")
|
||||||
Reference in New Issue
Block a user