PTP+Squared is a dynamic, self healing PTP distribution and synchronisation feature within TImebeat. At a glance this system can appear complicated and potentially tricky to get up and running, but if you follow this guide you will quickly realise that just isn't the case.
Before we start their are 2 concepts that need to be understood to make configuration seamless and near effortless when working at scale.
Concept 1:
Root nodes:
The root node can be viewed as many different things but ultimately this will be the device that injects the source of PTP into the Grid. If you use one of our G0kK-1 series devices we recommend that this become the root node. If not, however, then the root node will be the first device connected to your current GMC.
Concept 2:
Grid nodes:
Grid nodes are nodes within the deployment that are not connected to a direct source of UTC. This means they will be at least 1 hop from UTC at a minimum. Depending on how large the grid is and the specific configuration it is likely that a grid node will connect directly to a Root Node or be one hop away from the Root node. This is not a problem though as PTP+Squared will always take the best possible source so you will always get the best sync possible.
Click here to Skip straight to the Command line steps if you are familiar with the base config.
The best place to start when configuring PTP+Squared is to start with the root node.
Unlike the Grid nodes, a Root node requires a primary source. This is configured in the exact same way as using the basic configuration for a primary source we will go over it again below.
# ______ _ __ __
# /_ __/(_)____ ___ ___ / /_ ___ ____ _ / /_
# / / / // __ `__ \ / _ \ / __ \ / _ \ / __ `// __/
# / / / // / / / / // __// /_/ // __// /_/ // /_
# /_/ /_//_/ /_/ /_/ \___//_.___/ \___/ \__,_/ \__/
################### Timebeat Configuration AREA #########################
timebeat:
# Location of license key file for premium features
license.keyfile: '/etc/timebeat/timebeat.lic'
# Configuration for clock synchronisation
clock_sync:
# Clocksync
# Default is true enabling clock synchronisation, setting to false for monitoring purposes only (false will not adjust the clock)
adjust_clock: true
# To remove step limit restrictions comment out the below, this will allow the clock to be stepped indefinitely
step_limit: 15m # "s", "h" "d" can be used to denominate seconds, hours or days for step limit.
primary_clocks:
# PTP Config example
- protocol: ptp
domain: 0
# serve_unicast: true # (Enterprise Feature)
# max_unicast_subscribers: 0 # (Enterprise Feature)
# serve_multicast: true # (Enterprise Feature)
# server_only: true # (Enterprise Feature)
announce_interval: 1
sync_interval: 0
delayrequest_interval: 0
# unicast_master_table: ['1.2.3.4','2.3.4.5', '3.4.5.6']
# delay_strategy: e2e # other options which can be used are: 'p2p'
# hybrid_e2e: false # Send delay requests as unicast (enterprise profile)
# priority1: 128
# priority2: 128
# monitor_only: false
# use_layer2: false # Use ptp over ethernet instead of IP/UDP
interface: enp2s0
# profile: 'G.8275.2' # other options which can be used are: 'G.8275.1', 'G.8265.1' and 'enterprise-draft'
# logsource: 'Grandmaster Clock in NY4' # configurable name for source included in all logging records for source
# asymmetry_compensation: 0 # In nanoseconds. Static compensation for know asymmetry (+/- as the case maybe)
# max_packets_per_second: 0 # If inbound packet rate exceeds this a WRED algorithm is used
You can use the above config snippet for your primary source, further customisation can be done to fit your environment or requirements but the above is the basic config to get started.
Just make sure that all the elements in purple fit your PTP source configuration.
Step 2:
The Squared configuration for the root node:
This may look complicated at first but if you follow the steps below correctly it is simple and easily replicated.
ptpsquared:
# Enable PTPSquared functionality
enable: true
discovery:
# mdns: true
dht: true
# dht_seed_list: ["/ip4/10.101.101.23/tcp/65107/p2p/16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9"]
keypath: "/etc/timebeat/ptp2key.private"
# domains: [115,116]
interface: ens5f0
seats_to_offer: 4 # Number of seats available to consume time from this node
seats_to_fill: 3 # Number of seats this node seeks to fill from other nodes to consume time
concurrent_sources: 1 # Number of concurrent PTP sources used to steer our clock(s)
# active_sync_interval: -3 # sync interval for active sources
# active_delayrequest_interval: -3 # delay request interval for active sources
# monitor_sync_interval: 0 # sync interval for sources we monitor
# monitor_delayrequest_interval: 0 # delay request interval for sources we monitor
# capabilities: ["hqosc-1500"] # high-quality oscillator 1.5us drift per 24 hours
# preference_score: 0 # Used when requesting capacity from reservations
# reservations: ["1500:50%:115,116", "750:25%"] # List containing client seat reservations:
# :
# :
#
# debug: false # debug logging specifically for ptpsquared
# advanced:
# asymmetry_compensation: 0 # In nanoseconds. Static compensation for know asymmetry (+/- as the case maybe). All interfaces
# is_better_factor: 1.4 # How much "better" does a source need to be to be selected instead of a current source
# eos_weight: 1.0 # Weight attributed to error of source std dev calculation in determining hop cost
# All costs below default to zero which infers that cumulative error of source is the only factor
# determining the cost of a particular time source path
# base_hop_cost: 0.0 # The default cost of a single hop to downstream peers
# swts_cost: 0.0 # The cost of using software timestamping
# hwts_cost: 0.0 # The cost of using hardware timestamping
# Enable latency analysis functionality
# latency_analysis_enable: false
The basic configuration above can be used to get started. Just make sure again that you have the correct interface configured. This will be the interface that will be sending PTP to the grid nodes.
All the other items in the config section for PTP+squared can be modified to suit you but the default config as above will get you started for the Root Node.
(Please note if your root node is also your Grandmaster such as a G0kK-1 you may want to change concurrent sources to zero so that it is not affected by the grid in the event you lose GNSS or your direct source of UTC).
Configuring the Grid Node:
To configure the Grid node, you could see this as a slightly more complicated method but if you are lucky it is a lot easier to replicate at scale.
To configure the Grid node start by matching the above Root Node PTP+Squared config section. (please note, for a grid node you do not need to configure a primary source of time.)
ptpsquared:
# Enable PTPSquared functionality
enable: true
discovery:
# mdns: true
dht: true
dht_seed_list: ["/ip4/10.101.101.23/tcp/65107/p2p/16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9"]
keypath: "/etc/timebeat/ptp2key.private"
# domains: [115,116]
interface: ens5f0
seats_to_offer: 4 # Number of seats available to consume time from this node
seats_to_fill: 3 # Number of seats this node seeks to fill from other nodes to consume time
concurrent_sources: 1 # Number of concurrent PTP sources used to steer our clock(s)
# active_sync_interval: -3 # sync interval for active sources
# active_delayrequest_interval: -3 # delay request interval for active sources
# monitor_sync_interval: 0 # sync interval for sources we monitor
# monitor_delayrequest_interval: 0 # delay request interval for sources we monitor
# capabilities: ["hqosc-1500"] # high-quality oscillator 1.5us drift per 24 hours
# preference_score: 0 # Used when requesting capacity from reservations
# reservations: ["1500:50%:115,116", "750:25%"] # List containing client seat reservations:
# :
# :
#
# debug: false # debug logging specifically for ptpsquared
# advanced:
# asymmetry_compensation: 0 # In nanoseconds. Static compensation for know asymmetry (+/- as the case maybe). All interfaces
# is_better_factor: 1.4 # How much "better" does a source need to be to be selected instead of a current source
# eos_weight: 1.0 # Weight attributed to error of source std dev calculation in determining hop cost
# All costs below default to zero which infers that cumulative error of source is the only factor
# determining the cost of a particular time source path
# base_hop_cost: 0.0 # The default cost of a single hop to downstream peers
# swts_cost: 0.0 # The cost of using software timestamping
# hwts_cost: 0.0 # The cost of using hardware timestamping
# Enable latency analysis functionality
# latency_analysis_enable: false
You will notice that for the Grid node there is an extra line of config that has to be enabled in the discovery section:
discovery:
# mdns: true
dht: true
dht_seed_list: ["/ip4/10.101.101.23/tcp/65107/p2p/16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9"]
The dht_seed_list.
For this you will need to insert the IP address of the Root node. This will be on the same interface of the root node specified as the PTP output interface in the root node PTP+Squared config section.
You will also need to insert the Root node ID. To find the Root node ID you will need to access the Timebeat CLI. Once configured you will find the Root Node ID when passing the "show squared seats" command.
Once completed this will enable the grid once the Timebeat service starts on both devices.
Provided your Grid nodes have the same interface you can copy and paste this config into all of them and they will also join the grid.
Of course, you should follow all the other steps to set up timebeat such as data source/output configuration but we won't look into that in this guide.
The Command line steps to configure PTP+Squared:
(assumptions of below commands are that you are running as root, if you are not root just place sudo at the start of each command).
The Root Node:
Step 1: identify the interface you are receiving PTP on and the interface you wish to distribute PTP into the grid on (these can be the same if your networking is set up in this fashion).
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
2: eno1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
3: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
...
Step 2:
edit the timebeat.yml config file with a primary source of PTP:
# vi /etc/timebeat/timebeat.yml
ptpsquared:
# Enable PTPSquared functionality
enable: true
discovery:
# mdns: true
dht: true
# dht_seed_list: ["/ip4/10.101.101.23/tcp/65107/p2p/16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9"]
keypath: "/etc/timebeat/ptp2key.private"
# domains: [115,116]
interface: ens5f0
seats_to_offer: 4 # Number of seats available to consume time from this node
seats_to_fill: 3 # Number of seats this node seeks to fill from other nodes to consume time
concurrent_sources: 1 # Number of concurrent PTP sources used to steer our clock(s)
# active_sync_interval: -3 # sync interval for active sources
# active_delayrequest_interval: -3 # delay request interval for active sources
# monitor_sync_interval: 0 # sync interval for sources we monitor
# monitor_delayrequest_interval: 0 # delay request interval for sources we monitor
# capabilities: ["hqosc-1500"] # high-quality oscillator 1.5us drift per 24 hours
# preference_score: 0 # Used when requesting capacity from reservations
# reservations: ["1500:50%:115,116", "750:25%"] # List containing client seat reservations:
# :
# :
#
# debug: false # debug logging specifically for ptpsquared
# advanced:
# asymmetry_compensation: 0 # In nanoseconds. Static compensation for know asymmetry (+/- as the case maybe). All interfaces
# is_better_factor: 1.4 # How much "better" does a source need to be to be selected instead of a current source
# eos_weight: 1.0 # Weight attributed to error of source std dev calculation in determining hop cost
# All costs below default to zero which infers that cumulative error of source is the only factor
# determining the cost of a particular time source path
# base_hop_cost: 0.0 # The default cost of a single hop to downstream peers
# swts_cost: 0.0 # The cost of using software timestamping
# hwts_cost: 0.0 # The cost of using hardware timestamping
# Enable latency analysis functionality
# latency_analysis_enable: false
Step 3:
configure the CLI to be on:
cli:
# Enable the SSH based CLI interface. I.e. to access: ssh -p 65129 admin@127.0.0.1
enable: true
bind_port: 65129
bind_host: 127.0.0.1
# Server SSH key. If path specified key will be loaded or generated. Otherwise key will be random on startup
server_key: "/etc/timebeat/cli_id_rsa"
# Authorised SSH keys (yes... with an s not a z....)
# authorised_keys: "/etc/timebeat/authorised_keys"
# CLI username and password
username: "admin"
password: "password"
The above config can be copied in place but just uncomment the defaults and you will be good to go.
Step 4:
Set up the root node PTP+Squared config.
ptpsquared:
# Enable PTPSquared functionality
enable: true
discovery:
# mdns: true
dht: true
dht_seed_list: ["/ip4/10.101.101.23/tcp/65107/p2p/16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9"]
keypath: "/etc/timebeat/ptp2key.private"
# domains: [115,116]
interface: ens5f0
seats_to_offer: 4 # Number of seats available to consume time from this node
seats_to_fill: 3 # Number of seats this node seeks to fill from other nodes to consume time
concurrent_sources: 1 # Number of concurrent PTP sources used to steer our clock(s)
# active_sync_interval: -3 # sync interval for active sources
# active_delayrequest_interval: -3 # delay request interval for active sources
# monitor_sync_interval: 0 # sync interval for sources we monitor
# monitor_delayrequest_interval: 0 # delay request interval for sources we monitor
# capabilities: ["hqosc-1500"] # high-quality oscillator 1.5us drift per 24 hours
# preference_score: 0 # Used when requesting capacity from reservations
# reservations: ["1500:50%:115,116", "750:25%"] # List containing client seat reservations:
# :
# :
#
# debug: false # debug logging specifically for ptpsquared
# advanced:
# asymmetry_compensation: 0 # In nanoseconds. Static compensation for know asymmetry (+/- as the case maybe). All interfaces
# is_better_factor: 1.4 # How much "better" does a source need to be to be selected instead of a current source
# eos_weight: 1.0 # Weight attributed to error of source std dev calculation in determining hop cost
# All costs below default to zero which infers that cumulative error of source is the only factor
# determining the cost of a particular time source path
# base_hop_cost: 0.0 # The default cost of a single hop to downstream peers
# swts_cost: 0.0 # The cost of using software timestamping
# hwts_cost: 0.0 # The cost of using hardware timestamping
# Enable latency analysis functionality
# latency_analysis_enable: false
Step 5:
Start the timebeat service
# service timebeat start
Step 6:
Obtain the Root Node id from the CLI:
# ssh -p 65129 admin@127.0.0.1 admin@127.0.0.1's password: Welcome to Timebeat 1.4.8-enterprise CLI Timebeat# show squared seats
My ID is: 16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9 My RootDistance is: 0 Time sent from other peers to this peer: Current server peer(s): 0 Unfilled client seats: 0 Max seats: 0 Time sent from this peer to other peers: Current client peer(s): 0 Unfilled server seats: 0 Max seats: 0
Timebeat# exit
Connection to 127.0.0.1 closed.
(the purple string is your Root Node ID the above is just an example ID, and each root node is different).
That will be the root node complete.
The Grid Node:
ptpsquared:
# Enable PTPSquared functionality
enable: true
discovery:
# mdns: true
dht: true
dht_seed_list: ["/ip4/10.101.101.23/tcp/65107/p2p/16Uiu2HAmJiQvJQbja8pf5dKAZsSYxWmcDCxZaoYbMUL5X7GnXej9"]
keypath: "/etc/timebeat/ptp2key.private"
# domains: [115,116]
interface: ens5f0
# seats_to_offer: 4 # Number of seats available to consume time from this node
# seats_to_fill: 3 # Number of seats this node seeks to fill from other nodes to consume time
# concurrent_sources: 1 # Number of concurrent PTP sources used to steer our clock(s)
# active_sync_interval: -3 # sync interval for active sources
# active_delayrequest_interval: -3 # delay request interval for active sources
# monitor_sync_interval: 0 # sync interval for sources we monitor
# monitor_delayrequest_interval: 0 # delay request interval for sources we monitor
# capabilities: ["hqosc-1500"] # high-quality oscillator 1.5us drift per 24 hours
# preference_score: 0 # Used when requesting capacity from reservations
# reservations: ["1500:50%:115,116", "750:25%"] # List containing client seat reservations:
# :
# :
#
# debug: false # debug logging specifically for ptpsquared
# advanced:
# asymmetry_compensation: 0 # In nanoseconds. Static compensation for know asymmetry (+/- as the case maybe). All interfaces
# is_better_factor: 1.4 # How much "better" does a source need to be to be selected instead of a current source
# eos_weight: 1.0 # Weight attributed to error of source std dev calculation in determining hop cost
# All costs below default to zero which infers that cumulative error of source is the only factor
# determining the cost of a particular time source path
# base_hop_cost: 0.0 # The default cost of a single hop to downstream peers
# swts_cost: 0.0 # The cost of using software timestamping
# hwts_cost: 0.0 # The cost of using hardware timestamping
# Enable latency analysis functionality
# latency_analysis_enable: false
Step 1:
Step 1: identify the interface you are receiving PTP on and the interface you wish to distribute PTP into the grid on (these will be the same).
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
2: eno1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
3: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
...
Step 2:
edit the timebeat.yml config file with a primary source of PTP:
# vi /etc/timebeat/timebeat.yml
See the above example config.
The key items for a grid node in the config file will be to amend the dht_seed_list section to match your root node.
For this you will need the IP address of the Root Node (which we found earlier), and the Root Node ID (also found earlier).
Once complete just start the Timebeat service.