10-23-2020, 06:59 PM
Secure NTP with NTS
<div style="margin: 5px 5% 10px 5%;"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/10/secure-ntp-with-nts.png" width="1024" height="768" title="" alt="" /></div><div><p>Many computers use the Network Time Protocol (NTP) to synchronize their system clocks over the internet. NTP is one of the few unsecured internet protocols still in common use. An attacker that can observe network traffic between a client and server can feed the client with bogus data and, depending on the client’s implementation and configuration, force it to set its system clock to any time and date. Some programs and services might not work if the client’s system clock is not accurate. For example, a web browser will not work correctly if the web servers’ certificates appear to be expired according to the client’s system clock. Use Network Time Security (NTS) to secure NTP.</p>
<p> <span id="more-32025"></span> </p>
<p>Fedora 33<sup><a href="#footnote1">1</a></sup> is the first Fedora release to support NTS. NTS is a new authentication mechanism for NTP. It enables clients to verify that the packets they receive from the server have not been modified while in transit. The only thing an attacker can do when NTS is enabled is drop or delay packets. See <a href="https://tools.ietf.org/html/rfc8915" target="_blank" rel="noreferrer noopener">RFC8915</a> for further details about NTS.</p>
<p>NTP can be secured well with symmetric keys. Unfortunately, the server has to have a different key for each client and the keys have to be securely distributed. That might be practical with a private server on a local network, but it does not scale to a public server with millions of clients.</p>
<p>NTS includes a Key Establishment (NTS-KE) protocol that automatically creates the encryption keys used between the server and its clients. It uses Transport Layer Security (TLS) on TCP port 4460. It is designed to scale to very large numbers of clients with a minimal impact on accuracy. The server does not need to keep any client-specific state. It provides clients with cookies, which are encrypted and contain the keys needed to authenticate the NTP packets. Privacy is one of the goals of NTS. The client gets a new cookie with each server response, so it doesn’t have to reuse cookies. This prevents passive observers from tracking clients migrating between networks.</p>
<p>The default NTP client in Fedora is <em>chrony</em>. Chrony added NTS support in version 4.0. The default configuration hasn’t changed. Chrony still uses public servers from the <a href="https://www.pool.ntp.org">pool.ntp.org</a> project and NTS is not enabled by default.</p>
<p>Currently, there are very few public NTP servers that support NTS. The two major providers are Cloudflare and Netnod. The <a href="https://developers.cloudflare.com/time-services/nts/usage">Cloudflare servers</a> are in various places around the world. They use anycast addresses that should allow most clients to reach a close server. The <a href="https://www.netnod.se/time-and-frequency/how-to-use-nts">Netnod servers</a> are located in Sweden. In the future we will probably see more public NTP servers with NTS support.</p>
<p>A general recommendation for configuring NTP clients for best reliability is to have at least three working servers. For best accuracy, it is recommended to select close servers to minimize network latency and asymmetry caused by asymmetric network routing. If you are not concerned about fine-grained accuracy, you can ignore this recommendation and use any NTS servers you trust, no matter where they are located.</p>
<p>If you do want high accuracy, but you don’t have a close NTS server, you can mix distant NTS servers with closer non-NTS servers. However, such a configuration is less secure than a configuration using NTS servers only. The attackers still cannot force the client to accept arbitrary time, but they do have a greater control over the client’s clock and its estimate of accuracy, which may be unacceptable in some environments.</p>
<h2>Enable client NTS in the installer</h2>
<p>When installing Fedora 33, you can enable NTS in the <em>Time & Date</em> dialog in the <em>Network Time</em> configuration. Enter the name of the server and check the NTS support before clicking the <strong>+</strong> (Add) button. You can add one or more servers or pools with NTS. To remove the default pool of servers (<em>2.fedora.pool.ntp.org</em>), uncheck the corresponding mark in the <em>Use</em> column.</p>
<figure class="wp-block-image size-large"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/10/secure-ntp-with-nts.png" alt="" class="wp-image-32050" /><figcaption>Network Time configuration in<br />
Fedora installer</figcaption></figure>
<h2>Enable client NTS in the configuration file</h2>
<p>If you upgraded from a previous Fedora release, or you didn’t enable NTS in the installer, you can enable NTS directly in <em>/etc/chrony.conf</em>. Specify the server with the <em>nts</em> option in addition to the recommended <em>iburst</em> option. For example:</p>
<pre class="wp-block-preformatted">server time.cloudflare.com iburst nts
server nts.sth1.ntp.se iburst nts
server nts.sth2.ntp.se iburst nts</pre>
<p>You should also allow the client to save the NTS keys and cookies to disk,<br />
so it doesn’t have to repeat the NTS-KE session on each start. Add the<br />
following line to <em>chrony.conf</em>, if it is not already present:</p>
<pre class="wp-block-preformatted">ntsdumpdir /var/lib/chrony</pre>
<p>If you don’t want NTP servers provided by DHCP to be mixed with the servers you<br />
have specified, remove or comment out the following line in<br />
<em>chrony.conf</em>:</p>
<pre class="wp-block-preformatted">sourcedir /run/chrony-dhcp</pre>
<p>After you have finished editing <em>chrony.conf</em>, save your changes and restart the <em>chronyd</em> service:</p>
<pre class="wp-block-preformatted">systemctl restart chronyd</pre>
<h2>Check client status</h2>
<p>Run the following command under the root user to check whether the NTS key<br />
establishment was successful:</p>
<pre class="wp-block-preformatted" style="font-size: 60%"># <strong>chronyc -N authdata</strong>
Name/IP address Mode KeyID Type KLen Last Atmp NAK Cook CLen
=========================================================================
time.cloudflare.com NTS 1 15 256 33m 0 0 8 100
nts.sth1.ntp.se NTS 1 15 256 33m 0 0 8 100
nts.sth2.ntp.se NTS 1 15 256 33m 0 0 8 100</pre>
<p>The <em>KeyID</em>, <em>Type</em>, and <em>KLen</em> columns should have non-zero values. If they are zero, check the system log for error messages from <em>chronyd</em>. One possible cause of failure is a firewall is blocking the client’s connection to the server’s TCP port ( port 4460).</p>
<p>Another possible cause of failure is a certificate that is failing to verify because the client’s clock is wrong. This is a chicken-or-the-egg type problem with NTS. You may need to manually correct the date or temporarily disable NTS in order to get NTS working. If your computer has a real-time clock, as almost all computers do, and it’s backed up by a good battery, this operation should be needed only once.</p>
<p>If the computer doesn’t have a real-time clock or battery, as is common with<br />
some small ARM computers like the Raspberry Pi, you can add the <em>-s</em><br />
option to <em>/etc/sysconfig/chronyd</em> to restore time saved on the last<br />
shutdown or reboot. The clock will be behind the true time, but if the<br />
computer wasn’t shut down for too long and the server’s certificates were not<br />
renewed too close to their expiration, it should be sufficient for the time<br />
checks to succeed. As a last resort, you can disable the time checks with the<br />
<em>nocerttimecheck</em> directive. See the <em>chrony.conf(5)</em> man page<br />
for details.</p>
<p>Run the following command to confirm that the client is making NTP<br />
measurements:</p>
<pre class="wp-block-preformatted" style="font-size: 60%"># <strong>chronyc -N sources</strong>
MS Name/IP address Stratum Poll Reach LastRx Last sample ===============================================================================
^* time.cloudflare.com 3 6 377 45 +355us[ +375us] +/- 11ms
^+ nts.sth1.ntp.se 1 6 377 44 +237us[ +237us] +/- 23ms
^+ nts.sth2.ntp.se 1 6 377 44 -170us[ -170us] +/- 22ms</pre>
<p>The <em>Reach</em> column should have a non-zero value; ideally 377. The value 377 shown above is an octal number. It indicates that the last eight requests all had a valid response. The validation check will include NTS authentication if enabled. If the value only rarely or never gets to 377, it indicates that NTP requests or responses are getting lost in the network. Some major network operators are known to have middleboxes that block or limit rate of large NTP packets as a mitigation for amplification attacks that exploit the monitoring protocol of <em>ntpd</em>. Unfortunately, this impacts NTS-protected NTP packets, even though they don’t cause any amplification. The NTP working group is considering an alternative port for NTP as a workaround for this issue.</p>
<h2>Enable NTS on the server</h2>
<p>If you have your own NTP server running <em>chronyd</em>, you can enable server NTS support to allow its clients to be synchronized securely. If the server is a client of other servers, it should use NTS or a symmetric key for its own synchronization. The clients assume the synchronization chain is secured between all servers up to the primary time servers.</p>
<p>Enabling server NTS is similar to enabling HTTPS on a web server. You just need a private key and certificate. The certificate could be signed by the Let’s Encrypt authority using the <em>certbot</em> tool, for example. When you have the key and certificate file (including intermediate certificates), specify them in <em>chrony.conf</em> with the following directives:</p>
<pre class="wp-block-preformatted">ntsserverkey /etc/pki/tls/private/foo.example.net.key
ntsservercert /etc/pki/tls/certs/foo.example.net.crt</pre>
<p>Make sure the <em>ntsdumpdir</em> directive mentioned previously in the<br />
client configuration is present in <em>chrony.conf</em>. It allows the server<br />
to save its keys to disk, so the clients of the server don’t have to get new<br />
keys and cookies when the server is restarted.</p>
<p>Restart the <em>chronyd</em> service:</p>
<pre class="wp-block-preformatted">systemctl restart chronyd</pre>
<p>If there are no error messages in the system log from <em>chronyd</em>, it should be<br />
accepting client connections. If the server has a firewall, it needs to allow<br />
both the UDP 123 and TCP 4460 ports for NTP and NTS-KE respectively.</p>
<p>You can perform a quick test from a client machine with the following command:</p>
<pre class="wp-block-preformatted">$ <strong>chronyd -Q -t 3 'server foo.example.net iburst nts maxsamples 1'</strong>
2020-10-13T12:00:52Z chronyd version 4.0 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 +DEBUG)
2020-10-13T12:00:52Z Disabled control of system clock
2020-10-13T12:00:55Z System clock wrong by -0.001032 seconds (ignored)
2020-10-13T12:00:55Z chronyd exiting</pre>
<p>If you see a <em>System clock wrong</em> message, it’s working<br />
correctly.</p>
<p>On the server, you can use the following command to check how many NTS-KE<br />
connections and authenticated NTP packets it has handled:</p>
<pre class="wp-block-preformatted"># <strong>chronyc serverstats</strong>
NTP packets received : 2143106240
NTP packets dropped : 117180834
Command packets received : 16819527
Command packets dropped : 0
Client log records dropped : 574257223
NTS-KE connections accepted: 104
NTS-KE connections dropped : 0
Authenticated NTP packets : 52139</pre>
<p>If you see non-zero <em>NTS-KE connections accepted</em> and <em>Authenticated<br />
NTP packets</em>, it means at least some clients were able to connect to the<br />
NTS-KE port and send an authenticated NTP request.</p>
<p class="has-text-align-right"><em>— Cover photo by Louis. K on Unsplash —</em></p>
<hr class="wp-block-separator is-style-wide" />
<p style="font-size: 50%"><a id="footnote1">1</a>. The Fedora 33 <em>Beta</em> installer contains an older chrony prerelease which doesn’t work with current NTS servers because the NTS-KE port has changed. Consequently, in the Network Time configuration in the installer, the servers will always appear as not working. After installation, the chrony package needs to be updated before it will work with current servers.</p></p>
</div>
https://www.sickgaming.net/blog/2020/10/...-with-nts/
<div style="margin: 5px 5% 10px 5%;"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/10/secure-ntp-with-nts.png" width="1024" height="768" title="" alt="" /></div><div><p>Many computers use the Network Time Protocol (NTP) to synchronize their system clocks over the internet. NTP is one of the few unsecured internet protocols still in common use. An attacker that can observe network traffic between a client and server can feed the client with bogus data and, depending on the client’s implementation and configuration, force it to set its system clock to any time and date. Some programs and services might not work if the client’s system clock is not accurate. For example, a web browser will not work correctly if the web servers’ certificates appear to be expired according to the client’s system clock. Use Network Time Security (NTS) to secure NTP.</p>
<p> <span id="more-32025"></span> </p>
<p>Fedora 33<sup><a href="#footnote1">1</a></sup> is the first Fedora release to support NTS. NTS is a new authentication mechanism for NTP. It enables clients to verify that the packets they receive from the server have not been modified while in transit. The only thing an attacker can do when NTS is enabled is drop or delay packets. See <a href="https://tools.ietf.org/html/rfc8915" target="_blank" rel="noreferrer noopener">RFC8915</a> for further details about NTS.</p>
<p>NTP can be secured well with symmetric keys. Unfortunately, the server has to have a different key for each client and the keys have to be securely distributed. That might be practical with a private server on a local network, but it does not scale to a public server with millions of clients.</p>
<p>NTS includes a Key Establishment (NTS-KE) protocol that automatically creates the encryption keys used between the server and its clients. It uses Transport Layer Security (TLS) on TCP port 4460. It is designed to scale to very large numbers of clients with a minimal impact on accuracy. The server does not need to keep any client-specific state. It provides clients with cookies, which are encrypted and contain the keys needed to authenticate the NTP packets. Privacy is one of the goals of NTS. The client gets a new cookie with each server response, so it doesn’t have to reuse cookies. This prevents passive observers from tracking clients migrating between networks.</p>
<p>The default NTP client in Fedora is <em>chrony</em>. Chrony added NTS support in version 4.0. The default configuration hasn’t changed. Chrony still uses public servers from the <a href="https://www.pool.ntp.org">pool.ntp.org</a> project and NTS is not enabled by default.</p>
<p>Currently, there are very few public NTP servers that support NTS. The two major providers are Cloudflare and Netnod. The <a href="https://developers.cloudflare.com/time-services/nts/usage">Cloudflare servers</a> are in various places around the world. They use anycast addresses that should allow most clients to reach a close server. The <a href="https://www.netnod.se/time-and-frequency/how-to-use-nts">Netnod servers</a> are located in Sweden. In the future we will probably see more public NTP servers with NTS support.</p>
<p>A general recommendation for configuring NTP clients for best reliability is to have at least three working servers. For best accuracy, it is recommended to select close servers to minimize network latency and asymmetry caused by asymmetric network routing. If you are not concerned about fine-grained accuracy, you can ignore this recommendation and use any NTS servers you trust, no matter where they are located.</p>
<p>If you do want high accuracy, but you don’t have a close NTS server, you can mix distant NTS servers with closer non-NTS servers. However, such a configuration is less secure than a configuration using NTS servers only. The attackers still cannot force the client to accept arbitrary time, but they do have a greater control over the client’s clock and its estimate of accuracy, which may be unacceptable in some environments.</p>
<h2>Enable client NTS in the installer</h2>
<p>When installing Fedora 33, you can enable NTS in the <em>Time & Date</em> dialog in the <em>Network Time</em> configuration. Enter the name of the server and check the NTS support before clicking the <strong>+</strong> (Add) button. You can add one or more servers or pools with NTS. To remove the default pool of servers (<em>2.fedora.pool.ntp.org</em>), uncheck the corresponding mark in the <em>Use</em> column.</p>
<figure class="wp-block-image size-large"><img src="https://www.sickgaming.net/blog/wp-content/uploads/2020/10/secure-ntp-with-nts.png" alt="" class="wp-image-32050" /><figcaption>Network Time configuration in<br />
Fedora installer</figcaption></figure>
<h2>Enable client NTS in the configuration file</h2>
<p>If you upgraded from a previous Fedora release, or you didn’t enable NTS in the installer, you can enable NTS directly in <em>/etc/chrony.conf</em>. Specify the server with the <em>nts</em> option in addition to the recommended <em>iburst</em> option. For example:</p>
<pre class="wp-block-preformatted">server time.cloudflare.com iburst nts
server nts.sth1.ntp.se iburst nts
server nts.sth2.ntp.se iburst nts</pre>
<p>You should also allow the client to save the NTS keys and cookies to disk,<br />
so it doesn’t have to repeat the NTS-KE session on each start. Add the<br />
following line to <em>chrony.conf</em>, if it is not already present:</p>
<pre class="wp-block-preformatted">ntsdumpdir /var/lib/chrony</pre>
<p>If you don’t want NTP servers provided by DHCP to be mixed with the servers you<br />
have specified, remove or comment out the following line in<br />
<em>chrony.conf</em>:</p>
<pre class="wp-block-preformatted">sourcedir /run/chrony-dhcp</pre>
<p>After you have finished editing <em>chrony.conf</em>, save your changes and restart the <em>chronyd</em> service:</p>
<pre class="wp-block-preformatted">systemctl restart chronyd</pre>
<h2>Check client status</h2>
<p>Run the following command under the root user to check whether the NTS key<br />
establishment was successful:</p>
<pre class="wp-block-preformatted" style="font-size: 60%"># <strong>chronyc -N authdata</strong>
Name/IP address Mode KeyID Type KLen Last Atmp NAK Cook CLen
=========================================================================
time.cloudflare.com NTS 1 15 256 33m 0 0 8 100
nts.sth1.ntp.se NTS 1 15 256 33m 0 0 8 100
nts.sth2.ntp.se NTS 1 15 256 33m 0 0 8 100</pre>
<p>The <em>KeyID</em>, <em>Type</em>, and <em>KLen</em> columns should have non-zero values. If they are zero, check the system log for error messages from <em>chronyd</em>. One possible cause of failure is a firewall is blocking the client’s connection to the server’s TCP port ( port 4460).</p>
<p>Another possible cause of failure is a certificate that is failing to verify because the client’s clock is wrong. This is a chicken-or-the-egg type problem with NTS. You may need to manually correct the date or temporarily disable NTS in order to get NTS working. If your computer has a real-time clock, as almost all computers do, and it’s backed up by a good battery, this operation should be needed only once.</p>
<p>If the computer doesn’t have a real-time clock or battery, as is common with<br />
some small ARM computers like the Raspberry Pi, you can add the <em>-s</em><br />
option to <em>/etc/sysconfig/chronyd</em> to restore time saved on the last<br />
shutdown or reboot. The clock will be behind the true time, but if the<br />
computer wasn’t shut down for too long and the server’s certificates were not<br />
renewed too close to their expiration, it should be sufficient for the time<br />
checks to succeed. As a last resort, you can disable the time checks with the<br />
<em>nocerttimecheck</em> directive. See the <em>chrony.conf(5)</em> man page<br />
for details.</p>
<p>Run the following command to confirm that the client is making NTP<br />
measurements:</p>
<pre class="wp-block-preformatted" style="font-size: 60%"># <strong>chronyc -N sources</strong>
MS Name/IP address Stratum Poll Reach LastRx Last sample ===============================================================================
^* time.cloudflare.com 3 6 377 45 +355us[ +375us] +/- 11ms
^+ nts.sth1.ntp.se 1 6 377 44 +237us[ +237us] +/- 23ms
^+ nts.sth2.ntp.se 1 6 377 44 -170us[ -170us] +/- 22ms</pre>
<p>The <em>Reach</em> column should have a non-zero value; ideally 377. The value 377 shown above is an octal number. It indicates that the last eight requests all had a valid response. The validation check will include NTS authentication if enabled. If the value only rarely or never gets to 377, it indicates that NTP requests or responses are getting lost in the network. Some major network operators are known to have middleboxes that block or limit rate of large NTP packets as a mitigation for amplification attacks that exploit the monitoring protocol of <em>ntpd</em>. Unfortunately, this impacts NTS-protected NTP packets, even though they don’t cause any amplification. The NTP working group is considering an alternative port for NTP as a workaround for this issue.</p>
<h2>Enable NTS on the server</h2>
<p>If you have your own NTP server running <em>chronyd</em>, you can enable server NTS support to allow its clients to be synchronized securely. If the server is a client of other servers, it should use NTS or a symmetric key for its own synchronization. The clients assume the synchronization chain is secured between all servers up to the primary time servers.</p>
<p>Enabling server NTS is similar to enabling HTTPS on a web server. You just need a private key and certificate. The certificate could be signed by the Let’s Encrypt authority using the <em>certbot</em> tool, for example. When you have the key and certificate file (including intermediate certificates), specify them in <em>chrony.conf</em> with the following directives:</p>
<pre class="wp-block-preformatted">ntsserverkey /etc/pki/tls/private/foo.example.net.key
ntsservercert /etc/pki/tls/certs/foo.example.net.crt</pre>
<p>Make sure the <em>ntsdumpdir</em> directive mentioned previously in the<br />
client configuration is present in <em>chrony.conf</em>. It allows the server<br />
to save its keys to disk, so the clients of the server don’t have to get new<br />
keys and cookies when the server is restarted.</p>
<p>Restart the <em>chronyd</em> service:</p>
<pre class="wp-block-preformatted">systemctl restart chronyd</pre>
<p>If there are no error messages in the system log from <em>chronyd</em>, it should be<br />
accepting client connections. If the server has a firewall, it needs to allow<br />
both the UDP 123 and TCP 4460 ports for NTP and NTS-KE respectively.</p>
<p>You can perform a quick test from a client machine with the following command:</p>
<pre class="wp-block-preformatted">$ <strong>chronyd -Q -t 3 'server foo.example.net iburst nts maxsamples 1'</strong>
2020-10-13T12:00:52Z chronyd version 4.0 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 +DEBUG)
2020-10-13T12:00:52Z Disabled control of system clock
2020-10-13T12:00:55Z System clock wrong by -0.001032 seconds (ignored)
2020-10-13T12:00:55Z chronyd exiting</pre>
<p>If you see a <em>System clock wrong</em> message, it’s working<br />
correctly.</p>
<p>On the server, you can use the following command to check how many NTS-KE<br />
connections and authenticated NTP packets it has handled:</p>
<pre class="wp-block-preformatted"># <strong>chronyc serverstats</strong>
NTP packets received : 2143106240
NTP packets dropped : 117180834
Command packets received : 16819527
Command packets dropped : 0
Client log records dropped : 574257223
NTS-KE connections accepted: 104
NTS-KE connections dropped : 0
Authenticated NTP packets : 52139</pre>
<p>If you see non-zero <em>NTS-KE connections accepted</em> and <em>Authenticated<br />
NTP packets</em>, it means at least some clients were able to connect to the<br />
NTS-KE port and send an authenticated NTP request.</p>
<p class="has-text-align-right"><em>— Cover photo by Louis. K on Unsplash —</em></p>
<hr class="wp-block-separator is-style-wide" />
<p style="font-size: 50%"><a id="footnote1">1</a>. The Fedora 33 <em>Beta</em> installer contains an older chrony prerelease which doesn’t work with current NTS servers because the NTS-KE port has changed. Consequently, in the Network Time configuration in the installer, the servers will always appear as not working. After installation, the chrony package needs to be updated before it will work with current servers.</p></p>
</div>
https://www.sickgaming.net/blog/2020/10/...-with-nts/