Packet is awesome.

When we started planning our recent unplanned server move, we investigated options for having not one, but two sites, for the “hub” systems for the NTP Pool. With 4000 NTP servers and hundreds of millions of clients using the system, it really should be a given!

Evaluating our options on a ridiculously short timeframe, Packet stood out as an interesting choice, though we were a little apprehensive at first if their setup would be too unusual compared to more familiar options.

After a quick chat with some of the friendly staff at Packet, we were off to the races to see if we could get everything migrated in less than a week of nights and weekends. If we could, we’d be able to move the physical servers the following Sunday without downtime to any critical services, and get us closer to having proper redundancy.

Working with the Packet system has been fascinating and extremely productive. Despite having done this sort of work for several decades, it was a surprise how mixing familiar capabilities, APIs and abstractions opened new ways for quickly building and managing powerful, reliable and scalable infrastructure.

Continue Reading →

The beta site has been updated with new features for managing the accounts. Until now each server had to be associated with just one user login. In the new system servers are associated with an account that can have multiple user logins. If you can, please try it out and post bug reports or suggestions in the development forum or via email.

The NTP Pool consists of (as of this writing) more than 4000 NTP servers provided by the community, about 40 DNS servers and a good handful of “hub servers” running the website, databases, monitoring (for NTP, DNS, etc) and a bunch of other software to keep the system going. This spring we learned that the facility the systems were in was being decommissioned and we needed to find other arrangements, quickly.

Continue Reading →

As you might have seen in the news or from the US CERT, there’s an internal counter in the GPS messages that will “rollover” this week. Poorly implemented GPS receivers might lose track of time because of this. Many servers in the NTP Pool are using GPS signals to set the time, either directly or indirectly from another server that listens for GPS signals. The expected impact on users of the NTP Pool is very little or none at all.

Continue Reading →

Daniel Ziegenberg wrote a tutorial for Digital Ocean on configuring NTP for the NTP Pool on Ubuntu. Oliver Nadler has another tutorial covering non-Ubuntu, too.

There’s a new forum for discussion related to the NTP Pool at community.ntppool.org. Please come join us. There are a couple interesting threads about the recent leap second and lots of empty space for your questions or suggestions. :-)

Continue Reading →

Since last Tuesday some countries have seen an excessive number of queries to the NTP Pool.

After much detective work on nanog (conclusion) and the #NTP IRC channel it was determined to be a buggy Snapchat app update.

Continue Reading →

A network switch failed causing an outage for the management system and the NTP Pool website. The DNS and NTP services should only be minorly affected, even if the outage lasts a little while. I’ll update status website with updates.

The IETF has published a new version of NTP Best Current Practices documenting learned best practices on how to run NTP servers and clients.

If you know a little about the NTP protocol reading it will be a quick way to learn more about how it works “in the wild” (including on your own systems).

Continue Reading →

Thanks to the kind folks at statuspage.io we now have a system status page. It’s also available at an alternate domain.

Most of the data is updated manually, so it won’t be any better than the busy humans can manage, but it’ll be a better system for giving system updates than posting here (or on the dicussion mailing list).

Continue Reading →