NTP Pool servers hosted at NetActuate

The NTP Pool consists of (as of this writing) more than 4000 NTP servers provided by the community, about 40 DNS servers and a good handful of “hub servers” running the website, databases, monitoring (for NTP, DNS, etc) and a bunch of other software to keep the system going.

This spring we learned that the facility the systems were in was being decommissioned and we needed to find other arrangements, quickly. A few weeks later we had some options lined up and a long list of work to make it a smooth migration.

We worked at a frantic pace for a week to prepare for moving the physical servers by the deadline without any service impact (more on that later) and a Sunday in early may we got everything un-installed and moved to a NetActuate in Los Angeles.

By the end of the day we had the network up and the equipment racked and relatively neatly cabled (we counted it’s about the 5th move in about 20 years of maintaining various Perl community infrastructure and about 14 years of the NTP Pool, so we only get sporadic practice!).

When we first got in touch, NetActuate very quickly understood what we needed and figured out how to make it work on time and budget. Their services align unusually closely with the problem domain of the NTP Pool, with many of their customers using them to host anycast services across 70+ worldwide facilities.

NetActuate has been extremely easy and quick to work with and we are looking forward to a long and productive relationship with them. They are a small, friendly and highly technical company. Our experience is that everyone we talked to were experts at what they did and always the “real people” doing the work.

The hectic move also taught us that for something that so many of you in the community and beyond depend on working, we should get a bit more redundancy, so more on that in the next post.