eNom data center migration mistakenly knocks sites offline

yoursunny · January 2022

eNom domain registrar, which manages over 15 million domains, is experiencing a botched server migration resulting in all domains using its servers being down for over a 24 hour period.
Maintenance window has been extended multiple times, and the issue is still ongoing.

Customer websites are down.
Emails are not delivering.

"What's absolutely nuts about this is we can not change our NameServers due to eNom itself is down (going on 24 hours now). Never in my life have I been in a situation where I can't route around network issue at the root domain records level. We are totally trapped!"
eNom customer

One of the affected websites is Certitudes, a geocaching puzzle checker, which creates great difficulty for geocachers around the world.

vyas · January 2022

That is a lot of (bull) shit Tucows owned company dropped from its behind

cybertech · January 2022

time to migrate to NameCrane.

bsh · January 2022

@Francisco NameCrane?

vyas · December 2022

@vyas said:
That is a lot of (bull) shit Tucows owned company dropped from its behind

https://twitter.com/RetroTechDreams/status/1605705825104039938

Encoders · December 2022

this is why you don't put your NS along with your domain registrar kids

root · December 2022

@vyas

vyas · December 2022

@root said:
@vyas

Still the same year !!!

TBF I ran for a search on Tucows in Vanilla and posted in the first result.

Lazy posting on my part

AuroraZero · December 2022

@Encoders said:
this is why you don't put your NS along with your domain registrar kids

So this happens alot with domain registars then? I am curious honestly, been doing this a few days now and would like to know why this would any different then adding them to say cloudflare, he.net, etc...

FrankZ · December 2022

@AuroraZero said:

@Encoders said:
this is why you don't put your NS along with your domain registrar kids

So this happens alot with domain registars then? I am curious honestly, been doing this a few days now and would like to know why this would any different then adding them to say cloudflare, he.net, etc...

Realistically everybody can go down. I think the point of @Encoders comment as I read it was that when you have you dns at the registrar and they go down you can't change your dns servers because they are down. When you have your own dns servers or use an outside service and it goes down, for a longer then acceptable time, you can go to the registrar and change the name servers for your domain(s). If the registrar is down, and you use an outside service for your dns that is up, there is no problem resolving your domains because your dns servers are still shown at the root servers.

AuroraZero · December 2022

@FrankZ said:

@AuroraZero said:

@Encoders said:
this is why you don't put your NS along with your domain registrar kids

So this happens alot with domain registars then? I am curious honestly, been doing this a few days now and would like to know why this would any different then adding them to say cloudflare, he.net, etc...

Realistically everybody can go down. I think the point of @Encoders comment as I read it was that when you have you dns at the registrar and they go down you can't change your dns servers because they are down. When you have your own dns servers or use an outside service and it goes down, for a longer then acceptable time, you can go to the registrar and change the name servers for your domain(s). If the registrar is down, and you use an outside service for your dns that is up, there is no problem resolving your domains because your dns servers are still shown at the root servers.

Here is my point my friend, even if you change your DNS to your own DNS it has to go through the registar. So if your own DNS goes that ok you can change it but if the registar goes down you are still screwed. The main problem with DNS is the single point of failure at the registar. No matter what you do you have to use the registar to point your domain DNS to an outside source. So if the registar goes down they can not proxy that through to the DNS service you have selected.

FrankZ · December 2022

@AuroraZero said:

@FrankZ said:

@AuroraZero said:

@Encoders said:
this is why you don't put your NS along with your domain registrar kids

So this happens alot with domain registars then? I am curious honestly, been doing this a few days now and would like to know why this would any different then adding them to say cloudflare, he.net, etc...

Realistically everybody can go down. I think the point of @Encoders comment as I read it was that when you have you dns at the registrar and they go down you can't change your dns servers because they are down. When you have your own dns servers or use an outside service and it goes down, for a longer then acceptable time, you can go to the registrar and change the name servers for your domain(s). If the registrar is down, and you use an outside service for your dns that is up, there is no problem resolving your domains because your dns servers are still shown at the root servers.

Here is my point my friend, even if you change your DNS to your own DNS it has to go through the registar. So if your own DNS goes that ok you can change it but if the registar goes down you are still screwed. The main problem with DNS is the single point of failure at the registar. No matter what you do you have to use the registar to point your domain DNS to an outside source. So if the registar goes down they can not proxy that through to the DNS service you have selected.

That is not correct as far as I understand it. Your dns servers are not proxied through your registrar if you are not using your registrar as your dns service. The registrar is in effect holding your domain and makes submissions to .com or who ever on you behalf that need to go through the registrar, but the DNS root servers are where the DNS information is held for DNS routing, not at the registrar or .com. The root servers receive the DNS delegation information from the chain I described above but after that delegation is made to the root DNS servers there is not a registrar proxie involved in resolving your domain name for a user.

yoursunny · December 2022

@FrankZ said:
The registrar is in effect holding your domain and makes submissions to .com or who ever on you behalf that need to go through the registrar, but the DNS root servers are where the DNS information is held for DNS routing, not at the registrar or .com. The root servers receive the DNS delegation information from the chain I described above but after that delegation is made to the root DNS servers there is not a registrar proxie involved in resolving your domain name for a user.

If you are not using registrar name server,a downtime at the registrar won't affect the DNS resolution of your domain.
However, for a .com domain, the .com registry does participate in resolving your domain (step 2 below).

root-servers return the names and IP addresses of .com registry servers, called gtld-servers.
gtld-servers return the names and IP addresses of the name servers that your configured, e.g. Cloudflare.
The name servers of your domain return the A/AAAA records pointing to your web server.

The registry servers can go down, although it's unlikely for .com registry.
See also How I hijacked the top-level domain of a sovereign state.

FrankZ · December 2022

@yoursunny said:

@FrankZ said:
The registrar is in effect holding your domain and makes submissions to .com or who ever on you behalf that need to go through the registrar, but the DNS root servers are where the DNS information is held for DNS routing, not at the registrar or .com. The root servers receive the DNS delegation information from the chain I described above but after that delegation is made to the root DNS servers there is not a registrar proxie involved in resolving your domain name for a user.

If you are not using registrar name server,a downtime at the registrar won't affect the DNS resolution of your domain.
However, for a .com domain, the .com registry does participate in resolving your domain (step 2 below).

root-servers return the names and IP addresses of .com registry servers, called gtld-servers.

gtld-servers return the names and IP addresses of the name servers that your configured, e.g. Cloudflare.

The name servers of your domain return the A/AAAA records pointing to your web server.

The registry servers can go down, although it's unlikely for .com registry.
See also How I hijacked the top-level domain of a sovereign state.

I appreciate your more articulate and concise explanation. But I am not sure if you are agreeing or disagreeing with the premise of my statement, that eNom, namesilo, namecheap etc would not have any involvement in the DNS resolution for a user if their website and DNS servers were down, and you were not using eNom, namesilo, namecheap etc, DNS servers for resolution of your domain name.

yoursunny · December 2022

@FrankZ said:

@yoursunny said:

@FrankZ said:
The registrar is in effect holding your domain and makes submissions to .com or who ever on you behalf that need to go through the registrar, but the DNS root servers are where the DNS information is held for DNS routing, not at the registrar or .com. The root servers receive the DNS delegation information from the chain I described above but after that delegation is made to the root DNS servers there is not a registrar proxie involved in resolving your domain name for a user.

If you are not using registrar name server,a downtime at the registrar won't affect the DNS resolution of your domain.
However, for a .com domain, the .com registry does participate in resolving your domain (step 2 below).

root-servers return the names and IP addresses of .com registry servers, called gtld-servers.

gtld-servers return the names and IP addresses of the name servers that your configured, e.g. Cloudflare.

The name servers of your domain return the A/AAAA records pointing to your web server.

The registry servers can go down, although it's unlikely for .com registry.
See also How I hijacked the top-level domain of a sovereign state.

I appreciate your more articulate and concise explanation. But I am not sure if you are agreeing or disagreeing with the premise of my statement, that eNom, namesilo, namecheap etc would not have any involvement in the DNS resolution for a user if their website and DNS servers were down, and you were not using eNom, namesilo, namecheap etc, DNS servers for resolution of your domain name.

eNom et al are not involved in the critical path.

The statement "the DNS root servers are where the DNS information is held for DNS routing, not at the registrar or .com" is wrong, because .com registry (gtld-servers) are involved.

xleet · December 2022

Cloudflare offers domain registrar services, but you must use Cloudflare as the authoritative DNS servers for your domain. In return, they register and renew domain registrations at their cost. You must decide whether the financial benefits are worth the risk.

legendary · December 2022

@xleet said:
Cloudflare offers domain registrar services, but you must use Cloudflare as the authoritative DNS servers for your domain. In return, they register and renew domain registrations at their cost. You must decide whether the financial benefits are worth the risk.

You are not obligated to use cloudflare name servers. Just pay montlhy 250$ fee and you are free to change ns.

root · December 2022

I never had such issues, but I feel the need to ask, as we might learn something important.

If a registrar goes broke over night, what is the procedure for recovering the domain?
If a registrar has it's servers down for more than 24 hours, what is the procedure for transferring the domain (locked and without EPP code)?
What is the procedure of changing nameservers without access to registrar (if there is a backdoor procedure like this) in case their panel is down?
If access to registrar is lost (lost password and lost 2FA) what is the procedure to prove ownership and transfer the domain to another registrar?

yoursunny · December 2022

@root said:
If a registrar has it's servers down for more than 24 hours, what is the procedure for transferring the domain (locked and without EPP code)?
What is the procedure of changing nameservers without access to registrar (if there is a backdoor procedure like this) in case their panel is down?

The registrar is not obligated to provide you online access to a panel.
You can write a letter, slap on a stamp, and deliver it to your registrar.
They are obligated to comply with your request within five days.

If a registrar goes broke over night, what is the procedure for recovering the domain?

You may be able to access the panel or send a letter to request transfer, or the registry may automatically transfer affected domains to another registrar.
See also: My domain registrar has gone bankrupt but the domain is locked, what can I do?

If access to registrar is lost (lost password and lost 2FA) what is the procedure to prove ownership and transfer the domain to another registrar?

Prove ownership to the existing registrar with documentation such as your passport.
Once you regain the account, you can request a domain transfer normally.
See also: Domain Transfer, WHOIS Privacy, DNSSEC, and the Absence of Push-ups

xleet · December 2022

@legendary said:
You are not obligated to use cloudflare name servers. Just pay montlhy 250$ fee and you are free to change ns.

True, but if you switched to Cloudflare to get "at cost" domain renewal fees, then it would take many many domains before the registration fee savings covers that $250 per month. Otherwise, it does not make much sense to me.

eNom data center migration mistakenly knocks sites offline

Comments