I don't think you can get constant 25MB/s on HTML files - it would need to be a lot of images/videos, big images or like hundreds of requests per second.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Here's a friggin' surprise, not:
ec2-47-128-96-140.ap-southeast-1.compute.amazonaws.com
What really pisses me off are the number of 'legitimate' services that use Hetzner and Amazon compute. Otherwise, I'd block their complete ranges - tried it once and screwed up a lot of software updates. Both providers harbour real problematic nefarious assholes. (Not sitting on the fence. )
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@lesuser said: takes database and file backup and rsync it to another server every 30 minutes.
Have you monitored the size of these, let's face it, fairly frequent bundles? Do you periodically run full backups (say twice per day) and then differential ones? You may be better served by direct (differential) rsync of the files and remote database replication.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@lesuser What about blocking non-Cloudflare IPs with ufw?
for url in "https://www.cloudflare.com/ips-v4" "https://www.cloudflare.com/ips-v6"; do
curl -s $url | while read ip; do
sudo ufw allow from $ip to any port 443 comment "Cloudflare IP"
done
done
Seems like QN LAX having issues again. I'm trying to get into the facility so if anyone's there to let me in I'll give you guys some live updates on whether everything's on fire, otherwise I'll perform some maintenance while it's down anyway (mostly applying paste.)
Based on how slow they are at responding right now and probably receiving an insane volume of tickets though I'm not sure if I'll be able to get in. I'll still (maybe) try. With this strategy:
@VirMach said:
Seems like QN LAX having issues again. I'm trying to get into the facility so if anyone's there to let me in I'll give you guys some live updates on whether everything's on fire, otherwise I'll perform some maintenance while it's down anyway (mostly applying paste.)
Based on how slow they are at responding right now and probably receiving an insane volume of tickets though I'm not sure if I'll be able to get in. I'll still (maybe) try. With this strategy:
Looks like there's been a lot of complaints with QN lately.
You can block both in and out traffic for all ports using ufw and whitelist one port for Cloudflare ZeroTrust so you can access both HTTP and HTTPS with it.
I have VMs that are not available on the interweb on nodes LAXA031, LAXA029 and LAXA026. Others on nodes LAXA022 and LAXA019 are available. I can ping the VMs that are down from the ones that are up so ......
@FrankZ said:
I have VMs that are not available on the interweb on nodes LAXA031, LAXA029 and LAXA026. Others on nodes LAXA022 and LAXA019 are available. I can ping the VMs that are down from the ones that are up so ......
When I run mtr 172.98.13.** -w -c 10 with either --tcp or --udp towards my VM on LAXA030, I get a 100% packet loss...
Any updates on LA network situation? Even tho I did not received any notifications on my uptime monitors, some of my VPS is still not accessible to some parts of the world...
At this point I'm putting on my conspiracy theory hat.
Looks like they decided to consolidate IP addresses. We don't use fully QN addresses for servers, we actually have it set up in a way where we can quickly switch to our own, except I have to be able to access IPMI and that's another issue at this point related to the other partial routing problems.
Anyway, the way the people are talking about it, seems like the new people who took over just decided to do it this way. I can't think of any reason it'd be done this way unless they just bought QN for the IPv4 and really really just want people to move out so they have some extra space. Seems to be the datacenter version of buying a building and wanting to kick out the tenants to make more money except you have to do it in a shady way to make sure there's still "technically" connectivity and whatever else.
Like there's no way upper upper management just thought okay cool let's make sure we keep the same staff, do a major consolidation of an entire datacenter we purchased, and while we do that, let's also do maintenance on the network, and then also renumber everyone but like in a really weird way where we send no notifications out to maximize the headache for everyone. It'd be understandable (coming from personal experience) if these weren't literally their addresses. It's not like they have some issue with the people leasing it to them. Or that they're even switching them to some other addresses they're leasing. Like they definitely just wanted to have a bunch of /24 blocks and not have it partially used by one customer as a /28 or whatever, it doesn't even seem like the people doing the work understand what's going on. No mention of it whatsoever last time when we contacted them, like it'd be a really good time for them to just go "oh by the way we don't know when or why, but your IP blocks might get revoked WHILE we also have connectivity issues that makes it difficult for you to even access IPMI so maybe I don't know do whatever you want." Or even let me access the facility and do my own work. They closed my ticket for facility access with no reply.
I'll basically paraphrase this from the last time I spoke with someone in another company that was behaving in a self-destructive way. We can technically do our best to deal with it all, but most other companies that pay you probably aren't going to put up with it at all. And once that happens, unless a bigger fish just keeps the cycle of infinite acquisitions going, it's going to end as a 12 hour notice type deal like Dedipath.
That's my way of saying I'm looking into what we need to do as a more permanent solution to the problem (Edge Centres LLC.)
Oh god are they actually founded in 2021 and they're just buying cheap facilities and ruining them and just buying more? It's that type of company? Hahaha and their tagline or whatever, name, "AI based data center solutions for edge computing." They're Edging investors with their SEO.
@VirMach said: Oh god are they actually founded in 2021 and they're just buying cheap facilities and ruining them and just buying more?
Maybe vulture capitalism has made another appearance. I saw this going on up close in the '80s. Make a holding company, buy other companies, sell off all the parts, put off paying vendors for as long as you can, put the secondary companies into bankruptcy, profit.
@VirMach said: Their CEO's name is Jon the end is nigh.
@grasple said:
there is an elegance to buying up tier-7 DCs just to plunder their legacy IPv4 space, the dirt bag capitalism 2025 engenders
Someone at WHT brought up the prices IPv4 are going for. Suppose the new owners got a few million dollars for the IP space at a cost of their customers and reputation. I can't imagine the owners will break even on the deal considering how much they must've spent to purchase the business -- or even just to purchase the building.
@grasple said:
there is an elegance to buying up tier-7 DCs just to plunder their legacy IPv4 space, the dirt bag capitalism 2025 engenders
Someone at WHT brought up the prices IPv4 are going for. Suppose the new owners got a few million dollars for the IP space at a cost of their customers and reputation. I can't imagine the owners will break even on the deal considering how much they must've spent to purchase the business -- or even just to purchase the building.
maybe. or maybe they didn't "purchase" a building, but bought an interest in a lease, or their veiled corp financed the purchase of property and can default without recourse etc etc, maybe they paid an upfront price and offered the previous owners earnouts etc.
they definitely seem uninterested in running a DC well, so it is fun to speculate why they bought a DC.
@grasple said: maybe. or maybe they didn't "purchase" a building, but bought an interest in a lease, or their veiled corp financed the purchase of property and can default without recourse etc etc, maybe they paid an upfront price and offered the previous owners earnouts etc.
That's a fair point. QN probably had an unusual structured lease-back agreement, which makes sense if they don't want to be managing property all day. Either way, the money they’d get for the IP space pales in comparison to the value of the business they just destroyed.
I mean, selling off the IPs you assigned to your customers isn't great. Doing it without warning is profoundly stupid.
Virmach did get my VM on Node: NYCB032 migrated to Node: NYCB028 and it is back online with new IP. So that is a big
For those of you who were on NYCB032 check the billing panel to see if you are also up. I did not receive any email notice.
@VirMach Is there a particular reason there is no information at all on the network status page with respect to LAX2Z017 being down. From this thread I understand LAX has issues, but IDK why the status page would not be telling customers of that. Ticket creation just seems to hang when it tries to read the service screen.
@tetech said:
Meanwhile, is there a plan on what to do next with the non-functional VPSes in LA? Or still up in the air?
Still waiting for them to even begin to respond to our request properly. They should all be functional to some degree, just not globally, and too many variables plus we're pretty blind on it because of the previous issue I mentioned above.
@tulipyun said:
Quadranet hasn't updated its status page in three days.
VirMach apparently did better than they did, they emptied their status page about 6 hours ago.
They say “There are no Network Issues Currently”.
There's no useful information we could provide on the status page and we would be unable to properly communicate to everyone who is affected on what routes. The system already displays the relevant state of the service in regards to controls being down. As in what's stated there is accurate so there does not need to be a status update re-stating the same thing. I don't believe we've added status updates (usually, recently) any time there are control issues unless it's not actually a control issue and therefore it needs to be corrected/clarified.
If you guys disagree with that approach for the controls portion, let me know.
We already have a large ticket backlog which we're going to use instead in this case rather than referring customers to the network status page, and actually go through them one by one for each specific problem reported once (a) we actually have some communication from QN on the networking portion, and (b) once we have some eyes, aka, they need to complete our request so we have our access back.
If you guys disagree with that approach and think we should instead post a network status update and then close all tickets and refer everyone to the network status page, let me know, but I think that'd be the wrong way to do it here. We're not going to both post a network status update and go through tickets one by one though, it's one or the other. I know it's not ideal we're not replying to every ticket to confirm we've received it and we're not just doing nothing about it, except, well, we are doing nothing about it right now because we literally cannot do anything until QN fulfills our request. So we could reply to every single ticket and inform customers we are currently doing nothing about it but that will just also be bad and unhelpful and then we'll have everyone responding to complain which will derail all the tickets from focusing on the actual issue once we can do something about it since customers we'll have to treat the tickets both as potential complaints that need to be responded to, as well as reported issues.
@grasple said: maybe. or maybe they didn't "purchase" a building, but bought an interest in a lease, or their veiled corp financed the purchase of property and can default without recourse etc etc, maybe they paid an upfront price and offered the previous owners earnouts etc.
That's a fair point. QN probably had an unusual structured lease-back agreement, which makes sense if they don't want to be managing property all day. Either way, the money they’d get for the IP space pales in comparison to the value of the business they just destroyed.
I mean, selling off the IPs you assigned to your customers isn't great. Doing it without warning is profoundly stupid.
I really don't think they sold off any IP space, yet. I think the people that bought QN really thought it was an amazing idea to consolidate the addresses so they can make their own jobs easier in the future when they want to use/assign IP addresses to their customers under their primary brand or whatever. They probably had this as a thing they had to get done in Q4 2024, missed it, and tried just rushing it Q1 2025 instead. Some higher up probably said "I don't care, just get it done now" so they did.
But I guess no one thought that... well, no one did any thinking. They forgot there's customers that use these IP addresses for IPMI (we're not currently using them for IPMI, technically still possible to get in, it's just not going to be feasible without spending the entire day only doing that, but I won't get into it, still trying. I'll be focusing on that more heavily now that I know QN just intends on pretending to respond but not actually able to do anything because not enough staff members exist that actually can do things.)
They're definitely regretting it right now after finally maybe thinking about the implications of what they tried doing without thinking because now it's a lot of credits, a lot of overtime pay, and a lot of general headache for them even if they do not care about the customers and only care about themselves.
As for the value of the IP space paling in comparison, I really don't think so. They seem like the type of business where the entire value is just the IP addresses and everything else is seen as a liability, at least to someone purchasing them because they do not charge a lot for their services. The rest is probably just a "bonus" but they do still need to obviously maintain whatever they're receiving to keep the lights on at the facilities and keep everyone on payroll. Once they actually use the IP addresses for whatever other thing they're trying to do and get in the customers they want to get in, they'll probably be trying to think of ways to get rid of these existing customers. I've spoken with them, they really did not seem interested in having us as a customer when we reached out to their primary brand, in fact they straight up just said they didn't even want to give us a quote or discuss anything basically in the nicest way they could. They're focusing on getting big companies in. Like really big. I disagree with that approach, but that does seem to be their approach. So they really would probably care even less if you're renting a small old Xeon server from them.
I have a lot of thoughts on the matter so what I said isn't perfect and it's oversimplified and of course it's all speculation and just how I feel based on a lot of other things I didn't explain. It could be completely wrong.
make official announcement about LAX and update its progress
answer all tickets about LAX and point them to the announcement and close the ticket so all affected can monitor it on the correct page without draining more support resources
send email to all LAX affected so those who didnt even know if their vps was down, now know
Here's basically what I've verified so far though, it might be useful to you guys.
Nothing is "down." At least based on the spot checking I've done, if your VPS was online before any of this occurred, as in you do not require controls and nothing was modified/broken specifically by you inside your virtual server since then (and you locked yourself out and require VNC) then it should have connectivity. There's a small chance I missed some very specific thing, but servers are powered on, and the way it's set up, all virtual servers have connectivity.
There ARE severe routing problems, as in dead routes. So depending on where you are accessing the service from and if that specific routing is broken or not, it might appear as if you are completely down. What you could do if you really need access to your service, instead of waiting for us to reply to your ticket right now, and if you want to try something on your own, is to try accessing it from another route/location so a VPN, a different server, and it should work. I'd say there's above a 50% chance each time you try as long as it's fully randomized (so you aren't just trying from the same network just on a different VPS.)
I'm going to disappear for a while to really try to focus on some workarounds to try to do this ourselves as it's been over 12 hours where we haven't even received a pretend response from them and it doesn't seem like anything is going to get done as I mentioned above. This is just so we can try to get controls working again and assess everything further, it does not mean I can do anything about the routing issues on your VPS specifically, nor can I do anything if there are also routing issues for the controls. I mean I guess if it comes down to that I could also do further workarounds but let's get through this first hoop.
@FrankZ said:
I have VMs that are not available on the interweb on nodes LAXA031, LAXA029 and LAXA026. Others on nodes LAXA022 and LAXA019 are available. I can ping the VMs that are down from the ones that are up so ......
So for example, I've 100% verified that all the nodes you listed here are working in the way I described above. However, the routing issues are NOT the same on each that's tested, otherwise everything I described vaguely above would be much simpler. This means for example, a random IP tested on LAXA031 has different routing issues than a random IP tested on LAXA029, and so on. For example one can be accessed through Cogent in Denver, and the other can be accessed through HE in Los Angeles but not vice versa.
Comments
I don't think you can get constant 25MB/s on HTML files - it would need to be a lot of images/videos, big images or like hundreds of requests per second.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Here's a friggin' surprise, not:
ec2-47-128-96-140.ap-southeast-1.compute.amazonaws.com
What really pisses me off are the number of 'legitimate' services that use Hetzner and Amazon compute. Otherwise, I'd block their complete ranges - tried it once and screwed up a lot of software updates. Both providers harbour real problematic nefarious assholes. (Not sitting on the fence.
)
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Have you monitored the size of these, let's face it, fairly frequent bundles? Do you periodically run full backups (say twice per day) and then differential ones? You may be better served by direct (differential) rsync of the files and remote database replication.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Am I misremembering or has VirMach not been providing a stats chart for cpu usage on the statistics page?
@lesuser What about blocking non-Cloudflare IPs with ufw?
Seems like QN LAX having issues again. I'm trying to get into the facility so if anyone's there to let me in I'll give you guys some live updates on whether everything's on fire, otherwise I'll perform some maintenance while it's down anyway (mostly applying paste.)
Based on how slow they are at responding right now and probably receiving an insane volume of tickets though I'm not sure if I'll be able to get in. I'll still (maybe) try. With this strategy:
Looks like there's been a lot of complaints with QN lately.
https://lowendtalk.com/discussion/comment/4279444/#Comment_4279444
This thread is a good read.
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
@lesuser
You can block both in and out traffic for all ports using ufw and whitelist one port for Cloudflare ZeroTrust so you can access both HTTP and HTTPS with it.
This works for me.
https://microlxc.net/
Get yourself a luxury vehicle and you'll be the first one getting in.
We accept Karma donations for the last flan. 🍮 affbrr
LA is already in a quantum superposition state
I have VMs that are not available on the interweb on nodes LAXA031, LAXA029 and LAXA026. Others on nodes LAXA022 and LAXA019 are available. I can ping the VMs that are down from the ones that are up so ......
When I run
mtr 172.98.13.** -w -c 10with either--tcpor--udptowards my VM on LAXA030, I get a 100% packet loss...mtr 172.98.13.** -w -c 10mtr 172.98.13.** -w -c 10 --tcpnameserver 8.8.8.8
nameserver 8.8.4.4
Los Angeles It does have Settings, but they can't resolve domain names.
my LAX VM is online but control panel down. no sweat tho!
I bench YABS 24/7/365 unless it's a leap year.
7 x 7 x 7
Have you tried turning it off and on again?Any updates on LA network situation? Even tho I did not received any notifications on my uptime monitors, some of my VPS is still not accessible to some parts of the world...
You probably saw me somewhere...
VirMach Tokyo Node RYZE.TYO-C033.VMS, IPv6 network is broken after maintenance on 2025/01/25, please fix it, thank you!
RIP LA.
This multi node network issue is getting to be a bit long in the tooth.
Wondering when there is going to be an update.
EDIT: Found update here
Will QuadraNet repeat the mistakes of Dedipath? :'(
This may be a mistake of switching from /24 to /26 by @VirMach
Virmach Deals
hostcram says QN is DOWN down. seems like Virmach has a lot to move
I bench YABS 24/7/365 unless it's a leap year.
https://lowendtalk.com/discussion/202244/quadranet-la-is-offline-since-23-jan-2024
Looks like everyone has started to move out
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
Virmach has Psychz and Colocrossing new LA location to move to. no forever friends and foes?
I bench YABS 24/7/365 unless it's a leap year.
Would love to see a presence in Multacom too, they are truly Asia (and kinda China) Optimized.
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
https://status.quadranet.com/
highest possibility is Virmach will wait for QN to get back online.
I bench YABS 24/7/365 unless it's a leap year.
At this point I'm putting on my conspiracy theory hat.
Looks like they decided to consolidate IP addresses. We don't use fully QN addresses for servers, we actually have it set up in a way where we can quickly switch to our own, except I have to be able to access IPMI and that's another issue at this point related to the other partial routing problems.
Anyway, the way the people are talking about it, seems like the new people who took over just decided to do it this way. I can't think of any reason it'd be done this way unless they just bought QN for the IPv4 and really really just want people to move out so they have some extra space. Seems to be the datacenter version of buying a building and wanting to kick out the tenants to make more money except you have to do it in a shady way to make sure there's still "technically" connectivity and whatever else.
Like there's no way upper upper management just thought okay cool let's make sure we keep the same staff, do a major consolidation of an entire datacenter we purchased, and while we do that, let's also do maintenance on the network, and then also renumber everyone but like in a really weird way where we send no notifications out to maximize the headache for everyone. It'd be understandable (coming from personal experience) if these weren't literally their addresses. It's not like they have some issue with the people leasing it to them. Or that they're even switching them to some other addresses they're leasing. Like they definitely just wanted to have a bunch of /24 blocks and not have it partially used by one customer as a /28 or whatever, it doesn't even seem like the people doing the work understand what's going on. No mention of it whatsoever last time when we contacted them, like it'd be a really good time for them to just go "oh by the way we don't know when or why, but your IP blocks might get revoked WHILE we also have connectivity issues that makes it difficult for you to even access IPMI so maybe I don't know do whatever you want." Or even let me access the facility and do my own work. They closed my ticket for facility access with no reply.
I'll basically paraphrase this from the last time I spoke with someone in another company that was behaving in a self-destructive way. We can technically do our best to deal with it all, but most other companies that pay you probably aren't going to put up with it at all. And once that happens, unless a bigger fish just keeps the cycle of infinite acquisitions going, it's going to end as a 12 hour notice type deal like Dedipath.
That's my way of saying I'm looking into what we need to do as a more permanent solution to the problem (Edge Centres LLC.)
Oh god are they actually founded in 2021 and they're just buying cheap facilities and ruining them and just buying more? It's that type of company? Hahaha and their tagline or whatever, name, "AI based data center solutions for edge computing." They're Edging investors with their SEO.
Their CEO's name is Jon the end is nigh.
Fuck
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Maybe vulture capitalism has made another appearance. I saw this going on up close in the '80s. Make a holding company, buy other companies, sell off all the parts, put off paying vendors for as long as you can, put the secondary companies into bankruptcy, profit.
Just like what VirBot does when the customer doesn't pay $15 priority support fee.
We accept Karma donations for the last flan. 🍮 affbrr
there is an elegance to buying up tier-7 DCs just to plunder their legacy IPv4 space, the dirt bag capitalism 2025 engenders
Someone at WHT brought up the prices IPv4 are going for. Suppose the new owners got a few million dollars for the IP space at a cost of their customers and reputation. I can't imagine the owners will break even on the deal considering how much they must've spent to purchase the business -- or even just to purchase the building.
maybe. or maybe they didn't "purchase" a building, but bought an interest in a lease, or their veiled corp financed the purchase of property and can default without recourse etc etc, maybe they paid an upfront price and offered the previous owners earnouts etc.
they definitely seem uninterested in running a DC well, so it is fun to speculate why they bought a DC.
They bought IPs, AS8100, and crappy little office next to One Wilshire that was called a 'data center' - Been there myself.
Networking person, Kait, who knew anything left long ago to NetActuate.
It's like CC but handled more poorly. HP are already leasing those IPs on IPXO, but Buffalo's still running as fine as it can.
That's a fair point. QN probably had an unusual structured lease-back agreement, which makes sense if they don't want to be managing property all day. Either way, the money they’d get for the IP space pales in comparison to the value of the business they just destroyed.
I mean, selling off the IPs you assigned to your customers isn't great. Doing it without warning is profoundly stupid.
They had floors of an adjacent slap dash office building. Some of which had cooling issues.
Meanwhile, is there a plan on what to do next with the non-functional VPSes in LA? Or still up in the air?
Quadranet hasn't updated its status page in three days.
VirMach apparently did better than they did, they emptied their status page about 6 hours ago.
They say “There are no Network Issues Currently”.
mmhmm, more customers left, more free IP space
Virmach did get my VM on Node: NYCB032 migrated to Node: NYCB028 and it is back online with new IP. So that is a big
For those of you who were on NYCB032 check the billing panel to see if you are also up. I did not receive any email notice.
@VirMach Is there a particular reason there is no information at all on the network status page with respect to LAX2Z017 being down. From this thread I understand LAX has issues, but IDK why the status page would not be telling customers of that. Ticket creation just seems to hang when it tries to read the service screen.
The SLA credits will be insane.
We accept Karma donations for the last flan. 🍮 affbrr
thats only if you paid
I bench YABS 24/7/365 unless it's a leap year.
Still waiting for them to even begin to respond to our request properly. They should all be functional to some degree, just not globally, and too many variables plus we're pretty blind on it because of the previous issue I mentioned above.
There's no useful information we could provide on the status page and we would be unable to properly communicate to everyone who is affected on what routes. The system already displays the relevant state of the service in regards to controls being down. As in what's stated there is accurate so there does not need to be a status update re-stating the same thing. I don't believe we've added status updates (usually, recently) any time there are control issues unless it's not actually a control issue and therefore it needs to be corrected/clarified.
If you guys disagree with that approach for the controls portion, let me know.
We already have a large ticket backlog which we're going to use instead in this case rather than referring customers to the network status page, and actually go through them one by one for each specific problem reported once (a) we actually have some communication from QN on the networking portion, and (b) once we have some eyes, aka, they need to complete our request so we have our access back.
If you guys disagree with that approach and think we should instead post a network status update and then close all tickets and refer everyone to the network status page, let me know, but I think that'd be the wrong way to do it here. We're not going to both post a network status update and go through tickets one by one though, it's one or the other. I know it's not ideal we're not replying to every ticket to confirm we've received it and we're not just doing nothing about it, except, well, we are doing nothing about it right now because we literally cannot do anything until QN fulfills our request. So we could reply to every single ticket and inform customers we are currently doing nothing about it but that will just also be bad and unhelpful and then we'll have everyone responding to complain which will derail all the tickets from focusing on the actual issue once we can do something about it since customers we'll have to treat the tickets both as potential complaints that need to be responded to, as well as reported issues.
I really don't think they sold off any IP space, yet. I think the people that bought QN really thought it was an amazing idea to consolidate the addresses so they can make their own jobs easier in the future when they want to use/assign IP addresses to their customers under their primary brand or whatever. They probably had this as a thing they had to get done in Q4 2024, missed it, and tried just rushing it Q1 2025 instead. Some higher up probably said "I don't care, just get it done now" so they did.
But I guess no one thought that... well, no one did any thinking. They forgot there's customers that use these IP addresses for IPMI (we're not currently using them for IPMI, technically still possible to get in, it's just not going to be feasible without spending the entire day only doing that, but I won't get into it, still trying. I'll be focusing on that more heavily now that I know QN just intends on pretending to respond but not actually able to do anything because not enough staff members exist that actually can do things.)
They're definitely regretting it right now after finally maybe thinking about the implications of what they tried doing without thinking because now it's a lot of credits, a lot of overtime pay, and a lot of general headache for them even if they do not care about the customers and only care about themselves.
As for the value of the IP space paling in comparison, I really don't think so. They seem like the type of business where the entire value is just the IP addresses and everything else is seen as a liability, at least to someone purchasing them because they do not charge a lot for their services. The rest is probably just a "bonus" but they do still need to obviously maintain whatever they're receiving to keep the lights on at the facilities and keep everyone on payroll. Once they actually use the IP addresses for whatever other thing they're trying to do and get in the customers they want to get in, they'll probably be trying to think of ways to get rid of these existing customers. I've spoken with them, they really did not seem interested in having us as a customer when we reached out to their primary brand, in fact they straight up just said they didn't even want to give us a quote or discuss anything basically in the nicest way they could. They're focusing on getting big companies in. Like really big. I disagree with that approach, but that does seem to be their approach. So they really would probably care even less if you're renting a small old Xeon server from them.
I have a lot of thoughts on the matter so what I said isn't perfect and it's oversimplified and of course it's all speculation and just how I feel based on a lot of other things I didn't explain. It could be completely wrong.
you should do both:
I bench YABS 24/7/365 unless it's a leap year.
Here's basically what I've verified so far though, it might be useful to you guys.
Nothing is "down." At least based on the spot checking I've done, if your VPS was online before any of this occurred, as in you do not require controls and nothing was modified/broken specifically by you inside your virtual server since then (and you locked yourself out and require VNC) then it should have connectivity. There's a small chance I missed some very specific thing, but servers are powered on, and the way it's set up, all virtual servers have connectivity.
There ARE severe routing problems, as in dead routes. So depending on where you are accessing the service from and if that specific routing is broken or not, it might appear as if you are completely down. What you could do if you really need access to your service, instead of waiting for us to reply to your ticket right now, and if you want to try something on your own, is to try accessing it from another route/location so a VPN, a different server, and it should work. I'd say there's above a 50% chance each time you try as long as it's fully randomized (so you aren't just trying from the same network just on a different VPS.)
I'm going to disappear for a while to really try to focus on some workarounds to try to do this ourselves as it's been over 12 hours where we haven't even received a pretend response from them and it doesn't seem like anything is going to get done as I mentioned above. This is just so we can try to get controls working again and assess everything further, it does not mean I can do anything about the routing issues on your VPS specifically, nor can I do anything if there are also routing issues for the controls. I mean I guess if it comes down to that I could also do further workarounds but let's get through this first hoop.
So for example, I've 100% verified that all the nodes you listed here are working in the way I described above. However, the routing issues are NOT the same on each that's tested, otherwise everything I described vaguely above would be much simpler. This means for example, a random IP tested on LAXA031 has different routing issues than a random IP tested on LAXA029, and so on. For example one can be accessed through Cogent in Denver, and the other can be accessed through HE in Los Angeles but not vice versa.