@lesuser said:
One of my DediPath VPS starting with 47.xxx.xxx.xxx is still online while other one 45.xxx.xxx.xxx is offline.
In case it's useful to you so you can back up your stuff, you may just find spotty routing and might find a host you can copy your stuff to. I have a VPS with them in 45.42.214.0/23 and can route to/from some hosts, but not others:
traceroute to 8.8.4.4 (8.8.4.4), 30 hops max, 60 byte packets
1 45.42.215.1 2.298 ms 2.264 ms 2.288 ms
2 * * *
3 45.92.192.124 0.220 ms 0.228 ms 0.216 ms
4 74.201.164.149 0.261 ms 0.293 ms 64.74.240.241 0.238 ms
5 216.52.95.56 0.736 ms 0.723 ms 0.711 ms
6 198.232.115.145 1.511 ms * 1.646 ms
7 156.146.105.165 1.881 ms 1.911 ms 1.797 ms
8 * * *
9 8.8.4.4 2.405 ms 2.411 ms 2.437 ms
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
1 45.42.215.1 2.413 ms 2.380 ms 2.351 ms
2 45.11.82.30 12.637 ms 12.622 ms 12.604 ms
3 45.92.192.124 0.240 ms 0.223 ms 0.226 ms
4 74.201.164.149 4.122 ms 4.104 ms 4.127 ms
5 216.52.95.89 0.916 ms 216.52.95.25 0.732 ms 216.52.95.89 0.886 ms
6 * * *
7 * * *
8 * * *
9 * * *
odin@loki:1[~]$ ping -qnc 5 8.8.4.4
PING 8.8.4.4 (8.8.4.4) 56(84) bytes of data.
--- 8.8.4.4 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 1.907/1.942/1.985/0.029 ms
odin@loki:1[~]$ ping -qnc 5 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
--- 1.1.1.1 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4082ms
SolusVM was acting up, unfortunately it wasn't doing well trying to process all the changes, so notifications couldn't finish going on for the changes and they'll have to continue after re-racked.
@JoeMerit said:
My NY, Atlanta & Denver all died. I dont really mind because I backed everything up and moved some processes to other servers.
Kudos. This is a really good answer. There was a surprisingly long time to backup and move services given I thought everything DediPath was going to go down on Friday.
@JoeMerit said:
My NY, Atlanta & Denver all died. I dont really mind because I backed everything up and moved some processes to other servers.
A couple of them are not responding now, it makes me think it's possibly routing related since when you originally mentioned it, they were accessible from the master server. Going to check further.
--
Anyway, SolusVM is not having a good time right now, I'm unable to restore any of Denver. It was already struggling earlier today on the IPv4 changes hence why we couldn't finish them in time. About 40~ of them are ready and waiting for SVM to function.
Okay so the two in Atlanta and two in Denver that went offline are all on the same /22. This is the one we were moving to LAX, perhaps they misunderstood the instructions, could make sense. I could see how this would happen if they went further than I thought with the plan.
Only weird part is the IPMI isn't working either, I wonder if they sent Flex a de-announcement request or something.
(edit) Read through the communication, doesn't seem like they did that, RADB still shows FLEX. Okay, I'm getting some more coffee.
Okay there we go RPKI updated, sort of, routing issue. Contacting IPv4 provider to have it changed back and we'll just scrap the moving blocks idea moving forward. Probably my fault somewhere down the line.
@VirMach said: . Okay, I'm getting some more coffee.
Update on the coffee, I pressed the grind button, packed it down, put my cup and pressed the brew button and heard the wrong noise, turns out I pressed grind (again) and there's ground coffee everywhere. Yeahhhhh I may be exhausted.
SolusVM is acting terrible today and it might have been due to another weird update, some important functions aren't working well which makes loading in backups difficult, it basically won't assign it properly.
A good amount are in LAX already, we're most likely going to wait for it to come back online though. It was a routing issue caused by miscommunication and trying to move the block to LAX for migrations, it's not actually down. So it's similar to what happened when our /17 got pulled last time and should hopefully start showing back up soon. Then we can properly move it then if decided, and still have backups as well.
@Encoders said: oh my sides the setup fee is increased to $30. that's it, I'm going to renew my tokyo into yearly.
Thank you for your effort fighting those cheap mjj abusing resources in tokyo region. finally i can at ease knowing i have decent dns mirror in jp
We actually end up still getting orders for it and yeah it has a general positive effect and essentially the same revenue stream at a much lower volume. I mean anything's higher than negative $30 from a dispute on a $10 a year special.
@paradiser said:
I'm really happy with the Sj node from Virmach. If possible, please keep the node at the same datacenter.
@rafanake said:
can someone recommend me a more reliable service?
There are lots of alternatives, but I think you'll find find at least one these will have a track record of better than average combined value in pricing, reliability, consistency, and drama-free service in multiple locations and with IPv6 to boot:
Very bumpy last day, just wanted to comment to say I'm still around. I did get a little bit of sleep but running pretty short on energy and may have to sleep again soon, trying to get some stuff out the way before then.
Still trying to get NYC back up, the network status page accurately reflects where we last left off with an extra few obstacles in the way. Some of the new IPv4 blocks will still have limited routing for probably another day even after we get it back online.
The four servers in DEN and ATL still haven't got their routing back, the plan is to shift to loading in backups but we now also have about 10 other servers down from IPXO pulling IPv4 and not getting back to us on leasing the blocks (not that we expected them to actually lease it to us as they said in their public post, seemed like a marketing thing.) Hivelocity did reply back today on getting some of our unused IPv4 blocks to LAX so I'm trying to fit it into my hectic schedule to take servers down there so we have space to start loading in from backups for the 14-15 servers outside of NYC that have gone down. It will still be a very tight fit.
@VirMach said:
Very bumpy last day, just wanted to comment to say I'm still around. I did get a little bit of sleep but running pretty short on energy and may have to sleep again soon, trying to get some stuff out the way before then.
Still trying to get NYC back up, the network status page accurately reflects where we last left off with an extra few obstacles in the way. Some of the new IPv4 blocks will still have limited routing for probably another day even after we get it back online.
The four servers in DEN and ATL still haven't got their routing back, the plan is to shift to loading in backups but we now also have about 10 other servers down from IPXO pulling IPv4 and not getting back to us on leasing the blocks (not that we expected them to actually lease it to us as they said in their public post, seemed like a marketing thing.) Hivelocity did reply back today on getting some of our unused IPv4 blocks to LAX so I'm trying to fit it into my hectic schedule to take servers down there so we have space to start loading in from backups for the 14-15 servers outside of NYC that have gone down. It will still be a very tight fit.
Brother if you need a hand do not hesitate to reach out. I would be glad to assist you in any way you need. Even if it's just answering tickets to take some of this off you. At least you might be able to sleep for a bit.
@VirMach said:
Very bumpy last day, just wanted to comment to say I'm still around. I did get a little bit of sleep but running pretty short on energy and may have to sleep again soon, trying to get some stuff out the way before then.
Still trying to get NYC back up, the network status page accurately reflects where we last left off with an extra few obstacles in the way. Some of the new IPv4 blocks will still have limited routing for probably another day even after we get it back online.
The four servers in DEN and ATL still haven't got their routing back, the plan is to shift to loading in backups but we now also have about 10 other servers down from IPXO pulling IPv4 and not getting back to us on leasing the blocks (not that we expected them to actually lease it to us as they said in their public post, seemed like a marketing thing.) Hivelocity did reply back today on getting some of our unused IPv4 blocks to LAX so I'm trying to fit it into my hectic schedule to take servers down there so we have space to start loading in from backups for the 14-15 servers outside of NYC that have gone down. It will still be a very tight fit.
Brother if you need a hand do not hesitate to reach out. I would be glad to assist you in any way you need. Even if it's just answering tickets to take some of this off you. At least you might be able to sleep for a bit.
Most servers in NYC are in a state where we can continue. I'm wrapping up trying to get the rest up and then people will start getting IPv4 changes and service should begin being accessible again. Once we do that I'll update status page.
Hello ,
We've recently announced changes to our Uptime Monitoring Network via an email sent out to all of the affected users:
View the IP changes
You are now receiving this follow-up email because our system has detected that one or more of your Uptime Monitors cannot be accessed by our new monitoring IPs.
Our Singapore 7 (IP: 107.155.75.66) and Singapore 8 (IP: 210.16.67.82) nodes cannot reach the following Uptime Monitor(s):
virmach tokyo ryzen ping
In order to dig further into these errors, you can check each monitor's Location Fail Log.
hmm？
root@LooseUnsightly-VM:~# nexttrace 107.155.75.66 -g en
NextTrace v1.2.0.2 2023-09-02T11:46:50Z a0ab83c
[NextTrace API] prefered API IP - 104.26.8.231 - 1.16ms
IP Geo Data Provider: LeoMoeAPI
traceroute to 107.155.75.66, 30 hops max, 52 bytes packets
1 147.78.245.1 AS23959 [VIRMACH-JP] Tokyo Tokyo Japan owl.net
1.53 ms / 2.01 ms / 1.21 ms
2 *
3 211.14.4.177 AS9607 [JPNIC-NET] Tokyo Tokyo Japan bbtower.co.jp
as3258.xe-0-1-7.tedge0503.newote.bbtower.ad.jp 0.40 ms / 0.69 ms / 0.72 ms
4 211.14.4.222 AS9607 [JPNIC-NET] Tokyo Tokyo Japan bbtower.co.jp
xe-0-1-7.tedge0504.newote.bbtower.ad.jp 0.85 ms / 0.71 ms / 0.96 ms
5 172.29.0.10 * [RFC1918] LAN Address
0.86 ms / 0.97 ms / 0.82 ms
6 210.138.130.225 AS2497 [APNIC-AP] Tokyo Tokyo Japan iij.ad.jp
1.60 ms / 0.80 ms / 0.91 ms
7 58.138.98.73 AS2497 [IIJ] Tokyo Tokyo Japan iij.ad.jp
tky009bb00.IIJ.Net 7.49 ms / 4.73 ms / 9.96 ms
8 58.138.114.2 AS2497 [IIJ] Tokyo Tokyo Japan iij.ad.jp
tky009ix53.IIJ.Net 3.89 ms / 3.33 ms / 2.68 ms
9 69.174.126.169 AS3257 Tokyo Tokyo Japan gtt.net
ae3.cr0-tyo1.ip4.gtt.net 6.07 ms / 9.56 ms / 25.51 ms
10 *
11 *
12 *
13 *
14 *
15 *
16 *
17 *
18 *
19 *
20 *
21 *
22 *
23 *
24 *
25 *
26 *
27 *
28 *
29 *
30 *
MapTrace URL: https://api.leo.moe/tracemap/224982d0-fe33-5a7c-a012-653af190d822.html
@dedicados said:
Operation Timed Out After 90001 Milliseconds With 0 Bytes Received
on my storage server
Storage will take a bit longer, needs LACP for 2x10Gbps and that got done just need to fix configuration.
@SeaScoot said:
My NYCB020 VM shows up in VPS Panel. I can access with VNC but no DNS resolution yet.
ifcfg-eth0 looks good with DNS servers plus did NetworkManager restart.
Okay so it took a looooong time for fixing the last few but everything except NYCB019, NYCB043, and NYCB044 are actually functional now. Some still need to propagate for new IP addresses but I'd be surprised if that's not complete for all very soon, what you guys are seeing is that happening for ones that have been online already.
Now moving on to switching IP addresses, let's see if SolusVM plays nice (ha!)
@FrankZ said:
FYI: Both my Seattle VMs and my San Jose VMs just became non accessible.
Yeah Sabey still hasn't replied but it looks like they came into work today after the long weekend. Awesome.
San Jose going down though, I communicated to them over the weekend I sure hope they're moving us into that new cabinet and not doing this (powering down) to a newly paying customer, lol.
In other news, Hivelocity fixed nothing, and just contacted us threatening to terminate our account tomorrow because we didn't respond to abuse reports. Guess where the abuse reports were opened? On the legacy system. The one I had a scheduled phone call about and assured would be resolved. Yeah that one, the one that doesn't properly notify us. The one where you have to click "looking for old tickets?" to see.
(Edit) Holy crap, most of them are the SAME reports we confirmed we handled like a dozen times.
(Edit2) I just went through all of them, they're ALL the same as last time.
"But their reply does not actually explain the steps taken to resolve the case"
Here's what they're talking about by the way, this is the reply they describe as not actually explaining
This is only one reply in one place, let me know if you guys want me to go through every single time, all the times these IP addresses have been mentioned and all our responses, your scrollbar is going to have to work overtime.
Comments
In case it's useful to you so you can back up your stuff, you may just find spotty routing and might find a host you can copy your stuff to. I have a VPS with them in 45.42.214.0/23 and can route to/from some hosts, but not others:
It's time to ditch IPv4 completely and give everyone a routed /56.
We accept Karma donations for the last flan. 🍮 affbrr
SolusVM was acting up, unfortunately it wasn't doing well trying to process all the changes, so notifications couldn't finish going on for the changes and they'll have to continue after re-racked.
NY, denver gone offline
Yep, me too. DENZ001 NYCB015
NYCB028 is down
New York expected, Denver, not so much. Taking a look and then most likely loading in from backups.
All in NY are offline.
https://status.virm.ac
ServerStatus , slackvpn , linux mirrors
Recommended Providers: Servarica(aff) | HostBRR(aff) | SolidVPS(aff) | RackNerd(aff) | KuroIT(aff) | AlphaVPS(aff) | HostDare(aff)
My NY, Atlanta & Denver all died. I dont really mind because I backed everything up and moved some processes to other servers.
Atlanta's should still be online.
Denver, evaluating still, that one's unplanned.
NYC is scheduled maintenance, the hardware was handed over basically it's being moved into a new cabinet.
Kudos. This is a really good answer. There was a surprisingly long time to backup and move services given I thought everything DediPath was going to go down on Friday.
A couple of them are not responding now, it makes me think it's possibly routing related since when you originally mentioned it, they were accessible from the master server. Going to check further.
--
Anyway, SolusVM is not having a good time right now, I'm unable to restore any of Denver. It was already struggling earlier today on the IPv4 changes hence why we couldn't finish them in time. About 40~ of them are ready and waiting for SVM to function.
Okay so the two in Atlanta and two in Denver that went offline are all on the same /22. This is the one we were moving to LAX, perhaps they misunderstood the instructions, could make sense. I could see how this would happen if they went further than I thought with the plan.
Only weird part is the IPMI isn't working either, I wonder if they sent Flex a de-announcement request or something.
(edit) Read through the communication, doesn't seem like they did that, RADB still shows FLEX. Okay, I'm getting some more coffee.
Okay there we go RPKI updated, sort of, routing issue. Contacting IPv4 provider to have it changed back and we'll just scrap the moving blocks idea moving forward. Probably my fault somewhere down the line.
Update on the coffee, I pressed the grind button, packed it down, put my cup and pressed the brew button and heard the wrong noise, turns out I pressed grind (again) and there's ground coffee everywhere. Yeahhhhh I may be exhausted.
Get some sleep. Monday is coming oh so soon.
Monday is federal holiday.
We accept Karma donations for the last flan. 🍮 affbrr
NYC is all racked up and getting wired.
@VirMach, is Denver moving to L.A currently?
oh my sides the setup fee is increased to $30. that's it, I'm going to renew my tokyo into yearly.
Thank you for your effort fighting those cheap mjj abusing resources in tokyo region. finally i can at ease knowing i have decent dns mirror in jp
Fuck this 24/7 internet spew of trivia and celebrity bullshit.
I'm really happy with the Sj node from Virmach. If possible, please keep the node at the same datacenter.
SolusVM is acting terrible today and it might have been due to another weird update, some important functions aren't working well which makes loading in backups difficult, it basically won't assign it properly.
A good amount are in LAX already, we're most likely going to wait for it to come back online though. It was a routing issue caused by miscommunication and trying to move the block to LAX for migrations, it's not actually down. So it's similar to what happened when our /17 got pulled last time and should hopefully start showing back up soon. Then we can properly move it then if decided, and still have backups as well.
We actually end up still getting orders for it and yeah it has a general positive effect and essentially the same revenue stream at a much lower volume. I mean anything's higher than negative $30 from a dispute on a $10 a year special.
Adding an update on network status page.
if all goes well, by Christmas all services will be online again... with a few delays only.... and ofc new ip's
VirMach be like ....
NYCB004S Operation Timed Out After 90000 Milliseconds With 0 Bytes Received
Sultan Muda - Amazon Store
keep spirit 💪🏾
When NY will come back up?
What about LAX2Z019? Dead forever a data lost?
O Gzz virmach became the worst service i ever tried, they used to be good but this 2023, damn!
sometime BTW now and Christmas
can someone recommend me a more reliable service?
what location ? Let us know a description of what you are looking for and I will split it off into a new request thread.
There are lots of alternatives, but I think you'll find find at least one these will have a track record of better than average combined value in pricing, reliability, consistency, and drama-free service in multiple locations and with IPv6 to boot:
Dataplane.org's current server hosting provider list
There are not. He probably have flash deal for 3-8$/y and wont find them anywhere near that price point.
ServerStatus , slackvpn , linux mirrors
Recommended Providers: Servarica(aff) | HostBRR(aff) | SolidVPS(aff) | RackNerd(aff) | KuroIT(aff) | AlphaVPS(aff) | HostDare(aff)
Very bumpy last day, just wanted to comment to say I'm still around. I did get a little bit of sleep but running pretty short on energy and may have to sleep again soon, trying to get some stuff out the way before then.
Still trying to get NYC back up, the network status page accurately reflects where we last left off with an extra few obstacles in the way. Some of the new IPv4 blocks will still have limited routing for probably another day even after we get it back online.
The four servers in DEN and ATL still haven't got their routing back, the plan is to shift to loading in backups but we now also have about 10 other servers down from IPXO pulling IPv4 and not getting back to us on leasing the blocks (not that we expected them to actually lease it to us as they said in their public post, seemed like a marketing thing.) Hivelocity did reply back today on getting some of our unused IPv4 blocks to LAX so I'm trying to fit it into my hectic schedule to take servers down there so we have space to start loading in from backups for the 14-15 servers outside of NYC that have gone down. It will still be a very tight fit.
The VirMach grief cycle.
Would recommend.
Brother if you need a hand do not hesitate to reach out. I would be glad to assist you in any way you need. Even if it's just answering tickets to take some of this off you. At least you might be able to sleep for a bit.
The Yeti has left the building.
Ditto, let me know if I can help further
Most servers in NYC are in a state where we can continue. I'm wrapping up trying to get the rest up and then people will start getting IPv4 changes and service should begin being accessible again. Once we do that I'll update status page.
NY027 is recovered and ip changed.
NYCB040 is online, IP changed, but still can't be accessed via SSH after rebooting and reconfiguring.
My NYCB027 node is still locked.
My nodes are unlocked but they lack the new IPs (or making old one work:D)
// Because it seems we are reporting random stuff here
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
hmm？
Operation Timed Out After 90001 Milliseconds With 0 Bytes Received
on my storage server
My NYCB020 VM shows up in VPS Panel. I can access with VNC but no DNS resolution yet.
ifcfg-eth0 looks good with DNS servers plus did NetworkManager restart.
FYI: Both my Seattle VMs and my San Jose VMs just became non accessible.
Storage will take a bit longer, needs LACP for 2x10Gbps and that got done just need to fix configuration.
Okay so it took a looooong time for fixing the last few but everything except NYCB019, NYCB043, and NYCB044 are actually functional now. Some still need to propagate for new IP addresses but I'd be surprised if that's not complete for all very soon, what you guys are seeing is that happening for ones that have been online already.
Now moving on to switching IP addresses, let's see if SolusVM plays nice (ha!)
Yeah Sabey still hasn't replied but it looks like they came into work today after the long weekend. Awesome.
San Jose going down though, I communicated to them over the weekend I sure hope they're moving us into that new cabinet and not doing this (powering down) to a newly paying customer, lol.
In other news, Hivelocity fixed nothing, and just contacted us threatening to terminate our account tomorrow because we didn't respond to abuse reports. Guess where the abuse reports were opened? On the legacy system. The one I had a scheduled phone call about and assured would be resolved. Yeah that one, the one that doesn't properly notify us. The one where you have to click "looking for old tickets?" to see.
(Edit) Holy crap, most of them are the SAME reports we confirmed we handled like a dozen times.
(Edit2) I just went through all of them, they're ALL the same as last time.
"But their reply does not actually explain the steps taken to resolve the case"
Here's what they're talking about by the way, this is the reply they describe as not actually explaining
This is only one reply in one place, let me know if you guys want me to go through every single time, all the times these IP addresses have been mentioned and all our responses, your scrollbar is going to have to work overtime.
So Hivelocity migrations being added by tomorrow?