It's a "STORAGE-2T" plan, which I bought on sale in one of the other threads. It's specs don't quite match what is shown over here: https://virmach.com/cloud-storage-servers/
@VirMach said: By the way taking all recommendations for datacenters in Chicago at this point.
What are you looking for on space/power/bandwidth? You can DM me or message me on Skype/Discord/Telegram.
Do you have options without Path's protection? I looked at your stuff a bit ago and passed up on recommending you to several people due to your use of Path.
Is there a best guess estimate guesstimate when Chicago networking issues might be resolved? Have website (very non critical) in ChiTown and if it's most likely going to be up again shortly don't want to shuffle things around. Yes I am lazy
@JDMcPea said:
Is there a best guess estimate guesstimate when Chicago networking issues might be resolved? Have website (very non critical) in ChiTown and if it's most likely going to be up again shortly don't want to shuffle things around. Yes I am lazy
THNAKX!
Ditto; I mean I could shift it to a more production-y server but if CHI is likely to come back in the next day or two I have better things to be doing (mostly involving beer)
@fluttershy said: Do you have options without Path's protection? I looked at your stuff a bit ago and passed up on recommending you to several people due to your use of Path.
We only use Path if you order it - it's an add-on service only.
QuickPacket - Dedicated Servers in Ashburn, Los Angeles, Chicago
It's a "STORAGE-2T" plan, which I bought on sale in one of the other threads. It's specs don't quite match what is shown over here: https://virmach.com/cloud-storage-servers/
Seems like we're on the same node. My best server with VirMach too.
@VirMach Hi
About half a month ago, I had tickets for three service transfers. The tickets contain: #511984 #550315 #349556.
Among them, #511984 and #550315 were closed on May 16th, and #349556 is still in waiting status until now. But none of the service were transferred over.
This situation has bothered me for days. If you could check them in your free time, that would be great.
Thank you!
Chicago network was up for a little bit yesterday, but then went unavailable again after a short time. My VM on CHIZ003 is still not accessible at this time. No official update from VirMach since May 18th.
I have no inside information, but have the feeling a DC change is in the works for Chicago nodes.
Chicago network was up for a little bit yesterday, but then went unavailable again after a short time. My VM on CHIZ003 is still not accessible at this time. No official update from VirMach since May 18th.
I have no inside information, but have the feeling a DC change is in the works for Chicago nodes.
Last update I got was attack stopped so they removed nullroute, and tried and failed multiple times to get information on planning deracking and trying to move them elsewhere, and general information on the "attack" as well as some other things, which were basically all ignored. Ticket is getting updated on their end so I'm assuming staff notes, I asked for more information again but I don't expect much at this point. Trying to work it all in and maybe finalize things on our end then perhaps just get someone to show up to the facility to grab our servers if they continue ignoring everything.
@BNNY said:
Any updates on NYCB009SH? Can we expect the service back online this week?
We received these maybe Thursday, but of course as everything goes around here there was the Chicago thing, a few other things, hundreds of tickets, and so on. I've been working on them collectively and trying to get somewhere by tomorrow regarding those New York servers, hopefully.
We were supposed to get them last last Friday (12th or sooner) initially and I really wish that would've happened because I would've been more free that weekend to do more immediate work as initially planned.
@FrankZ said: No official update from VirMach since May 18th.
Comedically timed as usual. I think I basically brought the servers inside from the NYC fiasco, unboxed them and got them on workbench, and then it happened.
(edit) Very not good that it's been 3-4 days since then and felt like one. VirMach timewarp occuring.
@Keyi said: @VirMach Hi
About half a month ago, I had tickets for three service transfers. The tickets contain: #511984 #550315 #349556.
Among them, #511984 and #550315 were closed on May 16th, and #349556 is still in waiting status until now. But none of the service were transferred over.
This situation has bothered me for days. If you could check them in your free time, that would be great.
Thank you!
Hi @VirMach
I am still waiting, please take a look.
@VirMach ,
As someone who has worked in data centers most of my professional life I can sympathize with "the perfect storm" of simultaneously outages/issues and/or being understaffed. However, it's not very reassuring that the last update from your Network/Server Status page was from 5/18 and references only the NYC server issues. It would go a long way in increasing customer confidence if you could share some small daily update at least acknowledging the Chicago node outages / network issues on any one of your status pages or social media accounts.
I can appreciate the level of triage you are likely dealing with at the moment but I think it would lower the temperature for a lot of customers if you were able to share some update from your official sources vs. a separate forum. Before finding this forum post I opened ticket #147694. If you can share a wider update to the Network Status page regarding the Chicago issue then I am happy to close my individual request. Thank you.
@rkzed said: @FrankZ, does reinstall the VPS will automagically fix the missing disk issue?
my VPS has been down for a week and I can't do anything from my end
@somik no it's a separate issue that happened to my VPS. the disk is missing from VPS. My VPS was stable for the past few months then suddenly my VPS froze and when i rebooted it, it stuck at boot screen with an error message about missing boot disk. I tried booting with Gparted iso, it didn't detect my disk either.
@rkzed said: @FrankZ, does reinstall the VPS will automagically fix the missing disk issue?
my VPS has been down for a week and I can't do anything from my end
No, I don't think you should do that. I expect that will only create more issues.
As I understand it, the LVM (virtual disk) assigned to your VM has stopped being seen by the hypervisor. An analogy would be when a disk drops off on a dedicated server. I think you can see why reinstalling would probably not resolve the issue. Hope for a node reboot.
@FrankZ said: No, I don't think you should do that. I expect that will only create more issues. As I understand it, the LVM (virtual disk) assigned to your VM has stopped being seen by the hypervisor. An analogy would be when a disk drops off on a dedicated server. I think you can see why reinstalling would probably not resolve the issue. Hope for a node reboot
ah, I thought doing reinstall will overwrite the problematic config and fix by itself. i guess I have to wait them to fix it then..
@VirMach, take a peek at my ticket #518513, you wont believe what I wrote there, winkwink
Hey virmach ,Is it possible build a reverse proxy for client to take back some data from Chicago server?
The network seems down much longer as we expected.
@FrankZ said: No official update from VirMach since May 18th.
Comedically timed as usual. I think I basically brought the servers inside from the NYC fiasco, unboxed them and got them on workbench, and then it happened.
(edit) Very not good that it's been 3-4 days since then and felt like one. VirMach timewarp occuring.
I hate to badger you in your timewarp, but another weekend has snuck past and Chicago is still AWOL. Just a steer would be useful about now; if you're at the "we just need to move to a new DC" stage let us know and I can start rebuilding stuff rather than waiting to see if it will all magically reappear...
Happily I have backups; I just need to know if it's worth spending the time restoring them elsewhere or if the network will reappear before I've finished
@FrankZ said: No official update from VirMach since May 18th.
Comedically timed as usual. I think I basically brought the servers inside from the NYC fiasco, unboxed them and got them on workbench, and then it happened.
(edit) Very not good that it's been 3-4 days since then and felt like one. VirMach timewarp occuring.
I hate to badger you in your timewarp, but another weekend has snuck past and Chicago is still AWOL. Just a steer would be useful about now; if you're at the "we just need to move to a new DC" stage let us know and I can start rebuilding stuff rather than waiting to see if it will all magically reappear...
They keep updating us that null is removed then it's not, and ignoring all questions about shipping out servers. I bumped them on it again. They claim that 90% of the attack was hitting our IPs but still haven't provided me with any of the requested information. Current plan is to see if we can get any connectivity going for emergency migrations.
@ahnlak said:
Happily I have backups; I just need to know if it's worth spending the time restoring them elsewhere or if the network will reappear before I've finished
I'd recommend loading in backups elsewhere as it's not under my control and we're already heavily burdened on our end. I'd love to say I can fly to Chicago right now to grab them but I can't.
@BNNY said:
It seems that there is a new Chicago Datacenter with DigitalRealty online now. Any updates on the NYCB servers?
Running into various problems, I'll update the status page as soon as I have a solid ETA. They delays are completely on our side now, sorry. (For the NY servers.)
(edit) It's a lot to go over but a big part of it was logistics, we had to move away from one office that was set up specifically for builds into storage in the other and I didn't realize how disorganized and heavy it all was, I screwed up my back a little bit (picking up and accidentally dropping a heavy box) and slammed my left foot knee into metal so that explains about 2-3 days of the delay. I just couldn't physically move things around that needed to be moved.
@VirMach said:
Running into various problems, I'll update the status page as soon as I have a solid ETA. They delays are completely on our side now, sorry. (For the NY servers.)
I'm wondering if you can provide some backup data for NYCB009? My account renewed, so I still intend to use the service, but at the same time I would appreciate having the access to some of the server data that I didn't have a local back up of
Comments
Also very happy with mine!
Just ran YABS for you, @FrankZ:
## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ##
Yet-Another-Bench-Script
v2023-04-23
https://github.com/masonr/yet-another-bench-script
## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ##
Fri May 19 02:08:52 PM EDT 2023
Basic System Information:
Uptime : 0 days, 0 hours, 0 minutes
Processor : AMD Ryzen Threadripper 3960X 24-Core Processor
CPU cores : 3 @ 3799.946 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 3.8 GiB
Swap : 4.0 GiB
Disk : 1.8 TiB
Distro : Ubuntu 22.04.2 LTS
Kernel : 5.15.0-72-generic
VM Type : KVM
IPv4/IPv6 : ✔ Online / ❌ Offline
IPv4 Network Information:
ISP : Alibaba.com LLC
ASN : AS35913 DediPath
Host : Zenlayer Inc, Virtual Machine Solutions LLC
Location : Secaucus, New Jersey (NJ)
Country : United States
fio Disk Speed Tests (Mixed R/W 50/50):
iperf3 Network Speed Tests (IPv4):
Geekbench 5 Benchmark Test:
Test | Value
|
Single Core | 1098
Multi Core | 3054
Full Test | https://browser.geekbench.com/v5/cpu/21222092
@Wonder_Woman which plan is this? Can you share a link?
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
It's a "STORAGE-2T" plan, which I bought on sale in one of the other threads. It's specs don't quite match what is shown over here: https://virmach.com/cloud-storage-servers/
What are you looking for on space/power/bandwidth? You can DM me or message me on Skype/Discord/Telegram.
QuickPacket - Dedicated Servers in Ashburn, Los Angeles, Chicago
Do you have options without Path's protection? I looked at your stuff a bit ago and passed up on recommending you to several people due to your use of Path.
Is there a best guess estimate guesstimate when Chicago networking issues might be resolved? Have website (very non critical) in ChiTown and if it's most likely going to be up again shortly don't want to shuffle things around. Yes I am lazy
THNAKX!
Ditto; I mean I could shift it to a more production-y server but if CHI is likely to come back in the next day or two I have better things to be doing (mostly involving beer)
We only use Path if you order it - it's an add-on service only.
QuickPacket - Dedicated Servers in Ashburn, Los Angeles, Chicago
VirMach now has 3 people who replies on this forum!!!
Waiting for Chicago network back to online. Two hosts are down.
Seems like we're on the same node. My best server with VirMach too.
they always were 3 .... what are you talking about
Huh! I always saw only 2 people replying to the theads so I assumed only 2 people from VirMach are on this forum.
@somik said:
Which two or three people here do you expect are employees of VirMach ?
My eyes must be playing tricks on me cause I can only find 2 people now, VirMach and qps...
How did i count 3 people?!?
@qps does not work for VirMach. He is quickpacket.com and was talking to VirMach about co-lo.
@VirMach Hi
About half a month ago, I had tickets for three service transfers. The tickets contain: #511984 #550315 #349556.
Among them, #511984 and #550315 were closed on May 16th, and #349556 is still in waiting status until now. But none of the service were transferred over.
This situation has bothered me for days. If you could check them in your free time, that would be great.
Thank you!
Hey @VirMach this offer work with storage? :-) https://virmach.com/vps-price-match-refugee-offers/
Any news about Chicago network? Virmach
Any news about Virmach? @FrankZ
Ontario Dildo Inspector
Chicago network was up for a little bit yesterday, but then went unavailable again after a short time. My VM on CHIZ003 is still not accessible at this time. No official update from VirMach since May 18th.
I have no inside information, but have the feeling a DC change is in the works for Chicago nodes.
Any updates on NYCB009SH? Can we expect the service back online this week?
Last update I got was attack stopped so they removed nullroute, and tried and failed multiple times to get information on planning deracking and trying to move them elsewhere, and general information on the "attack" as well as some other things, which were basically all ignored. Ticket is getting updated on their end so I'm assuming staff notes, I asked for more information again but I don't expect much at this point. Trying to work it all in and maybe finalize things on our end then perhaps just get someone to show up to the facility to grab our servers if they continue ignoring everything.
We received these maybe Thursday, but of course as everything goes around here there was the Chicago thing, a few other things, hundreds of tickets, and so on. I've been working on them collectively and trying to get somewhere by tomorrow regarding those New York servers, hopefully.
We were supposed to get them last last Friday (12th or sooner) initially and I really wish that would've happened because I would've been more free that weekend to do more immediate work as initially planned.
Comedically timed as usual. I think I basically brought the servers inside from the NYC fiasco, unboxed them and got them on workbench, and then it happened.
(edit) Very not good that it's been 3-4 days since then and felt like one. VirMach timewarp occuring.
Hi @VirMach
I am still waiting, please take a look.
@VirMach ,
As someone who has worked in data centers most of my professional life I can sympathize with "the perfect storm" of simultaneously outages/issues and/or being understaffed. However, it's not very reassuring that the last update from your Network/Server Status page was from 5/18 and references only the NYC server issues. It would go a long way in increasing customer confidence if you could share some small daily update at least acknowledging the Chicago node outages / network issues on any one of your status pages or social media accounts.
I can appreciate the level of triage you are likely dealing with at the moment but I think it would lower the temperature for a lot of customers if you were able to share some update from your official sources vs. a separate forum. Before finding this forum post I opened ticket #147694. If you can share a wider update to the Network Status page regarding the Chicago issue then I am happy to close my individual request. Thank you.
It seems that there is a new Chicago Datacenter with DigitalRealty online now. Any updates on the NYCB servers?
@FrankZ, does reinstall the VPS will automagically fix the missing disk issue?
my VPS has been down for a week and I can't do anything from my end
You mean this comment form @FrankZ
https://lowendspirit.com/discussion/comment/140037/#Comment_140037
@somik no it's a separate issue that happened to my VPS. the disk is missing from VPS. My VPS was stable for the past few months then suddenly my VPS froze and when i rebooted it, it stuck at boot screen with an error message about missing boot disk. I tried booting with Gparted iso, it didn't detect my disk either.
No, I don't think you should do that. I expect that will only create more issues.
As I understand it, the LVM (virtual disk) assigned to your VM has stopped being seen by the hypervisor. An analogy would be when a disk drops off on a dedicated server. I think you can see why reinstalling would probably not resolve the issue. Hope for a node reboot.
ah, I thought doing reinstall will overwrite the problematic config and fix by itself. i guess I have to wait them to fix it then..
@VirMach, take a peek at my ticket #518513, you wont believe what I wrote there, wink wink
Hey virmach ,Is it possible build a reverse proxy for client to take back some data from Chicago server?
The network seems down much longer as we expected.
Another day without any offical updates?
welcome to the TRASHMACH
Drive by Moaning just for @Nekki
The Yeti has left the building.
Nekki is not the only one to appreciate your handy work.
My word, I've just made quite the mess in my undies!
The Yeti has left the building.
Wtf, did I miss a hand job?
I guess both Yeti and VirmAch has given Nekki a hard time.
Ontario Dildo Inspector
or a diarrhoea...
That's a very runny time...
I hate to badger you in your timewarp, but another weekend has snuck past and Chicago is still AWOL. Just a steer would be useful about now; if you're at the "we just need to move to a new DC" stage let us know and I can start rebuilding stuff rather than waiting to see if it will all magically reappear...
@Virmach is rDNS on Alibaba block gone ?
ServerStatus , slackvpn , linux mirrors
Recommended Providers: Servarica(aff) | HostBRR(aff) | SolidVPS(aff) | RackNerd(aff) | KuroIT(aff) | AlphaVPS(aff) | HostDare(aff)
NYCB009 still dead...
Virmach ,Please switch the Chicago network or take the host to other ISP.
I need get back the server data. :'(
Happily I have backups; I just need to know if it's worth spending the time restoring them elsewhere or if the network will reappear before I've finished
Shouldn't be, let me know.
They keep updating us that null is removed then it's not, and ignoring all questions about shipping out servers. I bumped them on it again. They claim that 90% of the attack was hitting our IPs but still haven't provided me with any of the requested information. Current plan is to see if we can get any connectivity going for emergency migrations.
I'd recommend loading in backups elsewhere as it's not under my control and we're already heavily burdened on our end. I'd love to say I can fly to Chicago right now to grab them but I can't.
Running into various problems, I'll update the status page as soon as I have a solid ETA. They delays are completely on our side now, sorry. (For the NY servers.)
(edit) It's a lot to go over but a big part of it was logistics, we had to move away from one office that was set up specifically for builds into storage in the other and I didn't realize how disorganized and heavy it all was, I screwed up my back a little bit (picking up and accidentally dropping a heavy box) and slammed my left foot knee into metal so that explains about 2-3 days of the delay. I just couldn't physically move things around that needed to be moved.
I'm wondering if you can provide some backup data for NYCB009? My account renewed, so I still intend to use the service, but at the same time I would appreciate having the access to some of the server data that I didn't have a local back up of