There are going to be a lot of upcoming features, some of which have already been implemented. The descriptions below are not finalized so it's possible the end result may be different or it may take several iterations to get there.
Automatically making sure you have a functional IPv4 address in any case, but this also means for any migrations or changes in the future, the system will know if something's wrong and correct it if necessary.
More specifically, an IP changing system for entire block swaps, auto notifying customers, letting you decide when you want to perform it within an allotted time period, and completing/cleaning up given whatever stage you're on for the change.
Spinning up a temporary secondary service, transferring data, and swapping back and forth between them. This can end up being used for both optional and required/planned migrations, outages, loading in from backups, and so on.
Automatic actions on severs overloading, disk issues, and other common problems which should drastically reduce any longer outages as a result of backlogs. This has been tested and it also performs much better than a human in some cases where the problem also affects our ability to communicate with the server.
Granular notifications. We'll be able to do things like let you know if your service specifically is affected on a node instead of having to contact everyone, and also be able to notify you of things like the old node coming back up so you know you can move back, or that there's an upcoming migration and you can move on your own.
These systems are created in a way where they can be stacked together and customized so they'll serve many functions. A more specific scenario example: your node goes down, ability to spin up a secondary service and load from available backups is displayed automatically, or perhaps we'll activate it with a longer grace period for a node facing a potentially long outage. Once the old node is back up, you'll be able to decide to stay on the new temporary service and make it your permanent service, or switch back to the original if available (and for example, you need to pull off some data, then you can switch back again if you wanted or stay on the old service, etc.)
A lot of these functionalities are completed fully and just in a testing phase. For example, we already have a working system for the spinning up a copy (but haven't yet implemented the ability to switch back and forth on the client area, only admin.) Most of them are pretty much in a functional/core state though if they were mentioned here.
Any feedback or specific requests/questions regarding these systems, let me know.
@VirMach said:
There are going to be a lot of upcoming features, some of which have already been implemented. The descriptions below are not finalized so it's possible the end result may be different or it may take several iterations to get there.
Automatically making sure you have a functional IPv4 address in any case, but this also means for any migrations or changes in the future, the system will know if something's wrong and correct it if necessary.
More specifically, an IP changing system for entire block swaps, auto notifying customers, letting you decide when you want to perform it within an allotted time period, and completing/cleaning up given whatever stage you're on for the change.
Spinning up a temporary secondary service, transferring data, and swapping back and forth between them. This can end up being used for both optional and required/planned migrations, outages, loading in from backups, and so on.
Automatic actions on severs overloading, disk issues, and other common problems which should drastically reduce any longer outages as a result of backlogs. This has been tested and it also performs much better than a human in some cases where the problem also affects our ability to communicate with the server.
Granular notifications. We'll be able to do things like let you know if your service specifically is affected on a node instead of having to contact everyone, and also be able to notify you of things like the old node coming back up so you know you can move back, or that there's an upcoming migration and you can move on your own.
These systems are created in a way where they can be stacked together and customized so they'll serve many functions. A more specific scenario example: your node goes down, ability to spin up a secondary service and load from available backups is displayed automatically, or perhaps we'll activate it with a longer grace period for a node facing a potentially long outage. Once the old node is back up, you'll be able to decide to stay on the new temporary service and make it your permanent service, or switch back to the original if available (and for example, you need to pull off some data, then you can switch back again if you wanted or stay on the old service, etc.)
A lot of these functionalities are completed fully and just in a testing phase. For example, we already have a working system for the spinning up a copy (but haven't yet implemented the ability to switch back and forth on the client area, only admin.) Most of them are pretty much in a functional/core state though if they were mentioned here.
Any feedback or specific requests/questions regarding these systems, let me know.
According to https://status.virm.ac/, recently LAXA008, LAXA009, and LAXA010 have been experiencing synchronized, bursts of high occupancy (peaks close to 100%) almost every day for about 15 minutes at a time, during which time it causes my VPS on one of the nodes to become nearly unavailable, triggering a bunch of anomaly alarms.
Should I request @VirMach to pay attention to this, or should I silently turn off the anomaly alerts?
Mon, 20 May 2024 19:48:00 -0700 FFME001, FFME002, FFME003, FFME004, FFME005, FFME006, and FFME007 will be physically migrated between 10PM local time on 05/24/2025 and set up in Amsterdam the next day (05/25/2025.) This migration may take approximately 12 to 18 hours . Your service will return online with the same data & same IP address, but network connectivity may have intermittent issues initially. We will enable a feature where you can activate a second temporary service since the downtime may be extended. This will appear on the left sidebar, and you can activate a service in any other available location, and then choose to either keep your old service or new service once maintenance is completed. You will have access to this tool for approximately two weeks. It has not yet been activated, but we're working hard to enable it for you. Once it is ready, you will receive an email from us. In any case, we'll attempt to email notifications at least 72 hours in advance for the maintenance/migration. We are providing this update as a "reported" incident to ensure it is more easily seen.
Rest in peace Frankfurt.
Also holy shit @VirMach please - 12 to 18 hours for physical migration where driving gonna take at least 6 hours alone?!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Mon, 20 May 2024 19:48:00 -0700 FFME001, FFME002, FFME003, FFME004, FFME005, FFME006, and FFME007 will be physically migrated between 10PM local time on 05/24/2025 and set up in Amsterdam the next day (05/25/2025.) This migration may take approximately 12 to 18 hours . Your service will return online with the same data & same IP address, but network connectivity may have intermittent issues initially. We will enable a feature where you can activate a second temporary service since the downtime may be extended. This will appear on the left sidebar, and you can activate a service in any other available location, and then choose to either keep your old service or new service once maintenance is completed. You will have access to this tool for approximately two weeks. It has not yet been activated, but we're working hard to enable it for you. Once it is ready, you will receive an email from us. In any case, we'll attempt to email notifications at least 72 hours in advance for the maintenance/migration. We are providing this update as a "reported" incident to ensure it is more easily seen.
Rest in peace Frankfurt.
Just curious--did you receive an email about this, or notice it on the network status page?
I'm seeing the new option to activate a secondary service! Trying to decide if I want to take advantage of the situation to permanently move my $3/year VPS back to the USA. (Looks like Miami, Chicago, NYC, Atlanta, LA, Amsterdam, and San Jose are the options.)
Welp, 30 minutes later nothing changed and primary service is still down.
35 minutes clicked "Refresh" button... WHMCS loaded -secondary server for this service ID... first one is gone from WHMCS, but still shows up in SolusVM panel.
Booted primary from SolusVM, seems still to be working.
Booted secondary from SolusVM - no disk error.
I guess data transfer failed
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Trying different service, progress bar still not moving. @VirMach javascript expects element with id dataprogress and there is no such thing in popup - there is one progress element, but no id attached.
clientarea.php?actio…etails&id=23941:250 Uncaught
TypeError: Cannot set properties of null (setting 'value')
at clientarea.php?actio…ils&id=23941:250:32
// Animate progress bar
var intervalId = setInterval(function() {
value += increment;
progress.value = value;
document.getElementById('dataprogress')
null
Okey, clicking Refresh detected that migration and showed nice(r) popup.
2-3 hours holly molly macaroni cheese
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
I created a secondary with no data migration and it was replying to pings after a few minutes. Had to use the rescue image to install my keys and change the password. The solus vm panel shows the secondary service has 5 TB of transfer quota instead of 100 GB the old one had!
Starting to think that AMSD031T36 is broken - I've tried to migrate two servers with data, but both ended with no disk fuck off. Tried reinstall one, no disk, fuck off.
I've migrated one more with data that didn't end on AMSD031T36 and everything is working - bah, I even see "Secondary service" button in WHCMS and it loads nice page showing -primary and -secondary, timeleft and switch button.
Why do I have a feeling in 6 days I am gonna get rekt - it will delete my failed-to-migrate primary and I will be left with two broken -secondary
This also totally nukes my plans to semi-manually dd migrate data from Frankfurt to Amsterdam after failed automatic migration - can't really migrate if the -secondary has no disk, lmao?
EDIT: correct size 'disk' shows up in rescue mode and fdisk, dd attempt it is! ;')
37547409408 bytes (38 GB, 35 GiB) copied, 500.026 s, 75.1 MB/s
1120+0 records in
1120+0 records out
37580963840 bytes (38 GB, 35 GiB) copied, 500.411 s, 75.1 MB/s
73400320+0 records in
73400320+0 records out
37580963840 bytes (38 GB, 35 GiB) copied, 489.096 s, 76.8 MB/s
@Jab said:
Why do I have a feeling in 6 days I am gonna get rekt - it will delete my failed-to-migrate primary and I will be left with two broken -secondary
I don’t trust that either that’s why I’m pretty sure I left that part of the code disabled for now so I can process them manually. But I don’t trust me either. Hmm
Starting to think that AMSD031T36 is broken - I've tried to migrate two servers with data, but both ended with no disk fuck off. Tried reinstall one, no disk, fuck off.
I've migrated one more with data that didn't end on AMSD031T36 and everything is working - bah, I even see "Secondary service" button in WHCMS and it loads nice page showing -primary and -secondary, timeleft and switch button.
>
I hate that server. I was going to have a look but I’ve mostly just been driving.
Frankfurt has been successfully destroyed. More driving, currently ahead of schedule in terms of the 12 to 18 hours downtime. I’ll make sure to break something so we’re back on track.
Wow, tried to create second service for one vm with data transfer, now I can't access it in the billing panel anymore 🥲
Cloudflare Timeout: Error code 524
Mon, 20 May 2024 19:48:00 -0700 FFME001, FFME002, FFME003, FFME004, FFME005, FFME006, and FFME007 will be physically migrated between 10PM local time on 05/24/2025 and set up in Amsterdam the next day (05/25/2025.) This migration may take approximately 12 to 18 hours . Your service will return online with the same data & same IP address, but network connectivity may have intermittent issues initially. We will enable a feature where you can activate a second temporary service since the downtime may be extended. This will appear on the left sidebar, and you can activate a service in any other available location, and then choose to either keep your old service or new service once maintenance is completed. You will have access to this tool for approximately two weeks. It has not yet been activated, but we're working hard to enable it for you. Once it is ready, you will receive an email from us. In any case, we'll attempt to email notifications at least 72 hours in advance for the maintenance/migration. We are providing this update as a "reported" incident to ensure it is more easily seen.
Rest in peace Frankfurt.
Also holy shit @VirMach please - 12 to 18 hours for physical migration where driving gonna take at least 6 hours alone?!
Hey @Jab, did you get an email notification about this or you just monitoring the Network Status page all the time?
Hello guys, I was keeping a database on a VPS from Virmach, which happened to be in Frankfurt. Initially, it was of little value so I didn't keep any backups or anything. I was planning to take it all off the next month and this mess happened.
Now the question is: how screwed am I? Is data loss likely at this point?
@nooneatall said:
Hello guys, I was keeping a database on a VPS from Virmach, which happened to be in Frankfurt. Initially, it was of little value so I didn't keep any backups or anything. I was planning to take it all off the next month and this mess happened.
Now the question is: how screwed am I? Is data loss likely at this point?
Comments
No, I don't have the permission to do that.
There are going to be a lot of upcoming features, some of which have already been implemented. The descriptions below are not finalized so it's possible the end result may be different or it may take several iterations to get there.
These systems are created in a way where they can be stacked together and customized so they'll serve many functions. A more specific scenario example: your node goes down, ability to spin up a secondary service and load from available backups is displayed automatically, or perhaps we'll activate it with a longer grace period for a node facing a potentially long outage. Once the old node is back up, you'll be able to decide to stay on the new temporary service and make it your permanent service, or switch back to the original if available (and for example, you need to pull off some data, then you can switch back again if you wanted or stay on the old service, etc.)
A lot of these functionalities are completed fully and just in a testing phase. For example, we already have a working system for the spinning up a copy (but haven't yet implemented the ability to switch back and forth on the client area, only admin.) Most of them are pretty much in a functional/core state though if they were mentioned here.
Any feedback or specific requests/questions regarding these systems, let me know.
@virmach you should build a new control panel. Let's name it Virmach Flan
It looks cool so far,
When will it be enabled?
High packet loss in LAXA018
Is it possible for @VirMach to offer a one-time migration service within the US for users who were migrated by the dedipath incident?
Maybe
please fix your broken Email 2FA,I can't receive your Email 2FA OTP code
I searched a lot and found that many people had encountered such problem
Do I need to register another account for submitting a ticket for this problem？
You need to ... "Let it go...." "Let it goo..."
https://microlxc.net/
According to https://status.virm.ac/, recently LAXA008, LAXA009, and LAXA010 have been experiencing synchronized, bursts of high occupancy (peaks close to 100%) almost every day for about 15 minutes at a time, during which time it causes my VPS on one of the nodes to become nearly unavailable, triggering a bunch of anomaly alarms.
Should I request @VirMach to pay attention to this, or should I silently turn off the anomaly alerts?
Rest in peace Frankfurt.
Also holy shit @VirMach please - 12 to 18 hours for physical migration where driving gonna take at least 6 hours alone?!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
something's happening!
I bench YABS 24/7/365 unless it's a leap year.
Oh, nooo
We're all gonna die!
Brace for impact!!
https://microlxc.net/
Just curious--did you receive an email about this, or notice it on the network status page?
I'm seeing the new option to activate a secondary service! Trying to decide if I want to take advantage of the situation to permanently move my $3/year VPS back to the USA. (Looks like Miami, Chicago, NYC, Atlanta, LA, Amsterdam, and San Jose are the options.)
Amazing how my $8/yr VPS can fund an automated VPS Control Panel
食之无味 弃之可惜 - Too arduous to relish, too wasteful to discard.
anyone experiencing serious network problem from LAXA018?
Secondary service is generating. Please do not interrupt the process with any actions.I love the rave effect :-D
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Welp, 30 minutes later nothing changed and primary service is still down.
35 minutes clicked "Refresh" button... WHMCS loaded
-secondaryserver for this service ID... first one is gone from WHMCS, but still shows up in SolusVM panel.Booted primary from SolusVM, seems still to be working.
Booted secondary from SolusVM - no disk error.
I guess data transfer failed
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Trying different service, progress bar still not moving.
@VirMach javascript expects element with id dataprogress and there is no such thing in popup - there is one
progresselement, but noidattached.Okey, clicking Refresh detected that migration and showed nice(r) popup.
2-3 hours holly molly macaroni cheese
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
I created a secondary with no data migration and it was replying to pings after a few minutes. Had to use the rescue image to install my keys and change the password. The solus vm panel shows the secondary service has 5 TB of transfer quota instead of 100 GB the old one had!
Starting to think that
AMSD031T36is broken - I've tried to migrate two servers with data, but both ended with no disk fuck off. Tried reinstall one, no disk, fuck off.I've migrated one more with data that didn't end on
AMSD031T36and everything is working - bah, I even see "Secondary service" button in WHCMS and it loads nice page showing -primary and -secondary, timeleft and switch button.Why do I have a feeling in 6 days I am gonna get rekt - it will delete my failed-to-migrate primary and I will be left with two broken -secondary
This also totally nukes my plans to semi-manually
ddmigrate data from Frankfurt to Amsterdam after failed automatic migration - can't really migrate if the -secondary has no disk, lmao?EDIT: correct size 'disk' shows up in rescue mode and fdisk,
ddattempt it is! ;')EDIT2:
"This was a triumph. I'm making a note here: HUGE SUCCESS."
Only needed to rename eth0 to ens3 in interface, thanks Debian & VirMach network auto-reconfig!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
I don’t trust that either that’s why I’m pretty sure I left that part of the code disabled for now so I can process them manually. But I don’t trust me either. Hmm
>
I hate that server. I was going to have a look but I’ve mostly just been driving.
Frankfurt has been successfully destroyed. More driving, currently ahead of schedule in terms of the 12 to 18 hours downtime. I’ll make sure to break something so we’re back on track.
Oh noooeeeeee
https://microlxc.net/
REST IN PEACE FRANKFURT
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Love how there was no communication prior to to the downtime. Not heard anything since the attempt in January.
Proper low-end!
Wow, tried to create second service for one vm with data transfer, now I can't access it in the billing panel anymore 🥲
Cloudflare Timeout: Error code 524
From the sounds of it, Virmach has been driving aimlessly around the world's DCs, so maybe he doesn't have roaming on his phone?
Hey @Jab, did you get an email notification about this or you just monitoring the Network Status page all the time?
No e-mail
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Yeah, I also didn't get any e-mails about the FFME downtime/migrations. I loved Frankfurt.
I also tried to use that "Secondary Service" button on 1 of my 3 in FFME without success.
First it gave: " {"status":"error","message":"Not eligible to activate secondary service."} "
And now that panel keeps giving a CloudFlare timeout.
I think you guys were supposed to use the Secondary Service before shit went down aka WHMCS gonna shit itself because nodes offline.
Yeah, I know, I am being very helpful.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
This migration may take approximately 12 to 18 hours.SOON™
mere minutes left for everything come back online, right? Right? RIGHT?!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Hey, what's up?
NOT Frankfurt.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
It's a very long 12-18 hours.
Do you know Virmach reached 10000 customers in 2016?
cmon @VirMach deliver the bad news already :-D
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
frankfurt was top!
Do you guys think @VirMach even made to Amsterdam or was arrested and put in jail for speeding on German autobahn?
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
he was in AMS first wasnt he?
I bench YABS 24/7/365 unless it's a leap year.
No, migration is from Frankfurt to Amsterdam.
Last sign of life was "I killed Frankfurt, time to drive to Amsterdam".
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
welp, new day new happenings!
I bench YABS 24/7/365 unless it's a leap year.
@VirMach please look into LAXA018, network issue still persist
Is there a recovery plan for TYOC005?
Hello guys, I was keeping a database on a VPS from Virmach, which happened to be in Frankfurt. Initially, it was of little value so I didn't keep any backups or anything. I was planning to take it all off the next month and this mess happened.
Now the question is: how screwed am I? Is data loss likely at this point?
you would have to assume so
I bench YABS 24/7/365 unless it's a leap year.
my frankfurt vps is killed and not yet reborn in amsterdamn!