after be migrated from DEN001 to LAXA007, SolusVM panel shows that the ip still the same as DEN001 before(66.59.xxxxxx), and not receive any new ip of la, the panel shows the number of ipv4 is 1, it just only 66.59.xxxx
i checked that the 66.59.xxxxx is empty route now, that means my LAXA007 vps can't be used, even it can be boot normally and view by vnc, but it doesn't have any network because there is no new IP.
is this normal? do I need to open a ticket to request for your help? @VirMach
@tuc said:
Yet not seeing any new IPs as of now. Am I out of the boat?
IIRC you are on LAXA031, I also am on the node. Have a new IPv4, switched over early, does not route. So you are not behind at all.
I don't think they are up to LAXA031 yet. Status page shows LAXA004, LAXA005, LAXA006, LAXA007 are confirmed ready.
I am on this. Only new IP is listed in the panel (no old IP). Neither new IP nor old IP work. Result: no connectivity.
Tampa is done, I don't think the routing's correct on Hivelocity's end. Still need to look into it further.
LAX2 is done and it seems like routing is correct. We had them do it wrong the last time for this location so I think they probably put a note there so they wouldn't do it like that again.
Chicago2 is waiting on Hivelocity, they set this up in a strange way, probably delayed to Monday.
Chicago1 & Miami, waiting on QuadraNet. Routing should be fine, it's related to the switches.
Looking into these, I also noticed a lot of strange issues, networking related, that are only happening with Hivelocity, all three locations. These existed before the any of the changes. We tried speaking to them about it a couple weeks ago and they just basically refused to help out and just said our switch is broken or misconfigured (they configured it for us.) This was for Chicago, where the network engineer basically definitely pretended like he checked their end, except if he actually did he'd see it was actually set up on their witch and not ours, so at least for that location he basically said "it's a problem with the switch" but it's... their switch doing the gateways.> @tetech said:
@tuc said:
Yet not seeing any new IPs as of now. Am I out of the boat?
IIRC you are on LAXA031, I also am on the node. Have a new IPv4, switched over early, does not route. So you are not behind at all.
I don't think they are up to LAXA031 yet. Status page shows LAXA004, LAXA005, LAXA006, LAXA007 are confirmed ready.
I am on this. Only new IP is listed in the panel (no old IP). Neither new IP nor old IP work. Result: no connectivity.
Looking into it, could be needing a restart server-side.
(edit) Nope, my fault here. Although the old ones should still be working, the new ones shouldn't have been changed yet. Going back to double checking this location now that the others are out of the way or waiting on DC.
@tuc said:
Yet not seeing any new IPs as of now. Am I out of the boat?
IIRC you are on LAXA031, I also am on the node. Have a new IPv4, switched over early, does not route. So you are not behind at all.
I don't think they are up to LAXA031 yet. Status page shows LAXA004, LAXA005, LAXA006, LAXA007 are confirmed ready.
I am on this. Only new IP is listed in the panel (no old IP). Neither new IP nor old IP work. Result: no connectivity.
Confirmed fixed. My VM on LAXA031 is working now with the new IP, yes it only shows the new IP, not the old one, but the new one works now.
Note: I am using a 255.255.255.128 subnet with the gateway as xx.xx.xx.129
Edit:
@gritty said: after be migrated from DEN001 to LAXA007, SolusVM panel shows that the ip still the same as DEN001
Confirmed. My VM that was in Denver and now is on LAXA007 has only one IPv4 and it is still the one from Denver and does not work.
@vgood said: now
09:54
Saturday, September 23, 2023
Eastern Time (ET)
Yep, my VM on LAX2Z019 still times out in the panel. I expect that VirMach is a little behind on this projected resolution time due to fixing new IP related issues. Let's give him a little while longer to resolve this issue.
This VM does have both the new and the old IP addresses, so that is some progress.
I am currently traveling in mostly remote areas until sometime in April 2024. Consequently DM's sent to me will go unanswered during this time.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
@tuc said:
Yet not seeing any new IPs as of now. Am I out of the boat?
IIRC you are on LAXA031, I also am on the node. Have a new IPv4, switched over early, does not route. So you are not behind at all.
I don't think they are up to LAXA031 yet. Status page shows LAXA004, LAXA005, LAXA006, LAXA007 are confirmed ready.
I am on this. Only new IP is listed in the panel (no old IP). Neither new IP nor old IP work. Result: no connectivity.
My VM on LAXA031 is working now with the new IP, yes it only shows the new IP, not the old one, but the new one works now.
Note: I am using a 255.255.255.128 subnet with the gateway as xx.xx.xx.129
I added a few other issues to my comment to make you feel better, dog face.
I am currently traveling in mostly remote areas until sometime in April 2024. Consequently DM's sent to me will go unanswered during this time.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
SolusVM is getting so frustrating to use that I actually started working on our own control panel. It's about 0.7% done.
But yeah, the changes went from probably taking an hour on the older version to 36 hours ETA on this new one, and even then with a bunch of problems. If we do it at the same rate we used to do them just fine, it'll crash the node basically. From... changing an IP address to the main IP address. I don't know, nothing making sense anymore with them.
Oh and by making API calls for the changes, now it's causing SolusVM to miss other API calls. Really wonderful.
✅ Tampa has been completed but has routing issues, we are waiting on the datacenter. Do not switch to new IPv4 yet.
Probably should have read that before switching but welp, it's an idler, so it gonna idle even moar!
Btw. VirMach not sure if "✅" works with "routing isssues, do not switch" :P
Yeah I thought about that but I didn't feel like finding a halfway between X and checkmark emoji, plus the ticket's already in for them to fix it and then I'd have to constantly check to make sure I don't forget to update the page when it's done.
Are you not able to manually add back in your old address? If you're not let me know because it means SolusVM did push an update breaking that. To be fair to them though in this case they would actually be making an improvement, technically. It's just that for something like 8 years they didn't do anything about that. I could see why it ends up overloading the nodes and taking so long now if they did actually add in a script outside of the database that goes in and every time the main IP is changed or additional IP is removed it probably does something wise like regenerating the entire list of restrictions for all IP addresses.
@tbnuser said:
My server on RYZE.LAX-A022.VMS dropped off the internet and I had to reconfigure the network with the new IP to get it to work again.
That's not good. It does look like possibly SolusVM now fully restricts you from being able to use the old address if it's just soft removed on SolusVM. I really do think it's the situation I stated above in that case, where it's literally regenerating all the specific restrictions to avoid IPv4 stealing after every change now, something it didn't used to do.
The problem is if we don't strip away the old address then we'd have to reconfigure everyone again at the end, plus most of them have been processed. I'll see if we can shift strategy and not remove the old address for any remaining changes, and just reconfigure at the end for all but that could get messier. I'll look into it further and see how we communicated everything to see what'd end up being better in this scenario.
Okay, we're going to stop doing the automatic changes. We'll just let people change it themselves, and then proceed with automatic changes on the 29th. I've skimmed through the message in the same way anyone would read a wall of text and it definitely sounds like it would be changing on the 29th. We'll just update the page with when each node is available to change, and won't change the main IPv4 until that day.
I'm going to focus on trying to reconfigure any that were hanged and put a notice where people can contact us for their old address if desired in this case, because another 30 hours of adding the old IPv4 back and trying to make sure people get the exact old address without it breaking anything else sounds risky for no reason.
Sometimes I always wonder why unfortunate things happen to good people. Although VirMach is doing a business here, but a lot of times I feel like it is a charity, things just broke randomly and VirMach still tried his best to keep things afloat.
When VirMach migrate away from CC, a lot of people say VirMach probably cannot survive with those insane deals (come on, $0.95/yr with IPv4? That cannot even cover the cost of IP), but yet they did. Now when DediPath encountered issue, VirMach still managed to resolve it so quickly and everything seems to be smooth on my side as a user.
Sure, there are times where support can be slow or poor, but based on my ~6-7 years of usage I don't really encounter major hiccups. Even with all the inflation, as far as I know VirMach never increased recurring price of any services. (Keep in mind the pricing are rock-bottom, literally unsustainable)
Thanks VirMach, just want to show my appreciation here after following probably thousand pages of forum posts (OGF/LES), hope that one day everything can back to normal and we can see VirMach 10th anniversary deals
@VirMach Solus re-config. network does the job improperly for vps setup using iso. Please just let it be an option for whoever needs. My vps are all OK now. Please don't touch them
@tuc said: @VirMach Solus re-config. network does the job improperly for vps setup using iso. Please just let it be an option for whoever needs. My vps are all OK now. Please don't touch them
I had that problem in the past, since Centos likes to do the ens3 thing (which RHEL thinks is a better way). However, Solus/Virmach's custom scripts seem to want to use eth0, eth0:0, etc. and screw things up. Now if I do a Centos install I change the network naming to eth0 specifically to try to mitigate that.
@VirMach said:
Okay, we're going to stop doing the automatic changes. We'll just let people change it themselves, and then proceed with automatic changes on the 29th. I've skimmed through the message in the same way anyone would read a wall of text and it definitely sounds like it would be changing on the 29th. We'll just update the page with when each node is available to change, and won't change the main IPv4 until that day.
I'm going to focus on trying to reconfigure any that were hanged and put a notice where people can contact us for their old address if desired in this case, because another 30 hours of adding the old IPv4 back and trying to make sure people get the exact old address without it breaking anything else sounds risky for no reason.
Yeah. That's the way I read it. Thought I had until 29th to change IPs - ideally that would have been during a low usage period. So this was a surprise. Once the new IP showed I lost all network connectivity. The weird thing was that the old IP was still in the ifcfg-eth0 file - so it didn't get changed automatically, but was also dead. I changed it manually (rebooted, etc.) but IP a still showed the same old IP which no longer worked. There are a bunch of files in /etc/sysconfig/network-scripts that I had never seen before so I renamed anything that looked like it could be read as a config file, and that seemed to have done the trick. Finally things seem to be working with the new IP so I hope things don't get reverted.
@tuc said: My vps are all OK now. Please don't touch them
@SeaScoot said: Finally things seem to be working with the new IP so I hope things don't get reverted.
I can't promise that, unfortunately, but we do try to avoid touching it unless necessary. In the case of these that got changed early, I've kept a list so we'll likely not do anything on September 29th for them, but reconfigurations will run today most likely for them if I can get it stop having issues.
Assuming it works correctly and your server pings, it'll skip over it. Otherwise if you have firewall blocking that, it'll reconfigure it. The script is more complicated than that and it also skips over some other scenarios as well to try to reduce impact. But it also breaks because it's the worst operation ever in terms of SolusVM API, so what may end up happening sometimes is if it fails really bad then it requires manual work and there's no way we can go one by one checking them before reconfiguring in that scenario and it'll just have to be mass reconfigured. Because if we don't we'll get a thousand tickets over the next week. This way we only get several hundred
Advanced probing feature by SolusVM. I wonder what it actually does... it wouldn't be very difficult for it to do it properly instead of breaking it.
@SeaScoot said: The weird thing was that the old IP was still in the ifcfg-eth0 file - so it didn't get changed automatically, but was also dead. I changed it manually (rebooted, etc.) but IP a still showed the same old IP which no longer worked. There are a bunch of files in /etc/sysconfig/network-scripts that I had never seen before so I renamed anything that looked like it could be read as a config file, and that seemed to have done the trick
Yeah, I really don't know what change they made, I read through all the changelogs. Supposedly the thing they "fixed"... was "fixed" in 2021 (it wasn't.)
I still suspect it's doing a halfway reconfigure very poorly. Because just doing the network update I mentioned shouldn't overload the node like that if it's done too quickly, and I noticed that whatever it was doing was causing some services to come online, which shouldn't be possible unless it's doing some reconfiguration. I haven't had time to dedicate to trying to figure out what the hell they did but it would have obviously been great if I did.
LAX2Z018 node has received an email about the change in IP address, but it has not provided the actual IP address.
Dear VirMach Customer,
Quick Version: Your service's IP address will be changed on September 29th. It is not ready yet, but we wanted to give you advanced notice that it will be ****.
I am currently traveling in mostly remote areas until sometime in April 2024. Consequently DM's sent to me will go unanswered during this time.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
Comments
after be migrated from DEN001 to LAXA007, SolusVM panel shows that the ip still the same as DEN001 before(66.59.xxxxxx), and not receive any new ip of la, the panel shows the number of ipv4 is 1, it just only 66.59.xxxx
i checked that the 66.59.xxxxx is empty route now, that means my LAXA007 vps can't be used, even it can be boot normally and view by vnc, but it doesn't have any network because there is no new IP.
is this normal? do I need to open a ticket to request for your help? @VirMach
I am on this. Only new IP is listed in the panel (no old IP). Neither new IP nor old IP work. Result: no connectivity.
Tampa is done, I don't think the routing's correct on Hivelocity's end. Still need to look into it further.
LAX2 is done and it seems like routing is correct. We had them do it wrong the last time for this location so I think they probably put a note there so they wouldn't do it like that again.
Chicago2 is waiting on Hivelocity, they set this up in a strange way, probably delayed to Monday.
Chicago1 & Miami, waiting on QuadraNet. Routing should be fine, it's related to the switches.
Looking into these, I also noticed a lot of strange issues, networking related, that are only happening with Hivelocity, all three locations. These existed before the any of the changes. We tried speaking to them about it a couple weeks ago and they just basically refused to help out and just said our switch is broken or misconfigured (they configured it for us.) This was for Chicago, where the network engineer basically definitely pretended like he checked their end, except if he actually did he'd see it was actually set up on their witch and not ours, so at least for that location he basically said "it's a problem with the switch" but it's... their switch doing the gateways.> @tetech said:
Looking into it, could be needing a restart server-side.
(edit) Nope, my fault here. Although the old ones should still be working, the new ones shouldn't have been changed yet. Going back to double checking this location now that the others are out of the way or waiting on DC.
Fixed.
Confirmed fixed. My VM on LAXA031 is working now with the new IP, yes it only shows the new IP, not the old one, but the new one works now.
Note: I am using a 255.255.255.128 subnet with the gateway as xx.xx.xx.129
Edit:
Confirmed. My VM that was in Denver and now is on LAXA007 has only one IPv4 and it is still the one from Denver and does not work.
Yep, my VM on LAX2Z019 still times out in the panel. I expect that VirMach is a little behind on this projected resolution time due to fixing new IP related issues. Let's give him a little while longer to resolve this issue.
This VM does have both the new and the old IP addresses, so that is some progress.
I am currently traveling in mostly remote areas until sometime in April 2024. Consequently DM's sent to me will go unanswered during this time.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
That's terrible, I love it keep going
URL Shortener | YetiNode | Come join us on the MetalVPS IRC channel!!! | Don't be a jerk, let YetiNode do the work.
so migration from Seattle to LA is not happening?
I added a few other issues to my comment to make you feel better, dog face.
I am currently traveling in mostly remote areas until sometime in April 2024. Consequently DM's sent to me will go unanswered during this time.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
Confirmed fixed. Thanks.
We'd be so lucky!
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Some VPS in LAX not getting new IP yet
SolusVM is getting so frustrating to use that I actually started working on our own control panel. It's about 0.7% done.
But yeah, the changes went from probably taking an hour on the older version to 36 hours ETA on this new one, and even then with a bunch of problems. If we do it at the same rate we used to do them just fine, it'll crash the node basically. From... changing an IP address to the main IP address. I don't know, nothing making sense anymore with them.
Oh and by making API calls for the changes, now it's causing SolusVM to miss other API calls. Really wonderful.
Probably should have read that before switching but welp, it's an idler, so it gonna idle even moar!
Btw. VirMach not sure if "✅" works with "routing isssues, do not switch" :P
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
Yeah I thought about that but I didn't feel like finding a halfway between X and checkmark emoji, plus the ticket's already in for them to fix it and then I'd have to constantly check to make sure I don't forget to update the page when it's done.
Are you not able to manually add back in your old address? If you're not let me know because it means SolusVM did push an update breaking that. To be fair to them though in this case they would actually be making an improvement, technically. It's just that for something like 8 years they didn't do anything about that. I could see why it ends up overloading the nodes and taking so long now if they did actually add in a script outside of the database that goes in and every time the main IP is changed or additional IP is removed it probably does something wise like regenerating the entire list of restrictions for all IP addresses.
My server on RYZE.LAX-A022.VMS dropped off the internet and I had to reconfigure the network with the new IP to get it to work again.
Virtfuuuuuuuusion
I bench YABS 24/7/365 unless it's a leap year.
Doesn't really have an API.
OMG that is absolutely horrifying. Makes me wet and leaves me wanting. Keep going, don't stop now for the love of Peter!
URL Shortener | YetiNode | Come join us on the MetalVPS IRC channel!!! | Don't be a jerk, let YetiNode do the work.
That's not good. It does look like possibly SolusVM now fully restricts you from being able to use the old address if it's just soft removed on SolusVM. I really do think it's the situation I stated above in that case, where it's literally regenerating all the specific restrictions to avoid IPv4 stealing after every change now, something it didn't used to do.
The problem is if we don't strip away the old address then we'd have to reconfigure everyone again at the end, plus most of them have been processed. I'll see if we can shift strategy and not remove the old address for any remaining changes, and just reconfigure at the end for all but that could get messier. I'll look into it further and see how we communicated everything to see what'd end up being better in this scenario.
Okay, we're going to stop doing the automatic changes. We'll just let people change it themselves, and then proceed with automatic changes on the 29th. I've skimmed through the message in the same way anyone would read a wall of text and it definitely sounds like it would be changing on the 29th. We'll just update the page with when each node is available to change, and won't change the main IPv4 until that day.
I'm going to focus on trying to reconfigure any that were hanged and put a notice where people can contact us for their old address if desired in this case, because another 30 hours of adding the old IPv4 back and trying to make sure people get the exact old address without it breaking anything else sounds risky for no reason.
Sometimes I always wonder why unfortunate things happen to good people. Although VirMach is doing a business here, but a lot of times I feel like it is a charity, things just broke randomly and VirMach still tried his best to keep things afloat.
When VirMach migrate away from CC, a lot of people say VirMach probably cannot survive with those insane deals (come on, $0.95/yr with IPv4? That cannot even cover the cost of IP), but yet they did. Now when DediPath encountered issue, VirMach still managed to resolve it so quickly and everything seems to be smooth on my side as a user.
Sure, there are times where support can be slow or poor, but based on my ~6-7 years of usage I don't really encounter major hiccups. Even with all the inflation, as far as I know VirMach never increased recurring price of any services. (Keep in mind the pricing are rock-bottom, literally unsustainable)
Thanks VirMach, just want to show my appreciation here after following probably thousand pages of forum posts (OGF/LES), hope that one day everything can back to normal and we can see VirMach 10th anniversary deals
@VirMach Solus re-config. network does the job improperly for vps setup using iso. Please just let it be an option for whoever needs. My vps are all OK now. Please don't touch them
thoroughly molests @tuc 's VPS
URL Shortener | YetiNode | Come join us on the MetalVPS IRC channel!!! | Don't be a jerk, let YetiNode do the work.
reconfig sets to eth0
but im using ens3
I bench YABS 24/7/365 unless it's a leap year.
I had that problem in the past, since Centos likes to do the ens3 thing (which RHEL thinks is a better way). However, Solus/Virmach's custom scripts seem to want to use eth0, eth0:0, etc. and screw things up. Now if I do a Centos install I change the network naming to eth0 specifically to try to mitigate that.
How are you... online?
Yeah. That's the way I read it. Thought I had until 29th to change IPs - ideally that would have been during a low usage period. So this was a surprise. Once the new IP showed I lost all network connectivity. The weird thing was that the old IP was still in the ifcfg-eth0 file - so it didn't get changed automatically, but was also dead. I changed it manually (rebooted, etc.) but IP a still showed the same old IP which no longer worked. There are a bunch of files in /etc/sysconfig/network-scripts that I had never seen before so I renamed anything that looked like it could be read as a config file, and that seemed to have done the trick. Finally things seem to be working with the new IP so I hope things don't get reverted.
I can't promise that, unfortunately, but we do try to avoid touching it unless necessary. In the case of these that got changed early, I've kept a list so we'll likely not do anything on September 29th for them, but reconfigurations will run today most likely for them if I can get it stop having issues.
Assuming it works correctly and your server pings, it'll skip over it. Otherwise if you have firewall blocking that, it'll reconfigure it. The script is more complicated than that and it also skips over some other scenarios as well to try to reduce impact. But it also breaks because it's the worst operation ever in terms of SolusVM API, so what may end up happening sometimes is if it fails really bad then it requires manual work and there's no way we can go one by one checking them before reconfiguring in that scenario and it'll just have to be mass reconfigured. Because if we don't we'll get a thousand tickets over the next week. This way we only get several hundred
Advanced probing feature by SolusVM. I wonder what it actually does... it wouldn't be very difficult for it to do it properly instead of breaking it.
Yeah, I really don't know what change they made, I read through all the changelogs. Supposedly the thing they "fixed"... was "fixed" in 2021 (it wasn't.)
I still suspect it's doing a halfway reconfigure very poorly. Because just doing the network update I mentioned shouldn't overload the node like that if it's done too quickly, and I noticed that whatever it was doing was causing some services to come online, which shouldn't be possible unless it's doing some reconfiguration. I haven't had time to dedicate to trying to figure out what the hell they did but it would have obviously been great if I did.
LAX2Z018 node has received an email about the change in IP address, but it has not provided the actual IP address.
Dear VirMach Customer,
Quick Version: Your service's IP address will be changed on September 29th. It is not ready yet, but we wanted to give you advanced notice that it will be ****.
@adef_1 - Look in SolusVM I expect it will show up there when it is assigned, it did for me.
I am currently traveling in mostly remote areas until sometime in April 2024. Consequently DM's sent to me will go unanswered during this time.
For staff assistance or support issues please use the helpdesk ticket system at https://support.lowendspirit.com/index.php?a=add
@VirNach, on LAXA-005, is 66.59.xxx.xxx will be permanent or 141.11.xxx.xx?