@Mamyyy said: idk but I've been looping ssh connect since the last hour without any sign of life.
Its 90001 milliseconds error in control panel.
You are correct that both the billing panel and SolusVM time out and steal is up to 50%. But I am connected right now, so it is having issues, but is not down. The top shown below is CDT
and I run a backup DNS server for that region on it and this is a graph of the response time to a DNS request from the four locations listed. Screenshot was taken two minutes ago .
EDIT: Maybe it is dropping VMs due to overloading and it just has not gotten to me yet.
EDIT2: OK, now I get no response as well. (11:35 CDT)
@foxcoo said:
TYOC002S offline, please check. I think someone is mining with it.Because I have noticed that the online CPU usage has now dropped significantly. :'(
@FrankZ said: back, accessible via the billing panel, SolusVM, and ssh.
We're going to reach out to some top users especially around Saturday night and figure out how it's being used and how it could potentially be optimized. That probably means anyone who noticed it today might be one of the top users as that's how usually things go. Once we deal with that if it doesn't calm down then we need to evaluate all the rest. The server's fine, the disks are fine, the controller is fine, and it's online, but it keeps getting stuck in scenario every Saturday because of the way it's being utilized.
Tokyo essentially has lots and lots of sporadic bursting patterns that can amplify eachother to the point where it's temporarily toast until it catches back up.
@VirMach will you please connect NYCB033X to Solus Panel? It was never connected and I'm running low on OS install, but it says
"If you run out, please use the control panel directly." There is NO Control Panel!
@tenpera said: @VirMach will you please connect NYCB033X to Solus Panel? It was never connected and I'm running low on OS install, but it says
"If you run out, please use the control panel directly." There is NO Control Panel!
I definitely need more details like your plan name and what functions you performed such as Ryzen migrate.
@foxcoo said:
TYOC002S offline, please check. I think someone is mining with it.Because I have noticed that the online CPU usage has now dropped significantly. :'(
@FrankZ said: back, accessible via the billing panel, SolusVM, and ssh.
We're going to reach out to some top users especially around Saturday night and figure out how it's being used and how it could potentially be optimized. That probably means anyone who noticed it today might be one of the top users as that's how usually things go. Once we deal with that if it doesn't calm down then we need to evaluate all the rest. The server's fine, the disks are fine, the controller is fine, and it's online, but it keeps getting stuck in scenario every Saturday because of the way it's being utilized.
Tokyo essentially has lots and lots of sporadic bursting patterns that can amplify eachother to the point where it's temporarily toast until it catches back up.
Weird. mine is still down.
This is the first time I'm experiencing this since it was provisioned in June.
cPanel/shared hosting IP changes. I spoke with our developer and we decided to change plans a little. Not that it matters, I don't think I ever shared the original plan.
I wanted some feedback from anyone who has a dedicated IP address on VPShared or VirMach cPanel plan. One of the nameserver IP addresses is changing and we're probably going to add a 3rd and 4th. Grabbing the exact IP you might have previously and then mapping it to the new one and changing it is going to be difficult to get down perfectly, especially since we have an unknown ETA for the LOA being processed. We do already have a smaller block which we can use for the main shared IP and communicate that sooner, and essentially tell everyone who has a dedicated IP with an A record to discontinue using it. While I'm setting up the new nameservers I'll also try to set up the Cloudflare module again if that's supported, which will allow you to switch over to that for the time being with a shared IP.
This way there's virtually zero downtime.
It also makes adding the dedicated IPv4 smoother and that way there's no rush on having to use it again. Plus if you forget to do it, you will immediately know it's broken and then have the new one already listed on your cPanel so even if the above communication is missed, it's where it should be and at the exact moment it breaks the new one is there.
Of course for people who really want their dedicated IP switched sooner we do have some extras we can manually switch it to earlier. I won't mention this part in the email as it'll just be unhelpful and potentially confusing to most people but you'd just make a priority ticket and we'll keep an eye out for them.
Created on: 26 September 2022 at 07:14 (Delivered after 2 seconds)
From: Virtual Machine Solutions LLC Technical Support tickets@virmach.com Using Virtual Machine Solutions LLC
(...)
This is a (7) day notice. The change will be completed on Thursday, September 29, 2022.
Running behind schedule, a lot it seems?
Also it still seems like not everything is done - like I got ticket about FFME002, but not about FFME001 and both uses the same address space so I expect both to be migrated ;')
I suggest to update template a this point - if someone will get this ticket for first time and will stop reading after "This is a 7 day notice" (I don't have time for this now, 7 days, I will read it Friday) - (s)he gonna get pissed in few days..
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
We're going to reach out to some top users especially around Saturday night and figure out how it's being used and how it could potentially be optimized. That probably means anyone who noticed it today might be one of the top users as that's how usually things go. Once we deal with that if it doesn't calm down then we need to evaluate all the rest. The server's fine, the disks are fine, the controller is fine, and it's online, but it keeps getting stuck in scenario every Saturday because of the way it's being utilized.
Tokyo essentially has lots and lots of sporadic bursting patterns that can amplify eachother to the point where it's temporarily toast until it catches back up.
CPU steal has been a constant issue since day 2 after provisioning.
Is it too difficult to suspend CPU/IO heavy users and network abusers?
That is not a suggestion, it is a piss-poorly worded question.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Reminds me: I need to top-up one of my mobile phone SIMs - it's only 3G though.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@VirMach said:
(edit) I'll try to locate the older backups and on the chance that you're insanely lucky and it's there, it happened to be the specific segment that got copied over fully intact, and so on. It'll take several days at the very least. Make sure you have a priority ticket if you already don't.
I think that's really the best I can hope for at this point, thank you for going the extra mile on this! My latest backup is (negligently on my part...) ~May, so if you have anything newer than that, I'll take it! Late June would be great, an obviously if there was something from that week I'd be over the moon.
RE: priority ticket, I created a very general one should I have a specific one for a backup restoration? The ticket I already created won't let me reply to it even to add context I missed when I created it heh.
Hey @VirMach I just wanted to make sure this was still on your radar. I created a new priority ticket specifically for a backup restoral since my original priority ticket was closed. I'd really like to get things back up and running before the 3 month anniversary of the botched migration (< 2 weeks away).
Only one example of which there are likely thousands at any given minute on a provider's network. I'm seeing similar with plex lusers in AMS.
Let's double the network blasts.. (rhetorical)
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@AlwaysSkint said:
Rant/
Continuously bombarding virtual NICs and computing resources by some ass that doesn't know how to configure dropbox and/or doesn't give a crap..
Only one example of which there are likely thousands at any given minute on a provider's network. I'm seeing similar with plex lusers in AMS.
Let's double the network blasts.. (rhetorical)
I've since blocked 'em and the system log is swamped, so can't tell you offhand. I did check against known port assignments though. The flattened network of AMS doesn't help - broadcasting across subnets.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
We're going to reach out to some top users especially around Saturday night and figure out how it's being used and how it could potentially be optimized. That probably means anyone who noticed it today might be one of the top users as that's how usually things go. Once we deal with that if it doesn't calm down then we need to evaluate all the rest. The server's fine, the disks are fine, the controller is fine, and it's online, but it keeps getting stuck in scenario every Saturday because of the way it's being utilized.
Tokyo essentially has lots and lots of sporadic bursting patterns that can amplify eachother to the point where it's temporarily toast until it catches back up.
CPU steal has been a constant issue since day 2 after provisioning.
Is it too difficult to suspend CPU/IO heavy users and network abusers?
The usage is extremely spiky in that location for almost every user. If that's how it's going to be used by most customers in that region then obviously to stop this we can't just start suspending everyone during peak times and it'd be better to reach out and try to smooth out the usage where we can first to at least avoid the server from locking up on Saturdays.
I've since blocked 'em and the system log is swamped, so can't tell you offhand. I did check against known port assignments though. The flattened network of AMS doesn't help - broadcasting across subnets.
Amsterdam will be fixed along with the IP changes that are coming this week, as in they'll be split off.
We already got a report for TYOC026 which I've been looking into and about to create a network status for, and then no reports for FFME006 but it looks like it's been overloading. It's not down yet but effectively unusable. I'll add it to the list and make the announcement alongside TYOC026.
@VirMach said: .. along with the IP changes that are coming this week, as in they'll be split off.
I was guessing/hoping that might happen.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
I also got ip replacement mail. But I don't really want my server location to get changed from Amsterdam/Frankfurt to US.(though IP change is fine, but not location)
I have already mentioned this in the tickets,
Hopefully it get noticed by @VirMach
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@Kaito said:
I also got ip replacement mail. But I don't really want my server location to get changed from Amsterdam/Frankfurt to US.(though IP change is fine, but not location)
I have already mentioned this in the tickets,
Hopefully it get noticed by @VirMach
Location won't change. Right now the IP addresses are not ours yet until they're assigned, no initially geolocation will be wrong.
Some of these Virmach folks need to find a new line of work. I've been doing this for over 30 years, and I've never seen critical service outages anywhere close to what Virmach is reporting IN REAL TIME AGAIN. Our server which weathered the last fiasco wasn't quite so lucky this time around. Here's a tip: SAVE YOUR MONEY AND GO ELSEWHERE!
@NerdUno said:
Some of these Virmach folks need to find a new line of work. I've been doing this for over 30 years, and I've never seen critical service outages anywhere close to what Virmach is reporting IN REAL TIME AGAIN. Our server which weathered the last fiasco wasn't quite so lucky this time around. Here's a tip: SAVE YOUR MONEY AND GO ELSEWHERE!
SEAZ009 is offline for 2-3 days and being worked on.
SEAZ010 is offline for 1-2 days and waiting on datacenter.
Nothing else is offline. Some nodes have partial issues but already addressed. Out of all the times you could've made this comment right now is the least "critical" and I'm assuming it just means you were recently affected on one of these nodes.
After it's back online and you're able to get your data off, create a ticket provide the ticket # here after clicking cancel and I'll provide you a refund so you can follow your own advice, save your money and go elsewhere.
The usage is extremely spiky in that location for almost every user. If that's how it's going to be used by most customers in that region then obviously to stop this we can't just start suspending everyone during peak times and it'd be better to reach out and try to smooth out the usage where we can first to at least avoid the server from locking up on Saturdays.
It's not just Saturday for me. and are you sure it's in that location instead of in this node? I have no problem with my Tokyo NVMe VPSes, just the storage one.
As someone mentioned here earlier, maybe there're some abusers doing mining stuffs/ torrenting all the time.
The usage is extremely spiky in that location for almost every user. If that's how it's going to be used by most customers in that region then obviously to stop this we can't just start suspending everyone during peak times and it'd be better to reach out and try to smooth out the usage where we can first to at least avoid the server from locking up on Saturdays.
It's not just Saturday for me. and are you sure it's in that location instead of in this node? I have no problem with my Tokyo NVMe VPSes, just the storage one.
As someone mentioned here earlier, maybe there're some abusers doing mining stuffs/ torrenting all the time.
Yes, I'll quote again:
The usage is extremely spiky in that location for almost every user
This means every day. The usage in that location is not like the others. 90% of the people have extremely spiky traffic/disk usage. As in, it appears most users are torrenting, using it as a VPN, or a private media server. Even the bandwidth usage is nothing like the other locations. So if you are someone who is not doing that, you're in the minority in Tokyo.
@foitin said: I have no problem with my Tokyo NVMe VPSes, just the storage one.
That's because of the "NVMe" part of that. Bandwidth usage will naturally be higher in proportion to the disk. We're seeing higher network usage in Tokyo NVMe versus other locations as well but since the nodes don't have 130TB disk that ends up being lower still.
Anyway, we'll most likely be contacting the highest users today and asking them what they're running and how they can reduce usage to at least ensure these people that were spiking the most on Saturday don't do it again next week, then we'll go from there and see how it improves. I've been monitoring everything for a couple days now so we should at least be able to make some level of improvement but it'll still be nowhere close to NYC or LAX.
I wanted to comment again that I have no ticket for FFME01 IP migration, but I have for FFME02 and FFME04... but I see secondary IP added to WHMCS, but no ticket, at all.
No idea if you want to check that. id=613983
[ and there is no new IP in panel and/or ticket for NYCB035 and NYCB036 (Dedipath), but there is IP for NYCB014 (PSINet/Cogent) - but this I will assume it's on purpose.
and nothing for DENZ001 too, but I think you mentioned this a later batch]
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Comments
NYCB035 offline
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
TYOC002S offline, please check. I think someone is mining with it.Because I have noticed that the online CPU usage has now dropped significantly. :'(
and it's back after 6 hours - not restarted as uptime is in days... seems like network died / nullrouted?
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Tokyo storage whatever the node name is is down
TYOC002S is not down, has no packet loss, but does have 30% CPU steal so it may not be very responsive.
EDIT: Graph added
idk but I've been looping ssh connect since the last hour without any sign of life.
Its 90001 milliseconds error in control panel.
You are correct that both the billing panel and SolusVM time out and steal is up to 50%. But I am connected right now, so it is having issues, but is not down. The top shown below is CDT
and I run a backup DNS server for that region on it and this is a graph of the response time to a DNS request from the four locations listed. Screenshot was taken two minutes ago .
EDIT: Maybe it is dropping VMs due to overloading and it just has not gotten to me yet.
EDIT2: OK, now I get no response as well. (11:35 CDT)
TYOC002S is back, accessible via the billing panel, SolusVM, and ssh.
So close
We're going to reach out to some top users especially around Saturday night and figure out how it's being used and how it could potentially be optimized. That probably means anyone who noticed it today might be one of the top users as that's how usually things go. Once we deal with that if it doesn't calm down then we need to evaluate all the rest. The server's fine, the disks are fine, the controller is fine, and it's online, but it keeps getting stuck in scenario every Saturday because of the way it's being utilized.
Tokyo essentially has lots and lots of sporadic bursting patterns that can amplify eachother to the point where it's temporarily toast until it catches back up.
@VirMach will you please connect NYCB033X to Solus Panel? It was never connected and I'm running low on OS install, but it says
"If you run out, please use the control panel directly." There is NO Control Panel!
I definitely need more details like your plan name and what functions you performed such as Ryzen migrate.
Weird. mine is still down.
This is the first time I'm experiencing this since it was provisioned in June.
cPanel/shared hosting IP changes. I spoke with our developer and we decided to change plans a little. Not that it matters, I don't think I ever shared the original plan.
I wanted some feedback from anyone who has a dedicated IP address on VPShared or VirMach cPanel plan. One of the nameserver IP addresses is changing and we're probably going to add a 3rd and 4th. Grabbing the exact IP you might have previously and then mapping it to the new one and changing it is going to be difficult to get down perfectly, especially since we have an unknown ETA for the LOA being processed. We do already have a smaller block which we can use for the main shared IP and communicate that sooner, and essentially tell everyone who has a dedicated IP with an A record to discontinue using it. While I'm setting up the new nameservers I'll also try to set up the Cloudflare module again if that's supported, which will allow you to switch over to that for the time being with a shared IP.
This way there's virtually zero downtime.
It also makes adding the dedicated IPv4 smoother and that way there's no rush on having to use it again. Plus if you forget to do it, you will immediately know it's broken and then have the new one already listed on your cPanel so even if the above communication is missed, it's where it should be and at the exact moment it breaks the new one is there.
Of course for people who really want their dedicated IP switched sooner we do have some extras we can manually switch it to earlier. I won't mention this part in the email as it'll just be unhelpful and potentially confusing to most people but you'd just make a priority ticket and we'll keep an eye out for them.
Any suggestions?
Created on: 26 September 2022 at 07:14 (Delivered after 2 seconds)
From: Virtual Machine Solutions LLC Technical Support tickets@virmach.com Using Virtual Machine Solutions LLC
(...)
This is a (7) day notice. The change will be completed on Thursday, September 29, 2022.
Running behind schedule, a lot it seems?
Also it still seems like not everything is done - like I got ticket about FFME002, but not about FFME001 and both uses the same address space so I expect both to be migrated ;')
I suggest to update template a this point - if someone will get this ticket for first time and will stop reading after "This is a 7 day notice" (I don't have time for this now, 7 days, I will read it Friday) - (s)he gonna get pissed in few days..
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
CPU steal has been a constant issue since day 2 after provisioning.
Is it too difficult to suspend CPU/IO heavy users and network abusers?
Why does my VPS traffic become 4g?
That is not a suggestion, it is a piss-poorly worded question.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Don't poke fun. If you were living under 4g's you'd type gibberish too.
Reminds me: I need to top-up one of my mobile phone SIMs - it's only 3G though.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
I misread it! It's 40g
Maybe I'm the Terminator
Presumably you are fixing the previous traffic bugs.
It may also be necessary to add a button about reverting to the default traffic.
@Ademan said:
Hey @VirMach I just wanted to make sure this was still on your radar. I created a new priority ticket specifically for a backup restoral since my original priority ticket was closed. I'd really like to get things back up and running before the 3 month anniversary of the botched migration (< 2 weeks away).
Cheers!
Rant/
Continuously bombarding virtual NICs and computing resources by some ass that doesn't know how to configure dropbox and/or doesn't give a crap..
Only one example of which there are likely thousands at any given minute on a provider's network. I'm seeing similar with plex lusers in AMS.
Let's double the network blasts.. (rhetorical)
/Rant
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
The Dropbox port has always been really annoying. What port(s) is Plex using now? I thought they got rid of their broadcast stuff.
I've since blocked 'em and the system log is swamped, so can't tell you offhand. I did check against known port assignments though. The flattened network of AMS doesn't help - broadcasting across subnets.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
The usage is extremely spiky in that location for almost every user. If that's how it's going to be used by most customers in that region then obviously to stop this we can't just start suspending everyone during peak times and it'd be better to reach out and try to smooth out the usage where we can first to at least avoid the server from locking up on Saturdays.
Amsterdam will be fixed along with the IP changes that are coming this week, as in they'll be split off.
Some of my VMs are down from the FFME006 node since yesterday. Can't access the CP as well.
TYOC026 maybe has disk issue partially. 1 of my 2 servers there is offline due disk failed
We already got a report for TYOC026 which I've been looking into and about to create a network status for, and then no reports for FFME006 but it looks like it's been overloading. It's not down yet but effectively unusable. I'll add it to the list and make the announcement alongside TYOC026.
@VirMach
Is the bandwidth problem with the vps still solved? Just restore the default.
Tokyo maintenance window created for all servers.
hope this fixes things with TYO
I bench YABS 24/7/365 unless it's a leap year.
I was guessing/hoping that might happen.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
I also got ip replacement mail. But I don't really want my server location to get changed from Amsterdam/Frankfurt to US.(though IP change is fine, but not location)
I have already mentioned this in the tickets,
Hopefully it get noticed by @VirMach
Not now, Kaito!
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
The fuck you @Kaito is talking about? Where the hell you found location change?
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Forgive me, I am not really an advanced user. I tried to check the ip location from online sites.
Maybe I am wrong about how the ip allotment works.
LOL @ IP change location
good morning
^ Why?
On second thoughts, don't bother.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Location won't change. Right now the IP addresses are not ours yet until they're assigned, no initially geolocation will be wrong.
Some of these Virmach folks need to find a new line of work. I've been doing this for over 30 years, and I've never seen critical service outages anywhere close to what Virmach is reporting IN REAL TIME AGAIN. Our server which weathered the last fiasco wasn't quite so lucky this time around. Here's a tip: SAVE YOUR MONEY AND GO ELSEWHERE!
SEAZ009 is offline for 2-3 days and being worked on.
SEAZ010 is offline for 1-2 days and waiting on datacenter.
Nothing else is offline. Some nodes have partial issues but already addressed. Out of all the times you could've made this comment right now is the least "critical" and I'm assuming it just means you were recently affected on one of these nodes.
After it's back online and you're able to get your data off, create a ticket provide the ticket # here after clicking cancel and I'll provide you a refund so you can follow your own advice, save your money and go elsewhere.
It's not just Saturday for me. and are you sure it's in that location instead of in this node? I have no problem with my Tokyo NVMe VPSes, just the storage one.
As someone mentioned here earlier, maybe there're some abusers doing mining stuffs/ torrenting all the time.
Yes, I'll quote again:
This means every day. The usage in that location is not like the others. 90% of the people have extremely spiky traffic/disk usage. As in, it appears most users are torrenting, using it as a VPN, or a private media server. Even the bandwidth usage is nothing like the other locations. So if you are someone who is not doing that, you're in the minority in Tokyo.
That's because of the "NVMe" part of that. Bandwidth usage will naturally be higher in proportion to the disk. We're seeing higher network usage in Tokyo NVMe versus other locations as well but since the nodes don't have 130TB disk that ends up being lower still.
Anyway, we'll most likely be contacting the highest users today and asking them what they're running and how they can reduce usage to at least ensure these people that were spiking the most on Saturday don't do it again next week, then we'll go from there and see how it improves. I've been monitoring everything for a couple days now so we should at least be able to make some level of improvement but it'll still be nowhere close to NYC or LAX.
I wanted to comment again that I have no ticket for FFME01 IP migration, but I have for FFME02 and FFME04... but I see secondary IP added to WHMCS, but no ticket, at all.
No idea if you want to check that.
id=613983[ and there is no new IP in panel and/or ticket for NYCB035 and NYCB036 (Dedipath), but there is IP for NYCB014 (PSINet/Cogent) - but this I will assume it's on purpose.
and nothing for DENZ001 too, but I think you mentioned this a later batch]
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.