VirMach - Complain - Moan - Praise - Chit Chat - Flan

1268270272273274

Comments

  • As an aside, the network status page is really nice.

  • WSSWSS OG Guru Meditation Error

    @skorous said: As an aside, the network status page is really nice.

    "Aside from that Mrs. Lincoln, how was the show?"

    "It's a hard life- to be a stick insect." - Karl Pilkington

  • @WSS said:

    @skorous said: As an aside, the network status page is really nice.

    "Aside from that Mrs. Lincoln, how was the show?"

    LOL

  • imokimok OG Not Administrator

    @somik said:

    @imok said:

    @WSS said:

    @imok said:

    @somik said:

    I know I'll regret asking, but how much are you paying for the dedi with 4tb nvme?

    I don't remember for real. Around $400 I think.
    And it is 2x4TB NVMe.

    Did I kill your drive @virmach?

    $400/mo?

    WTF I'm not rich. It's per year.

    Rich people... Paid $400 but "cant remember" :lol:

    But $400 per year for 8TB nvme storage... that's quite a good deal...

    It's been a long time and I was trying to sleep 😅 (BTW I saw a light at night a bit blurry, I thought it was the moon, but it was not supposed to be on that side of the sky, and right now I'm not sure if it was real or a dream).

    I will check the actual price.

    @somik said:

    @eliphas said:

    @somik said:
    But $400 per year for 8TB nvme storage... that's quite a good deal...

    I wait for the $7/y/TB.
    Nah, ok, will buy 2TB at $14/y/TB.
    Spinning rust? oh yeaahhhh just put raidz-360 or something like that.

    What are you taking about man? I got 2TB of spinning rust dedi from CC for $100/year and I am happy with it. . . well, I was until I saw @imok got 8TB of nvme for $400/year :scream:

    And 4x3TB HDD.

    And a /28 (it's pending but I trust @virmach will deliver any time in the future).

    @VirMach said:

    @imok said: @virmach about the dead 4TB nvme in my dedi in OKH, does it have a replacement? Just asking to see if I should open a ticket or not.

    They exist.

    I don't know if I made my comment earlier or not but if you want to speed up the process can you do some of the general confirmations for me? Like does it show up in BIOS, what does OS display, what kernel parameters set?

    Nice. I'll see what I can get this weekend without burning the server.

  • Can't reach VPS after IP change and reboot (I had gotten an email saying there would be a node migration and new IP address). Did all the stuff with "reconfigure network", can't reach by VNC, little popup in the corner of the client area says "Powered off Node NYCB046" but it's unclear whether that means the node is powered off or that the VM running on the node is (virtually) powered off. Any idea if this node is down? If yes, no big deal, it's a super cheap VPS so I can just wait til the node is back online. If the node is up then I might need help troubleshooting the VM.

  • @skorous said:
    Speaking for NYCB015, ping.pe has some routes fine and others 100% loss. The interesting part to me is that I appear to be able to get to one server but not the other on the same subnet in Chicago at both Hosthatch and TNA Hosting.

    Yeah, it's strange.

    I'm still investigating on our end but I've spent a lot of time on it without anything fruitful coming from my investigation. I'm just trying to make sure it's nothing that could theoretically be on our end, I still don't believe it is since all we're doing is sending them over to our provider's routers. My guess is that it's potentially not a problem on their end either but could be an issue at Arelion.

    I'm sure once it's figured out it'll seem like it was obvious and I encourage anyone to chime in, I just have no idea what could be causing the situation you're describing and that's probably what's making it so difficult to diagnose, it's difficult to find a pattern when a situation such as "it reaches one server but not the other even though they're on the same subnet."

    @dgc1980 said:
    mine on NYCB044 also in unreachable from anywhere, it is not even able to ping the gateway .1 but I can ping the gateway from outside.

    That sounds extra weird, I'll look into it.

    @willie said:
    Can't reach VPS after IP change and reboot (I had gotten an email saying there would be a node migration and new IP address). Did all the stuff with "reconfigure network", can't reach by VNC, little popup in the corner of the client area says "Powered off Node NYCB046" but it's unclear whether that means the node is powered off or that the VM running on the node is (virtually) powered off. Any idea if this node is down? If yes, no big deal, it's a super cheap VPS so I can just wait til the node is back online. If the node is up then I might need help troubleshooting the VM.

    It's not down. I haven't yet changed the main nodes to the new blocks as I did not want to create additional issues. If it says powered off, and you cannot use VNC, it sounds like an issue your VPS is facing after it attempted to reboot during the reconfigure, as in a SVM bug. That's just a guess.

    Message me your IP and I can confirm.

    We have a system to automatically catch these but with the other overlapping issues going on, it's not really "built" for this environment and therefore is pretty useless right now until it is reworked. This is because SVM doesn't provide API for these particular issues so we have to "guess" that it's happening based on other data. I haven't thought of a good way of accounting for it when there's also routing issues.

  • I've been able to confirm something, after going through the data.

    Let's say there are 20 different test points, and let's say that each block can reach 10 test points, but not the other 10 test points. We have 10x new /24 subnets (we actually have 11x but to keep it simple we're saying 10.)

    The pattern is that 5x of the new /24 subnets (let's call them "Group A") can reach 10 test points (let's call them "Reachable A"), and not the other 10 test points ("Unreachable A"). Then, the other 5x /24 subnets ("Group B") can reach 10 test points ("Reachable B") and not the other 10 test points ("Unreachable B"). The discovery is that Unreachable A = Reachable B and Unreachable B = Reachable A.

    And there appear to be two Arelion interconnects.

    "Group A" and "Group B" have no relation to any internal grouping on our end. They can be on any VLAN, etc.

    This was my original "guess" on what could be happening, but I've finally pretty much confirmed it as best as we can on our end. It's similar to what happened in San Jose about two years ago where there were two edge routers and they had different configurations. I've already reported this as a possibility so at this point I'm going to just wait for this to be fixed upstream. I didn't want to "box" myself in so I started looking into all the other possibilities, which made it difficult to figure out since there were two different distinct clusters with different patterns that might make it appear "random" if you're not careful.


    Just a fun fact though, this means if you have two services in NYC on different nodes, it's theoretically statistically probable that you could be in a situation where you do have full internet access if you just combine them together.

  • @VirMach said: Message me your IP and I can confirm.

    Thanks, PM'd.

  • vyasvyas OG
    edited May 30

    I get the alert for 2 of my NY VPS'es:

    New subnets on NYC do not have global reachability. This is an upstream issue that we are waiting on to get resolved.

    NYCB014

    NYCB015

    In the spirit of "Lets break just because something is working fine" I wait for resolution

  • @VirMach TYOC033 IPV6 is unavailable

  • @VirMach said:

    @skorous said:
    Speaking for NYCB015, ping.pe has some routes fine and others 100% loss. The interesting part to me is that I appear to be able to get to one server but not the other on the same subnet in Chicago at both Hosthatch and TNA Hosting.

    Yeah, it's strange.

    I'm still investigating on our end but I've spent a lot of time on it without anything fruitful coming from my investigation. I'm just trying to make sure it's nothing that could theoretically be on our end, I still don't believe it is since all we're doing is sending them over to our provider's routers. My guess is that it's potentially not a problem on their end either but could be an issue at Arelion.

    I'm sure once it's figured out it'll seem like it was obvious and I encourage anyone to chime in, I just have no idea what could be causing the situation you're describing and that's probably what's making it so difficult to diagnose, it's difficult to find a pattern when a situation such as "it reaches one server but not the other even though they're on the same subnet."

    @dgc1980 said:
    mine on NYCB044 also in unreachable from anywhere, it is not even able to ping the gateway .1 but I can ping the gateway from outside.

    That sounds extra weird, I'll look into it.

    FWIW, that's my situation too - can mtr the gateway fine from the outside but no further, and the VPS itself considers the gateway "Destination Host Unreachable"

  • OMG my AMS instance might be usable after a year?

  • qpsqps ProviderOG

    @VirMach said: Arelion

    We've had some problems with their routers hanging on to stale routes well after they are no longer announced. If you are seeing something weird with Arelion, it could be related to this issue.

    QuickPacket - Dedicated Servers in Ashburn, Los Angeles, Chicago

  • edited May 31

    Dear @VirMach

    There is any update with the following nodes(?):

    ATLZ033 (IP: 66.59.196.XY) (Ticket ID: #968802 - Submitted: 10/15/2025 (13:22)
    Screenshot: https://freeimage.host/i/C3YlpaV

    SJCZ006 (IP: 91.186.210.XY) (Has no ticket)
    Screenshot: https://freeimage.host/i/C3Y0bxn

    Both of mentioned VM / node unreachable, offline for many-many months.
    There is any ETA for solution?

    Edit:
    My VPS from NYCB046 node (I've got new IP address for this VPS) also unreachable.
    It worked fine before the IP change. (Networking config has been reconfigured, and it probably OK).
    IP range: 185.227.252.XXX
    Screenshot: https://iili.io/C3YNrcF.png

    Thank You very much for the help, answer!
    Titus

  • Your new IP is still being assigned. Please try again in a few minutes.
    i was thought i could replace my IP on tyoc037 before maintenance🤦

  • I don't know where my SanJose vm is. It went L.A, and then Japan, and now the i.p showing is 1.0.3.xxx, which is in China.
    It might end up in North Korea.

  • somiksomik OG Hostbusters

    @tenpera said:
    I don't know where my SanJose vm is. It went L.A, and thæn Japan, and now the i.p showing is 1.0.3.xxx, which is in China.
    It might end up in North Korea.

    Your VM went to more countries thæn I have :lol:

    I speak fluent sarcasm and broken logic. | I would agree with you, but thæn we’d both be wrong.

  • JabJab TOP Member 2027

    I got like 4 e-mails about new IP's. My previous post bragging about me missing on all the fun is now invalid.
    Thanks VirMach!

    Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.

  • @taizi said:
    Your new IP is still being assigned. Please try again in a few minutes.
    i was thought i could replace my IP on tyoc037 before maintenance🤦

    Bug fixed.

  • msattmsatt ProviderOG

    Looks like things are stuck "Migration queued" - 2 VMs both with multiple IPs on AMSD018.
    Duplicate emails received (very short notice) about IP change but only gave a single IP.
    VMs are still running on 'old' ips.

    Mike

    Get your FREE VPS if you develop Open Source software

  • edited May 31

    @msatt said:
    Looks like things are stuck "Migration queued" - 2 VMs both with multiple IPs on AMSD018.
    Duplicate emails received (very short notice) about IP change but only gave a single IP.
    VMs are still running on 'old' ips.

    Mike

    This particular batch was a last minute change, mostly as a precaution due to previous node issues. I decided to use it as an opportunity to cluster everyone across more stable and emptier nodes. This did cause an additional delay on an already-behind-schedule change. I do apologize.

    Theoretically this could ave been just an IP change and it would have been easier, it was a decision made on purpose.

    The migration to IP change pipeline (and reverse) was reworked and unified, to add flexibility for some particular cases. That's an unintended side effect, it shouldn't be displaying that in this case. (It technically "should" be displaying it as coded, just an oversight.) I just need to add in a check to ensure it's not scheduled too far into the future when it shows that splash. It's obviously not helpful in that situation and was only intended for more immediate changes.

    The additional IPs issue was my mistake, I used a feature on the system incorrectly but obviously it shouldn't even "allow" me to make that kind of mistake. It was just because I wanted to combine a particular type of scheduled migration WITH prior communication WITH no delay on change after migration, and that combination only supports single changes. Looking into it now. EDIT -- Actually I do believe I already fixed this issue pre-emptively, it just didn't get worked into the communication, as in it doesn't pre-reserve additional IPs/communicate them in this case. I'll double check.

    I've already been looking into the duplicate communication bug, it likely has to do with a failsafe that was added some days ago to ensure we don't "miss" any communications as another bug caused some to never land in the past. I'll have to rethink it and clean it up properly for future cases. EDIT -- Yes, this was it.

  • Looks like my VM migrated (the wizard said it was completed), is now on AMSD030X, but it doesn't get the new IP (panel does not show the new IP). Manually setting the IP doesn't help. However the DHCP that hands out/offers the old IP, responds from .2 IP in the new IP range ... :sweat_smile:

  • cybertechcybertech OGBenchmark King YABS 24/7/365

    my tokyo new IP was automatically applied without any hiccups 👍🏻

    I bench YABS 24/7/365 unless it's a leap year.

  • @flips said:
    Looks like my VM migrated (the wizard said it was completed), is now on AMSD030X, but it doesn't get the new IP (panel does not show the new IP). Manually setting the IP doesn't help. However the DHCP that hands out/offers the old IP, responds from .2 IP in the new IP range ... :sweat_smile:

    In your case it's not done yet, being worked on now. (As in I'm aware.) Some delay.

    Luckily not many affected.

  • edited June 1

    My VM is seemingly migrated on AMSD013 (panel shows the new IP) but can't connect to the Internet. I ran 'Reconfigure networking' in the control panel which supposedly completed successfully but still can't reach the Internet. Are other people having the same issue?

  • Tokyo has never adopted IPv6? It has been waiting for many years.

  • FrankZFrankZ ModeratorOG
    edited June 2

    @cybertech said:
    my tokyo new IP was automatically applied without any hiccups 👍🏻

    same here today node TYOC035
    VPS on AMSD026 still shows in migration queue in the panel, but is working fine under old IP. but is dead now.

  • @VirMach said:

    @flips said:
    Looks like my VM migrated (the wizard said it was completed), is now on AMSD030X, but it doesn't get the new IP (panel does not show the new IP). Manually setting the IP doesn't help. However the DHCP that hands out/offers the old IP, responds from .2 IP in the new IP range ... :sweat_smile:

    In your case it's not done yet, being worked on now. (As in I'm aware.) Some delay.

    Luckily not many affected.

    Could you please handle the ticket #447726, the automatic IP change caused me an unnecessary disaster?

  • msattmsatt ProviderOG

    Wow - lots of IPs !!!
    id=722166 AMSD018 > AMSD035 now has 12 ipv4 (sbe 4) . None appear to be working and are in the 'old' ip range i.e. 88.214.21.x
    id=722545 AMSD018 > AMSD027 moved with 4 new working IPs (should only be 2) plus old ips.
    Lost IPv6 on both :(

    Get your FREE VPS if you develop Open Source software

  • edited June 1

    @M0nst3r said:
    My VM is seemingly migrated on AMSD013 (panel shows the new IP) but can't connect to the Internet. I ran 'Reconfigure networking' in the control panel which supposedly completed successfully but still can't reach the Internet. Are other people having the same issue?

    same
    @VirMach ticket #466068

  • too many Capcha, every time fails, give it up.

  • edited June 2

    "Please ensure your virtual server is using the new IP before the old IP is removed. Our system will attempt to update the network configuration on your VPS automatically on finalization. If that fails for any reason, you can run "Reconfigure Network" from your control panel or set the new IP manually via VNC."

    Only an IP address is specified. Last time I checked, IP address is only one of multiple parameters needed. Fine, I wait for it to "do the right thing", but this is a very silly email. Reconfigure Network does not change the IP address.

  • My TYCO038 seems to have been migrated to TACO032, but it still cannot be accessed. When I try to access it, I get a message saying “Migration is in queue”. This situation has persisted for over a week now.

  • @VirMach I have a VPS in AMSD026 and status is unreachable, old IP still unchanged

  • "Luckily not many affected."

  • edited June 2

    RYZE.TYO-C031.VMS was working an hour ago, but suddenly the system crashed. The backend panel shows no hard drive. Reinstalling the system didn't work.

    The machine ID is 635215
    (https://billing.virmach.com/clientarea.php?action=productdetails&id=635215)

  • edited June 2

    All of my VPS from location USA/NY received an 'upcoming IP address change' notification while they have a working IP address at the moment. Meanwhile I have two VM (ATLZ033, SJCZ006) without valid, working IP address, many-many months (technically they are unreachable, unusable). And my friendly questions, request - about the two mentioned node - always ignored by support (in ticket, and here also). :) :/

  • @msatt said: Wow - lots of IPs !!!
    id=722166 AMSD018 > AMSD035 now has 12 ipv4 (sbe 4)

    Hey! Can You give me only TWO of them? :)
    Coz I have two VPS without working IPv4. (I joking only).

  • Yay! The networking on NYC metro node NYCB044 on subnet 185.213.19.0/24 just started working!

  • @toor said:
    Yay! The networking on NYC metro node NYCB044 on subnet 185.213.19.0/24 just started working!

    mine also, Thank @Virmach <3

  • @VirMach said:

    @flips said:
    Looks like my VM migrated (the wizard said it was completed), is now on AMSD030X, but it doesn't get the new IP (panel does not show the new IP). Manually setting the IP doesn't help. However the DHCP that hands out/offers the old IP, responds from .2 IP in the new IP range ... :sweat_smile:

    In your case it's not done yet, being worked on now. (As in I'm aware.) Some delay.

    Luckily not many affected.

    In addition to the IP switch about to happen (a few days ago), seems there's some other issue with the node:

    Hope your figure out the issues … :)

  • @Virmach I previously paid $3 to migrate my service, but it hasn't taken effect. Could you help me migrate the service to any available node? Thank you.
    Service ID: 618406

  • TYOC035 IPV6 is unavailable
    @VirMach

  • edited June 4

    @msatt said:
    Wow - lots of IPs !!!
    id=722166 AMSD018 > AMSD035 now has 12 ipv4 (sbe 4) . None appear to be working and are in the 'old' ip range i.e. 88.214.21.x
    id=722545 AMSD018 > AMSD027 moved with 4 new working IPs (should only be 2) plus old ips.
    Lost IPv6 on both :(

    I've been babysitting this all day and it's not going well, a lot to clean up. The node had some new weird situations the system did not account for properly and it got delayed into missing the IP change window, except some smart recovery system that works on regular IP changes was not very smart when combined with migrations, and it basically went "oh the IP that I needed isn't here, it must mean it's already used" and there's a fallback to pick available IPs, that broke out of the gating and gave you more on the old block since it never actually migrated.

    @fran said:
    @VirMach I have a VPS in AMSD026 and status is unreachable, old IP still unchanged

    Same batch as above, a bit different. I'm trying to make sure it doesn't go down the same path as AMSD018 so once that's done it should be smoother.

  • @VirMach
    Hello, my VPS on TYO037 went offline after the new IP assigned, I didn't ask for an early assignment before the maintenance, and I can't do anything about it now. Anything wrong?

  • @toor said:
    Yay! The networking on NYC metro node NYCB044 on subnet 185.213.19.0/24 just started working!

    Sadly it's still not working on NYCB045; can't even ping the gateway from the VPS. Is this a known issue @VirMach or should I just accept that it's especially cursed and give up / reinstall / sacrifice a goat?

  • flipsflips OG
    edited June 4

    My VPS at AMSD030X came up for a few hours, but then something happened, so it's down again.

    Running ifup ens3 returned unknown interface ens3, so I figured some script had changed my /etc/network/interfaces, and sure enough, it was replaced by SolusVM (setting up static for eth0) …

    Adding GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0" to /etc/default/grub, running update-grub and rebooting got it back up … :sunglasses:
    (Doesn't seem like IPv6 works, but I'm not very stressed about that …) :innocent:

  • Hey Virmach, any chance that you'll get reverse DNS delegations for the new IP blocks in NYC once things have calmed down? It'd be cool if I could fix my rDNS.

  • TYOC040 TYOC037 downed?

    main IP down, node down, WHMCS status: Active

  • qpsqps ProviderOG

    @dude_naw said: new IP blocks in NYC

    If your new IP is a QuickPacket-owned IP, we can delegate RDNS to @VirMach on request, he just needs to ticket or DM me on Discord when he has a chance and we can get it setup on our side (and then VirMach needs to set it up on his side).

    QuickPacket - Dedicated Servers in Ashburn, Los Angeles, Chicago

Sign In or Register to comment.