VirMach
VirMach
About
- Username
- VirMach
- Joined
- Visits
- 2,824
- Last Active
- Roles
- Member, Hosting Provider
- Thanked
- 9380
Comments
-
NYC storage node, I had to do a lot of weird setups for this one to make it work as we faced problems such as a disk going missing, the wrong switch, and so on in the background. Right now it's LACP aggregated 3Gbps. Originally was supposed to have…
-
(Quote) Very unlucky user or very lucky user based on preference, when it happened, etc. Maybe your services are getting migrated to Ryzen, maybe they're all broken.
-
I was going to get a flight to San Jose for tomorrow morning but I'm still waiting to see how we can even get DC access since it's the first time. I might still go on Tuesday and see if I can beat the DC hands to it for SJCZ004 at this point, only a…
-
I did also finally get some switch access, so you will either see more nodes up in NYC that we can use for migrations, or you'll see me break the networking.
-
(Quote) All systems are existing systems that have been tested, some more than others. The newer nodes I've actually spent less time testing, not as a result of being careless, but in that a lot of the issues were already ironed out so it was a quic…
-
Okay after working on this all day, outside of the two nodes somehow still down (SJCZ004 and NYCB036) I've counted these specifically and we've gone down from something like 2% of people after the most recent round of migrations to exactly 0.91% of …
-
(Quote) There's about 3-5% of the VMs offline, but the node as a whole isn't. Those are on our queue.
-
(Quote) It's been renamed to NYCB036 and I've been painfully aware of it being down. I have no access to IPMI and no one seems to care. I've mentioned it at least 4 times and it hasn't even been marked as request put in to the DC yet. Also marked on…
-
(Quote) 04:40:12 up 6 days, 4:12, 1 user, load average: 8.04, 8.25, 7.43
-
(Quote) We completed PSU swap here, so we have to see what's wrong at this point. It's just shutting off and not throwing out kernel errors so what's left that could be possible I can think of is CPU suddenly overheating or loose cables elsewhere.
-
SEAZ008 finally back up, god that took a long time. DC hands plugged in IPMI to LAN and LAN to IPMI, finally corrected. A lot of others have been fixed as well, all of Atlanta (DC hands gave the port numbers in reverse order) and all of NYC by now e…
-
(Quote) I'd scream at CC but I've given up on that. They placed a permanent nullroute on the main IP. This is probably the 30th time they've done this, from a single website being malicious. I'm trying to get a server up and just move it at this poi…
-
(Quote) Not sure what you mean by that, on my end the /24 is functional and previous issue(s) corrected. I actually fully forgot what happened with this one, my brain can only store so much when it comes to literally everything breaking at some poin…
-
(Quote) Expected. This is one of the reasons I said IPv6 may break and it's not yet officially delivered and we put it on the backburner for some locations. It was only set up to function on the same VLAN, we have to go through and split it up now, …
-
It's crazy how hard the NIC was trying to keep up with the previous network configuration, this screenshot says it all. I almost had to check and make sure the change didn't just knock everyone offline or crash something. (Image)
-
(Quote) Yep, it's actually crazy how much of a massive improvements it made for Tokyo. Other regions improved maybe 20-30% after the change but Tokyo is probably an easy 70% across the board, that's lower load, CPU usage, improved ping and transfer …
-
(Quote) All of NYC has also had this applied. DC had wrong ports labeled for about 7 servers so those are currently offline (networking-wise.) Around 6 can be fixed more quickly, one definitely can't because it has to wait on network engineer.
-
Oh by the way, we signed on for Chicago, Tampa, and Los Angeles with Hivelocity. Pretty excited about that. Basically it happened the second QN made me lose my mind over the nullroutes. Not that we're moving all of LA to Hivelocity, it'd just be nic…
-
(Quote) I'm lucky enough to still have my parents around and every time we speak they remind me of this, and even though I'm a grown man now I still get a kick out of their reaction when I say something along the lines of "okay, I'm taking a tw…
-
(Quote) This is knowledge from a long time ago and it's based on hearsay from someone who used to work for us, but in the past, we couldn't do static routing with SolusVM. I don't remember the specific nor do I want to dive into it right now to veri…
-
(Quote) DALZ007 as well as a few others have been waiting probably nearly 2 weeks for networking to set up the VLAN properly. The way the teams function and how they do networking is not ideal in my opinion and we only have issues with this one part…
-
Tokyo is finally getting its long-overdue network overhaul. We had a good discussion about it and I want to kind of share what we believe is occurring right now and why. Turns out, pretty much every network engineer I've spoken with agrees that lar…
-
(Quote) All I can immediately tell you is that it's not a node-wide issue and only 1 single person's service is "offline" and many are using networking and racking up bandwidth usage, with their service pinging. You might have to install a…
-
(Quote) Very vague so I'm unable to even figure out if you're on a location facing problems such as Seattle or if it's unique to your service. We'll allow people to request credits as long as they're very clear about it and basically, concise, when …
-
Seattle finally got the switch configuration changes we requested. SEAZ004 and SEAZ008 having some issues with it (indirectly) so waiting on hands request for that, otherwise Seattle networking is finally decent. ATLZ007 had a disk issue that's bee…
-
(Quote) Some weird DNS issue that just went away. I don't remember who we set up for DNS but it's possible it briefly had a burst of connection issues in that route. Unrelated to SolusVM. It's WHMCS, it couldn't resolve Google either.
-
(Quote) 09:29:04 up 29 days, 20:51
-
(Quote) I was already expecting their DDoS protection to be pretty bad, as in an attack would leak through. Never would I have guessed that it's so good that it doesn't let ANYTHING through. They've solved the universal problem of denial of service …
-
SolusVM has migrated to either an Epyc or Ryzen servers, I don't remember. We got it like a year ago probably at this point and never used it until now. Let's see if this improves anything or if we're still stuck with PHP/MySQL bottlenecks.
-
(Quote) It could be an IP conflict or IP change. If it continues let me know. We've brought people on, we actually have someone helping but unfortunately they're not at the level we need them to be to really make an impact. There's also someone I'm…
-
(Quote) Yeah looks like syncs got absolutely gutted by everything I said above as well.
-
(Quote) The templates were all fixed for all migrations up until yesterday or so, but only on SolusVM. Those aren't tied to their original package anymore. WHMCS is next if I can squeeze it in.
-
(Quote) This was supposed to be a quick migration but QuadraNet has been nullrouting our IP addresses all day for doing the transfers. I guess their DDoS protection is some script they set up to deny you service, the 14 year old developer must have …
-
Side note on multi-IP, at this point it's going to require a lot of work to sort through all of it and that's after we set up more nodes. Multi-IP will most likely for most people require another migration as well. Originally this wasn't going to be…
-
(Quote) Bottom of the list at this point to be frank. I understand it's very important to some, we just have to make sure people have a functional service first, a functional IP second, that the networking actually functions well, more builds, coord…
-
Template Syncs - Ongoing, many more re-synced. OS installs should work better. QuadraNet's "DDoS Protection" is essentially just hefty false positives though so for Los Angeles I might have to literally drive down a hard drive and load the…
-
Networking Update - Just essentially waiting on DC hands at this point for NYC, Dallas, Atlanta, San Jose, Seattle, Phoenix, and Denver. Once these switch over networking should improve drastically and it should also have a positive impact on the CP…
-
(Quote) We're ramping up the abuse script. It's what it is called. I didn't say boot loop after migrating is abuse. Abuse script will just power it down, not suspend. I don't see the harm in powering down something stuck in a boot loop. I was just …
-
(Quote) Were these always offline after Ryzen Migrate button?
-
(Quote) This has an issue with a software getting stuck and duplicating its process over and over until it overloads and we have to reboot it. We made some changes, if it happens again we'll try to catch it earlier this time to avoid a reboot.