Next Steps:
We will provide a detailed update as soon as possible. We deeply regret any inconvenience this may have caused and appreciate your patience.
Posted 2 hours ago. Jan 23, 2025 - 07:09 PST
ASAP
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@VirMach said:
I read the title and started laughing. I'll read the rest later but there's probably a good chance if you were in Tokyo, Chicago, or Los Angeles there was some comedic timing.
@VirMach said:
I read the title and started laughing. I'll read the rest later but there's probably a good chance if you were in Tokyo, Chicago, or Los Angeles there was some comedic timing.
@tulipyun said: However, the network problems previously encountered have not improved.
I assume that portion of it may take a little longer. The physical migration might have happened a little sooner than originally planned and last I heard they were waiting on IIJ for provisioning, I assume for additional capacity.
Timeline originally looked like "February" (in December.) I'll try to get back more details after this is done.
Chicago - Never got to post this but I was writing something about how based on who & how it was said, Chicago sounded like it was going to be a while. That was about 9 hours ago. Still haven't got any other update from them.
Tokyo - Four servers still not back online, they're aware/working on it. Everything else back.
@virmach , Excuse me, my registered email is with skiff.com, but they have stopped all services, including email forwarding. Since I cannot change the email address, I have added a new email address in the "Contact" section with control panel. Will this affect my ability to receive important emails from Virmach in the future?
@cosmossofa said: @virmach , Excuse me, my registered email is with skiff.com, but they have stopped all services, including email forwarding. Since I cannot change the email address, I have added a new email address in the "Contact" section with control panel. Will this affect my ability to receive important emails from Virmach in the future?
IDK, but @VirMach did help me change my primary e-mail not that long ago. I should think you can create a ticket to disappear the old e-mail from your account and replace it with a new one. I think I would recommend that. I don't think they want bounces anyway.
@cosmossofa said: @virmach , Excuse me, my registered email is with skiff.com, but they have stopped all services, including email forwarding. Since I cannot change the email address, I have added a new email address in the "Contact" section with control panel. Will this affect my ability to receive important emails from Virmach in the future?
IDK, but @VirMach did help me change my primary e-mail not that long ago. I should think you can create a ticket to disappear the old e-mail from your account and replace it with a new one. I think I would recommend that. I don't think they want bounces anyway.
VirMach suspended my VPS yesterday due to bandwidth over usage. I got 1.95TB of bandwidth in this package. This VPS just hosts one website which is behind Cloudflare and according to CF, I have only used 17GB bandwidth from 1st Jan till 24 Jan.
Do you guys know if there is some bandwidth report or logs in VirMach control panel to check? Very strange that this website which
has never crossed 100GB in a month in last 2 years, all of a sudden crossed 1.95TB in 24 days.
I suspect it could be due to one plugin which caches JS and images and was taking up 30GB space, but I am not sure. I have disabled that plugin.
Or are there some logs/utility available in Debian to check?
This would allow you to establish if it was short period of time (so like DDOS, hacked server sending shit) or was constant over time - so maybe this plugin.
If you don't have firewall set to discard everything except Cloudflare using Cloudflare for "17GB of bandwidth" is kinda meh - your machine is still on the internet and talking to other things outside of Cloudflare.
--
For Debian - there is nothing installed by default to monitor traffic - if you installed vnstat earlier then you could also have nice stats https://humdi.net/vnstat/ - but that need to be installed earlier as is collects data as it go.
HetrixTools agent deployed? They have Network graph too.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
This graph is only showing today's data. Can't seem to find custom date range. I have now installed vnstat so let's see. I am using ufw and have always blocked all ports except 21, 22, 53, 80, 123, 443, 465, 587, 993 for both incoming and outgoing.
I have noticed a massive increase in bots & scrapers, on a few of the sites that I host. Could be that.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Note: forum screwed the format.. SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
Ban the feckers, ban 'em all !!!
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
That's your main issue - Ubuntu on a server, pah! Replace ufw with csf (if possible on that OS). Install modsecurity; free rules being better than none.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@lesuser said: I added a WAF in Cloudflare to block all these user agents so let's see.
You might want to add them on the VM directly, because as you stated above, the traffic did not appear to come thru cloudflare.
Although the suggestion above is a good practice, I do not expect this will solve your issue. The traffic shows outgoing, and not reflected in cloudflare, so I expect the traffic is originating from your VM for reasons other than requests to your website..
@lesuser said: I added a WAF in Cloudflare to block all these user agents so let's see.
You might want to add them on the VM directly, because as you stated above, the traffic did not appear to come thru cloudflare.
I think they are coming from Cloudflare but I am not very well versed in Linux related things so I am not sure. Here is one of the entries from nginx access logs. Let me see how can I define these rules in nginx as .htaccess works in apache.
I also have a cron job running which takes database and file backup and rsync it to another server every 30 minutes. If this job becomes wonky then the other server from another provider should have been suspended also as it has much lower bandwidth that this one.
Comments
If your LA appears to be online: Don't reboot it.
It appears that the server is still online, but is not accessible from most locations around the world
141.11.95.0/24
This prefix is not visible in the DFZ right now
We are losing millions.
We accept Karma donations for the last flan. 🍮 affbrr
I honestly thought quadranet had a better network.. single homed cogent? Damn.
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
the strong willed are pleased to exist in the DFFZ and peer via the Akashic records
https://status.quadranet.com/incidents/v2ybsgnl1bnr
The Ultimate Speedtest Script | Get Instant Alerts on new LES/LET deals | Cheap VPS Deals | VirMach Flash Sales Notifier
FREE KVM VPS - FreeVPS.org | FREE LXC VPS - MicroLXC
declining to re-assure customers in the outage status that they are multi-homed == single-homed cogent, ho ho ho.
Posted 2 hours ago. Jan 23, 2025 - 07:09 PST
ASAP
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
to be fair, if the update was "Cogent is still fucked and it's still our only upstream", I would also be reluctant to post it
Chicago is down. Node CHIZ001 and CHIZ035.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Chicago CHIZ036 down here.
Seems like a network issue. We can hope Qnet fixes quickly.
Doesn't look as though Qnet are in Chicago..
https://status.quadranet.com/
Confusing: https://virmach.com/network/
https://status.equinix.com/ <- clean bill of health for CHI
Yo! @VirMach
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
TYOC030 down again
Fuck this 24/7 internet spew of trivia and celebrity bullshit.
TYO migration to new DC seems to be happening.
Seems like core switch issue (not ours.)
Expected, network status page.
Chicago is off.
Tokyo was down more than 5 hour
its migrating to a new Datacentre
I bench YABS 24/7/365 unless it's a leap year.
i didn't get info about migrating, also check my email too. Moving where ?
lol so it's not just me
same as mine. Down more than 5 hour
https://lowendspirit.com/discussion/comment/203810/#Comment_203810
TLDR - moving to NTT but not NTT
I bench YABS 24/7/365 unless it's a leap year.
The servers in Tokyo are back online.
However, the network problems previously encountered have not improved.
It is still routing through NTT.
Not all back up yet.
I assume that portion of it may take a little longer. The physical migration might have happened a little sooner than originally planned and last I heard they were waiting on IIJ for provisioning, I assume for additional capacity.
Timeline originally looked like "February" (in December.) I'll try to get back more details after this is done.
Chicago - Never got to post this but I was writing something about how based on who & how it was said, Chicago sounded like it was going to be a while. That was about 9 hours ago. Still haven't got any other update from them.
Tokyo - Four servers still not back online, they're aware/working on it. Everything else back.
so fragrant after migrate
I bench YABS 24/7/365 unless it's a leap year.
@virmach , Excuse me, my registered email is with skiff.com, but they have stopped all services, including email forwarding. Since I cannot change the email address, I have added a new email address in the "Contact" section with control panel. Will this affect my ability to receive important emails from Virmach in the future?
IDK, but @VirMach did help me change my primary e-mail not that long ago. I should think you can create a ticket to disappear the old e-mail from your account and replace it with a new one. I think I would recommend that. I don't think they want bounces anyway.
Thanks , i will try .
Chicago is back.
Good evening
Hi imok
dnscry.pt - Public DNSCrypt resolvers hosted by LowEnd providers • Need a free NAT LXC? -> https://microlxc.net/
any flashes
I bench YABS 24/7/365 unless it's a leap year.
VirMach suspended my VPS yesterday due to bandwidth over usage. I got 1.95TB of bandwidth in this package. This VPS just hosts one website which is behind Cloudflare and according to CF, I have only used 17GB bandwidth from 1st Jan till 24 Jan.
Do you guys know if there is some bandwidth report or logs in VirMach control panel to check? Very strange that this website which
has never crossed 100GB in a month in last 2 years, all of a sudden crossed 1.95TB in 24 days.
I suspect it could be due to one plugin which caches JS and images and was taking up 30GB space, but I am not sure. I have disabled that plugin.
Or are there some logs/utility available in Debian to check?
Powerful AMD Ryzen VPS (aff)
This would allow you to establish if it was short period of time (so like DDOS, hacked server sending shit) or was constant over time - so maybe this plugin.
If you don't have firewall set to discard everything except Cloudflare using Cloudflare for "17GB of bandwidth" is kinda meh - your machine is still on the internet and talking to other things outside of Cloudflare.
--
For Debian - there is nothing installed by default to monitor traffic - if you installed vnstat earlier then you could also have nice stats https://humdi.net/vnstat/ - but that need to be installed earlier as is collects data as it go.
HetrixTools agent deployed? They have Network graph too.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
This graph is only showing today's data. Can't seem to find custom date range. I have now installed
vnstatso let's see. I am usingufwand have always blocked all ports except 21, 22, 53, 80, 123, 443, 465, 587, 993 for both incoming and outgoing.Powerful AMD Ryzen VPS (aff)
I have noticed a massive increase in bots & scrapers, on a few of the sites that I host. Could be that.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Full VPS Control PanelHere's my last 30 days traffic. From 25 Dec 2024 to 25 Jan 2025.
Powerful AMD Ryzen VPS (aff)
^ maxed out for a whole week doesn't look good.
.htaccess snippet, as an example:
BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
BrowserMatchNoCase "petalbot" bad_bot
BrowserMatchNoCase "PetalBot" bad_bot
BrowserMatchNoCase "PerplexityBot" bad_bot
BrowserMatchNoCase "openai" bad_bot
BrowserMatchNoCase "GPTBot" bad_bot
BrowserMatchNoCase "SemrushBot" bad_bot
BrowserMatchNoCase "BLEXBot" bad_bot
SetEnvIfNoCase User-Agent .ahrefsbot. bad_bot
SetEnvIfNoCase User-Agent .semrush. bad_bot
SetEnvIfNoCase User-Agent .bytespyder. bad_bot
SetEnvIfNoCase User-Agent .bytedance. bad_bot
SetEnvIfNoCase User-Agent .claudebot. bad_bot
SetEnvIfNoCase User-Agent .magesift. bad_bot
SetEnvIfNoCase User-Agent .otbot. bad_bot
SetEnvIfNoCase User-Agent .petalbot. bad_bot
SetEnvIfNoCase User-Agent .Owler. bad_bot
SetEnvIfNoCase User-Agent .spider. bad_bot
SetEnvIfNoCase User-Agent .perplexity. bad_bot
Order Deny,Allow
Deny from env=bad_bot
Note: forum screwed the format..
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_botBan the feckers, ban 'em all !!!
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
That's your main issue - Ubuntu on a server, pah! Replace ufw with csf (if possible on that OS). Install modsecurity; free rules being better than none.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
25 MB/s, 24/7 for 4 days
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
But why this bandwidth is not reflected in Cloudflare?
Powerful AMD Ryzen VPS (aff)
I added a WAF in Cloudflare to block all these user agents so let's see.
Powerful AMD Ryzen VPS (aff)
You might want to add them on the VM directly, because as you stated above, the traffic did not appear to come thru cloudflare.
Although the suggestion above is a good practice, I do not expect this will solve your issue. The traffic shows outgoing, and not reflected in cloudflare, so I expect the traffic is originating from your VM for reasons other than requests to your website..
Because you have multiple entry points, Cloudflare is only one of them
I think they are coming from Cloudflare but I am not very well versed in Linux related things so I am not sure. Here is one of the entries from nginx access logs. Let me see how can I define these rules in nginx as
.htaccessworks in apache.47.128.96.140 - - [23/Jan/2025:01:41:40 -0500] "GET /product/clothing/track-suit/ HTTP/1.1" 200 33439 "-" "Mozilla/5.0 (compatible; Bytespider; [email protected]) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36"Powerful AMD Ryzen VPS (aff)
I also have a cron job running which takes database and file backup and rsync it to another server every 30 minutes. If this job becomes wonky then the other server from another provider should have been suspended also as it has much lower bandwidth that this one.
Powerful AMD Ryzen VPS (aff)
Modsecurity with standard and custom rules really does solve so many day to day issues. I don't leave home without it.
@lesuser here is a link to a how to do modsecurity for Nginx
Based on your comments I’d guess the machine got owned and is spewing traffic for someone else.
Restore from unrelated backups on a new machine at your new provider I guess.