So, this happened... Completely not a BF deal, but I realised I could consolidate a ton of my VPS and get a beast of a dedi from Hetzner instead on a server auction.
The NVMe disks are a bit weird - both are 12,658 hours, so about 17 months old. The first looks like it's had a reasonably busy life - 57.6TB written for a percentage used of 11%. The second has been barely used, with only 87.3 GB ever written to it!
Amount of RAM is also insane, although sadly it's not ECC. It wasn't advertised as ECC though, so I guess that's on me.
Less than 50€/m I suppose. Have seen those lately but only with 64GB RAM though. No idea what to do with this combination of RAM and tons of RAM but it's an impressive setup. Nice catch.
€56.04 and they obviously have quite a few of them as the price isn't changing, but the time keeps adding a few hours every now and then, so I'm guessing people are buying them then. Actually, it was my first time on Hetzner anything, but it did seem like they had some other i9-9900K's with different disk configurations at a decent chunk more, so I'm guessing this is near the end of the discounting. The ones with 64GB are €54.84, so I figured it was definitely worth getting the better one.
It's definitely a bit expensive than the VPSes I've earmarked for cancellation / transfer, which would have taken me to AX41, but with the setup fee on that, I figured I'd go for this instead.
Hah, now I wonder if I should have waited more especially as it was kind of an impulse buy after I was annoyed at myself for missing the Pebble Host dedi - because I'd already worked out which one it was and added it to my basket ready for the deal and then in my excitement, I just clicked on the OGF link and I ended up with 2 and by the time I'd faffed around reducing the quantity back to 1, I'd already missed it. This one is better, but cost per power is about the same, and getting that one would have actually saved me money overall instead of increasing my costs.
Hah, now I wonder if I should have waited more especially as it was kind of an impulse buy after I was annoyed at myself for missing the Pebble Host dedi - because I'd already worked out which one it was and added it to my basket ready for the deal and then in my excitement, I just clicked on the OGF link and I ended up with 2 and by the time I'd faffed around reducing the quantity back to 1, I'd already missed it. This one is better, but cost per power is about the same, and getting that one would have actually saved me money overall instead of increasing my costs.
BF is the wrong day to start saving money. You did everything right and actually 56€/m seems like a good deal. The 64GB boxes I've noticed were around 50€/m and 6€/m for another 64GB is an amazing deal imo.
@webcraft said:
BF is the wrong day to start saving money.
Wat, you drunk? Its perfect.
Except, you have to verify every "DEAL" you see, after some time, you just need to look at the specs and price and your brain goes, wow what a scam.
The 5$/y MXRoute, saves money, end of story.... I could continue this one. @jarland stop scamming us.
@webcraft said:
BF is the wrong day to start saving money.
Wat, you drunk? Its perfect.
Except, you have to verify every "DEAL" you see, after some time, you just need to look at the specs and price and your brain goes, wow what a scam.
The 5$/y MXRoute, saves money, end of story.... I could continue this one. @jarland stop scamming us.
I mean it like this, if there wasn't a BF special you probably wouldn't have bought it. Just like @ralf bought the 129GB machine though he knows he won't use the extra RAM compared to a 64GB machine.
@webcraft said:
BF is the wrong day to start saving money.
Wat, you drunk? Its perfect.
Except, you have to verify every "DEAL" you see, after some time, you just need to look at the specs and price and your brain goes, wow what a scam.
The 5$/y MXRoute, saves money, end of story.... I could continue this one. @jarland stop scamming us.
I mean it like this, if there wasn't a BF special you probably wouldn't have bought it. Just like @ralf bought the 129GB machine though he knows he won't use the extra RAM compared to a 64GB machine.
Yes, we get weak, over good priced shit, yea.
He could use the additional 64GB as High Speed NVMe doh.
@webcraft said:
BF is the wrong day to start saving money.
Wat, you drunk? Its perfect.
Except, you have to verify every "DEAL" you see, after some time, you just need to look at the specs and price and your brain goes, wow what a scam.
The 5$/y MXRoute, saves money, end of story.... I could continue this one. @jarland stop scamming us.
I mean it like this, if there wasn't a BF special you probably wouldn't have bought it. Just like @ralf bought the 129GB machine though he knows he won't use the extra RAM compared to a 64GB machine.
At the prices I was looking at, the extra 64GB was only an extra $1.20, so I might as well take it even if I don't need it, because at least it'll end up getting used as a disk cache or something.
IncogNET 512MB $25/year
Due to it being a Tor exit, Cloudflare shows a challenge for yabs.sh redirect, so I have to use the full script URI.
For the same reason, yabsdb.com submission fails.
Going to do this a bit differently (and I've already posted the original YABS anyway).
I thought it'd be interesting to see how much of an impact virtualisation had on a YABS, so I made a VM using all 16 threads. The first observation was that when I had 16 vCores, GB5 terminated early on the multi-core Horizon Detection test. I tried a few things, thinking it was the CPU host-passthrough that was the problem, but eventually discovered it was fine running with fewer cores. Even 15 was fine, as long as there was one free vCore, but as soon as I used 16, it failed.
What's interesting about this is that the small 4k IO performance is awful in the VM, about 1/4 speed. But, what I didn't expect is that the blocks that are page size and up are somewhat quicker on the VM. I'm not sure why this is, but maybe the readbacks are more likely to end up in RAM cache.
I'd like to do a follow-up experiment sometime where the drive is a raw LVM partition as that's how my KS-LE-1 is set up for VM. This time I decided to just mount an LVM partition on /var/lib/libvirt/images. I'm still undecided whether I prefer lvextend or qemu-img resize to extend a VM's drive.
Networking
Performance is pretty much the same on host and VM. Here's the host, with the IPv6 results removed as I didn't pass them through to the VM:
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 929 Mbits/sec | 929 Mbits/sec | 19.2 ms
Scaleway | Paris, FR (10G) | 931 Mbits/sec | 935 Mbits/sec | 16.4 ms
NovoServe | North Holland, NL (40G) | busy | 936 Mbits/sec | 11.9 ms
Uztelecom | Tashkent, UZ (10G) | 873 Mbits/sec | 4.88 Mbits/sec | 87.0 ms
Clouvider | NYC, NY, US (10G) | 715 Mbits/sec | 389 Mbits/sec | 83.4 ms
Clouvider | Dallas, TX, US (10G) | 860 Mbits/sec | 312 Mbits/sec | 125 ms
Clouvider | Los Angeles, CA, US (10G) | 830 Mbits/sec | 268 Mbits/sec | 145 ms
VM:
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 941 Mbits/sec | 935 Mbits/sec | 20.3 ms
Scaleway | Paris, FR (10G) | 934 Mbits/sec | 934 Mbits/sec | 16.6 ms
NovoServe | North Holland, NL (40G) | 938 Mbits/sec | 936 Mbits/sec | 11.1 ms
Uztelecom | Tashkent, UZ (10G) | 802 Mbits/sec | 290 Mbits/sec | 191 ms
Clouvider | NYC, NY, US (10G) | 894 Mbits/sec | 411 Mbits/sec | 84.0 ms
Clouvider | Dallas, TX, US (10G) | 865 Mbits/sec | 401 Mbits/sec | 122 ms
Clouvider | Los Angeles, CA, US (10G) | 832 Mbits/sec | 269 Mbits/sec | 149 ms
Geekbench
Some loss to be expected as I'm using fewer vCores, but it's still not too bad. Slightly better than 15/16 of GB5 score.
Host first:
Geekbench 5 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1441
Multi Core | 7982
Full Test | https://browser.geekbench.com/v5/cpu/18922424
and now the VM:
Geekbench 5 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1436
Multi Core | 7633
Full Test | https://browser.geekbench.com/v5/cpu/18967843
Final notes
I'm not intending to use this VM for anything in the real-word, but I thought it was an interesting experiment.
I added this to my linux command line: iommu=pt intel_iommu=on isolcpus=1,2,3,4,5,6,7,9,10,11,12,13,14,15.
In the VM XML file, I also added this because I hate the generic CPU ID names: <cpu mode='host-passthrough' check='none' migratable='on'> <feature policy='disable' name='pdcm'/> <feature policy='disable' name='mpx'/> </cpu>
I plan to leave CPU 0/8 for the host, wireguard and routing.
I'll use 7/15 for haproxy.
I'll use 1-5/9-13 as a general pool for any ancillary VMs, like mail spool, package builder, etc.
My main web app will use 6/14 dedicated to it as its CPU 0,1 but also use that pool.
Depending on work load, I might change that balance a bit and dedicate more to the web app, or hopefully reduce the number of cores I share with the pool when I discover what the sweet spot is. I'm currently using 4 threads for the app, but there are a number of other helpers running to sync the database elsewhere too.
I scratched my head for a long time looking at the commands in the Github repository. These commands: "curl -sL yabs.sh | bash" or "wget -qO- yabs.sh | bash"
What had me confused was, "How do wget and curl know where to find the yabs.sh shell script file?"
It took a little head-scratching, but I finally figured out that they used a trick. It turns out that "yabs.sh" serves two purposes. It is the filename of the shell script. It is also a URL. The trick is that the URL (yabs.sh) redirects to the Github page so you don't notice it. (In case anyone cares, ".sh" is the country code for St. Helena, a British island in the South Atlantic.)
So ... https://yabs.sh redirects to the Github page. That's the trick. The "-s" and "-q" in the commands stand for "silent mode" and "quiet mode" respectively. I speculate that they were included to hide the trick, which shows up in the more verbose output. Note that I omitted the "-q" in the command below, to expose the more verbose output:
After that, there is a download progress bar, and then the script launches and runs the benchmark tests.
Edit, later:
I know this post will get lost in the overall thread. I wondered whether anyone cares, which is why I posted it here. It seemed pointless to start a new thread for a minor triviality like this. Maybe it was clearly obvious to everyone else, other than me. Sort of a "Yeah? So? What's the point?" situation.
@chimichurri said:
Hosthatch BF2022 2TB Amsterdam, 110 dollars every 3 years
... @dosai Big thanks for the tip! Just like you said, network is fast, my man
I clearly wasn't paying enough attention to everything that was going on! I thought the Polar Bear was the best value 2TB backup option!
@ralf said: Going to do this a bit differently (and I've already posted the original YABS anyway).
Hey @ralf - just wanted to say this was a very awesome and interesting read! Thanks for taking the time to test out the host vs a host-sized VM and compare the two results! If I'm not mistaken, I believe @Not_Oles has done some lengthy experiments testing YABS VM performance vs host performance as well in the past and had similar findings.
@xleet said: It took a little head-scratching, but I finally figured out that they used a trick
Heh -- yes, it is a trick! Can't say it was my idea, though, as I "borrowed" the idea from another widely popular benchmarking script (bench.sh). Appreciate the explanation to all those who may also be wondering the same thing as you
Comments
So, this happened... Completely not a BF deal, but I realised I could consolidate a ton of my VPS and get a beast of a dedi from Hetzner instead on a server auction.
The NVMe disks are a bit weird - both are 12,658 hours, so about 17 months old. The first looks like it's had a reasonably busy life - 57.6TB written for a percentage used of 11%. The second has been barely used, with only 87.3 GB ever written to it!
Amount of RAM is also insane, although sadly it's not ECC. It wasn't advertised as ECC though, so I guess that's on me.
whistle noice. Roughly how much is that a month?
Less than 50€/m I suppose. Have seen those lately but only with 64GB RAM though. No idea what to do with this combination of RAM and tons of RAM but it's an impressive setup. Nice catch.
€56.04 and they obviously have quite a few of them as the price isn't changing, but the time keeps adding a few hours every now and then, so I'm guessing people are buying them then. Actually, it was my first time on Hetzner anything, but it did seem like they had some other i9-9900K's with different disk configurations at a decent chunk more, so I'm guessing this is near the end of the discounting. The ones with 64GB are €54.84, so I figured it was definitely worth getting the better one.
It's definitely a bit expensive than the VPSes I've earmarked for cancellation / transfer, which would have taken me to AX41, but with the setup fee on that, I figured I'd go for this instead.
Hah, now I wonder if I should have waited more especially as it was kind of an impulse buy after I was annoyed at myself for missing the Pebble Host dedi - because I'd already worked out which one it was and added it to my basket ready for the deal and then in my excitement, I just clicked on the OGF link and I ended up with 2 and by the time I'd faffed around reducing the quantity back to 1, I'd already missed it. This one is better, but cost per power is about the same, and getting that one would have actually saved me money overall instead of increasing my costs.
BF is the wrong day to start saving money. You did everything right and actually 56€/m seems like a good deal. The 64GB boxes I've noticed were around 50€/m and 6€/m for another 64GB is an amazing deal imo.
Wat, you drunk? Its perfect.
Except, you have to verify every "DEAL" you see, after some time, you just need to look at the specs and price and your brain goes, wow what a scam.
The 5$/y MXRoute, saves money, end of story.... I could continue this one.
@jarland stop scamming us.
Free NAT KVM | Free NAT LXC | Bobr
ITS WEDNESDAY MY DUDES
I mean it like this, if there wasn't a BF special you probably wouldn't have bought it. Just like @ralf bought the 129GB machine though he knows he won't use the extra RAM compared to a 64GB machine.
Yes, we get weak, over good priced shit, yea.
He could use the additional 64GB as High Speed NVMe doh.
Free NAT KVM | Free NAT LXC | Bobr
ITS WEDNESDAY MY DUDES
At the prices I was looking at, the extra 64GB was only an extra $1.20, so I might as well take it even if I don't need it, because at least it'll end up getting used as a disk cache or something.
BTW I think the $10 mxroute deal was great value anyway, but maybe @Neoon should check to see if this is still available: https://lowendspirit.com/discussion/comment/111058/#Comment_111058
Liga Hosting CM deal Amsterdam €4.99/m
Amsterdam send/receive at 12pm
Liga Hosting CM Deal Seattle €1.69/m
HostHatch 10TB Storage VM 2022 NL (2 years recurring)
Jaysus! Blistering speeds.
LigaHosting Seattle CM sale.
ServerStatus , slackvpn <-- openVPN auto install script for Slackware 15
IncogNET 512MB $25/year
Due to it being a Tor exit, Cloudflare shows a challenge for
yabs.shredirect, so I have to use the full script URI.For the same reason,
yabsdb.comsubmission fails.MicroNode by NatVPS Instance in Korea
EvolutionHost Developer SSD, normally €10/month
iHostART KVM 1GB, normally €5.40/year
iHostART OpenVZ 50GB HDD, €5.50/year
IPv6 is via WireGuard tunnel to iHostART KVM 1GB.
Accepting submissions for IPv6 less than /64 Hall of Incompetence.
Advin Servers
KVM Standard S Black Friday (2 CPU 16GB RAM)
Los Angeles
@Mason you probably never expected this to happen... but looks it can happen and you need one more if in code, lmao.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
Got to keep the templates small when we sell packages with 3Gb disk!
Yeah.... in retrospect, I should have known this issue would arise when I accepted the PR last week. Will get it patched up soon(tm)
Head Janitor @ LES • About • Rules • Support
the LXC templates even come without syslog-ng.
Free NAT KVM | Free NAT LXC | Bobr
ITS WEDNESDAY MY DUDES
Going to do this a bit differently (and I've already posted the original YABS anyway).
I thought it'd be interesting to see how much of an impact virtualisation had on a YABS, so I made a VM using all 16 threads. The first observation was that when I had 16 vCores, GB5 terminated early on the multi-core Horizon Detection test. I tried a few things, thinking it was the CPU host-passthrough that was the problem, but eventually discovered it was fine running with fewer cores. Even 15 was fine, as long as there was one free vCore, but as soon as I used 16, it failed.
Summary
First up host top-level summary:
and the VM:
FIO
The next section is what really surprised me. First up the host, which is using an ext3 FS on an LVM 2-disk mirror:
And now the VM, which is running on a ext3 FS, on a qcow2 file mounted on an ext3 FS on an LVM 2-disk mirror:
What's interesting about this is that the small 4k IO performance is awful in the VM, about 1/4 speed. But, what I didn't expect is that the blocks that are page size and up are somewhat quicker on the VM. I'm not sure why this is, but maybe the readbacks are more likely to end up in RAM cache.
I'd like to do a follow-up experiment sometime where the drive is a raw LVM partition as that's how my KS-LE-1 is set up for VM. This time I decided to just mount an LVM partition on
/var/lib/libvirt/images. I'm still undecided whether I preferlvextendorqemu-img resizeto extend a VM's drive.Networking
Performance is pretty much the same on host and VM. Here's the host, with the IPv6 results removed as I didn't pass them through to the VM:
VM:
Geekbench
Some loss to be expected as I'm using fewer vCores, but it's still not too bad. Slightly better than 15/16 of GB5 score.
Host first:
and now the VM:
Final notes
I'm not intending to use this VM for anything in the real-word, but I thought it was an interesting experiment.
I added this to my linux command line:
iommu=pt intel_iommu=on isolcpus=1,2,3,4,5,6,7,9,10,11,12,13,14,15.In the VM XML file, I also added this because I hate the generic CPU ID names:
<cpu mode='host-passthrough' check='none' migratable='on'><feature policy='disable' name='pdcm'/><feature policy='disable' name='mpx'/></cpu>I plan to leave CPU 0/8 for the host, wireguard and routing.
I'll use 7/15 for haproxy.
I'll use 1-5/9-13 as a general pool for any ancillary VMs, like mail spool, package builder, etc.
My main web app will use 6/14 dedicated to it as its CPU 0,1 but also use that pool.
Depending on work load, I might change that balance a bit and dedicate more to the web app, or hopefully reduce the number of cores I share with the pool when I discover what the sweet spot is. I'm currently using 4 threads for the app, but there are a number of other helpers running to sync the database elsewhere too.
Fascinating to see it broken down that way and also you info on how you plan to use the cores. Thanks for sharing!
LigaHosting KVM SSD - US CB - LES CB
Free Hosting at YetiNode | Cryptid Security | URL Shortener | LaunchVPS | ExtraVM | Host-C | In the Node, or Out of the Loop?
Monster @ $100 Year
HostCram LLC - Web Hosting Built For Speed, Reliability, Security & Uptime! [We operate AS39618]
Hosthatch BF2022 2TB Amsterdam, 110 dollars every 3 years
@dosai Big thanks for the tip! Just like you said, network is fast, my man
Contribute your idling VPS/dedi (link), Android (link) or iOS (link) devices to medical research
I was trying out YABS for the first time. Here is where you find the shell script, more about that later:
https://github.com/masonr/yet-another-bench-script
I scratched my head for a long time looking at the commands in the Github repository. These commands: "curl -sL yabs.sh | bash" or "wget -qO- yabs.sh | bash"
What had me confused was, "How do wget and curl know where to find the yabs.sh shell script file?"
It took a little head-scratching, but I finally figured out that they used a trick. It turns out that "yabs.sh" serves two purposes. It is the filename of the shell script. It is also a URL. The trick is that the URL (yabs.sh) redirects to the Github page so you don't notice it. (In case anyone cares, ".sh" is the country code for St. Helena, a British island in the South Atlantic.)
So ... https://yabs.sh redirects to the Github page. That's the trick. The "-s" and "-q" in the commands stand for "silent mode" and "quiet mode" respectively. I speculate that they were included to hide the trick, which shows up in the more verbose output. Note that I omitted the "-q" in the command below, to expose the more verbose output:
After that, there is a download progress bar, and then the script launches and runs the benchmark tests.
Edit, later:
I know this post will get lost in the overall thread. I wondered whether anyone cares, which is why I posted it here. It seemed pointless to start a new thread for a minor triviality like this. Maybe it was clearly obvious to everyone else, other than me. Sort of a "Yeah? So? What's the point?" situation.
I clearly wasn't paying enough attention to everything that was going on! I thought the Polar Bear was the best value 2TB backup option!
Hey @ralf - just wanted to say this was a very awesome and interesting read! Thanks for taking the time to test out the host vs a host-sized VM and compare the two results! If I'm not mistaken, I believe @Not_Oles has done some lengthy experiments testing YABS VM performance vs host performance as well in the past and had similar findings.
Head Janitor @ LES • About • Rules • Support
Heh -- yes, it is a trick! Can't say it was my idea, though, as I "borrowed" the idea from another widely popular benchmarking script (bench.sh). Appreciate the explanation to all those who may also be wondering the same thing as you
Head Janitor @ LES • About • Rules • Support