Thanks for your email with ID info. I appreciate your email because it convinces me that you are making a serious request which deserves the time I am investing in trying to administer access to the server.
May I please share a few thoughts which came up in my mind?
Port 81: Could we use a high port instead of a privileged port?
What exactly is meant by "private proxy?" I originally imagined it meant a proxy just for use by our friendly Neighbors on our fine server. But maybe "private proxy" means something more or different than a proxy just for all of us?
If somebody interested in security like @AuroraZero asked what is the threat model here, how might you answer?
There are providers here at LES who seem especially interested in privacy. How was the decision made to try our fine server instead of considering some of the providers who seem especially interested in privacy?
Sharing access with others via PM: Should all users of our server comply with the OP Rules in the same excellent way that you are?
Do we need explicit consent from @Hosteroid and @cmeerw, or is silence enough because "silence denotes consent?"
The last thing I want to do is cause trouble for @Hosteroid. How can we be sure that running a proxy on our fine server will not cause trouble for Hosteroid?
Does it make any sense for us to consider other proxies in addition to or instead of shadowsocks-libev? For example, what about Tinyproxy?
Do we know that shadowsocks-libev still works on Debian 12? Maybe you have used shadowsocks-libev recently?
Debian 8, 9 or higher
You can build shadowsocks-libev and all its dependencies by script:
mkdir -p ~/build-area/
cp ./scripts/build_deb.sh ~/build-area/
cd ~/build-area
./build_deb.sh # See https://github.com/shadowsocks/shadowsocks-libev/blob/master/scripts/build_deb.sh
What logging does shadowsocks-libev do?
Are you up on the source code of shadowsocks-libev? Are you okay with posting here in our thread about building shadowsocks-libev (or maybe about installing with Docker) and also posting about how everything works? Are you okay with answering questions about proxies from me and from others?
If we need to use Docker, are you okay with posting here about Docker and with answering questions both about Docker and about the shadowsocks-libev running inside Docker? Are @cmeerw and me going to be able to see inside the Docker container and post about what is there?
I'm looking forward to having you with us on our server because I think proxies are something I could learn more about!
@itsdeadjim said: You can still try removing packages like before (you probably dont need quick3d) but I am not sure how far you can go from here.
I haven't the faintest idea about what's really going on. But, just for fun, I will change the qt6-qtquick3d PLIST by adding the file presently in the build directory but not on the PLIST and also by renaming the .so file to be consistent with the PKG directory.
I'm just having a little fun introducing myself to the huge and complex pkgsrc build process for Wireshark. I don't expect to use the resulting Wireshark, even if it does eventually compile. If there is yet another bump in the build process, I might not continue further. But it's been a lot of fun so far!
=> Checking file-check results for qt6-qtmultimedia-6.8.0nb3
ERROR: ************************************************************
ERROR: The following files are in /usr/pkgsrc/multimedia/qt6-qtmultimedia/work/.destdir/usr/pkg but not in the PLIST:
ERROR: /usr/pkgsrc/multimedia/qt6-qtmultimedia/work/.destdir/usr/pkg/qt6/include/QtMultimedia/6.8.0/QtMultimedia/private/qsymbolsresolveutils_p.h
*** Error code 1
Stop.
bmake[2]: stopped making "reinstall" in /usr/pkgsrc/multimedia/qt6-qtmultimedia
*** Error code 1
Stop.
bmake[1]: stopped making "reinstall" in /usr/pkgsrc/multimedia/qt6-qtmultimedia
*** Error code 1
Stop.
bmake: stopped making "all" in /usr/pkgsrc/net/wireshark
Even installing Wireshark with apt-get would require quite a few packages.
I would track down what, and why they're deemed necessary - as X11 is not a requirement for wireshark. It runs on OpenWRT. Afraid I don't care enough to delve deeper.
Oh wow, so many questions~~~ this is why I thought those would come at an earlier step #1... I might have not applied if I had known this would come.
@Not_Oles said: Port 81: Could we use a high port instead of a privileged port?
I am using the same port on a network of other servers of mine~ and I do not think it is used for any relevant service.
What exactly is meant by "private proxy?" I originally imagined it meant a proxy just for use by our friendly Neighbors on our fine server. But maybe "private proxy" means something more or different than a proxy just for all of us?
I just meant a proxy protected by password~ (as opposed to a public one for everyone opened to all). I have no idea of the (other) requirements other than just being a LES member to get all the settings. I imagined that only interested people would ask for the access, I can't imagine people just joining LES to get this private proxy access considering all the browser extensions that exist already... This is why: I feel all those questions are a bit overkill~ my simple mind just imagined the start of a custom/tradition of sharing a similar service (be it from their own servers).
If somebody interested in security like @AuroraZero asked what is the threat model here, how might you answer?
I do not understand the "threat model" so I guess it would be:
from a server load point of view? Shadowsocks is very well written and uses very little RAM & CPU, the only resources would really be bandwidth/traffic
from a security point of view? It is password protected, it does not care about DDoS
from an encryption point of view? It is using AES
from a legal perspective? It cannot be more dangerous than all those hosting providers here on LES, can it?
There are providers here at LES who seem especially interested in privacy. How was the decision made to try our fine server instead of considering some of the providers who seem especially interested in privacy?
I do not really need this myself~ I think it is just a nice addition: out of 65.5K TCP ports and 65.5K UDP ports of (potential) services that could run (131K ports in total) just 2x ports are enough to implement this...
Sharing access with others via PM: Should all users of our server comply with the OP Rules in the same excellent way that you are?
Most people can order a server without showing an ID... I do not think it would be relevant to apply the same OP Rules for that.
Do we need explicit consent from @Hosteroid and @cmeerw , or is silence enough because "silence denotes consent?"
I have no idea.... aren't you in charge of this?
The last thing I want to do is cause trouble for @Hosteroid. How can we be sure that running a proxy on our fine server will not cause trouble for Hosteroid?
I really doubt anyone would ask access Hosteroid' server to hack the Pentagon considering the amount of VPN/proxy offers out there already... there are even public list of proxies out there (accidently left opened by unexperienced admins and discovered by bots). Surely those "offers" are more valuable for doing something nasty than asking me or you for the access settings?...
Does it make any sense for us to consider other proxies in addition to or instead of shadowsocks-libev? For example, what about Tinyproxy?
I don't know about Tinyproxy, I've looked it up and there are no encryption layer (apart maybe from the standard SSL).
Do we know that shadowsocks-libev still works on Debian 12? Maybe you have used shadowsocks-libev recently?
This is why Docker is appropriate: it doesn't care about the underlying OS (I didn't even know you were running Debian 12, and you can change this in the future without a problem).
I do not think there is any, especially considering those logs would be within the Docker container (which would make it a pain to examine). In terms of disk I believe the usage is around 50MB~.
Are you up on the source code of shadowsocks-libev? Are you okay with posting here in our thread about building shadowsocks-libev (or maybe about installing with Docker) and also posting about how everything works? Are you okay with answering questions about proxies from me and from others?
If we need to use Docker, are you okay with posting here about Docker and with answering questions both about Docker and about the shadowsocks-libev running inside Docker? Are @cmeerw and me going to be able to see inside the Docker container and post about what is there?
I did not examine the code of shadowsocks-libev, I just trust the developers. I do not mind writing a tutorial~. The container would only be using port 81 on both TCP & UDP, there cannot be any other usage than shadowsocks as no backdoors could exists because no other ports could create an access to the container anyway... The port forwarding to/from the container is easily viewable from a ps aux command.
We could make a rule that such access should NOT be used for scrapers (and obviously any other nasty stuff, you can include NO torrenting in that, I do not even think it's possible to torrent with Shadowsocks but maybe I am wrong as I didn't even try it myself~). This is NOT the usage I had in mind ANYWAY... Maybe this is too sensitive of a topic/service~ I can see from the questioning it triggers that it seems quite a concern already.. Maybe it's better that I just drop this idea now?
If I get the strength/time one day I might just offer it myself to the LES community from my own servers: again I just think there is very little demand for this, considering the numerous offers out there, with free or $1/m servers available around...
@WSS said:
I would track down what, and why they're deemed necessary - as X11 is not a requirement for wireshark. It runs on OpenWRT. Afraid I don't care enough to delve deeper.
Presumably, wireshark can be built without the GUI (which is what the OpenWRT build does - so you only get TShark there).
@jcn50 said: I really doubt anyone would ask access Hosteroid' server to hack the Pentagon considering the amount of VPN/proxy offers out there already... there are even public list of proxies out there (accidently left opened by unexperienced admins and discovered by bots). Surely those "offers" are more valuable for doing something nasty than asking me or you for the access settings?...
Maybe the value could be that this fine server is not on any blacklist so far
@jcn50 said: This is why Docker is appropriate: it doesn't care about the underlying OS (I didn't even know you were running Debian 12, and you can change this in the future without a problem).
There is even a Debian package, but why does the README not recommend using that?
@cmeerw said: Maybe the value could be that this fine server is not on any blacklist so far
A shadowsocks service is not a SMTP relay service. I do not even know what happens on the email headers side when sending an email, I actually never tried~.
@cmeerw said: There is even a Debian package, but why does the README not recommend using that?
@cmeerw said: Maybe the value could be that this fine server is not on any blacklist so far
A shadowsocks service is not a SMTP relay service. I do not even know what happens on the email headers side when sending an email, I actually never tried~.
Not all blacklists are only about SMTP (and email spam), but some of them also cover other undesirable behaviour from a particular IP.
@WSS said:
I would track down what, and why they're deemed necessary - as X11 is not a requirement for wireshark. It runs on OpenWRT. Afraid I don't care enough to delve deeper.
@jcn50 said: Maybe this is too sensitive of a topic/service~ I can see from the questioning it triggers that it seems quite a concern already.. Maybe it's better that I just drop this idea now?
@jcn50 said: I guess I will just silently evaporate from this thread...
Thanks for opening and participating in our excellent discussion!
Thanks to Hosteroid for donating our excellent server!
@cmeerw said:
So at the moment we have a 64 GB root volume (ext4) and a 64 GB data volume (btrfs).
@cmeerw wanted btrfs and did excellent work resetting the partitions and the file systems on our Hosteroid LES Community Server. Thanks @cmeerw!
I wondered whether the fio IOPS test in Yabs might show a difference between our ext4 and btrfs volumes. So I ran Yabs on both the ext4 and the btrfs volumes. The full Yabs output for both tests is given below.
The Yabs Read Write Total 4K IOPS test results seem to be about 381.2k on ext4 and about 156.9k on btrfs.
If the Yabs IOPS data is at all significant in this context, could someone please explain to me the causes of the difference in IOPS?
By the way, switching our attention for a moment from local disk read and write speeds to network send and receive speeds, please also note also that our server's IPv4 and IPv6 send speeds almost always exceed, and often significantly exceed the receive speeds. I heard that higher send speeds cost more because, with servers, it typically is the sender (not the recipient) who pays for network traffic, and the sender usually has to pay more for faster upstream providers. Maybe, based on the below Yabs results, we can say that our Hosteroid LES Community Server serves pretty fast compared to almost all the speed test servers used by Yabs! Thanks @Hosteroid!
Here's the Yabs from the /root directory, which I guess is on the "root volume (ext4)."
root@hlcs:~# date; pwd
Thu Dec 19 03:00:19 AM UTC 2024
/root
root@hlcs:~# curl -sL yabs.sh | bash
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2024-12-17 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu Dec 19 03:00:32 AM UTC 2024
Basic System Information:
---------------------------------
Uptime : 4 days, 9 hours, 41 minutes
Processor : Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
CPU cores : 8 @ 799.968 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 31.3 GiB
Swap : 3.8 GiB
Disk : 126.9 GiB
Distro : Debian GNU/Linux 12 (bookworm)
Kernel : 6.1.0-28-amd64
VM Type : NONE
IPv4/IPv6 : ✔ Online / ✔ Online
IPv6 Network Information:
---------------------------------
ISP : M247 Europe SRL
ASN : AS9009 M247 Europe SRL
Host : HOSTEROID Bucharest
Location : Bucharest, București (B)
Country : Romania
fio Disk Speed Tests (Mixed R/W 50/50) (Partition /dev/mapper/M247--X10E--9N--vg-root):
---------------------------------
Block Size | 4k (IOPS) | 64k (IOPS)
------ | --- ---- | ---- ----
Read | 761.59 MB/s (190.3k) | 425.99 MB/s (6.6k)
Write | 763.60 MB/s (190.9k) | 428.23 MB/s (6.6k)
Total | 1.52 GB/s (381.2k) | 854.23 MB/s (13.3k)
| |
Block Size | 512k (IOPS) | 1m (IOPS)
------ | --- ---- | ---- ----
Read | 750.23 MB/s (1.4k) | 953.74 MB/s (931)
Write | 790.09 MB/s (1.5k) | 1.01 GB/s (993)
Total | 1.54 GB/s (3.0k) | 1.97 GB/s (1.9k)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | busy | 614 Mbits/sec | 37.7 ms
Eranium | Amsterdam, NL (100G) | 924 Mbits/sec | 923 Mbits/sec | 32.6 ms
Uztelecom | Tashkent, UZ (10G) | 866 Mbits/sec | 469 Mbits/sec | 114 ms
Leaseweb | Singapore, SG (10G) | 784 Mbits/sec | 579 Mbits/sec | 220 ms
Clouvider | Los Angeles, CA, US (10G) | 796 Mbits/sec | 290 Mbits/sec | 180 ms
Leaseweb | NYC, NY, US (10G) | 852 Mbits/sec | 598 Mbits/sec | 109 ms
Edgoo | Sao Paulo, BR (1G) | 708 Mbits/sec | 141 Mbits/sec | 236 ms
iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 905 Mbits/sec | 717 Mbits/sec | 37.8 ms
Eranium | Amsterdam, NL (100G) | 911 Mbits/sec | 908 Mbits/sec | 32.7 ms
Uztelecom | Tashkent, UZ (10G) | 857 Mbits/sec | 430 Mbits/sec | 114 ms
Leaseweb | Singapore, SG (10G) | 733 Mbits/sec | 648 Mbits/sec | 220 ms
Clouvider | Los Angeles, CA, US (10G) | busy | 292 Mbits/sec | 180 ms
Leaseweb | NYC, NY, US (10G) | 842 Mbits/sec | 579 Mbits/sec | 109 ms
Edgoo | Sao Paulo, BR (1G) | 720 Mbits/sec | 184 Mbits/sec | 235 ms
Geekbench 6 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1510
Multi Core | 5028
Full Test | https://browser.geekbench.com/v6/cpu/9500963
YABS completed in 12 min 49 sec
root@hlcs:~#
Here's the Yabs from /home, which I guess is on the "64 GB data volume (btrfs)."
root@hlcs:/home# date;pwd
Thu Dec 19 03:18:54 AM UTC 2024
/home
root@hlcs:/home# curl -sL yabs.sh | bash
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2024-12-17 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Thu Dec 19 03:19:05 AM UTC 2024
Basic System Information:
---------------------------------
Uptime : 4 days, 9 hours, 59 minutes
Processor : Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
CPU cores : 8 @ 2090.588 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 31.3 GiB
Swap : 3.8 GiB
Disk : 126.9 GiB
Distro : Debian GNU/Linux 12 (bookworm)
Kernel : 6.1.0-28-amd64
VM Type : NONE
IPv4/IPv6 : ✔ Online / ✔ Online
IPv6 Network Information:
---------------------------------
ISP : M247 Europe SRL
ASN : AS9009 M247 Europe SRL
Host : HOSTEROID Bucharest
Location : Bucharest, București (B)
Country : Romania
fio Disk Speed Tests (Mixed R/W 50/50) (Partition /dev/mapper/M247--X10E--9N--vg-data):
---------------------------------
Block Size | 4k (IOPS) | 64k (IOPS)
------ | --- ---- | ---- ----
Read | 313.39 MB/s (78.3k) | 393.98 MB/s (6.1k)
Write | 314.21 MB/s (78.5k) | 396.05 MB/s (6.1k)
Total | 627.60 MB/s (156.9k) | 790.03 MB/s (12.3k)
| |
Block Size | 512k (IOPS) | 1m (IOPS)
------ | --- ---- | ---- ----
Read | 402.37 MB/s (785) | 394.32 MB/s (385)
Write | 423.75 MB/s (827) | 420.58 MB/s (410)
Total | 826.13 MB/s (1.6k) | 814.90 MB/s (795)
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 917 Mbits/sec | 689 Mbits/sec | 37.7 ms
Eranium | Amsterdam, NL (100G) | 924 Mbits/sec | 922 Mbits/sec | 32.8 ms
Uztelecom | Tashkent, UZ (10G) | 716 Mbits/sec | 483 Mbits/sec | 114 ms
Leaseweb | Singapore, SG (10G) | 781 Mbits/sec | 392 Mbits/sec | 220 ms
Clouvider | Los Angeles, CA, US (10G) | 798 Mbits/sec | 248 Mbits/sec | 180 ms
Leaseweb | NYC, NY, US (10G) | 851 Mbits/sec | 584 Mbits/sec | 109 ms
Edgoo | Sao Paulo, BR (1G) | 655 Mbits/sec | 107 Mbits/sec | 236 ms
iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 597 Mbits/sec | 566 Mbits/sec | 37.6 ms
Eranium | Amsterdam, NL (100G) | 910 Mbits/sec | 907 Mbits/sec | 32.6 ms
Uztelecom | Tashkent, UZ (10G) | 857 Mbits/sec | 447 Mbits/sec | 114 ms
Leaseweb | Singapore, SG (10G) | 739 Mbits/sec | 659 Mbits/sec | 220 ms
Clouvider | Los Angeles, CA, US (10G) | 787 Mbits/sec | 239 Mbits/sec | 179 ms
Leaseweb | NYC, NY, US (10G) | 841 Mbits/sec | 635 Mbits/sec | 109 ms
Edgoo | Sao Paulo, BR (1G) | 719 Mbits/sec | 259 Mbits/sec | 236 ms
Geekbench 6 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1514
Multi Core | 5044
Full Test | https://browser.geekbench.com/v6/cpu/9501121
YABS completed in 12 min 45 sec
root@hlcs:/home#
Here is /etc/fstab.
root@hlcs:~# date
Thu Dec 19 03:33:40 AM UTC 2024
root@hlcs:~# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# systemd generates mount units based on this file, see systemd.mount(5).
# Please run 'systemctl daemon-reload' after making changes here.
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/M247--X10E--9N--vg-root / ext4 errors=remount-ro 0 1
# /boot was on /dev/nvme0n1p1 during installation
UUID=cea09f74-374a-4d7a-9161-b8eae6669bf9 /boot ext4 defaults 0 2
/dev/mapper/M247--X10E--9N--vg-data /mnt/data btrfs defaults 0 2
/dev/mapper/M247--X10E--9N--vg-swap none swap sw 0 0
# bind mounts
/mnt/data/home /home none defaults,bind 0 0
root@hlcs:~#
As you guys can see, we have some extra disk space. We would welcome a few more of you to share it. If you are interested, please check the How To Apply section in the OP.
I asked Google Gemini whether the Linux kernel caching was independent of the file system in use. Gemini's response was:
Yes, dm-cache in the Linux kernel operates below the filesystem layer. It's a device mapper target, meaning it sits between the filesystem and the underlying block device (like a hard drive or SSD). The filesystem interacts with dm-cache as if it were a regular block device, unaware of the caching mechanisms happening underneath.
I asked Google Gemini what to do next to investigate the differences in the Yabs IOPS results. These are Gemini's suggested steps:
mount Command Output: Share the output of the mount command so we can see the mount options for both filesystems.
btrfs filesystem show Output: Share the output of btrfs filesystem show for the btrfs filesystem to see its configuration.
fio Command: Share the exact fio command you're using to run the benchmark.
btrfs filesystem df Output: Share the output of btrfs filesystem df /mount/point of the btrfs partition to check for free space and fragmentation.
Can anyone here suggest any corrections or additions to Gemini's suggestions?
How can we determine why, on the same same machine, with the same drive (just different Partitions), with the same test, and with no other workload, the ext4 4k RW IOPS would be so different -- 381 on ext4 and 156 on btrfs?
@Not_Oles said:
I asked Google Gemini whether the Linux kernel caching was independent of the file system in use. Gemini's response was:
Yes, dm-cache in the Linux kernel operates below the filesystem layer. It's a device mapper target, meaning it sits between the filesystem and the underlying block device (like a hard drive or SSD). The filesystem interacts with dm-cache as if it were a regular block device, unaware of the caching mechanisms happening underneath.
I asked Google Gemini what to do next to investigate the differences in the Yabs IOPS results. These are Gemini's suggested steps:
Is there a description somewhere how the YABS IOPS tests work? (in terms of files created, operations done on those files, ...)
fio Command: Share the exact fio command you're using to run the benchmark.
I guess, and then read up on what fio actually does.
How can we determine why, on the same same machine, with the same drive (just different Partitions), with the same test, and with no other workload, the ext4 4k RW IOPS would be so different -- 381 on ext4 and 156 on btrfs?
Thanks @cmeerw! I appreciate your trying to teach me! Maybe I should say "guide me" toward learning rather than "teach me?" In other words, better teaching than merely "teaching?" /s
@Not_Oles said: How can we determine why, on the same same machine, with the same drive (just different Partitions), with the same test, and with no other workload, the ext4 4k RW IOPS would be so different -- 381 on ext4 and 156 on btrfs?
Comparing btrfs with ext4 speed is like comparing a Formula 1 car with a Cesna airplane. A Cesna is not a slower car. btrfs is a copy-on-write filesystem, with extensive metadata maintaining and data checksums, which ext4 lacks. In some cases btrfs can be much slower than ext4 https://www.phoronix.com/review/linux-50-filesystems/2
IMHO those differences in yabs, especially in smaller blocks, makes total sense
@Not_Oles said: How can we determine why, on the same same machine, with the same drive (just different Partitions), with the same test, and with no other workload, the ext4 4k RW IOPS would be so different -- 381 on ext4 and 156 on btrfs?
Comparing btrfs with ext4 speed is like comparing a Formula 1 car with a Cesna airplane. A Cesna is not a slower car. btrfs is a copy-on-write filesystem, with extensive metadata maintaining and data checksums, which ext4 lacks. In some cases btrfs can be much slower than ext4 https://www.phoronix.com/review/linux-50-filesystems/2
IMHO those differences in yabs, especially in smaller blocks, makes total sense
Well... btrfs has a lot of features, but I am unlikely to ever use every feature of it. The question then is how much should I pay for features I don't use? Ideally, copy-on-write should only affect performance if I actually use it for some snapshot or similar. Similarly, if data checksums have a significant performance impact, maybe it should be possible to turn them off (I haven't actually checked if they can be).
@cmeerw said: Well... btrfs has a lot of features, but I am unlikely to ever use every feature of it. The question then is how much should I pay for features I don't use? Ideally, copy-on-write should only affect performance if I actually use it for some snapshot or similar. Similarly, if data checksums have a significant performance impact, maybe it should be possible to turn them off (I haven't actually checked if they can be).
Both can be turned off for data (I am not sure about metadata), but then there remain very few reasons to use btrfs. I have no idea by how much these will improve performance. By the way, copy on write is not only about snapshots, but also ensures consistency during crashes/power failures.
mount -o nodatacow,nodatasum ....
I am quite sure that btrfs can be fine-tuned to handle specific work loads much more effectively (i.e. to give better 4k yabs) but I have no experience with it, I prefer zfs tbh.
@itsdeadjim said: Comparing btrfs with ext4 speed is like comparing a Formula 1 car with a Cesna airplane. A Cesna is not a slower car. btrfs is a copy-on-write filesystem, with extensive metadata maintaining and data checksums, which ext4 lacks. In some cases btrfs can be much slower than ext4 https://www.phoronix.com/review/linux-50-filesystems/2
Your comment helped me understand what information I am looking for.
What I am looking for is a simple list comparing the steps and the relative time expended on each step to write to and to read from a file in ext4, btrfs, and xfs. The list should include "extra" steps beyond just writing and reading
From a quick skim, the paper covers Ext4, XFS, and Btrfs. It's largely about minimizing performance variation of repeated tests within each of these file systems. The context is practical workloads such as mail server, file server, and Web server. The ideas behind the paper are that modern, large systems need predictable performance, and that performance variations of repeated sequences within each individual file system can be significantly and undesirably higher than expected.
The article doesn't seem to contain the simple, time aware, architectural comparison that I am looking for.
Comments
Hi again @jcn50!
Thanks for your email with ID info. I appreciate your email because it convinces me that you are making a serious request which deserves the time I am investing in trying to administer access to the server.
May I please share a few thoughts which came up in my mind?
Port 81: Could we use a high port instead of a privileged port?
What exactly is meant by "private proxy?" I originally imagined it meant a proxy just for use by our friendly Neighbors
on our fine 
server. But maybe "private proxy" means something more or different than a proxy just for all of us?
If somebody interested in security like @AuroraZero asked what is the threat model here, how might you answer?
There are providers here at LES who seem especially interested in privacy. How was the decision made to try our fine server instead of considering some of the providers who seem especially interested in privacy?
Sharing access with others via PM: Should all users of our server comply with the OP Rules in the same excellent way that you are?
Do we need explicit consent from @Hosteroid
and @cmeerw 
, or is silence enough because "silence denotes consent?"
The last thing I want to do is cause trouble for @Hosteroid. How can we be sure that running a proxy on our fine
server will not cause trouble for Hosteroid?
Does it make any sense for us to consider other proxies in addition to or instead of shadowsocks-libev? For example, what about Tinyproxy?
Do we know that shadowsocks-libev still works on Debian 12? Maybe you have used shadowsocks-libev recently?
Instead of Docker, could we possibly consider following something like the build instructions from the RREADME.md at https://github.com/shadowsocks/shadowsocks-libev ?
What logging does shadowsocks-libev do?
Are you up on the source code of shadowsocks-libev? Are you okay with posting here in our thread about building shadowsocks-libev (or maybe about installing with Docker) and also posting about how everything works? Are you okay with answering questions about proxies from me and from others?
If we need to use Docker, are you okay with posting here about Docker and with answering questions both about Docker and about the shadowsocks-libev running inside Docker? Are @cmeerw and me going to be able to see inside the Docker container and post about what is there?
I'm looking forward to having you with us on our server because I think proxies are something I could learn more about!
Thanks! Best wishes!
Tom
I hope everyone gets the servers they want!
@itsdeadjim
I looked at https://releng.netbsd.org/bulktracker/x11/qt6-qtbase. The only Linux on that page seems to be Rocky, which seems to have "failed" and also "indirect-failed".
I haven't the faintest idea about what's really going on. But, just for fun, I will change the qt6-qtquick3d PLIST by adding the file presently in the build directory but not on the PLIST and also by renaming the .so file to be consistent with the PKG directory.
I'm just having a little fun introducing myself to the huge and complex pkgsrc build process for Wireshark. I don't expect to use the resulting Wireshark, even if it does eventually compile. If there is yet another bump in the build process, I might not continue further. But it's been a lot of fun so far!
Thanks @Hosteroid!
Thanks @itsdeadjim!
I hope everyone gets the servers they want!
Okay, that's it for this time.
Even installing Wireshark with apt-get would require quite a few packages.
I hope everyone gets the servers they want!
Why do you have X11 depends?
My pronouns are like/subscribe.
@WSS
Great to see you!
X11 dependencies are present, I suppose, because Wireshark is a graphical application. And graphical applications want X11.
Probably you have a better idea.
It's getting late here, so sleep now for me. But I will look for your reply, if any, in the morning.
Hope you are doing great! Best wishes! Nice that you are back!
I hope everyone gets the servers they want!
I would track down what, and why they're deemed necessary - as X11 is not a requirement for wireshark. It runs on OpenWRT. Afraid I don't care enough to delve deeper.
My pronouns are like/subscribe.
Oh wow, so many questions~~~ this is why I thought those would come at an earlier step #1... I might have not applied if I had known this would come.
I am using the same port on a network of other servers of mine~ and I do not think it is used for any relevant service.
I just meant a proxy protected by password~ (as opposed to a public one for everyone opened to all). I have no idea of the (other) requirements other than just being a LES member to get all the settings. I imagined that only interested people would ask for the access, I can't imagine people just joining LES to get this private proxy access considering all the browser extensions that exist already... This is why: I feel all those questions are a bit overkill~ my simple mind just imagined the start of a custom/tradition of sharing a similar service (be it from their own servers).
I do not understand the "threat model" so I guess it would be:
I do not really need this myself~ I think it is just a nice addition: out of 65.5K TCP ports and 65.5K UDP ports of (potential) services that could run (131K ports in total) just 2x ports are enough to implement this...
Most people can order a server without showing an ID... I do not think it would be relevant to apply the same OP Rules for that.
I have no idea.... aren't you in charge of this?
I really doubt anyone would ask access Hosteroid' server to hack the Pentagon considering the amount of VPN/proxy offers out there already... there are even public list of proxies out there (accidently left opened by unexperienced admins and discovered by bots). Surely those "offers" are more valuable for doing something nasty than asking me or you for the access settings?...
I don't know about Tinyproxy, I've looked it up and there are no encryption layer (apart maybe from the standard SSL).
This is why Docker is appropriate: it doesn't care about the underlying OS (I didn't even know you were running Debian 12, and you can change this in the future without a problem).
The deployment with Docker would take 1~2 min.
I do not think there is any, especially considering those logs would be within the Docker container (which would make it a pain to examine). In terms of disk I believe the usage is around 50MB~.
I did not examine the code of shadowsocks-libev, I just trust the developers. I do not mind writing a tutorial~. The container would only be using port 81 on both TCP & UDP, there cannot be any other usage than shadowsocks as no backdoors could exists because no other ports could create an access to the container anyway... The port forwarding to/from the container is easily viewable from a
ps auxcommand.We could make a rule that such access should NOT be used for scrapers (and obviously any other nasty stuff, you can include NO torrenting in that, I do not even think it's possible to torrent with Shadowsocks but maybe I am wrong as I didn't even try it myself~). This is NOT the usage I had in mind ANYWAY... Maybe this is too sensitive of a topic/service~ I can see from the questioning it triggers that it seems quite a concern already.. Maybe it's better that I just drop this idea now?
If I get the strength/time one day I might just offer it myself to the LES community from my own servers: again I just think there is very little demand for this, considering the numerous offers out there, with free or $1/m servers available around...
I have opened a poll~, let's see the demand...
Presumably, wireshark can be built without the GUI (which is what the OpenWRT build does - so you only get TShark there).
Maybe the value could be that this fine server is not on any blacklist so far
There is even a Debian package, but why does the README not recommend using that?
A shadowsocks service is not a SMTP relay service. I do not even know what happens on the email headers side when sending an email, I actually never tried~.
I don't know~~ one can only guess & my assumption is that libpcre3 will one day be deprecated for Debian as well.
At the moment the poll is favoring the "No" side~ so I guess I will just silently evaporate from this thread...
Not all blacklists are only about SMTP (and email spam), but some of them also cover other undesirable behaviour from a particular IP.
Google found discussion of Wireshark and OpenWRT at https://openwrt.org/docs/guide-user/firewall/misc/tcpdump_wireshark. One of the example commands from that discussion is
The -k and -i options are explained on the man page:
It looks like Wireshark in the OpenWRT example might be passing its output over ssh, and so it's "really" tshark, maybe?
Neither NetBSD pkgsrc nor pkgsrc-wip seem to offer the option to install tshark. The Wireshark website offers hints:
https://ask.wireshark.org/question/12584/how-to-build-and-install-tshark-without-wireshark/
https://www.wireshark.org/docs/wsdg_html_chunked/ChToolsCMake.html
Maybe I could figure out some more and eventually help add tshark to pkgsrc-wip?
Hey, maybe we LESbians could meet up at the Sharkfest!
Thanks again @WSS! Again, it's great that you are back! I hope you continue to stick around!
I hope everyone gets the servers they want!
Thanks for opening and participating in our excellent discussion!
Thanks to Hosteroid for donating our excellent server!
Best wishes and kindest regards!
I hope everyone gets the servers they want!
@cmeerw wanted btrfs and did excellent work resetting the partitions and the file systems on our Hosteroid LES Community Server. Thanks @cmeerw!
I wondered whether the
fioIOPS test in Yabs might show a difference between our ext4 and btrfs volumes. So I ran Yabs on both the ext4 and the btrfs volumes. The full Yabs output for both tests is given below.The Yabs Read Write Total 4K IOPS test results seem to be about 381.2k on ext4 and about 156.9k on btrfs.
If the Yabs IOPS data is at all significant in this context, could someone please explain to me the causes of the difference in IOPS?
By the way, switching our attention for a moment from local disk read and write speeds to network send and receive speeds, please also note also that our server's IPv4 and IPv6 send speeds almost always exceed, and often significantly exceed the receive speeds. I heard that higher send speeds cost more because, with servers, it typically is the sender (not the recipient) who pays for network traffic, and the sender usually has to pay more for faster upstream providers. Maybe, based on the below Yabs results, we can say that our Hosteroid LES Community Server serves pretty fast compared to almost all the speed test servers used by Yabs! Thanks @Hosteroid!
Here's the Yabs from the
/rootdirectory, which I guess is on the "root volume (ext4)."Here's the Yabs from
/home, which I guess is on the "64 GB data volume (btrfs)."Here is /etc/fstab.
As you guys can see, we have some extra disk space. We would welcome a few more of you to share it. If you are interested, please check the How To Apply section in the OP.
Thanks again @Hosteroid!
Thanks again @cmeerw! 
Thanks again LES! 
I hope everyone gets the servers they want!
I asked Google Gemini whether the Linux kernel caching was independent of the file system in use. Gemini's response was:
I asked Google Gemini what to do next to investigate the differences in the Yabs IOPS results. These are Gemini's suggested steps:
Can anyone here suggest any corrections or additions to Gemini's suggestions?
How can we determine why, on the same same machine, with the same drive (just different Partitions), with the same test, and with no other workload, the ext4 4k RW IOPS would be so different -- 381 on ext4 and 156 on btrfs?
Thanks everyone!
Thanks Hosteroid! 
I hope everyone gets the servers they want!
Is there a description somewhere how the YABS IOPS tests work? (in terms of files created, operations done on those files, ...)
I guess, and then read up on what
fioactually does.Maybe should also try with XFS then?
Just created an XFS volume:
and added it to
/etc/fstab(mounted on/mnt/xfs)Thanks @cmeerw!
I appreciate your trying to teach me! 
Maybe I should say "guide me" toward learning rather than "teach me?" In other words, better teaching than merely "teaching?" /s
I hope everyone gets the servers they want!
Ran Yabs on the new xfs partition.
4k RW IOPS from Yabs:
381k on ext4
156k on btrfs
375k on xfs
Full Yabs:
I hope everyone gets the servers they want!
Comparing btrfs with ext4 speed is like comparing a Formula 1 car with a Cesna airplane. A Cesna is not a slower car. btrfs is a copy-on-write filesystem, with extensive metadata maintaining and data checksums, which ext4 lacks. In some cases btrfs can be much slower than ext4 https://www.phoronix.com/review/linux-50-filesystems/2
IMHO those differences in yabs, especially in smaller blocks, makes total sense
Well... btrfs has a lot of features, but I am unlikely to ever use every feature of it. The question then is how much should I pay for features I don't use? Ideally, copy-on-write should only affect performance if I actually use it for some snapshot or similar. Similarly, if data checksums have a significant performance impact, maybe it should be possible to turn them off (I haven't actually checked if they can be).
I'm curious how ZFS compares to this trio. There's zfs-dkms available in contrib for Debian.
Both can be turned off for data (I am not sure about metadata), but then there remain very few reasons to use btrfs. I have no idea by how much these will improve performance. By the way, copy on write is not only about snapshots, but also ensures consistency during crashes/power failures.
I am quite sure that btrfs can be fine-tuned to handle specific work loads much more effectively (i.e. to give better 4k yabs) but I have no experience with it, I prefer zfs tbh.
>
@itsdeadjim
Thanks for your comment and the link!
Your comment helped me understand what information I am looking for.
What I am looking for is a simple list comparing the steps and the relative time expended on each step to write to and to read from a file in ext4, btrfs, and xfs. The list should include "extra" steps beyond just writing and reading
Yeah, add ZFS to the list. Maybe BSD ffs and kfs (from Plan 9) too.
I hope everyone gets the servers they want!
Google (regular search, not Scholar) found this USENIX Fast'17 pdf:
On the Performance Variation in Modern Storage Stacks
https://www.fsl.cs.stonybrook.edu/docs/evos/evos-instability-fast17.pdf
From a quick skim, the paper covers Ext4, XFS, and Btrfs. It's largely about minimizing performance variation of repeated tests within each of these file systems. The context is practical workloads such as mail server, file server, and Web server. The ideas behind the paper are that modern, large systems need predictable performance, and that performance variations of repeated sequences within each individual file system can be significantly and undesirably higher than expected.
The article doesn't seem to contain the simple, time aware, architectural comparison that I am looking for.
I hope everyone gets the servers they want!