Alpine Linux 3.23 - 23mb ram | 71mb disk - guide

AnthonySmith · 6:35PM

Minimising Alpine 3.23 on a KVM VPS ran on TierHive

This guide documents how to strip a freshly deployed Alpine 3.23 (should be fine on 3.22 also) VPS down to the minimum RAM and disk footprint without breaking it.

For those reading it running it outside of TierHive, please note this targets KVM-based VPS instances with a virtio-blk disk (/dev/vda), a single network interface, and a static IP assigned at deployment. If your disk is virtio-scsi (/dev/sda) there is one step that differs, noted inline.

Before

Fresh deploy, cloud-init has run, nothing changed yet, cant run apk, not enough free ram.

              total        used        free      shared  buff/cache   available
Mem:             91          38          25           2          28          47
Swap:             0           0           0

Filesystem      Size      Used Available Use% Mounted on
/dev/vda       953.0M    164.4M    741.7M  18% /

Stage 1: Kernel Module Blacklist and Initramfs

The Alpine virt kernel loads modules that have no purpose on a headless KVM VPS: USB controllers, graphics drivers, cloud-specific network drivers for AWS and GCP, I2C buses, input devices, and more. Blacklisting them stops them loading on boot.

Some modules, however, are listed in the kernel modules= boot parameter and load before the blacklist is read. They also need removing from the initramfs features list. Do all three together so only one mkinitfs run is needed.

Create the blacklist:

cat > /etc/modprobe.d/blacklist-unnecessary.conf << 'EOF'
# Graphics (headless server)
blacklist drm
blacklist drm_kms_helper
blacklist simpledrm
blacklist virtio_gpu
blacklist fb

# KVM (not nesting VMs)
blacklist kvm
blacklist kvm_amd
blacklist kvm_intel

# Legacy devices
blacklist floppy
blacklist cdrom
blacklist sr_mod
blacklist isofs

# HID/input (headless)
blacklist hid
blacklist usbhid
blacklist hid_generic
blacklist psmouse
blacklist mousedev

# Wrong cloud drivers (not GCP/AWS)
blacklist gve
blacklist ena

# Force block DRM (blacklist alone does not work, ACPI triggers it)
install drm /bin/true
install drm_kms_helper /bin/true
install simpledrm /bin/true
install fb /bin/true

# USB (not needed on VPS)
blacklist usbcore
blacklist xhci_hcd
blacklist xhci_pci
blacklist usb_common

# I2C (not needed)
blacklist i2c_core
blacklist i2c_smbus
blacklist i2c_piix4

# Input (headless)
blacklist evdev
blacklist button

# Misc not needed
blacklist loop
blacklist ata_generic
blacklist i6300esb
blacklist qemu_fw_cfg

# Memory ballooning
blacklist virtio_balloon

# Hard block loop device (blacklist entry alone is not always sufficient)
install loop /bin/true
EOF

Strip the initramfs down to what a KVM virtio-blk instance actually needs. The default includes USB, CDROM, NVMe, RAID, SCSI, and cloud-specific drivers that will never be used:

sed -i 's/^features=.*/features="base ext4 virtio"/' /etc/mkinitfs/mkinitfs.conf

If your disk is virtio-scsi (/dev/sda) keep scsi in the features list: features="base ext4 scsi virtio"

Fix the bootloader. The modules= parameter loads drivers early, before the blacklist runs. Remove usb-storage, ena, and gve from it. Also add the tuning parameters now so they are in place for the reboot at the end of this guide.

For /boot/extlinux.conf:

sed -i 's/,usb-storage,ext4,ena,gve/,ext4 ipv6.disable=1 audit=0 nowatchdog/' /boot/extlinux.conf

For /etc/update-extlinux.conf (persists the change across kernel updates):

sed -i 's/,usb-storage,ext4,ena,gve/,ext4/' /etc/update-extlinux.conf
sed -i 's/default_kernel_opts="/default_kernel_opts="ipv6.disable=1 audit=0 nowatchdog /' /etc/update-extlinux.conf

The parameters added:

ipv6.disable=1 disables IPv6 at kernel level, removing the associated threads and memory allocations. Skip this if you use IPv6.
audit=0 disables the Linux audit subsystem. It serves no purpose on a VPS, runs a kernel thread, and pre-allocates slab memory.
nowatchdog disables the softlockup and hardlockup detectors.

Rebuild the initramfs:

mkinitfs

Stage 2: Replace OpenSSH with Dropbear

Dropbear is a minimal SSH server designed for low-resource systems. It is significantly smaller than OpenSSH and links against far fewer libraries. On a NAT VPS with a single exposed port, the switch must be done atomically: stop sshd and start dropbear in one command or you will lose access.

apk add dropbear
rc-service sshd stop && rc-service dropbear start
rc-update del sshd default
rc-update add dropbear default
apk del openssh openssh-client-common openssh-client-default openssh-keygen openssh-server openssh-server-common openssh-server-common-openrc openssh-server-pam openssh-sftp-server

Your session might drop. Reconnect on the same port as before.

Stage 3: Remove Cloud-Init and Python

Cloud-init runs once at first boot to configure the instance from the provider metadata. After that it does nothing. It pulls in Python 3 and a large set of dependencies. Since it has already run and the instance is configured, all of it can be removed, if you want to keep python, remove the '|py3-|python3|pyc' part in the command.

apk del $(grep "^P:" /lib/apk/db/installed | sed 's/^P://' | grep -E "^(cloud-init|cloud-utils|py3-|python3|pyc)")

Stage 4: Package Cleanup

Remove packages that serve no purpose on a running VPS. This covers NTP replacement, redundant shell and user management tools, hardware management utilities for hardware that does not exist, and network drivers for other cloud platforms.

Replace chrony with busybox ntpd. Chrony is a full-featured NTP implementation. Busybox includes a lightweight ntpd applet that requires no additional package:

rc-service chronyd stop
rc-update del chronyd default
rc-update add ntpd default
apk del chrony chrony-openrc

NTP is optional on KVM. The guest clock is disciplined by the hypervisor via kvm-clock. On TierHive and similar platforms where the end user has no control over VM suspension or migration, the hypervisor keeps the clock accurate and ntpd adds no value. To skip NTP entirely, do not add the ntpd service.

Remove packages with no runtime use:

apk del bash sudo doas nvme-cli syslinux mtools numactl curl e2fsprogs-extra partx qemu-guest-agent qemu-guest-agent-openrc

qemu-guest-agent enables live snapshots and guest introspection from the hypervisor. If your hosting platform uses QEMU guest operations, keep it.

Remove orphaned libraries left behind by the packages above. Some will be retained by apk because the kernel package depends on them, which is expected:

apk del readline gdbm mpdecimal sqlite-libs yaml p11-kit libtasn1 gnutls nettle gmp libidn2 libunistring libexpat libedit libffi shadow tzdata libseccomp libncursesw libpanelw ncurses-terminfo-base

Remove dhcpcd. Once a VPS has a static IP assigned at deployment, the DHCP client is not needed:

apk del dhcpcd dhcpcd-openrc

Clear the package cache:

rm -rf /var/cache/apk/*

Stage 5: Service Cleanup

Disable services that have nothing to do on a KVM VPS:

rc-update del acpid boot
rc-update del hwclock boot
rc-update del swap boot

acpid handles ACPI events such as power button presses. The hypervisor manages power state on a VPS, not the guest.
hwclock syncs the hardware clock at boot and shutdown. On KVM the RTC is virtualised and managed by the hypervisor.
swap checks for and activates swap devices. There is no swap.

Stage 6: System Tuning

Fix IPv6 sysctl errors

With ipv6.disable=1 set, the kernel no longer has IPv6 sysctl keys. The default Alpine sysctl file tries to set them anyway and produces errors at boot. Comment them out:

sed -i '/net\.ipv6/s/^/# /' /usr/lib/sysctl.d/00-alpine.conf

Prevent debugfs and tracefs from mounting

These kernel debug filesystems expose internal state and are not needed on a production VPS. Note that the memory for the tracing framework is allocated at kernel initialisation regardless; this only stops the filesystems from being accessible:

sed -i 's/mount -n -t debugfs/: #mount -n -t debugfs/' /etc/init.d/sysfs
sed -i 's/mount -n -t tracefs/: #mount -n -t tracefs/' /etc/init.d/sysfs

Sysctl tuning

The default network socket buffers are sized for servers under heavy load, not minimal VPS instances. Reduce them along with a few other settings:

cat > /etc/sysctl.d/10-minvps.conf << 'EOF'
# Reduce network socket buffers
net.core.rmem_default = 32768
net.core.wmem_default = 32768
net.core.rmem_max = 131072
net.core.wmem_max = 131072
net.core.netdev_max_backlog = 64
net.core.somaxconn = 128

# Reclaim inode and dentry caches more aggressively under memory pressure
vm.vfs_cache_pressure = 500

# Reduce PID table overhead
kernel.pid_max = 4096

# Dirty page writeback thresholds
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10

# Disable watchdog
kernel.watchdog = 0
EOF

Switch syslogd to in-memory circular buffer

By default syslogd writes to /var/log/messages on disk. Switching to a circular buffer stores logs in memory instead. They remain accessible via logread. This removes the ongoing disk writes and the associated page cache overhead:

sed -i 's/SYSLOGD_OPTS="-t"/SYSLOGD_OPTS="-t -C64"/' /etc/conf.d/syslog

Reduce block device read-ahead

The kernel defaults to 8MB of read-ahead on the block device. On virtual storage this is wasted memory. 128KB is more than sufficient:

echo 128 > /sys/block/vda/queue/read_ahead_kb

cat > /etc/local.d/readahead.start << 'EOF'
#!/bin/sh
echo 128 > /sys/block/vda/queue/read_ahead_kb
EOF

chmod +x /etc/local.d/readahead.start
rc-update add local default

Reboot

reboot

After

              total        used        free      shared  buff/cache   available
Mem:             91          23          51           0          17          63
Swap:             0           0           0

Filesystem      Size      Used Available Use% Mounted on
/dev/vda       953.0M     71.9M    834.1M   8% /

RAM down from 38MB to 23MB. Disk down from 164.4MB to 71.9MB.

What Is Still Running

The only userspace processes after boot are syslogd (circular buffer mode), dropbear, and two getty processes: one on ttyS0 for serial console access and one on tty1 for the browser-based console panel.

The loaded kernel modules are exactly what the system requires: the virtio stack (virtio-blk, virtio-net, virtio-rng), the ext4 filesystem stack (ext4, jbd2, mbcache, crc16), hardware AES acceleration (aesni-intel, ghash-clmulni-intel, gf128mul), rng-core, net-failover, failover, and af-packet for raw socket support.

Notes for tinkerers:

Two kernel threads will appear in ps aux that look surprising: [scsi_eh_0] and [scsi_eh_1]. These are SCSI error handler threads compiled into the virt kernel. They are dormant and cannot be removed without a custom kernel.

Similarly, [watchdogd] persists despite nowatchdog in the cmdline. There is no /dev/watchdog device present and no watchdog module loaded. The thread does nothing.

I have done this on the VPS that runs https://backtogeek.com if you want to have a look as a performance indication, the whole thing runs on the 128mb, 1gb disk, low priority tier (the $0.10 /month $0.000135 /hour one) its a nothing site, just playing with rust to see how compact i can get a bespoke thingy running with tls support, it uses about 25mb in total to run that site, i will finish it one day and release it (open source) as a hackernews clone/micro blog platform

Anyway... hopefully that injects some low-end spirit into you!

More to come, its not meant to be a tierhive promotion, but it's what I have to work on, and it does not get much more low end than tierhive, this should work on pretty much any kvm host, if any hosts want to provide a VPS for me to test future crap on, I am happy to use your links instead!

version on the tierhive blog: https://tierhive.com/blog/tierhive-howto/how-to-run-alpine-with-just-23mb-ram

Coming soon, similar guide for Debian 13

Radi · 6:59PM

Looks awesome!

Chievo · 7:00PM

Wow another level achieved ! Congrats

ZizzyDizzyMC · 7:10PM

This is genuinely impressive and excellent knowledge even if it's not on your host. Appreciate the effort here.

AnthonySmith · 7:20PM

@ZizzyDizzyMC said:
This is genuinely impressive and excellent knowledge even if it's not on your host. Appreciate the effort here.

Amazing what the price of ram will do haha.