Free LES Community Server from Hosteroid via MetalVPS!

Not_Oles · June 21

The next step is installing the sources, which is covered in the OpenBSD FAQ - Building the System from Source.

OpenBSD and NetBSD use CVS as their source code version management software. OpenBSD and NetBSD both recommend avoding running CVS as root. So we want initially to

set up an unprivileged CVS user.

Then, we need to

set up additional source directories /usr/xenocara (for X11), and /usr/ports (for packages).

Next, we need to select an OpenBSD CVS server from the list at OpenBSD Anonymous CVS.

Finally, we want our unprivileged user to

fetch the OpenBSD-current, xenocara, and ports sources.

I will post a little more about each of these steps as I go through them. Thanks!

cmeerw · June 23

So the update seems to have overwritten /etc/rc.conf - the main thing that I had changed in there was setting httpd_flags= (from httpd_flags=NO - the old file was backed up in /var/backups/etc_rc.conf.backup). I have made that change again now.

@Not_Oles do you happen to remember any warning related to that?

Not_Oles · June 23

@cmeerw As you know, in NetBSD there is etcupdate, which is a specific command designed to adjust upgrades to the files in etc.

But, in OpenBSD, when somebody runs sysupgrade -s it's not clear to me, yet, how etc is protected. Or if etc is protected. In particular, I don't recall seeing a warning about etc/rc.conf, but a warning could have been displayed prominently and I might nevertheless have missed it.

If I understand correctly, you want http_flags= set to nothing (not even set to "") in /etc/rc.conf. So I will check that setting when I do something that seems like it might change /etc/rc.conf.

I will take another look at the sysupgrade man page and documentation to see what I can find about how sysupgrade -s treats /etc. Anything else, please let me know. Apologies for any inconvenience! Thanks!

Not_Oles · June 24

@Not_Oles said: I will take another look at the sysupgrade man page and documentation to see what I can find about how sysupgrade -s treats /etc.

Looking again at the sysupgrade(8) man page, which is available online at https://man.openbsd.org/sysupgrade.8, there seems to be nothing said about what happens to /etc when sysupgrade is used.

The OpenBSD Following -current and using snapshots FAQ also doesn't seem to say anything about /etc.

When I previously used sysupgrade at Hetzner Cloud, there were no issues that I recall.

Nevertheless, it seems obvious that hlcs wasn't in the default state, so I should have looked for any changes @cmeerw made which might have needed attention. My bad!

@cmeerw If there is anything else that I need to fix, I sincerely hope and trust that you would have mentioned it. So, moving onward!

Not_Oles · June 24

@Not_Oles said: we want initially to

set up an unprivileged CVS user.

@cmeerw Okay to name the unprivileged CVS user "ucvsu"?

cmeerw · June 25

@Not_Oles said:

@Not_Oles said: we want initially to

set up an unprivileged CVS user.

@cmeerw Okay to name the unprivileged CVS user "ucvsu"?

Sure.

Not_Oles · June 25

Guys, @cmeerw Does this seem reasonable?

Plan for creating a build user, creating directories, and fetching OpenBSD sources for hlcs
-------------------------------------------------------------------------------------------
June 25, 2025

References:

https://www.openbsd.org/faq/current.html # Scroll down to see note re login class
https://www.openbsd.org/faq/faq5.html#Bld
https://www.openbsd.org/anoncvs.html
man useradd
man login.conf
man id

Commands:

# useradd -G wsrc,wobj -L build ucvsu # _u_nprivileged _cvs_ _u_ser
  # Looks like we already have login class "build":
  hlcs$ grep build /etc/login.conf
  build:\
  pbuild:\
  hlcs$ 
# cd /usr
# mkdir ports xenocara
# chown root:wsrc ports xenocara
# chmod 775 ports xenocara
# su ucvsu
$ cvs -qd [email protected]:/cvs checkout -P src
$ cvs -qd [email protected]:/cvs checkout -P ports
$ cvs -qd [email protected]:/cvs checkout -P xenocara

Notes:

Nearby CVS mirror updated hourly from main server:
hlcs$ ping -c 2 ftp.hostserver.de      
PING ftp.hostserver.de (217.31.80.35): 56 data bytes
64 bytes from 217.31.80.35: icmp_seq=0 ttl=246 time=29.643 ms
64 bytes from 217.31.80.35: icmp_seq=1 ttl=246 time=29.597 ms

--- ftp.hostserver.de ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 29.597/29.620/29.643/0.023 ms
hlcs$ 

Expected ftp.hostserver.de fingerprint:
(ED25519) SHA256:KeG1InAfAnQKqpCewmXw/Egb+4UZZuIMGdlYVG+uxNg
Confirmed the fingerprint:
hlcs$ ssh ftp.hostserver.de
The authenticity of host 'ftp.hostserver.de (217.31.80.35)' can't be established.
ED25519 key fingerprint is: SHA256:KeG1InAfAnQKqpCewmXw/Egb+4UZZuIMGdlYVG+uxNg
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
Host key verification failed.
hlcs$ 

# We're already running clang 19.1.
hlcs$ clang --version
OpenBSD clang version 19.1.7
Target: amd64-unknown-openbsd7.7
Thread model: posix
InstalledDir: /usr/bin
hlcs$ 

# We already have some of the needed directories:
hlcs$ ls -l /usr
total 120
drwxr-xr-x   7 root   wheel   512 Jun 18 15:16 X11R6
drwxr-xr-x   2 root   wheel  6144 Jun 19 15:39 bin
drwxr-xr-x   2 root   wheel  1024 Jun 19 04:12 games
drwxr-xr-x  31 root   bin    3072 Jun 19 04:12 include
drwxr-xr-x   6 root   wheel  3584 Jun 19 15:39 lib
drwxr-xr-x   5 root   wheel   512 Jun 18 14:30 libdata
drwxr-xr-x   7 root   wheel  1024 Jun 19 15:39 libexec
drwxr-xr-x  11 root   wheel   512 Jun 18 15:16 local
drwxr-xr-x   2 root   wheel   512 Jun 18 14:30 mdec
drwxrwx---   2 build  wobj    512 May 29 12:28 obj # <--
drwxr-xr-x   2 root   wheel  4096 Jun 19 15:39 sbin
drwxr-xr-x  18 root   wheel   512 Jun 18 14:30 share
drwxrwxr-x   2 root   wsrc    512 May 29 12:28 src # <--
drwxrwx---   2 build  wobj    512 May 29 12:28 xobj # <--
hlcs$

Not_Oles · June 26

Plan for building OpenBSD-current from source code for hlcs

June 25, 2025

Guys, @cmeerw How about this plan for compiling and installing OpenBSD-current? Assuming that the sources are fetched successfully as described in the previous post. . . . Thanks!

References

From https://www.openbsd.org/anoncvs.html :

After upgrading your source tree, you should read the comments at the top of /usr/src/Makefile before attempting a build. Also, you should build a new kernel before doing a make build if possible. In some cases it may be necessary to rebuild and install the config utility before you can build the kernel. If config GENERIC fails this is probably the case.

From https://www.openbsd.org/faq/faq5.html#wsrc :

Building OpenBSD
At this point you are ready to build OpenBSD from source.
If you are building -current, review changes and special build instructions listed on this page.

Follow the detailed instructions in steps 2 and 3 of release(8).

Further Reading on the Build Process
mk.conf(5)
src/Makefile
/usr/share/mk/bsd.README
config(8)
options(4)

Step 1: Build a new kernel and reboot

From: https://man.openbsd.org/release :

# cd /sys/arch/amd64/compile/GENERIC.MP
# make obj
# make config
# make && make install
# reboot

Step 2: Build and install a new userland and reboot

From: https://man.openbsd.org/release :

# cd /usr/src
# make obj && make build
# sysmerge
# cd /dev && ./MAKEDEV all
# reboot

Thanks to @Hosteroid for our lovely hlcs! Thanks to @cmeerw for great sysadmin help!

Guys, if you would like an account on hlcs, please review the How to Apply section of the first post in this thread!

cmeerw · June 26

@Not_Oles said: # cd /sys/arch/amd64/compile/GENERIC.MP
# make obj
# make config
# make && make install
# reboot

Shouldn't the compile steps ideally be done under an unprivileged user, and then only switch to root for make install and reboot?

Not_Oles · June 26

@cmeerw said: Shouldn't the compile steps ideally be done under an unprivileged user, and then only switch to root for make install and reboot?

Yes! Especially, haha, immediately after we went through making the unprivileged user!

So,

$ make obj
$ make config
$ make
# make install
# reboot

Also, things might go less quickly without the -j option!

Probably I want to build xenocara. We might not want to build all the ports.

Thanks for fixing my mistakes! Much appreciated! More before too long. . . .

Not_Oles · June 27

Hosteroid says,

. . . maintenance at our Bucharest M247 Data Centre on 5th July 2025, starting at 11:00 GMT + 3. . . .

Monitor our live status page for real-time updates and detailed progress: https://www.hosteroid.uk/serverstatus.php

Not_Oles · June 28

Hi!

ucvsu is now present on hlcs! Here are a few highlights.

I forgot to include the -m option in the above plan!

hlcs# useradd -G wsrc,wobj -L build ucvsu
useradd: Warning: home directory `/home/ucvsu' doesn't exist, and -m was not specified
hlcs#

So I copied /etc/skel/.* manually, and changed the owners manually. The result is below.

hlcs# pwd
/home/ucvsu
hlcs# ls -alR  
.:
total 72
drwxr-xr-x  3 ucvsu  ucvsu  512 Jun 27 23:54 .
drwxr-xr-x  6 root   wheel  512 Jun 27 23:46 ..
-rw-r--r--  1 ucvsu  ucvsu   87 Jun 27 23:54 .Xdefaults
-rw-r--r--  1 ucvsu  ucvsu  761 Jun 27 23:54 .cshrc
-rw-r--r--  1 ucvsu  ucvsu  101 Jun 27 23:54 .cvsrc
-rw-r--r--  1 ucvsu  ucvsu  281 Jun 27 23:54 .login
-rw-r--r--  1 ucvsu  ucvsu  175 Jun 27 23:54 .mailrc
-rw-r--r--  1 ucvsu  ucvsu  199 Jun 27 23:54 .profile
drwx------  2 ucvsu  ucvsu  512 Jun 27 23:54 .ssh

./.ssh:
total 24
drwx------  2 ucvsu  ucvsu  512 Jun 27 23:54 .
drwxr-xr-x  3 ucvsu  ucvsu  512 Jun 27 23:54 ..
-rw-------  1 ucvsu  ucvsu  940 Jun 27 23:57 authorized_keys
hlcs#

I checked /etc/passwd and /etc/group:

hlcs# grep ucvsu /etc/passwd                                                                     
ucvsu:*:1002:1002::/home/ucvsu:/bin/ksh
hlcs# grep ucvsu /etc/group   
wsrc:*:9:ucvsu
wobj:*:21:ucvsu
ucvsu:*:1002:
hlcs#

I checked to make sure login works as expected on IPv4 and IPv6:

chronos@penguin:~/servers/hosteroid$ ssh [email protected] 
OpenBSD 7.7-current (GENERIC.MP) #22: Wed Jun 18 08:57:04 MDT 2025

Welcome to OpenBSD: The proactively secure Unix-like operating system.

  [ . . . ]

hlcs$ whoami
ucvsu
hlcs$ exit
Connection to xxx.xxx.xxx.xxx closed.
chronos@penguin:~/servers/hosteroid$

hlcs$ who am i
ucvsu    ttyp0    Jun 28 00:35   (xxxx:xxxx:xxxx::xxxx)
hlcs$

If ucvsu needs anything more, will somebody please speak up!?

@cmeew Your ssh public key is included, so you should be able to login as ucvsu if you wish.

I haven't done any doas configuration. If you want anything, please let me know, or just do it and let me know.

@babywhale You haven't been active, so I commented out your key from /root/.ssh/authorized_keys and /home/ucvsu/.ssh/authorized_keys. But any time you want access restored, please ask, and I will restore your access right away. Thanks for all your help with everything! Especially BashVM!

I made ssh keys for ucvsu, even though making ssh keys also wasn't in the plan! No password for these keys. @cmeerw if you want a password added, please let me know.

Next up is to try CVS!

Thanks to @Hosteroid for our fun, community server! Thanks to @cmeerw for admin skills! Thanks to any of you guys who want to join -- please review the How To Apply section of the OP!

Best!

Tom

somik · June 28

I have no idea whats going on here anymore... and the messages are too long to be worth the time to read. Summarize?

Not_Oles · June 28

@somik

Intel E3 dedi dual booted with Debian and OpenBSD-current. An unprivileged Concurrent Version System (CVS) user presently is being added to facilitate downloading and re-compiling the OpenBSD system from source code.

The server is kindly donated by @Hosteroid. @cmeerw is helping with system administration.

If you want to use the server for something, kindly let us know. It's free for awesome LESbians like you!

somik · June 28

Naa, I'm good. I am playing with the E5 server running proxmox I have at home. Was wondering what's going on since I saw a lot of messages flying around here. I did see messages about openbsd being installed/booted, but did not think it was being dual booted with debian.

Not_Oles · June 28

Made the additional needed directories:

hlcs# cd /usr
hlcs# mkdir -p ports xenocara
hlcs# chgrp wsrc ports/ xenocara/                                                           
hlcs# chmod 775 ports/ xenocara/                                                        
hlcs# ls -l
total 136
drwxr-xr-x   7 root   wheel   512 Jun 18 15:16 X11R6
drwxr-xr-x   2 root   wheel  6144 Jun 19 15:39 bin
drwxr-xr-x   2 root   wheel  1024 Jun 19 04:12 games
drwxr-xr-x  31 root   bin    3072 Jun 19 04:12 include
drwxr-xr-x   6 root   wheel  3584 Jun 19 15:39 lib
drwxr-xr-x   5 root   wheel   512 Jun 18 14:30 libdata
drwxr-xr-x   7 root   wheel  1024 Jun 19 15:39 libexec
drwxr-xr-x  11 root   wheel   512 Jun 18 15:16 local
drwxr-xr-x   2 root   wheel   512 Jun 18 14:30 mdec
drwxrwx---   2 build  wobj    512 May 29 12:28 obj
drwxrwxr-x   2 root   wsrc    512 Jun 28 15:37 ports
drwxr-xr-x   2 root   wheel  4096 Jun 19 15:39 sbin
drwxr-xr-x  18 root   wheel   512 Jun 18 14:30 share
drwxrwxr-x   2 root   wsrc    512 May 29 12:28 src
drwxrwxr-x   2 root   wsrc    512 Jun 28 15:37 xenocara
drwxrwx---   2 build  wobj    512 May 29 12:28 xobj
hlcs#

Not_Oles · June 28

Maybe the sources now are present on hlcs!

To me, it's really nice to have the sources immediately at hand, just a cd away! Thanks to the OpenBSD developers!

hlcs$ cvs -qd [email protected]:/cvs checkout -P src 
  [ . . . ] # The fun is in here, but there is too much to post! :)
hlcs$ ls src/                                                
CVS            bin            games          lib            sbin           usr.bin
Makefile       distrib        gnu            libexec        share          usr.sbin
Makefile.cross etc            include        regress        sys
hlcs$ cvs -qd [email protected]:/cvs checkout -P xenocara
  [. . . ]
hlcs$ ls xenocara/                                                                      
3RDPARTY  MODULES   README.md data      distrib   driver    font      proto     util
CVS       Makefile  app       dist      doc       etc       lib       share     xserver
hlcs$ cvs -qd [email protected]:/cvs checkout -P ports    
  [ . . . ]
hlcs$ ls ports
CVS            books          education      japanese       multimedia     sysutils
Makefile       cad            emulators      java           net            telephony
README         chinese        fonts          korean         news           tests
archivers      comms          games          lang           plan9          textproc
astro          converters     geo            mail           print          wayland
audio          databases      graphics       math           productivity   www
benchmarks     devel          infrastructure meta           security       x11
biology        editors        inputmethods   misc           shells
hlcs$

Thanks to @Hosteroid for the nice community server! Thanks to @cmeerw for sysadmin help!

Anybody want to hop on? Try OpenBSD's web server? Add AI? Something else? Something more? Something MOARRRRRR? Please review the How To Apply section of the OP. Thanks!

Next up is compiling! Unless I still am missing something?

Not_Oles · June 28

Compiling our new OpenBSD-current kernel!

Checking for updates.

hlcs$ cd /usr/src
hlcs$ cvs -q up -Pd -A
  [ . . . ]
hlcs$

Compiling the new kernel first, as recommended in https://man.openbsd.org/release.8.

hlcs$ cd sys/arch/amd64/compile/GENERIC.MP/
hlcs$ make obj
/usr/src/sys/arch/amd64/compile/GENERIC.MP/obj -> /usr/obj/sys/arch/amd64/compile/GENERIC.MP
hlcs$ make config
config  -b /usr/src/sys/arch/amd64/compile/GENERIC.MP/obj  -s /usr/src/sys /usr/src/sys/arch/amd64/conf/GENERIC.MP
hlcs$ make -j 4
  [ . . . ]
ld -T ld.script -X --warn-common -nopie -o bsd ${SYSTEM_HEAD} vers.o ${OBJS}
text    data    bss     dec     hex
27006746        495936  1343488 28846170        1b8285a
mv bsd bsd.gdb
ctfstrip -S -o bsd bsd.gdb
hlcs$ echo $?
0
hlcs$

Installing and rebooting.

hlcs# cd /usr/src/sys/arch/amd64/compile/GENERIC.MP
hlcs# make install
mkdir -p -m 700 /usr/share/relink/kernel
rm -rf /usr/share/relink/kernel/GENERIC.MP /usr/share/relink/kernel.tgz
mkdir /usr/share/relink/kernel/GENERIC.MP
tar -chf - Makefile makegap.sh ld.script *.o |  tar -C /usr/share/relink/kernel/GENERIC.MP -xf -
[[ ! -f /bsd ]] || cmp -s bsd /bsd || ln -f /bsd /obsd
[[ ! -f /etc/bsd.re-config ]] || config -e -c /etc/bsd.re-config -f bsd
install -F -m 700 bsd /bsd && sha256 -h /var/db/kernel.SHA256 /bsd
hlcs# echo $?
0
hlcs# date; ls -l /*bsd
Sat Jun 28 22:35:53 UTC 2025
-rwx------  1 root  wheel  32179683 Jun 28 22:34 /bsd
-rwx------  1 root  wheel  32141799 Jun 19 15:39 /obsd
hlcs# reboot

Here we are, up again.

hlcs$ date; uptime; uname -a
Sat Jun 28 22:40:19 UTC 2025
10:40PM  up 2 mins, 1 user, load averages: 0.15, 0.08, 0.03
OpenBSD hlcs.my.domain 7.7 GENERIC.MP#0 amd64
hlcs$

Thanks @Hosteroid! Thanks @cmeerw! Thanks OpenBSD! Thanks LES!

Next is compiling and installing our new userland, sysmerge, makedev, and rebooting. . . .

Not_Oles · June 29

Compiling our new OpenBSD userland!

hlcs$ who am i
ucvsu    ttyp0    Jun 28 22:39   (xxx.xxx.xxx.xxx)
hlcs$ cd /usr/src
hlcs$ make obj
  [ . . . ]
hlcs$ make -j 4 build
umask 007; exec make do-build
do-build must be called by root
*** Error 1 in . (Makefile:84 'do-build': @if [[ `id -u` -ne 0 ]]; then  echo do-build must be called by root >&2;  false;  fi)
*** Error 2 in /usr/src (Makefile:74 'build')
hlcs$

hlcs# make -j 4 build 
./llvm/llvm/lib/Target/AMDGPU/R600.td
/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/obj/../../../llvm-tblgen/llvm-tblgen -gen-callingconv  -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/include -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU  -o R600GenCallingConv.inc /usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU/R600.td
*** Signal 6 in target 'AMDGPUGenAsmMatcher.inc'
LLVM ERROR: out of memory
Allocation failed
*** Signal SIGABRT in gnu/usr.bin/clang/include/llvm/AMDGPU (Makefile:100 'AMDGPUGenGlobalISel.inc')
*** Signal SIGABRT (Makefile:51 'AMDGPUGenAsmMatcher.inc')
*** Error 2 in gnu/usr.bin/clang (<bsd.subdir.mk>:48 'all')
*** Error 2 in gnu/usr.bin (<bsd.subdir.mk>:48 'all')
*** Error 2 in gnu (<bsd.subdir.mk>:48 'all')
*** Error 2 in . (<bsd.subdir.mk>:48 'all')
*** Error 2 in . (Makefile:97 'do-build')
*** Error 2 in /usr/src (Makefile:74 'build')
hlcs#

I wonder how much memory is needed?

Could it be that simply restarting the compile (keeping -j 4) might work the second time around? (BTW, I know we have 8 threads. I just used 4 not to be greedy, so @cmeerw would have compute power if needed.)

What I did was restart the compile single threaded (make without the -j option), and the compile proceeded.

I will post again as the compile proceeds or when it completes. Thanks!

Not_Oles · June 29

Okay, similar error:

===> gnu/usr.bin/clang/include/llvm/AMDGPU                                 
/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/obj/../../../llvm-tblgen/llvm-tblgen -gen-
asm-matcher  -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/include -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU  -o AMDGPUGenAsmMatcher.inc /usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU/AMDGPU.td
LLVM ERROR: out of memory
Allocation failed
*** Signal SIGABRT in gnu/usr.bin/clang/include/llvm/AMDGPU (Makefile:51 'AMDGPUGenAsmMatcher.inc')
*** Error 2 in gnu/usr.bin/clang (<bsd.subdir.mk>:48 'all': @for entry in include/llvm/Config libLLVMSupport libLLVMTableGen llvm-min-tblgen...)
*** Error 2 in gnu/usr.bin (<bsd.subdir.mk>:48 'all': @for entry in clang cvs binutils binutils-2.17 perl texinfo; do  set -e; if test -d /u...)
*** Error 2 in gnu (<bsd.subdir.mk>:48 'all': @for entry in lib usr.bin usr.sbin; do  set -e; if test -d /usr/src/gnu/${entry}.amd64; then  ...)
*** Error 2 in . (<bsd.subdir.mk>:48 'all': @for entry in lib include bin libexec sbin usr.bin usr.sbin share games gnu sys; do  set -e; if ...)
*** Error 2 in . (Makefile:97 'do-build')
*** Error 2 in /usr/src (Makefile:74 'build')
hlcs#

More soon. It's getting late here, so sleep for now. @cmeerw If you want to play with this, maybe look inside the tmux session that root has running. If not, that's okay too. I guess it's not impossible that there might be a new commit which fixes this? I haven't checked yet. I remember you making some per process limit changes. I have to look at that again. Of course, all ideas always appreciated. Thanks!

cmeerw · June 29

@Not_Oles said: LLVM ERROR: out of memory

Could be another case where the ulimit needs to be adjusted?

I have done ulimit -d 25165824 in that tmux session and done another make build

BTW, to get all 8 threads, sysctl hw.smt=1 is needed (probably need to add that to /etc/sysctl.conf...)

Not_Oles · June 29

Hi @cmeew! Thanks for your help! I woke up to:

===> gnu/usr.bin/clang/include/llvm/AMDGPU
/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/obj/../../../llvm-tblgen/llvm-tblgen -gen-asm-matcher  -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/include -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU  -o AMDGPUGenAsmMatcher.inc /usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU/AMDGPU.td
LLVM ERROR: out of memory
Allocation failed
*** Signal SIGABRT in gnu/usr.bin/clang/include/llvm/AMDGPU (Makefile:51 'AMDGPUGenAsmMatcher.inc')
*** Error 2 in gnu/usr.bin/clang (<bsd.subdir.mk>:48 'all': @for entry in include/llvm/Config libLLVMSupport libLLVMTableGen llvm-min-tblgen...)
*** Error 2 in gnu/usr.bin (<bsd.subdir.mk>:48 'all': @for entry in clang cvs binutils binutils-2.17 perl texinfo; do  set -e; if test -d /u...)
*** Error 2 in gnu (<bsd.subdir.mk>:48 'all': @for entry in lib usr.bin usr.sbin; do  set -e; if test -d /usr/src/gnu/${entry}.amd64; then  ...)
*** Error 2 in . (<bsd.subdir.mk>:48 'all': @for entry in lib include bin libexec sbin usr.bin usr.sbin share games gnu sys; do  set -e; if ...)
*** Error 2 in . (Makefile:97 'do-build')
*** Error 2 in /usr/src (Makefile:74 'build')
hlcs#

@cmeerw said: I have done ulimit -d 25165824 in that tmux session and done another make build

Checking the entry for ulimit in the ksh man page, it says, "limits may not be increased once they are set."

The man page also says,

-d n
Impose a size limit of n kilobytes on the size of the data area.

Is 25165824 kilobytes not enough? What about exiting tmux and starting another tmux session to get around the shell's inability to increase limits, and then trying ulimit -d 32425444 or maybe ulimit -d unlimited?

For convenient reference, here are all the soft and hard limits from inside the current tmux session:

hlcs# ulimit -a
time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         25165824
stack(kbytes)        8192
lockedmem(kbytes)    87381
memory(kbytes)       32425444
nofiles(descriptors) 128
processes            1310
hlcs# ulimit -aH
time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         25165824
stack(kbytes)        32768
lockedmem(kbytes)    262144
memory(kbytes)       32430652
nofiles(descriptors) 1024
processes            1310
hlcs#

And from outside the tmux session:

hlcs# ulimit -a
time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         4194304
stack(kbytes)        8192
lockedmem(kbytes)    87381
memory(kbytes)       32425444
nofiles(descriptors) 128
processes            1310
hlcs# ulimit -aH
time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         4194304
stack(kbytes)        32768
lockedmem(kbytes)    262144
memory(kbytes)       32430652
nofiles(descriptors) 1024
processes            1310
hlcs#

Physical hardware limit (bytes):

hlcs# sysctl hw.physmem
hw.physmem=34292264960
hlcs#

Thanks!

cmeerw · June 29

Just tried running that command in a normal shell, and that seems to work. BTW, /usr/bin/time -l reports

       31.70 real        29.59 user         2.05 sys
   1519720  maximum resident set size
      3500  average shared memory size
   1166415  average unshared data size
       354  average unshared stack size
    416053  minor page faults
         1  major page faults
         0  swaps
         0  block input operations
      1349  block output operations
         0  messages sent
         0  messages received
         0  signals received
        18  voluntary context switches
       188  involuntary context switches

So that's well below the original 4 GB limit.

Not_Oles · June 29

@cmeew FWIW, the build doesn't seem to work for me with 4 threads:

hlcs# date
Sun Jun 29 19:56:07 UTC 2025
hlcs# cd /usr/src
hlcs# make -j 4 build
  [ . . . ]
Target/AMDGPU  -o AMDGPUGenPreLegalizeGICombiner.inc /usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU/AMDGPUGISel.td
LLVM ERROR: out of memory
Allocation failed
*** Signal 6 in target 'AMDGPUGenAsmMatcher.inc'
*** Signal SIGABRT in gnu/usr.bin/clang/include/llvm/AMDGPU (Makefile:51 'AMDGPUGenAsmMatcher.inc')
*** Error 2 in gnu/usr.bin/clang (<bsd.subdir.mk>:48 'all')
*** Error 2 in gnu/usr.bin (<bsd.subdir.mk>:48 'all')
*** Error 2 in gnu (<bsd.subdir.mk>:48 'all')
*** Error 2 in . (<bsd.subdir.mk>:48 'all')
*** Error 2 in . (Makefile:97 'do-build')
*** Error 2 in /usr/src (Makefile:74 'build')
hlcs#

Or with one thread (and /usr/bin/time -l):

hlcs# /usr/bin/time -l make build
  [ . . . ]
===> gnu/usr.bin/clang/include/llvm/AMDGPU
/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/obj/../../../llvm-tblgen/llvm-tblgen -gen-asm-matcher  -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/include -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU  -o AMDGPUGenAsmMatcher.inc /usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU/AMDGPU.td
LLVM ERROR: out of memory
Allocation failed
*** Signal SIGABRT in gnu/usr.bin/clang/include/llvm/AMDGPU (Makefile:51 'AMDGPUGenAsmMatcher.inc')
*** Error 2 in gnu/usr.bin/clang (<bsd.subdir.mk>:48 'all': @for entry in include/llvm/Config libLLVMSupport libLLVMTableGen llvm-min-tblgen...)
*** Error 2 in gnu/usr.bin (<bsd.subdir.mk>:48 'all': @for entry in clang cvs binutils binutils-2.17 perl texinfo; do  set -e; if test -d /u...)
*** Error 2 in gnu (<bsd.subdir.mk>:48 'all': @for entry in lib usr.bin usr.sbin; do  set -e; if test -d /usr/src/gnu/${entry}.amd64; then  ...)
*** Error 2 in . (<bsd.subdir.mk>:48 'all': @for entry in lib include bin libexec sbin usr.bin usr.sbin share games gnu sys; do  set -e; if ...)
*** Error 2 in . (Makefile:97 'do-build')
*** Error 2 in /usr/src (Makefile:74 'build')
     2451.30 real      1663.10 user       540.88 sys
   1506160  maximum resident set size
     64664  average shared memory size
     38558  average unshared data size
       112  average unshared stack size
 159380599  minor page faults
    316406  major page faults
         0  swaps
      8590  block input operations
    446758  block output operations
         0  messages sent
        20  messages received
     78391  signals received
   1382205  voluntary context switches
      7150  involuntary context switches
hlcs#

What am I missing? Thanks!

Not_Oles · June 30

Trying, without tmux,

hlcs# date
Mon Jun 30 00:46:13 UTC 2025
hlcs# w
12:46AM  up 1 day,  2:08, 1 user, load averages: 0.05, 0.03, 0.00
USER    TTY FROM              LOGIN@  IDLE WHAT
root     p0 xxx.xxx.xxx.xxx  12:46AM     0 w 
hlcs# cd /usr/src
hlcs# ulimit -d 25165824
hlcs# /usr/bin/time -l make build
  [ . . . ]

I will post again to let you know what happens. . . . Thanks!

Not_Oles · June 30

Nope!

===> gnu/usr.bin/clang/include/llvm/AMDGPU
/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/obj/../../../llvm-tblgen/llvm-tblgen -gen-asm-matcher  -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/include -I/usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU  -o AMDGPUGenAsmMatcher.inc /usr/src/gnu/usr.bin/clang/include/llvm/AMDGPU/../../../../../llvm/llvm/lib/Target/AMDGPU/AMDGPU.td
LLVM ERROR: out of memory
Allocation failed
*** Signal SIGABRT in gnu/usr.bin/clang/include/llvm/AMDGPU (Makefile:51 'AMDGPUGenAsmMatcher.inc')
*** Error 2 in gnu/usr.bin/clang (<bsd.subdir.mk>:48 'all': @for entry in include/llvm/Config libLLVMSupport libLLVMTableGen llvm-min-tblgen...)
*** Error 2 in gnu/usr.bin (<bsd.subdir.mk>:48 'all': @for entry in clang cvs binutils binutils-2.17 perl texinfo; do  set -e; if test -d /u...)
*** Error 2 in gnu (<bsd.subdir.mk>:48 'all': @for entry in lib usr.bin usr.sbin; do  set -e; if test -d /usr/src/gnu/${entry}.amd64; then  ...)
*** Error 2 in . (<bsd.subdir.mk>:48 'all': @for entry in lib include bin libexec sbin usr.bin usr.sbin share games gnu sys; do  set -e; if ...)
*** Error 2 in . (Makefile:97 'do-build')
*** Error 2 in /usr/src (Makefile:74 'build')
     2451.85 real      1663.40 user       540.36 sys
   1506228  maximum resident set size
     64701  average shared memory size
     38481  average unshared data size
       112  average unshared stack size
 159430126  minor page faults
    316512  major page faults
         0  swaps
      8590  block input operations
    446750  block output operations
         0  messages sent
        20  messages received
     78392  signals received
   1377363  voluntary context switches
      7186  involuntary context switches
hlcs#

cmeerw · June 30

@Not_Oles said: What am I missing? Thanks!

Sorry, I meant that just running that llvm-tblgen from the command line worked (but it doesn't seem to work when being run using make) - and I have no idea either why we are seeing that difference.

cmeerw · June 30

@cmeerw said:

@Not_Oles said: What am I missing? Thanks!

Sorry, I meant that just running that llvm-tblgen from the command line worked (but it doesn't seem to work when being run using make) - and I have no idea either why we are seeing that difference.

Just having a look at what make build actually does:

hlcs# make -n build
umask 007; exec make do-build
hlcs# make -n do-build
if [[ `id -u` -ne 0 ]]; then  echo do-build must be called by root >&2;  false;  fi
cd /usr/src/share/mk && exec make install
exec make cleandir
exec make includes
cd /usr/src/lib &&  su build -c 'exec make' &&  NOMAN=1 exec make install
/sbin/ldconfig -R
cd /usr/src/gnu/lib &&  su build -c 'exec make' &&  NOMAN=1 exec make install
/sbin/ldconfig -R
su build -c 'exec make' &&  exec make install
/bin/sh /usr/src/distrib/sets/makeetcset /usr/src make

So it's not actually using the ulimit set in the root shell, but the one it gets for the build user - and that user has the default datasize-max=1536M and datasize-cur=1536M.

There is an entry for build in /etc/login.conf:

#
# Building LLVM in base requires higher limits
#
build:\
        :datasize-max=1843M:\
        :datasize-cur=1843M:\
        :tc=default:

But the build user doesn't use that login class:

build:*:21:21::0:0:base and xenocara build:/var/empty:/bin/ksh

Not_Oles · June 30

@cmeerw Okay, that's all right! Thanks for taking a little dive into OpenBSD with me!

Do you want to stick with the present OpenBSD / Debian dual boot install?

Might there be something else that you want to run on hlcs?

Any other LESbians want to try? It's free! Please review "How To Apply" in the OP.

Thanks to @Hosteroid for our fun Community server!

cmeerw · June 30

@cmeerw said: But the build user doesn't use that login class:

I have set the login class for the build user to build now and restarted a make build

@Not_Oles said: Do you want to stick with the present OpenBSD / Debian dual boot install?

At least we should get the OpenBSD build done before switching to anything else.

Free LES Community Server from Hosteroid via MetalVPS!

Comments

Plan for building OpenBSD-current from source code for hlcs