Hetzner EX series (Intel) and virtualization

hostaspacehostaspace
in Help

As EX-44 come with Intel Core i5-13500 which have 6 P-core and 8 E-core the price/performance is very lucrative to opt-in even with ECC addon added.

Do anyone uses proxmox on this CPU???, can proxmox handle both P & E cores also what about iGPU, does it work properly with passthrough?

EX44 Hardware data:

   CPU1: 13th Gen Intel(R) Core(TM) i5-13500 (Cores 20)
   Memory:  64125 MB
   Disk /dev/nvme0n1: 512 GB (=> 476 GiB) doesn't contain a valid partition tabl                                                                                        e
   Disk /dev/nvme1n1: 512 GB (=> 476 GiB) doesn't contain a valid partition tabl                                                                                        e
   Total capacity 953 GiB with 2 Disks

Network data:
   eth0  LINK: yes
         MAC:  c8::78
         IP:   
         IPv6: 2a01:
         RealTek RTL-8169 Gigabit Ethernet driver

lspci

00:00.0 Host bridge: Intel Corporation Device 4640 (rev 02)
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 02)
00:02.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:06.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #0 (rev 02)
00:0a.0 Signal processing controller: Intel Corporation Platform Monitoring Technology (rev 01)
00:14.0 USB controller: Intel Corporation Device 7a60 (rev 11)
00:14.2 RAM memory: Intel Corporation Device 7a27 (rev 11)
00:15.0 Serial bus controller: Intel Corporation Device 7a4c (rev 11)
00:16.0 Communication controller: Intel Corporation Device 7a68 (rev 11)
00:17.0 SATA controller: Intel Corporation Device 7a62 (rev 11)
00:1a.0 PCI bridge: Intel Corporation Device 7a48 (rev 11)
00:1c.0 PCI bridge: Intel Corporation Device 7a38 (rev 11)
00:1c.2 PCI bridge: Intel Corporation Device 7a3a (rev 11)
00:1d.0 PCI bridge: Intel Corporation Device 7a36 (rev 11)
00:1f.0 ISA bridge: Intel Corporation Device 7a06 (rev 11)
00:1f.4 SMBus: Intel Corporation Device 7a23 (rev 11)
00:1f.5 Serial bus controller: Intel Corporation Device 7a24 (rev 11)
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO
02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)

lscpu -e

CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ MHZ
0 0 0 0 0:0:0:0 yes 4800.0000 800.0000 789.2880
1 0 0 0 0:0:0:0 yes 4800.0000 800.0000 2500.0000
2 0 0 1 4:4:1:0 yes 4800.0000 800.0000 745.1880
3 0 0 1 4:4:1:0 yes 4800.0000 800.0000 2500.0000
4 0 0 2 8:8:2:0 yes 4800.0000 800.0000 800.2580
5 0 0 2 8:8:2:0 yes 4800.0000 800.0000 2500.0000
6 0 0 3 12:12:3:0 yes 4800.0000 800.0000 800.0240
7 0 0 3 12:12:3:0 yes 4800.0000 800.0000 2500.0000
8 0 0 4 16:16:4:0 yes 4800.0000 800.0000 800.1250
9 0 0 4 16:16:4:0 yes 4800.0000 800.0000 2500.0000
10 0 0 5 20:20:5:0 yes 4800.0000 800.0000 801.2800
11 0 0 5 20:20:5:0 yes 4800.0000 800.0000 2500.0000
12 0 0 6 24:24:6:0 yes 3500.0000 800.0000 801.4310
13 0 0 7 25:25:6:0 yes 3500.0000 800.0000 805.4110
14 0 0 8 26:26:6:0 yes 3500.0000 800.0000 799.0580
15 0 0 9 27:27:6:0 yes 3500.0000 800.0000 800.6880
16 0 0 10 28:28:7:0 yes 3500.0000 800.0000 806.7300
17 0 0 11 29:29:7:0 yes 3500.0000 800.0000 2500.0000
18 0 0 12 30:30:7:0 yes 3500.0000 800.0000 2500.0000
19 0 0 13 31:31:7:0 yes 3500.0000 800.0000 2500.0000

What about performance in compare with AMD Ryzen 5 3600 come with AX-41 and cost cheaper.

Tagged:

Comments

  • CyberneticTitanCyberneticTitan OG

    iGPU should work via SR-IOV but it's kind of a recent thing: https://www.michaelstinkerings.org/gpu-virtualization-with-intel-12th-gen-igpu-uhd-730/

    Before 11th gen, there was GVT-g.

    Thanked by (1)hostaspace
  • hostaspacehostaspace

    Yea.. may be @somik @yoursunny or God himself throw some 🕯️.

  • somiksomik OG

    @hostaspace said:
    Yea.. may be @somik

    Who dares summon me!!!

    :lol:

    Anyway, Proxmox works fine on intel 12th and 13th gen CPUs, as long as you use a kernel version of 6.2 or higher. I have been told that kernel 6.1 onward supports P and E core CPUs, but since you need to update, might as well update to 6.2 or higher... Proxmox comes with Kernel 5.15 (i think) so if you do not know how to build/install PVE kernel 6.2, skip proxmox.

    Although there seems to be no issues with P and E cores, igpu MAY have issues.

    Best way to avoid these issues is to install debian on the server, followed by proxmox installation on debian, instead of installing proxmox from ISO.

    @Not_Oles was running VMs on Intel 13900K so if you are able to run VM, proxmox can manage it as well.

    Thanked by (4)FrankZ Khalequzzaman hostaspace hornet

    Artificial intelligence is no match for our natural stupidity.

    Time flies like an arrow; fruit flies like a banana.

  • Not_OlesNot_Oles Hosting ProviderContent Writer
    edited June 2023

    Hi @hostaspace and @somik,

    To control Performance and Efficiency Cores on 13th Gen as in EX44 and EX101 from Hetzner, Intel has "Thread Director." There is a description of Thread Director in How 13th Gen Intel® Core™ Processors Work.

    Intel submitted Linux kernel patches for "Thread Director." The initial set of Thread Director patches might be here.

    The integrated graphics is a separate question from the fast and slow cores. My EX101 seems to have been received from Hetzner without integrated graphics enabled. I'm unsure whether disabled iGPU is a motherboard default or whether Hetzner turns off iGPU, but I have just started reading the motherboard manual in preparation for trying to enable the graphics and maybe do some OS installs. I don't know if the EX44 uses the same motherboard, but my EX101 has an ASRock Rack W680D4U. This motherboard's instructions may be found at https://www.asrockrack.com/general/productdetail.asp?Model=W680D4U#Specifications.

    Passthrough is something I haven't yet looked into very much. VMs seem to run fine under Debian sid, but I haven't yet tried to enable passthrough. Here is some iGPU info from my EX101 which is running Debian sid, not Proxmox:

    root@fsn ~ # ls -la /dev/dri
ls: cannot access '/dev/dri': No such file or directory
root@fsn ~ # lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] (rev 04)
root@fsn ~ # dmesg | grep i915
[    2.345121] i915 0000:00:02.0: [drm] VT-d active for gfx access
[    2.345249] i915 0000:00:02.0: vgaarb: deactivate vga console
[    2.345288] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[    2.346138] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    2.346304] i915 0000:00:02.0: firmware: failed to load i915/adls_dmc_ver2_01.bin (-2)
[    2.346310] i915 0000:00:02.0: firmware: failed to load i915/adls_dmc_ver2_01.bin (-2)
[    2.346312] i915 0000:00:02.0: Direct firmware load for i915/adls_dmc_ver2_01.bin failed with error -2
[    2.346314] i915 0000:00:02.0: [drm] Failed to load DMC firmware i915/adls_dmc_ver2_01.bin. Disabling runtime power management.
[    2.346316] i915 0000:00:02.0: [drm] DMC firmware homepage: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915
[    3.014977] i915 0000:00:02.0: [drm] failed to retrieve link info, disabling eDP
[    3.015142] i915 0000:00:02.0: firmware: failed to load i915/tgl_guc_70.bin (-2)
[    3.015146] i915 0000:00:02.0: firmware: failed to load i915/tgl_guc_70.bin (-2)
[    3.015150] i915 0000:00:02.0: firmware: failed to load i915/tgl_guc_70.1.1.bin (-2)
[    3.015153] i915 0000:00:02.0: firmware: failed to load i915/tgl_guc_70.1.1.bin (-2)
[    3.015157] i915 0000:00:02.0: firmware: failed to load i915/tgl_guc_69.0.3.bin (-2)
[    3.015160] i915 0000:00:02.0: firmware: failed to load i915/tgl_guc_69.0.3.bin (-2)
[    3.015161] i915 0000:00:02.0: GuC firmware i915/tgl_guc_70.bin: fetch failed with error -2
[    3.015163] i915 0000:00:02.0: Please file a bug on drm/i915; see https://gitlab.freedesktop.org/drm/intel/-/wikis/How-to-file-i915-bugs for details.
[    3.015165] i915 0000:00:02.0: [drm] GuC firmware(s) can be downloaded from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915
[    3.090223] i915 0000:00:02.0: [drm] GuC firmware i915/tgl_guc_70.bin version 0.0.0
[    3.090298] i915 0000:00:02.0: GuC initialization failed -2
[    3.090300] i915 0000:00:02.0: Enabling uc failed (-5)
[    3.090301] i915 0000:00:02.0: Failed to initialize GPU, declaring it wedged!
[    3.090660] i915 0000:00:02.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_init+0xb3/0x2f0 [i915]
[    3.378817] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
[    3.379774] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[    3.380092] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[    3.380259] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[    3.380343] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
root@fsn ~ #

    Proxmox uses an Ubuntu kernel and a Debian userland. Last time I checked, Proxmox had a precompiled, newer Ubuntu kernel version available in Proxmox' Github repo, so maybe you could use that if they're still making the new version available. Or, of course, you can compile your own kernel.

    Hetzner has a really cool, fast "Proxmox on top of Debian" install via Hetzner Installimage. Here is a tutorial about Proxmox via Installimage.

    As I'm sure you both know, there is a new Debian release just out and also a new Proxmox version to go with the new Debian.

    Good luck!

    Tom

    Thanked by (2)FrankZ hostaspace

    I hope everyone gets the servers they want!

  • rubenmdhrubenmdh OG

    @somik said:

    @hostaspace said:
    Yea.. may be @somik

    Who dares summon me!!!

    :lol:

    Anyway, Proxmox works fine on intel 12th and 13th gen CPUs, as long as you use a kernel version of 6.2 or higher. I have been told that kernel 6.1 onward supports P and E core CPUs, but since you need to update, might as well update to 6.2 or higher... Proxmox comes with Kernel 5.15 (i think) so if you do not know how to build/install PVE kernel 6.2, skip proxmox.

    Although there seems to be no issues with P and E cores, igpu MAY have issues.

    Best way to avoid these issues is to install debian on the server, followed by proxmox installation on debian, instead of installing proxmox from ISO.

    @Not_Oles was running VMs on Intel 13900K so if you are able to run VM, proxmox can manage it as well.

    PVE 8 (now in beta) ships with Linux 6.2 by default. In any case, you can also install the 6.2 opt-in kernel on PVE 7.
    https://forum.proxmox.com/threads/opt-in-linux-6-2-kernel-for-proxmox-ve-7-x-available.124189/

    Thanked by (2)Not_Oles hostaspace
  • somiksomik OG

    @rubenmdh said:

    @somik said:

    @hostaspace said:
    Yea.. may be @somik

    Who dares summon me!!!

    :lol:

    Anyway, Proxmox works fine on intel 12th and 13th gen CPUs, as long as you use a kernel version of 6.2 or higher. I have been told that kernel 6.1 onward supports P and E core CPUs, but since you need to update, might as well update to 6.2 or higher... Proxmox comes with Kernel 5.15 (i think) so if you do not know how to build/install PVE kernel 6.2, skip proxmox.

    Although there seems to be no issues with P and E cores, igpu MAY have issues.

    Best way to avoid these issues is to install debian on the server, followed by proxmox installation on debian, instead of installing proxmox from ISO.

    @Not_Oles was running VMs on Intel 13900K so if you are able to run VM, proxmox can manage it as well.

    PVE 8 (now in beta) ships with Linux 6.2 by default. In any case, you can also install the 6.2 opt-in kernel on PVE 7.
    https://forum.proxmox.com/threads/opt-in-linux-6-2-kernel-for-proxmox-ve-7-x-available.124189/

    Thanks for the update. With the commands mentioned, it should be very easy to install PVE Kernel 6.2 on current gen proxmox to use with Intel 12th and 13th gen CPUs.

    apt update
apt install pve-kernel-6.2
reboot
    Thanked by (3)rubenmdh Not_Oles hostaspace

    Artificial intelligence is no match for our natural stupidity.

    Time flies like an arrow; fruit flies like a banana.

  • hostaspacehostaspace

    @somik said: apt update

    apt install pve-kernel-6.2
    reboot

    will try and update :) ..

  • somiksomik OG

    @hostaspace said:

    @somik said: apt update

    apt install pve-kernel-6.2
    reboot

    will try and update :) ..

    Do let us know how it goes. Extra confirmation never hurts :)

    Artificial intelligence is no match for our natural stupidity.

    Time flies like an arrow; fruit flies like a banana.

Sign In or Register to comment.