I've set up the SSDs with ZFS RAID10 and I'm not really sure what to do with the NVMe.

IA says "It can be used as a ZFS cache (L2ARC) if you are going to work with ZFS."

In your experience, will that ZFS cache make things faster? I have 128GB RAM BTW.

I got a cheap storages boxes like 1TB NAT for 10$/y and I had to put it into use.
However not fully trusting them with raw vm images, so PGP will do.

The backups are pgp encrypted before they are getting pulled by the storage server.
The proxmox host has never any access to the backup server and the backup server never has any access to the vm images since they are encrypted.

Whatever gets compromised, your should still be safe, that's the idea.
Downside is obviously, disk usage, for every backup we do in Proxmox, we have to do another encrypted copy and wait for the backup server to pull it before we can clear the disk space.

This can easily be done by using the Proxmox hooks, so you can configure your backup schedule like you normaly would do, only the STOREID has to match.
The only thing you have to tweak is, the backup server when it has to pull the backups.

1. Install the hook.

I do daily/weekly offside backups and keep them for 4 weeks.
Example for weekly, script can be modded though.

https://pastebin.com/raw/31jMWKTz
Wouldn't format properly...

Put the script to /usr/local/bin/vzdump-hook.sh and make it executable.
Don't forget to create the user "weekly" and the folder structure (/mnt/weekly/dump/ or whatever folder you want to use)
You also should install and ssh key for the backup server to be able to login into the user "weekly".

In my case, Proxmox creates usuable backups to /mnt/weekly/dump and the encrypted ones are put to /home/weekly/backups for pulling.

2. Edit /etc/vzdump.conf

Replace
#script: FILENAME
with
script: /usr/local/bin/vzdump-hook.sh

3. Either generate or import a existing pgp key

gpg --gen-key

or

gpg --import mahkey

Make SURE you backup this key.

4. Make a test backup and check if the encrypted backup is there.
There should be zero errors in the proxmox backup log.

If you do, you might have to trust that pgp key.
gpg --edit-key mahkey
and type "trust"

5. Setup a cronjob on the remote storage server

5 5 * * * /home/weekly/backups.sh

You might have to adjust the time.
rsync will do.

#!/bin/bash
set -e
if pgrep -fl backup.sh &>/dev/null; then
        rsync -Pav -e "ssh -i weekly" weekly@mahserverip:/home/weekly/backups/* /home/weekly/backups/
        find /home/weekly/backups/* -mtime +30 -exec rm {} +
fi

Don't forgot to make it executable and do a testrun.

6. Profit.

Thanks for reading my TED TALK.

I need a dedicated server for a month. Do you think I can find something for $60 or less?

E3 or E5, just not that old.

Not sure if an hourly paid VPS would run Proxmox without issues.

Code available at:
https://pastebin.com/cdrxhUSU

nat_manager.py Quick Start Guide

nat_manager.py is a Python script designed to manage NAT (Network Address Translation) and port forwarding rules for VMs and containers in a Proxmox environment. The script utilizes iptables to configure NAT rules and allows for easy addition, removal, listing, updating, exporting, and importing of port mappings.

This guide provides step-by-step instructions for setting up the network, using the script, and provides examples for common operations.

Network Setup in Proxmox

To use nat_manager.py effectively, you need to set up a bridge network (vmbr1) on your Proxmox server. This bridge will use a private IP range and manage the NAT and port forwarding for your VMs and containers.

1. Configure the Bridge Network (vmbr1)

Edit the /etc/network/interfaces file to configure the bridge network interface vmbr1:

sudo nano /etc/network/interfaces

Add the following configuration:

auto vmbr1
iface vmbr1 inet static
    address 10.0.0.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0

    post-up iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o <YOUR_PUBLIC_INTERFACE> -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s 10.0.0.0/24 -o <YOUR_PUBLIC_INTERFACE> -j MASQUERADE

• <YOUR_PUBLIC_INTERFACE>: Replace this with your network interface that has a public IP (e.g., enp0s3).

2. Enable IP Forwarding

To ensure IP forwarding is enabled permanently, add the following line to /etc/sysctl.conf:

net.ipv4.ip_forward = 1

Apply the changes:

sudo sysctl -p

3. Restart Networking Service

Restart the networking service to apply the changes:

sudo systemctl restart networking

4. Install Required Packages

To ensure iptables rules persist across reboots, install iptables-persistent and other required packages:

sudo apt-get update
sudo apt-get install iptables-persistent python3 python3-pip sqlite3 -y

• iptables-persistent: Allows iptables rules to be saved and restored on boot.
• python3 and sqlite3: Required for running the nat_manager.py script.

Using nat_manager.py

Run nat_manager.py using Python3. Below are the various usage instructions for managing NAT and port forwarding rules for your VMs and containers.

python3 nat_manager.py -h
usage: nat_manager.py [-h]
                      {add,remove,list,update,reserve,unreserve,list-reserved,export,import,backup,restore,rebuild-db}
                      ...

NAT Manager Script

positional arguments:
  {add,remove,list,update,reserve,unreserve,list-reserved,export,import,backup,restore,rebuild-db}
                        Available actions
    add                 Add port mappings for a container
    remove              Remove port mappings for a container
    list                List port mappings
    update              Update port mappings for a container
    reserve             Reserve ports for the host machine
    unreserve           Unreserve ports
    list-reserved       List reserved ports
    export              Export port mappings to a JSON file
    import              Import port mappings from a JSON file
    backup              Backup current configuration
    restore             Restore configuration from backup
    rebuild-db          Rebuild the database from existing iptables rules

options:
  -h, --help            show this help message and exit

1. Add Port Mappings

To add NAT port forwarding rules for a VM or container with an internal IP address (e.g., 10.0.0.5):

sudo python3 nat_manager.py add <container_ip> --mode <automatic|manual> --num-ports <N>

• Parameters:
  • <container_ip>: Internal IP address of the VM/container (e.g., 10.0.0.5).
  • --mode: Mode for adding ports, automatic (default) or manual.
  • --num-ports <N>: Number of ports to forward (default: 6).

Examples:

• Automatic Mode:

  sudo python3 nat_manager.py add 10.0.0.5 --mode automatic --num-ports 4
  
  

  This command automatically assigns 4 external ports (starting from 50000) to forward traffic to standard internal ports (e.g., 22, 80, 443, 8080) on 10.0.0.5.

• Manual Mode:

  sudo python3 nat_manager.py add 10.0.0.5 --mode manual --external-ports 50000 50001 --internal-ports 22 80 --protocols tcp udp
  
  

  This command manually assigns external ports 50000 (TCP) and 50001 (UDP) to forward to internal ports 22 (SSH) and 80 (HTTP) on 10.0.0.5.

2. Remove Port Mappings

To remove all port forwarding rules associated with a specific container IP:

sudo python3 nat_manager.py remove <container_ip>

• Example:

  sudo python3 nat_manager.py remove 10.0.0.5
  
  

  This command removes all port mappings associated with the IP 10.0.0.5.

3. List Current Port Mappings

To list all current port mappings or those for a specific container IP:

sudo python3 nat_manager.py list [container_ip]

• Examples:

  • List All Mappings:

  sudo python3 nat_manager.py list
  
  

  Lists all port mappings currently configured on the Proxmox server.

  • List Mappings for a Specific Container:

  sudo python3 nat_manager.py list 10.0.0.5
  
  

  Lists the port mappings for the container with IP 10.0.0.5.

4. Update Port Mappings

To update existing port mappings for a VM or container:

sudo python3 nat_manager.py update <container_ip>

• Examples:

  • Interactive Mode:

  sudo python3 nat_manager.py update 10.0.0.5
  
  

  This command will prompt you to update the internal ports or protocols for each external port currently mapped to 10.0.0.5. Leave input blank to keep the current mapping.

  • Non-Interactive Mode:

  sudo python3 nat_manager.py update 10.0.0.5 --external-ports 50000 50001 --internal-ports 2222 8081 --protocols tcp udp
  
  

  This command updates the external port 50000 to forward to internal port 2222 (TCP) and 50001 to forward to 8081 (UDP) on 10.0.0.5.

5. Export and Import Port Mappings

You can export current port mappings to a JSON file for backup purposes or import them from a JSON file.

• Export Port Mappings:

  sudo python3 nat_manager.py export /path/to/export.json
  
  

  This command exports the current port mappings to export.json.

• Import Port Mappings:

  sudo python3 nat_manager.py import /path/to/export.json
  
  

  This command imports port mappings from export.json.

6. Backup and Restore Configuration

You can backup the current configuration of iptables and port mappings or restore from a backup.

• Backup Current Configuration:

  sudo python3 nat_manager.py backup
  
  

  This creates a backup of the current iptables rules and port mappings database.

• Restore Configuration from Backup:

  sudo python3 nat_manager.py restore <timestamp>
  
  

  Replace <timestamp> with the desired backup timestamp (e.g., backup_20230917123045).

7. Rebuild the Database from Existing iptables Rules

If the SQLite database is lost or out of sync with iptables rules, you can rebuild it:

sudo python3 nat_manager.py rebuild-db

This command scans existing iptables rules and reconstructs the database for consistency.

Important Notes

• IP Forwarding: Ensure IP forwarding is enabled by adding net.ipv4.ip_forward = 1 to /etc/sysctl.conf and running sudo sysctl -p.
• Save iptables Rules: To ensure the rules persist after reboot, use iptables-save > /etc/iptables/rules.v4 and iptables-restore < /etc/iptables/rules.v4.

• Check iptables-persistent: Ensure iptables-persistent is installed and enabled to manage rule persistence:

  sudo apt-get install iptables-persistent -y
  sudo netfilter-persistent save
  
  

Network Configuration for VM/Container in Proxmox

When creating a VM or container in Proxmox that will use NAT:

1. Assign an Internal IP Address:

   • Assign an IP within the vmbr1 subnet, such as 10.0.0.5.
   • This IP will be used for internal communication and NAT port forwarding.

2. Connect to vmbr1 Network Bridge:

   • Ensure the VM/container network interface is attached to vmbr1 to use the internal network managed by NAT.
   • In Proxmox, select vmbr1 as the network bridge when creating or configuring the VM/container.

3. Configure Gateway (Optional):

   • Set the gateway to 10.0.0.1 (the vmbr1 address) to route all outbound traffic through the Proxmox host.

This setup allows VMs/containers to communicate internally using 10.0.0.x IPs and be accessed externally via port forwarding rules defined by nat_manager.py.

Do anyone uses proxmox on this CPU???, can proxmox handle both P & E cores also what about iGPU, does it work properly with passthrough?

EX44 Hardware data:

   CPU1: 13th Gen Intel(R) Core(TM) i5-13500 (Cores 20)
   Memory:  64125 MB
   Disk /dev/nvme0n1: 512 GB (=> 476 GiB) doesn't contain a valid partition tabl                                                                                        e
   Disk /dev/nvme1n1: 512 GB (=> 476 GiB) doesn't contain a valid partition tabl                                                                                        e
   Total capacity 953 GiB with 2 Disks

Network data:
   eth0  LINK: yes
         MAC:  c8::78
         IP:   
         IPv6: 2a01:
         RealTek RTL-8169 Gigabit Ethernet driver

lspci

lscpu -e

What about performance in compare with AMD Ryzen 5 3600 come with AX-41 and cost cheaper.

Facts
* I have used two different providers for this, same issue
* Using mikrotik initially and then switched to opnsense as firewall. I know, I am not using both of them at the same time

Configuration
vmbr0 -> primary IP for the proxmox (works brilliant)
vmbr1 -> additional public IP for the firewall (mikrotik / opnsense)
vmbr2 -> LAN interface

Problem?
vmbr0 works brilliantly, maybe because it's BridgePort to ens18 (primary)
vmbr1 doesn't have internet even though the IP address, Subnet and Gateway are correct. Can't really BridgePort it to ens18 as well

Any pointers? please do share

found this while searching for a proxmox module that works with Blesta. it's being developed by the same people behind Proxmox VPS for whmcs - modulesgarden.

Might be interesting for providers using Blesta so though of sharing here. (the default shipped Proxmox module is outdated)

I. Before We Start

We need to obtain our basic network configuration from our provider. Or, if we are running our own host node, we need to assign basic network configuration to ourselves. Our basic network configuration might look something like this:

For IPv6, one might expect something like:

But occasionally, IPv6 could be something like:

Notes:

• The /28 in the IPv4 address and the longer netmask are different ways of providing the same information about the size of the local, directly connected network. It suffices for us to have this information in one format or the other. We don't need both formats because the information is the same. Also, the broadcast IP might not be provided, since it isn't strictly necessary.
• For the second format of the IPv6 address, what happened to the /64? 😱 The /128 in the second form of the IPv6 address might seem clueless to IPv6 fans expecting a /64. Also, the second format of the IPv6 address includes a gateway6 address. The gateway6 address might seem strange to some IPv6 fans, but we need the gateway6 for our minimal, static configuration. More on all this below.

II. Introduction

In the previous post of this series we finished using the Proxmox web GUI to install our new Debian KVM VPS via the Debian netinst installer iso image. The final step in Part II was removing the netinst install iso image from the emulated cdrom and then reooting our new VM, which came up from its own internal filesystem:

In today's post, we continue from this exact place where we left Part II -- connected to our newly installed and newly rebooted KVM via the Proxmox web GUI. In this post, we will accomplish the networking configuration which was skipped in Part II because the Debian netinst iso doesn't automatically configure out of band IP addresses.

There are three network configuration and related tasks we will accomplish today:

• First, we go "inside" our VM through the Proxmox web GUI's emulated "physical" console connection and set up networking. In Debian, networking setup requires that we adjust the file /etc/network/interfaces to tell our VM its network address and the address of its gateway to the internet.
• Second, we edit the file /etc/resolv.conf to tell our VM the numerical addresses of Domain Name System ("DNS") servers it can use to translate human readable Uniform Resource Identifiers (URI) into numerical Internet Protocaol ("IP") addresses.
• Third, we set up /etc/apt/sources.list to tell our system's Aptitude software package manager ("APT") where to get software updates and the additional software packages we will want to install.

Section III, Quick Setup, runs quickly through all three of today's tasks in "recipe style."

Section IV offers additional context on our setup environment.

Sections V, VI, and VII provide additional details on today's three setup tasks.

Section VIII discusses security.

Section IX discusses backup.

When we finish the Quick Setup, our new Debian KVM VPS should be connected to the internet, DNS should work, and we should be able to use the Debian package system to add whatever additional software we want.

When we finish all of today's post, we should have reasonable context within which to understand our Debian VM's networking setup.

III. Quick Setup

Logged into our VM through the Proxmox web GUI, we run the command ip link show. This command will give us the name of our network interface, probably something like "ens18."

As root or with sudo, we edit the text of the file /etc/network/interfaces so that it contains the minimum necessary information:

auto ens18
iface ens18 inet static
  address IPv4_ADDRESS/CIDR
  gateway GATEWAY_ADDRESS

iface ens18 inet6 static
  address IPv6_ADDRESS/CIDR
  gateway GATEWAY6_ADDRESS

Using our example network configuration, our minimal /etc/network/interfaces looks like this:

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
  gateway 172.16.164.1

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
  gateway fe80:xxxx:xxxx:xxxx::3

Second, we edit the /etc/resolv.conf file so that it looks like this:

nameserver 1.1.1.1
nameserver 8.8.8.8
nameserver 2606:4700:4700::1111
nameserver 2001:4860:4860::8888

Third, we edit /etc/apt/sources.list so that it looks like this:

deb http://deb.debian.org/debian buster main contrib non-free

deb http://deb.debian.org/debian-security/ buster/updates main contrib non-free

deb http://deb.debian.org/debian buster-updates main contrib non-free

Finally, we restart networking so that our new configuration takes effect:

systemctl restart networking

At this point, we should have both IPv4 and IPv6 connectivity, and DNS and APT both should work.

IV. More Context

• Virtualized Console Connection

The Proxmox web GUI virtualizes a wired console connection. In other words, our web browser does connect over the internet to our Proxmox server, but, the view from inside our new KVM is the same as though a wired connection was attached. Our new KVM thinks it's talking over a wired connection to a physical console. From inside our new KVM, there is, as yet, no network connection.

By default, the Proxmox web GUI works via VNC. In the Proxmox wiki on serial terminal Proxmox warns that VNC might

not have the features you need (i.e. easy copy/paste between other terminals)

or it might be

impossible to capture all [kernel messages, standard output, or error] messages on [the] VNC screen.

Yep, copy / paste commands do not seem to work in the Proxmox KVM virtual console.

Also, if you enjoy using the vi editor, you might find what looks like a "Send-Esc" button among the set of choices within the set exposed by the top button on the console VNC control bar. Use of the real keyboard Escape key results in exiting full screen. However, a second real Esc seems to produce the expected mode change, despite that maybe we no longer can see too well without returning to full screen.

• No DHCP, No SLAAC

These days most network setups use Dynamic Host Configuration Protocol (DHCP) to autoconfigure IPv4 networking. The machine on which networking is to be configured asks for and receives from a DHCP server all the needed information for the networking setup.

It is possible to configure DHCP so that it always returns the same IP address to each VM, but, since our entire Proxmox network is static, it may be simpler to set up networking manually--the traditional way for servers.

Stateless Address Autoconfiguration ("SLAAC") provides automatic configuration of IPv6 addresses. SLAAC requires a /64, which is why people say, for IPv6, that a /64 is expected and that less than a /64 is clueless. However, it remains possible to hand configure a single static IPv6 address, as we are doing here.

What if, for whatever reason, we simply do not want to use SLAAC? What if our provider doesn't receive enough IPv6 addresses from his provider to allow passing on to each VPS its own /64? What if our provider's provider charges an extra fee for extra IPv6 addresses, but we do not want to pay our provider's pass through of his provider's extra fee? What if we simply choose to use single, static IPs as is traditional for servers?

• No Cloud-Init

As mentioned in the previous post of this series, most VM network setups these days are done with Cloud-Init. Proxmox supports Cloud-Init, which enables both networking and ssh access to virtual machines to be set up on the Proxmox hypervisor and outside of the VM. Cloud-init can use DHCP. Here, however, we have chosen the simplest possible manual configuration with static IPs.

• Our Static, Routed Configuration And Out of Band Gateway From Our Provider's Provider

Here, our single, static IPv4 and single, static IPv6 are each derived from a routed subnet assigned to our server node. However, our internet gateway IPv4 address is not included among our server's routed group of IPv4s. This is called an "out of band" gateway.

Besides routed subnets, it also is possible for a datacenter to assign to servers non-routed, individual IP addresses. Data for these non-routed IPs moves between the datacenter switch and server nodes via the "link layer." Hetzner has a tutorial on Debian network configuration which includes discussion of "bridged configuration" for non-routed IPs.

• Systemd in Debian Networking

Since about 2014, networking is setup on Debian with systemd. The choice of systemd initially was and has continued to be divisive. Nevertheless systemd has remained as the Debian default.

There are at least two basic variations of Debian's systemd network arrangement. The first--which seems to be the default variation for Debian systemd network configuration--at least with the netinst iso--is using systemd's networking.service. For example, by using systemctl, we can confirm that networking.service is what is being used on our Node:

root@Proxmox-VE ~ # systemctl status networking.service
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: 
   Active: active (exited) since Wed 2021-06-02 19:13:13 UTC; 1 weeks 2 days ago
     Docs: man:interfaces(5)
 Main PID: 791 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   Memory: 0B
   CGroup: /system.slice/networking.service

 [ . . . ]
root@Proxmox-VE ~ # 

Our test KVM also seems to think its networking is controlled by systemd:

root@debian-kvm:~# systemctl status networking
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Wed 2021-06-16 01:20:45 UTC; 4min 51s ago
     Docs: man:interfaces(5)
  Process: 448 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
 Main PID: 448 (code=exited, status=0/SUCCESS)

Jun 16 01:20:45 debian-kvm systemd[1]: Starting Raise network interfaces...
Jun 16 01:20:45 debian-kvm systemd[1]: Started Raise network interfaces.
root@debian-kvm:~#

As we can see, systemd networking.service calls the traditional debian ifup and ifdown.

root@debian-kvm:~# cat /lib/systemd/system/networking.service
[Unit]
Description=Raise network interfaces
Documentation=man:interfaces(5)
DefaultDependencies=no
Requires=ifupdown-pre.service
Wants=network.target
After=local-fs.target network-pre.target apparmor.service systemd-sysctl.service systemd-modules-load.service ifupdown-pre.service
Before=network.target shutdown.target network-online.target
Conflicts=shutdown.target

[Install]
WantedBy=multi-user.target
WantedBy=network-online.target

[Service]
Type=oneshot
EnvironmentFile=-/etc/default/networking
ExecStart=/sbin/ifup -a --read-environment
ExecStop=/sbin/ifdown -a --read-environment --exclude=lo
RemainAfterExit=true
TimeoutStartSec=5min
root@debian-kvm:~# 

The second Debian systemd possibility--not the default on Debian netinst.iso and not used here--is systemd-networkd. Sahitya Maruvada has a simple, clear, Debian systemd-networkd introduction, Working with systemd-networkd. The systemd-networkd wiki page and the systemd.network manpage also are available.

• Official Debian Network Setup Instructions

Official Debian network setup instructions include the Wiki, the Handbook, manual pages such as man interfaces, /etc/network/interfaces examples online, and sometimes locally:

# less /usr/share/doc/ifupdown/examples/network-interfaces

• The ip Command Usually Is Available Even Though Networking Setup Varies Among Linux Distributions

Setting up networking, DNS name resolution, and software package management is very different in different Linux distributions. Therefore, we should not assume that the steps taken below would be exactly the same with a different Linux distribution than Debian.

Nevertheless, despite the different distributions' differing network setup systems, the ip command, supplied by the iproute2 collection, usually is available these days. Please see also Red Hat's IP Command Cheat Sheet

Because the ip command often is available, networking can be configured in many distributions, including Debian, by running a sequence of ip commands. The net effect of the sequence of ip commands can be to get the network functioning on most distributions without touching that individual distribution's network setup scheme.

Here's an example of the ip command used in the context of an iPXE boot. Note that the first command in the linked example requires knowledge of the name of the interface. We can list the names of the interfaces on our system by running the ip link show command.

One issue with using a sequence of ip commands is that the network setup fails to persist across reboots. However, we can place the ip command sequence inside a script which will be run automagically every time the server reboots. The sequence of ip commands in a script reminds us of the days before systemd, when scripts controlled all parts of the boot process including network setup.

Our KVM VPS's internal network configuration that we will be using below is similar to how LXC containers are configured in Proxmox. As will be seen below, Proxmox's LXC containers' network configuration adopts a variant of the "scripted ip command" approach, which also works inside Proxmox's KVM VPSes.

V. Our VM's Network Setup

• Interfaces

Our original /etc/network/interfaces file, the one installed by the netinst.iso, might look like this:

debian@debian-kvm:~$ cd /etc/network
debian@debian-kvm:/etc/network$ cat interfaces.original
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback
debian@debian-kvm:/etc/network$ 

Note that, in the default from the netinst.iso, /etc/network/interfaces.d is empty, so sourcing its files does nothing to the configuration.

debian@debian-kvm:/etc/network$ ls interfaces.d
debian@debian-kvm:/etc/network$ 

Now, let's edit /etc/network/interfaces to match our example network information from the above Before We Start section.

debian@debian-kvm:/etc/network$ cat interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
  gateway 172.16.164.1

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
  gateway fe80:xxxx:xxxx:xxxx::3

debian@debian-kvm:/etc/network$ 

The minimum required information does not include comments (lines beginning with #). Maybe we can make the rash and short-sighted assumption that we are not going to install anything which will want a file included from interfaces.d. The loopback interface might no longer be required (please see lines 17 and 18 in this file from Debian sources). Thus, for our example setup, the minimum /etc/network/interfaces might be:

debian@debian-kvm:/etc/network$ cat interfaces

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
  gateway 172.16.164.1

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
  gateway fe80:xxxx:xxxx:xxxx::3

debian@debian-kvm:/etc/network$ 

When configuring Debian LXC containers, Proxmox configures their /etc/network/interfaces files using added post-up and pre-down routes. Similarly, just for fun, instead of giving the gateway addresses in our /etc/network/interfaces,, we can manually add routes. Except for the initial post-up and pre-down these added lines mirror ip route commands that we could run manually to set up or take down networking without touching the /etc/network/interfaces file.

debian@debian-kvm:/etc/network$ cat interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
     post-up ip route add 172.16.164.1 dev ens18
     post-up ip route add default via 172.16.164.1 dev ens18
     pre-down ip route del default via 172.16.164.1 dev ens18
     pre-down ip route del 172.16.164.1 dev ens18

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
     post-up ip route add fe80:xxxx:xxxx:xxxx::3  dev ens18
     post-up ip route add default via fe80:xxxx:xxxx:xxxx::3  dev ens18
     pre-down ip route del default via fe80:xxxx:xxxx:xxxx::3  dev ens18
     pre-down ip route del fe80:xxxx:xxxx:xxxx::3  dev ens18

debian@debian-kvm:/etc/network$ 

VI. Our VM's DNS

We might want to add more or different nameservers to /etc/resolv.conf. Our Quick Setup configuration, above, includes IPs from Cloudflare and from Google.

VII. Our VM's Apt Setup

The Debian wiki instructions for configuring apt are at https://wiki.debian.org/SourcesList. There also is a man page. The configuration shown above, in Section III Quick Setup, is from the SourcesList Debian wiki page.

The Debian Security Information page says:

You can use apt to easily get the latest security updates. This requires a line such as
deb http://security.debian.org/debian-security buster/updates main contrib non-free

Many of the larger providers offer Debian mirrors. For example, Debian packages and security updates are available from the Hetzner Debian Mirror

After /etc/sources.list is edited, we update our system's package repositories as follows:

apt-get upgrade && apt-get dist-upgrade -y

We can see exactly which packages are installed by looking at the logs in /var/log/apt.

We may wish to install openssh-server so that we can connect to our VM via ssh in addition to our Proxmox VNC connection. With ssh we regain cut and paste functionality while enjoying lower apparent latency!

apt-get install openssh-server

The Kennedy article, mentioned below in Section VII, has some good tips for ssh server configuration.

VIII. Security

Google suggests its first choice among essential server security articles. This article from 2013, by Bryan Kennedy, seems to provide still-good advice, except that, nowadays, many people prefer to use ed25519 keys

IX. Backup

After all this work, we certainly want to make an offline backup of our new VM. We can use Proxmox to make the backup and then download a a copy from the host node's /var/lib/vz/dump directory.

Introduction

In Part I of this series, we downloaded the Debian netinst install iso. We then created a KVM VPS with the iso attached, and, finally, we successfully booted the iso.

In today's post, we're going to install our KVM with Debian 10 from the newly booted iso. But first, a bit of context on installing.

Context

• Why the Debian minimal netinst iso?

Debian themselves say, "we think that in many cases the minimal CD image is better — above all, you only download the packages that you selected for installation on your machine. . . ."

What we gain from this series is a well-proven, widely used, minimal, highly extensible, open-source server operating system.

• What about networking?

The biggest difference between installing on our VPS and installing on our personal laptop or desktop might be network configuration. On personal devices, we are used to automatic network configuration happening behind the scenes via Dynamic Host Configuration Protocol (DHCP). We turn on our device, it gets its own IP address and internet connection without our having to do much.

On servers, however, the server's IP address and internet connection sometimes are set by hand instead of automatically via DHCP. Traditionally, server network settings are done from a console physically connected to the running server. Obviously, however, if our server is at a remote location, we cannot have a wired connection. Also, since networking hasn't yet been set up inside the server, we can't connect directly to our remote server over the internet, either.

As might be expected, the Debian minimal netinst iso is set up to configure networking automatically via DHCP. Thus, when we try the networking step of the install, that step will fail. The netinst iso will succeed, however, in installing a minimal Debian system without networking. In Part III of this series, covering Post Install Configuration, we will use the Proxmox web GUI and VNC to go inside our minimal system and set up networking by hand.

• Alternative installation methods

It might be worth mentioning a few of the many other excellent methods of server installation which, although frequently used, are not selected here because they might be even more complex than our "simple" method.

• First, Debian unattended Installation using a preseed file will not work here because no networking is set up to use for obtaining the preseed file.
• Cloud-init is "the industry standard multi-distribution method for cross-platform cloud instance initialization." However, the Proxmox Cloud-Init Support wiki article says, despite the convenience of ready-made images, "we usually recommended to prepare the images by yourself," because "you will know exactly what you have installed." Also, for a special perspective on Cloud-Init, you might enjoy watching Cloud-Init: The Good Parts.
• Proxmox supports Templates. It's possible to create templates with Packer. If interested, you can check Creating proxmox templates with packer.

Before We Start

We need to begin today at the exact stage where we left Part I. Our Debian Installer should be booted and running on our VPS.

We also will need the server's hostname (which can be Debian) plus the username (which also can be Debian) and the real name for the user account which the installer will create. It's also convenient to have on hand two previously generated good passwords, one for the root account and another for the new user account.

Debian Installer Steps

• Select Install

• Language

• Location

• Keyboard

• DHCP Tries and Fails

• Select "Do Not Configure Network at this Time"

• Hostname

• Enter and Confirm the Root Password

• User's Real Name

• Username

• User Password

• Time Zone

• Partitioning Method

• Disk to Partition

• Partitioning Scheme

• Confirm Partitioning

• Write Changes to Disks

• Confirm No Additional Install Media

• Confirm No Network Mirror

• Package Usage Survey

• Choose Additional Software

• Dual Boot

• Grub

• Installation Complete

In the Proxmox web GUI, we select VPS > Hardware > CD/DVD Drive. Press edit and select "Do not use any media." Then, we return to our "Installation Complete" screen by selecting Console, which should reappear just as we left it. Finally, we click the "Continue" button, which should reboot the VPS.

In Part I, we did not install Qemu Agent. Therefore, rebooting from the Proxmox web GUI (outside our VPS) as opposed to rebooting from the console (inside our VPS) might not work. However, if it is necessary to stop the server from the web GUI, we can use the web GUI's Stop command found on the drop-down menu of the Shutdown button.

• Successful Reboot

I have no clue where and what to look for.

Any idea appreciated.

The Proxmox itself is installed with the SYS provided image and is an as standard install as possible: no zfs, no ceph, no cluster. I have 4 HDD-s in a raid 10 configuration (software raid, option selected during the Proxmox installation). The first thing I did after installation was to register an ACME account and obtain a Let's Encrypt certificate for the domain where the node is. No LXC containers, just KVM. Cannot remember now if the issue started happening before creating the very first VM.

I tried on another server (read and write) and it works fine.

showmount -e remoteserver replies:

clnt_create: RPC: Port mapper failure - Timed out

I did pve-firewall stop and nothing changed.

Any ideas?

Can anyone tell me how I can get something like this to work?
My plan is to install Proxmox on it and then create one VM for pfsense and another for the windows server, but I read somewhere that running your primary router/firewall on a VM is a bad idea.
Is that true? Is there any other way to get this to work?