How do you guys do Proxmox in a VM

terrorgen

For those of you with a fat VPS that wants to run multiple LXC and nested KVM and installed Proxmox, how do you manage your networking?

Do you run a virtual router, assign the public facing IP address to that router, and use that virtual router to route between all other VMs? Or, do you use PVE's internal firewall to set routing rules?

Do you use a separate out of band IP address for Proxmox's Web UI? What do you do if you don't have additional IP addresses?

I am thinking to set up Proxmox on one of my VPS and these came into mind. I did reach out to the provider to see if they can assign me an additional /128 to assign to Proxmox but was refused one.

It's only a /128, come on!

Falzo

you are talking about IPv6 ? (because you mention /128)

there are lot of options anyway. I like to use static bridges/iptables.
you can use external IPs from small VPSses to GRE tunnel those directly into your proxmox VM and add them to a bridge, so that it can become available directly in a guest VM. you can use the main IP on the fat host VPS for proxmox then.

terrorgen

@Falzo said: you are talking about IPv6 ? (because you mention /128)

Yes, I reckon providers won't be willing to assign a second IPv4 address without additional charges but a /128 should be as cheap as a grain of salt.

Tunneling sounds like an interesting approach!

havoc

Proxmox website has info on a config for vps

VSYS_Host

If you will success with getting additional IPs from your hosting provider - then it depends on how they will provide you. For example, it can be routed, binded to MAC or even provided with additional virtual NIC per IP. If you limited with 1 IP, just use routed networking with 192.168 IPs and DNAT neccessary ports to proper VM.

msatt

I only use one public ipv4 and then use openwrt in proxmox.
Openwrt provides interfaces to 4 virtual bridges configured as 4 seperate subnets (my 'internal network'). Zerotier running in openwrt then provides my external access to whatever subnet I wish.
Depending upon configuration, subnets either have internet access via NAT or are blocked.
I use iptables to forward designated ports to internal servers when needed e.g. 80,443 etc.

loay

nat_manager.py - manage NAT port forwarding for Proxmox VMs and containers:

https://lowendspirit.com/discussion/8311/nat-manager-py-manage-nat-port-forwarding-for-proxmox-vms-and-containers

AuroraZero

I have some scripts that auto assign ports, and nat rules when I add a VM, working on automating the vm add as well then will make a plugin. Should be integrated into Glaciergate soon.

host_c

@terrorgen said: Do you run a virtual router, assign the public facing IP address to that router, and use that virtual router to route between all other VMs? Or, do you use PVE's internal firewall to set routing rules?

I would personally do option A, as it gives me a better control, plus I do not have to do routing and nat in debian, as I prefer a BSD to do that, either that you go with a GUY one or CLI.

This will allow me to set up my internal "LAN" the way I want it, it is more symmetrical to real life setups, where you have your router + switching + running boxes ( servers or vm's )

Both work, atho option A gives you a more tuned control in my opinion.

terrorgen

@host_c said:

@terrorgen said: Do you run a virtual router, assign the public facing IP address to that router, and use that virtual router to route between all other VMs? Or, do you use PVE's internal firewall to set routing rules?

I would personally do option A, as it gives me a better control, plus I do not have to do routing and nat in debian, as I prefer a BSD to do that, either that you go with a GUY one or CLI.

This will allow me to set up my internal "LAN" the way I want it, it is more symmetrical to real life setups, where you have your router + switching + running boxes ( servers or vm's )

Both work, atho option A gives you a more tuned control in my opinion.

if you misconfigured your virtual router you lost access to the web interface though. What would you do then?

host_c

@terrorgen said: if you misconfigured your virtual router you lost access to the web interface though. What would you do then?

since this is a LXC , you should have at least access to the VPS via VNC right?
In that case, set up a 1 / hour backup, and just restore last ours backup.
Or 1 backup of the LXC / day and keep 7 days. As a router it will probably use ~2GB to 3GB of storage max, so backups will not be that big, and you just loose X hours or days of modifications.

remy

Bridge network and NAT forwarding if I need IPv4 + I assign an IPv6 address to the VM.
I use iptables for port forwarding, only because I know it better than proxmox's firewall

All of my most powerful VPSs use proxmox. It's so much easier for backups
I also have 2 storage VMs running proxmox backup server
It's not such a low-end setup anymore, but at least I'm not too worried when a VM is unavailable for several days

Peppery

I've got a homelab and 2 Kimsufi's running Proxmox. Most of my services don't need to be exposed externally, so I set up Wireguard tunnels between home and OVH, and added static routes for the bridge subnets so the VMs are reachable from my whole LAN. No virtual router needed, and I firewall the Proxmox web UI to allow only the internal network so no separate management IP required.