Proxmox 7 node on OVH unreachable after reboot

miumiu
edited July 2021 in Help

Hello, can somebody help me resolve following maddness pls?

Proxmox host node is OVH network dedicated (SoYouStart)

/etc/network/interfaces:

auto lo
iface lo inet loopback

iface eno3 inet manual

auto vmbr0
iface vmbr0 inet static

  address 54.38.192.11/24
  gateway 54.38.192.254
  bridge-ports eno3
  bridge-stp off
  bridge-fd 0

1) After node server full reboot is network UNREACHABLE

2) But when i run SSH command:

service networking restart

network is reachable and work fine:

# ip route
default via 54.38.192.254 dev vmbr0
54.38.192.0/24 dev vmbr0 proto kernel scope link src 54.38.192.11

I do not know resolve this madness WHY after reboot mentioned config does not work = network unreachable, but after additional manual networking restart the same config works fine....

BTW: i do not want use DHCP config = default (automatic OVH install) config:

auto lo
iface lo inet loopback

iface eno3 inet manual

auto vmbr0
iface vmbr0 inet dhcp
  bridge-ports eno3
  bridge-stp off
  bridge-fd 0

I want use STATIC config, not DHCP

many thanks in advance for each attempt to help resolve this madness

Good day and Goodbye

Thanked by (1)AlwaysSkint
«1

Comments

  • AbdullahAbdullah Hosting ProviderOG

    is that the only network config file?

    dirty fix - just add an init script in /etc/init.d/ to restart networking at every reboot.

    Thanked by (1)miu
  • update 2: after searching on net i found that several ppl report the same problem after upgrade on PROXMOX 7 (unreachable network). So probably version 7 is still pretty bugged. (maybe reason why OVH still not offers Proxmox 7 template?) I tried several changes in config but not resolved this (not found exact reason and clean solution), without success

    Good day and Goodbye

  • Not_OlesNot_Oles Hosting ProviderContent Writer
    edited July 2021

    Hi @miu!

    Please answer a few questions from a clueless guy:

    (A) In your original post,

    @miu said: 1) After node server full reboot is network UNREACHABLE

    2) But when i run SSH command:

    service networking restart

    network is reachable and work fine:

    # ip route
    default via 54.38.192.254 dev vmbr0
    54.38.192.0/24 dev vmbr0 proto kernel scope link src 54.38.192.11

    How do you run the ssh command when the network is unreachable?


    (B) What do the logs say about bringing the network up as part of the reboot? Is there something that failed?


    (C) In your

    @miu said: /etc/network/interfaces:

    auto lo
    iface lo inet loopback

    iface eno3 inet manual

    auto vmbr0
    iface vmbr0 inet static

    address 54.38.192.11/24
    gateway 54.38.192.254
    bridge-ports eno3
    bridge-stp off
    bridge-fd 0

    interface eno3 is set to "manual."

    In man (5) interfaces, it says,

    "The manual Method
           This method may be used to define interfaces for which  no  configuration  is
           done  by  default.  Such interfaces can be configured manually by means of up
           and down commands or /etc/network/if-*.d scripts.
    

    Further down in the vmbr0 section, your /etc/network interfaces also has the line, bridge-ports eno3. What is supposed to happen when ifup reaches that bridge-ports eno3line following reboot? Could it be that there is a failure at that bridge-ports eno3 line because eno3 is called but eno3 is not present because manual configuration was specified for eno3?

    I have seen other similar configurations which specify bridge-ports none instead of bridge-ports eno3. Proxmox' Routed Configuration example has bridge-ports none:

    auto lo
    iface lo inet loopback
    
    auto eno1
    iface eno1 inet static
            address  198.51.100.5/24
            gateway  198.51.100.1
            post-up echo 1 > /proc/sys/net/ipv4/ip_forward
            post-up echo 1 > /proc/sys/net/ipv4/conf/eno1/proxy_arp
    
    
    auto vmbr0
    iface vmbr0 inet static
            address  203.0.113.17/29
            bridge-ports none
            bridge-stp off
            bridge-fd 0
    

    Maybe your issue could be resolved by changing bridge-ports eno3 to bridge-ports none?


    In summary:

    • How can you run an ssh command when the network is unreachable?
    • What do the logs say?
    • What is supposed to happen when ifup reaches the "bridge-ports xxx" line in /etc/network/interfaces, and could having the interface xxx non-functioning cause ifup to fail?

    Could your issue be resolved by changing bridge-ports eno3 to bridge-ports none?

    Hope this is helpful! Best wishes from Mexico! :)

    Thanked by (1)miu

    I hope everyone gets the servers they want!

  • miumiu
    edited July 2021

    Hello Oles, firstly thank you for your response!

    @Not_Oles said:
    Hi @miu!

    Please answer a few questions from a clueless guy:

    How do you run the ssh command when the network is unreachable?

    ANSWER @ A) Remotely Through IPMI console


    (B) What do the logs say about bringing the network up as part of the reboot? Is there something that failed?


    (C) In your

    @miu said: /etc/network/interfaces:

    auto lo
    iface lo inet loopback

    iface eno3 inet manual

    auto vmbr0
    iface vmbr0 inet static

    address 54.38.192.11/24
    gateway 54.38.192.254
    bridge-ports eno3
    bridge-stp off
    bridge-fd 0

    interface eno3 is set to "manual."

    In man (5) interfaces, it says,

    "The manual Method
           This method may be used to define interfaces for which  no  configuration  is
           done  by  default.  Such interfaces can be configured manually by means of up
           and down commands or /etc/network/if-*.d scripts.
    

    Further down in the vmbr0 section, your /etc/network interfaces also has the line, bridge-ports eno3. What is supposed to happen when ifup reaches that bridge-ports eno3line following reboot? Could it be that there is a failure at that bridge-ports eno3 line because eno3 is called but eno3 is not present because manual configuration was specified for eno3?

    I have seen other similar configurations which specify bridge-ports none instead of bridge-ports eno3. Proxmox' Routed Configuration example has bridge-ports none:

    auto lo
    iface lo inet loopback
    
    auto eno1
    iface eno1 inet static
            address  198.51.100.5/24
            gateway  198.51.100.1
            post-up echo 1 > /proc/sys/net/ipv4/ip_forward
            post-up echo 1 > /proc/sys/net/ipv4/conf/eno1/proxy_arp
    
    
    auto vmbr0
    iface vmbr0 inet static
            address  203.0.113.17/29
            bridge-ports none
            bridge-stp off
            bridge-fd 0
    

    Maybe your issue could be resolved by changing bridge-ports eno3 to bridge-ports none?

    ANSWER @ B & C:

    Thanks for your suggestion - this i not tried (change bridge-ports from ens 3 to none)
    From time reasons after several hours of frustration i reinstalled whole node from ISO 6.4 - it works FINE (network) with my above mentioned origin static config
    Maybe i will try your suggested change next time when install another Proxmox node from v7 ISO, then i will can update this

    thanks! regards miu

    Thanked by (1)Not_Oles

    Good day and Goodbye

  • miumiu
    edited July 2021

    So just now i can conclude only:

    auto lo
    iface lo inet loopback
    
    iface eno3 inet manual
    
    auto vmbr0
    iface vmbr0 inet static
    
      address 54.38.192.11/24
      gateway 54.38.192.254
      bridge-ports eno3
      bridge-stp off
      bridge-fd 0
    

    works on Proxmox 6.4 + SYS well (network works without any troubles)

    but on new Proxmox v7 + SYS not at all (network UNREACHABLE)

    Thanked by (1)Not_Oles

    Good day and Goodbye

  • @miu said:
    but on new Proxmox v7 + SYS not at all (network UNREACHABLE)

    Thanked by (1)miu
  • @miu i would look at the networking service if its started at reboot.
    look at the kernel logs in the console etc ...
    but if its a bug in v.7 then you have to live with it until update.

    Thanked by (2)Not_Oles miu
  • Not_OlesNot_Oles Hosting ProviderContent Writer

    @ehab said:
    @miu i would look at the networking service if its started at reboot.
    look at the kernel logs in the console etc ...
    but if its a bug in v.7 then you have to live with it until update.

    @ehab Yeah. for sure, but @miu can't look at the logs after the machine is reinstalled. :)

    I hope everyone gets the servers they want!

  • @ehab said:

    @miu said:
    but on new Proxmox v7 + SYS not at all (network UNREACHABLE)

    u r right - exactly ma face after suffering from this issue! :-D

    Thanked by (1)ehab

    Good day and Goodbye

  • I'm using a similar network config with Proxmox VE 7.0-9 and no problems at all.

    But I didn't use the ISO. Installed on top of Debian.

    Thanked by (1)miu
  • Somehow I missed that 7 was available - still running 6.4 on my boxes. Thanks for the heads-up.

    Thanked by (1)miu

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • havochavoc OGContent Writer

    Reddit is full of people bombing their boxes with 7 upgrade. I've decided I'll wait a month or 5. There isn't really anything I need in 7...

    Thanked by (4)AlwaysSkint Falzo miu Ouji
  • edited July 2021

    I'll run the tests and make sure that I have done some updates on my Proxmox Backup Server.
    @miu Re. networking issue; I wonder if this is the cause:

    Linux Bridge MAC-Address Change

    With Proxmox VE 7 / Debian Bullseye, a new systemd version is used, that changes how the MAC addresses of Linux network bridge devices are calculated:

    MACAddressPolicy=persistent was extended to set MAC addresses based on the device name. Previously addresses were only based on the ID_NET_NAME_* attributes, which meant that interface names would never be generated for virtual devices. Now a persistent address will be generated for most devices, including in particular bridges.
    
    -- https://www.freedesktop.org/software/systemd/man/systemd.net-naming-scheme.html#v241
    

    A unique and persistent MAC address is now calculated using the bridge name and the unique machine-id (/etc/machine-id), which is generated at install time.

    Please either ensure that any ebtable or similar rules that use the previous bridge MAC-Address are updated or configure the desired bridge MAC-Address explicitly, by switching to ifupdown2 and adding hwaddress to the respective entry in /etc/network/interfaces.

    Thanked by (2)Not_Oles miu

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • Not_OlesNot_Oles Hosting ProviderContent Writer
    edited July 2021

    @miu said: i not tried (change bridge-ports from ens 3 to none)

    @miu I ran a little test.

    My server (running Debian sid, not Proxmox) had bridge-ports none.

    Just for fun I tried changing the "none" to "enp7s0", which is the name of my server's primary interface, corresponding to "eno3" on your server.

    systemctl restart networking following this change broke networking, which was the hoped-for result.

    I reverted via the rescue system.

    Conceivably, my little test supports the possibility that changing "eno3" to "none" on your system might have fixed the network problem?

    Does anybody know how likely it is for my clueless guess to be right?

    Friendly greetings from Mexico! :)

    I hope everyone gets the servers they want!

  • edited July 2021

    @Not_Oles said:

    @miu said: i not tried (change bridge-ports from ens 3 to none)

    I ran a little test.

    My server (running Debian sid, not Proxmox) had bridge-ports none.

    Just for fun I tried changing the "none" to "enp7s0", which is the name of my server's primary interface, corresponding to "eno3" on your server.

    systemctl restart networking following this change broke networking, which was the hoped-for result.

    I reverted via the rescue system.

    Conceivably, my little test supports the possibility that changing "eno3" to "none" on your system might have fixed the network problem?

    Does anybody know how likely it is for my clueless guess to be right?

    Friendly greetings from Mexico! :)

    your mixing things up here old friend.

    he does not try to actually route (additional) IPs yet. he sets the initial bridge vmbr0 on the main IP (therefore ens3 config is empty/manual) and with that he MUST have something bridging to that interface - because it does not get an IP and therefore nothing can be routed to it.

    if you'd later on add an additional bridge for a routed config you also do not route towards ens3 then but vmbr0. small but important difference to what you are suggesting.

    TL;DR; I suggest to try and not mix up 'bridged config' and 'routed config' just yet ;-)

    PS: with soyoustart/OVH the routed config does not work well anyway. but you'll get virtual macs for all additional IPs individually (even if bought in subnets). so the bridged config is correct and the better choice.

    mine looks exactly the same, but I haven't looked at proxmox 7 yet and also always only install on top of debian 🤷‍♂️

    Thanked by (2)Not_Oles miu
  • bdlbdl OG
    edited July 2021

    I had this exact issue with 7 on a Hosthatch VPS.

    MaxKVM VPSes upgraded without issue.

    I ended up rolling back to 6.4, definitely was a bug in pve 7. Didn't seem related to the MAC address changes in bullseye.

  • hey @Not_Oles why not upgrade now and worry later

  • miumiu
    edited July 2021

    Good day good people! :)

    UPDATE from another forum (credit @ polluxtroy3758) = what is in journalctl log on OVH-brands PVE7 server:

    bridge port does not exist

    -- Boot 1d7b66b4b10e4870b9c19a354be86dcf --
    Jul 28 10:00:14 pve systemd[1]: Starting Network initialization...
    Jul 28 10:00:14 pve networking[617]: networking: Configuring network interfaces
    Jul 28 10:00:16 pve networking[675]: error: vmbr0: bridge port enp1s0 does not exist
    Jul 28 10:00:16 pve systemd[1]: Finished Network initialization.
    -- Boot fd6fb88a3e094183baf8a14c81f98d36 --
    Jul 28 10:48:46 pve systemd[1]: Starting Network initialization...
    Jul 28 10:48:46 pve networking[622]: networking: Configuring network interfaces
    Jul 28 10:48:48 pve networking[718]: error: vmbr0: bridge port enp1s0 does not exist
    Jul 28 10:48:48 pve systemd[1]: Finished Network initialization.
    
    Thanked by (1)Not_Oles

    Good day and Goodbye

  • miumiu
    edited July 2021

    and here is (mine) working network log from SYS with PVE6.4 (installed from ISO) with bridged (above mentioned) config, bridge port found and works as should:

    -- Logs begin at Tue 2021-07-27 20:23:58 CEST, end at Wed 2021-07-28 13:49:12 CEST. --
    Jul 27 20:24:03 pve systemd[1]: Starting Raise network interfaces...
    Jul 27 20:24:04 pve ifup[1059]: Waiting for vmbr0 to get ready (MAXWAIT is 2 seconds).
    Jul 27 20:24:04 pve systemd[1]: Started Raise network interfaces.

    Thanked by (1)Not_Oles

    Good day and Goodbye

  • no response from Proxmox staff on it yet (on their forum where i reported issue), when will be then i will update this thread also

    Thanked by (2)Abdullah Not_Oles

    Good day and Goodbye

  • Not_OlesNot_Oles Hosting ProviderContent Writer

    @miu Thanks for the updates! Especially with the logs. Could you please post or PM me your Proxmox forum post link?

    I hope everyone gets the servers they want!

  • miumiu
    edited July 2021

    @Not_Oles said:
    @miu Thanks for the updates! Especially with the logs. Could you please post or PM me your Proxmox forum post link?

    Hello Oles (Tom)
    SURE (hopefully i do not break any rules when post here URL from another forum)
    https://forum.proxmox.com/threads/proxmox-node-on-ovh-soyoustart-unreachable-after-each-reboot.93470/

    I will be glad when this may help to someone in future

    For now as i have mentioned i have resolved it so that i have reinstalled machine from ISO 6.4 (no time, not own ability fix issue)

    Thanked by (2)Not_Oles mikho

    Good day and Goodbye

  • Not_OlesNot_Oles Hosting ProviderContent Writer

    @miu Thanks again! Watching the thread on the Proxmox forum. :)

    Thanked by (1)miu

    I hope everyone gets the servers they want!

  • Nothing for thanking ;-) Unfortunately non-paid (customer) support there is not very well (slow and sometime w/o response of staff ever).. But i understand that for complete and good support i should purchase paid licence

    Good day and Goodbye

  • @miu said:
    Good day good people! :)

    UPDATE from another forum (credit @ polluxtroy3758) = what is in journalctl log on OVH-brands PVE7 server:

    bridge port does not exist

    -- Boot 1d7b66b4b10e4870b9c19a354be86dcf --
    Jul 28 10:00:14 pve systemd[1]: Starting Network initialization...
    Jul 28 10:00:14 pve networking[617]: networking: Configuring network interfaces
    Jul 28 10:00:16 pve networking[675]: error: vmbr0: bridge port enp1s0 does not exist
    Jul 28 10:00:16 pve systemd[1]: Finished Network initialization.
    -- Boot fd6fb88a3e094183baf8a14c81f98d36 --
    Jul 28 10:48:46 pve systemd[1]: Starting Network initialization...
    Jul 28 10:48:46 pve networking[622]: networking: Configuring network interfaces
    Jul 28 10:48:48 pve networking[718]: error: vmbr0: bridge port enp1s0 does not exist
    Jul 28 10:48:48 pve systemd[1]: Finished Network initialization.
    

    from that the network device simply is not ready when the networkis tried to be brought up. could be that when you use dhcp it only fires at later point in time because of different dependencies and therefore does not see the same issues.

    might even be unrelated to OVH/SYS specifics, but could also be related to drivers and whatnot... interesting. a workaround could probably be to fiddle with systemd and have it bring up networking only later.

    Thanked by (1)miu
  • miumiu
    edited July 2021

    @Falzo said:

    @miu said:
    Good day good people! :)

    UPDATE from another forum (credit @ polluxtroy3758) = what is in journalctl log on OVH-brands PVE7 server:

    bridge port does not exist

    -- Boot 1d7b66b4b10e4870b9c19a354be86dcf --
    Jul 28 10:00:14 pve systemd[1]: Starting Network initialization...
    Jul 28 10:00:14 pve networking[617]: networking: Configuring network interfaces
    Jul 28 10:00:16 pve networking[675]: error: vmbr0: bridge port enp1s0 does not exist
    Jul 28 10:00:16 pve systemd[1]: Finished Network initialization.
    -- Boot fd6fb88a3e094183baf8a14c81f98d36 --
    Jul 28 10:48:46 pve systemd[1]: Starting Network initialization...
    Jul 28 10:48:46 pve networking[622]: networking: Configuring network interfaces
    Jul 28 10:48:48 pve networking[718]: error: vmbr0: bridge port enp1s0 does not exist
    Jul 28 10:48:48 pve systemd[1]: Finished Network initialization.
    

    from that the network device simply is not ready when the networkis tried to be brought up. could be that when you use dhcp it only fires at later point in time because of different dependencies and therefore does not see the same issues.

    might even be unrelated to OVH/SYS specifics, but could also be related to drivers and whatnot... interesting. a workaround could probably be to fiddle with systemd and have it bring up networking only later.

    Additionally - bridged config with DHCP also does not work on OVH with PVE7 (ISO installation)

    auto lo
    iface lo inet loopback
    
    iface eno3 inet manual
    
    auto vmbr0
    iface vmbr0 inet dhcp
      bridge-ports eno3
      bridge-stp off
      bridge-fd 0
    

    (on 6.4 ISO works)

    Thanked by (1)Falzo

    Good day and Goodbye

  • @miu said: Additionally - bridged config with DHCP also does not work on OVH with PVE7 (ISO installation)

    oh, okay, I misread that and thought this at least would work. however, still thinking systemd simply tries to start networking to early then while the real device is not ready.

  • Thank you for all your contribution, time and suggestions, appreciated by me!

    Good day and Goodbye

  • ialexpwialexpw OGServices Provider

    @miu said:
    Thank you for all your contribution, time and suggestions, appreciated by me!

    Had this exact problem when I moved to PVE7 (upgraded from 6.4). I messed with my interfaces for a while as the network appeared to be up after a restart for ~5 minutes before dying.

    I'll probably have a note somewhere as to what happened on my side as I cannot remember what I changed, my interfaces file is as below now though;

    auto lo
    iface lo inet loopback
    
    auto eth0
    iface eth0 inet manual
    
    auto vmbr0
    iface vmbr0 inet static
            address 91.xx.xx.xx/xx
            gateway 91.xx.xx.xx
            bridge-ports eth0
            bridge-stp off
            bridge-fd 0
    
    auto vmbr1
    iface vmbr1 inet static
            address 10.10.10.1/24
            bridge-ports none
            bridge-stp off
            bridge-fd 0
    

    Which seems to be working - will update if I can find the logs from before! This is a dedi with Skynode.

    Thanked by (3)ehab miu Not_Oles

    Syuh - sftp, rsync & rclone accessible storage from 50GB to TBs.

  • edited July 2021

    @ialexpw said:
    I'll probably have a note somewhere as to what happened on my side as I cannot remember what I changed

    i can help you remember

    Thanked by (1)ialexpw
Sign In or Register to comment.