BGP routes not installed into Linux kernel routing table

terrorgenterrorgen OG
in Help

Hello LESbians who are proficient in BGP,

I am observing that my BIRD is not exporting any routes to the kernel.

Looks like it is receiving routes from the upstream alright:

~ terrorgen@awesomebox
❯ sudo birdc s p a upstream
BIRD 2.14 ready.
Name       Proto      Table      State  Since         Info
upstream BGP        ---        up     00:18:11.949  Established
  BGP state:          Established
    Neighbor address: 2001:db8:0:2::1
    Neighbor AS:      65000
    Local AS:         65001
    Neighbor ID:      1.2.3.4
    Local capabilities
      Multiprotocol
        AF announced: ipv6
      Route refresh
      Graceful restart
        Restart time: 120
        AF supported: ipv6
        AF preserved:
      4-octet AS numbers
      Enhanced refresh
      Long-lived graceful restart
    Neighbor capabilities
      Multiprotocol
        AF announced: ipv6
      Route refresh
      Graceful restart
      4-octet AS numbers
    Session:          external multihop AS4
    Source address:   2001:db8:3::30
    Hold timer:       75.387/90
    Keepalive timer:  21.792/30
  Channel ipv6
    State:          UP
    Table:          master6
    Preference:     100
    Input filter:   (unnamed)
    Output filter:  (unnamed)
    Routes:         195136 imported, 4 exported, 195134 preferred
    Route change stats:     received   rejected   filtered    ignored   accepted
      Import updates:         517095          0          0          0     517095
      Import withdraws:        70578          0        ---          0      70578
      Export updates:         517100     517093          3        ---          4
      Export withdraws:        70578        ---        ---        ---          0
    BGP Next hop:   2001:db8:3::30
    IGP IPv6 table: master6

But upon inspecting the individual routes, many of them are marked "!". According to the manual, that means "failure to install to kernel".

~ terrorgen@awesomebox 7s
❯ sudo birdc s r table master6
BIRD 2.14 ready.
Table master6:
::/0                 unicast [upstream 10:44:46.318 from 2001:db8:0:2::1] ! (100/?) [AS65000i]
        via 2001:db8:3::1 on enp3s0
2620:129:2000::/42   unicast [upstream 10:44:48.950 from 2001:db8:0:2::1] ! (100/?) [AS27016i]
        via 2001:db8:3::1 on enp3s0
2804:505c::/32       unicast [upstream 10:44:49.979 from 2001:db8:0:2::1] ! (100/?) [AS268384i]
        via 2001:db8:3::1 on enp3s0
2804:385c:a200::/40  unicast [upstream 10:44:49.960 from 2001:db8:0:2::1] ! (100/?) [AS262687i]
        via 2001:db8:3::1 on enp3s0
2804:d84:f000::/36   unicast [upstream 10:44:49.333 from 2001:db8:0:2::1] ! (100/?) [AS28171i]
        via 2001:db8:3::1 on enp3s0
2a04:26c0::/29       unicast [upstream 10:44:50.916 from 2001:db8:0:2::1] ! (100/?) [AS60532i]
        via 2001:db8:3::1 on enp3s0
2804:5a64::/34       unicast [upstream 10:44:50.720 from 2001:db8:0:2::1] ! (100/?) [AS268770i]
        via 2001:db8:3::1 on enp3s0
2804:5d3c:c000::/34  unicast [upstream 10:44:50.014 from 2001:db8:0:2::1] ! (100/?) [AS268959i]
        via 2001:db8:3::1 on enp3s0
2001:44b8:4040::/48  unicast [upstream 10:44:46.961 from 2001:db8:0:2::1] * (100/?) [AS7545i]
        via 2001:db8:3::1 on enp3s0
2803:bc40:8155::/48  unicast [upstream 10:44:49.180 from 2001:db8:0:2::1] ! (100/?) [AS52468i]
        via 2001:db8:3::1 on enp3s0
240a:aded::/32       unicast [upstream 10:44:48.046 from 2001:db8:0:2::1] * (100/?) [AS146215i]
        via 2001:db8:3::1 on enp3s0
2402:3a80:19ec::/46  unicast [upstream 10:44:47.367 from 2001:db8:0:2::1] * (100/?) [AS38266i]
        via 2001:db8:3::1 on enp3s0
2a00:1d58:f812::/48  unicast [upstream 10:44:50.213 from 2001:db8:0:2::1] ! (100/?) [AS47524?]
        via 2001:db8:3::1 on enp3s0
2620:12f:f000::/44   unicast [upstream 10:44:49.021 from 2001:db8:0:2::1] ! (100/?) [AS43?]
        via 2001:db8:3::1 on enp3s0
2800:bf0:2206::/48   unicast [upstream 10:44:48.906 from 2001:db8:0:2::1] ! (100/?) [AS52257i]
        via 2001:db8:3::1 on enp3s0
2001:4430:c20e::/47  unicast [upstream 10:44:46.931 from 2001:db8:0:2::1] * (100/?) [AS17853i]
        via 2001:db8:3::1 on enp3s0
...

Looking at systemd-journald this is the message being generated (restarted BIRD just to get the full message):

~ terrorgen@awesomebox 11s
❯ sudo systemctl restart bird2; sudo journalctl -fu bird2
Dec 04 10:50:36 awesomebox bird[773]: Shutdown completed
Dec 04 10:50:36 awesomebox systemd[1]: bird2.service: Deactivated successfully.
Dec 04 10:50:36 awesomebox systemd[1]: Stopped BIRD Internet Routing Daemon.
Dec 04 10:50:36 awesomebox systemd[1]: bird2.service: Consumed 10.171s CPU time, received 8.5M IP traffic, sent 17.9M IP traffic.
Dec 04 10:50:36 awesomebox systemd[1]: Starting BIRD Internet Routing Daemon...
Dec 04 10:50:36 awesomebox systemd[1]: Started BIRD Internet Routing Daemon.
Dec 04 10:50:36 awesomebox bird[1047]: Started
Dec 04 10:50:38 awesomebox bird[1047]: Netlink: File exists
Dec 04 10:50:38 awesomebox bird[1047]: Kernel dropped some netlink messages, will resync on next scan.
Dec 04 10:50:39 awesomebox bird[1047]: Kernel dropped some netlink messages, will resync on next scan.
Dec 04 10:50:39 awesomebox bird[1047]: Kernel dropped some netlink messages, will resync on next scan.
Dec 04 10:50:39 awesomebox bird[1047]: Kernel dropped some netlink messages, will resync on next scan.
(multiple identical messages)
Dec 04 10:50:41 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:41 awesomebox bird[1047]: ...
Dec 04 10:50:42 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:42 awesomebox bird[1047]: ...
Dec 04 10:50:43 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:43 awesomebox bird[1047]: ...
Dec 04 10:50:44 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:44 awesomebox bird[1047]: ...
Dec 04 10:50:45 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:45 awesomebox bird[1047]: ...
Dec 04 10:50:46 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:46 awesomebox bird[1047]: ...
Dec 04 10:50:47 awesomebox bird[1047]: Netlink: No route to host
Dec 04 10:50:47 awesomebox bird[1047]: ...

Wonder if you have any pointers?

Note: 2001:db8:3::30 is my VPS address, upstream is using multihop. Immediate nexthop router is 2001:db8:3::1 and BGP router is at 2001:db8:0:2::1.

The all seeing eye sees everything...

Comments

  • GeekWandererGeekWanderer

    How does the kernel section of your bird.conf look?

  • terrorgenterrorgen OG

    @GeekWanderer said:
    How does the kernel section of your bird.conf look?

    quite boiler plate...

    protocol kernel {
        metric 1024;
        ipv6 {
                import none;
                export where source ~ [RTS_BGP]; 
        };
        merge paths yes;
}

    The all seeing eye sees everything...

  • NeoonNeoon OG
    edited December 2023

    bird has multiple tables, you have to explicitly tell it to do that.

    I am not an expert in bird though.
    Maybe @FHR can help.

    Free NAT KVM | Free NAT LXC | Bobr
    ITS WEDNESDAY MY DUDES

  • terrorgenterrorgen OG

    there is a default master4 and master6 table. If you don't specify the table in the config the default tables are used.

    That being said, this is not the first nor the only BGP session I have. All sessions are written with the snippet above and there isn't a problem.

    The all seeing eye sees everything...

  • FHRFHR Hosting ProviderOG

    Is this actually a valid route installed in the kernel (aka visible in ip -6 r)?
    2001:db8:3::1 on enp3s0 If netlink is complaining about no route to host while trying to install a route it sort of points towards that not being the case; i.e. Bird is trying to insert a route that is not resolveable by the kernel.

    SkylonHost.com High Bandwidth European Cloud KVM | AS202297

  • terrorgenterrorgen OG

    that is the statically configured default route:
    default via 2001:db8:3::1 dev enp3s0 proto static metric 1024 pref medium

    The all seeing eye sees everything...

  • c1vhostingc1vhosting

    Pls share your entire bird config

    C1V Hosting: Low cost Italian Cloud & Data Center Solutions 🚀 | Contact us for special offers. | Our deals on Telegram

  • terrorgenterrorgen OG 
    router id 4.3.2.1;
log syslog all;

function is_self_net() -> bool {
        return net ~ 2001:db8:abcd::/48;
}

protocol device {
        scan time 10;
}

protocol static sss {
        ipv6;
        route 2001:db8:abcd::/48 unreachable;
}

protocol kernel {
        metric 1024;
        ipv6 {
                import none;
                export where source ~ [RTS_BGP]; 
        };
        merge paths yes;
}

template bgp peers
{
        local as 65001;
        ipv6 {
                import filter {
                        if (is_self_net()) then reject;
                        accept;
                };
                export where proto = "sss";
        };
        graceful restart on;
}

protocol bgp upstream from peers
{
        multihop;
        source address 2001:db8:3::30;
        neighbor 2001:db8:0:8::1 as 65000;
}

    The all seeing eye sees everything...

  • c1vhostingc1vhosting
    edited December 2023

    protocol kernel {
    ipv6 {
    import all;
    export all; };
    }

    Thanked by (1)Otus9051

    C1V Hosting: Low cost Italian Cloud & Data Center Solutions 🚀 | Contact us for special offers. | Our deals on Telegram

  • terrorgenterrorgen OG
    edited December 2023

    @c1vhosting said:
    protocol kernel {
    ipv6 {
    import all;
    export all; };
    }

    Why?
    It is potentially dangerous to just import and export all. Plus, it doesn't make any difference to the problem anyways.

    Thanked by (1)Otus9051

    The all seeing eye sees everything...

  • dwightdwight

    @terrorgen said:
    Why?
    It is potentially dangerous to just import and export all. Plus, it doesn't make any difference to the problem anyways.

    kernel export all isn't that dangerous. Anyway, I guess this article will help you: https://blog.fhrnet.eu/2018/08/15/bird-unreachable-routes-on-multihop-bgp-sessions/

  • terrorgenterrorgen OG

    @dwight said:

    @terrorgen said:
    Why?
    It is potentially dangerous to just import and export all. Plus, it doesn't make any difference to the problem anyways.

    kernel export all isn't that dangerous. Anyway, I guess this article will help you: https://blog.fhrnet.eu/2018/08/15/bird-unreachable-routes-on-multihop-bgp-sessions/

    Thanks but unfortunately, that doesn't describe my problem though. I can reach the multi hop router fine.

    The all seeing eye sees everything...

  • GeekWandererGeekWanderer

    What did you determine with this? Find a resolution?

  • terrorgenterrorgen OG

    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Thanked by (1)GeekWanderer

    The all seeing eye sees everything...

  • AuroraZeroAuroraZero ModeratorHosting Provider

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    Free Hosting at YetiNode | Cryptid Security | URL Shortener | LaunchVPS | ExtraVM | Host-C | In the Node, or Out of the Loop?

  • terrorgenterrorgen OG

    @AuroraZero said:

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    IKR?

    The all seeing eye sees everything...

  • AuroraZeroAuroraZero ModeratorHosting Provider

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    IKR?

    How did that even work?

    Free Hosting at YetiNode | Cryptid Security | URL Shortener | LaunchVPS | ExtraVM | Host-C | In the Node, or Out of the Loop?

  • terrorgenterrorgen OG

    @AuroraZero said:

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    IKR?

    How did that even work?

    Might need someone who truly understands the Linux kernel network stack to explain it to us.

    The all seeing eye sees everything...

  • AuroraZeroAuroraZero ModeratorHosting Provider

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    IKR?

    How did that even work?

    Might need someone who truly understands the Linux kernel network stack to explain it to us.

    What I am saying is it should not have even partially worked.

    Someone here should know I hope because it may not even be replacable.

    Thanked by (1)terrorgen

    Free Hosting at YetiNode | Cryptid Security | URL Shortener | LaunchVPS | ExtraVM | Host-C | In the Node, or Out of the Loop?

  • terrorgenterrorgen OG

    @AuroraZero said:

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    IKR?

    How did that even work?

    Might need someone who truly understands the Linux kernel network stack to explain it to us.

    What I am saying is it should not have even partially worked.

    Someone here should know I hope because it may not even be replacable.

    Oh, yes, very true. Why does the kernel choose to accept some prefixes and deny some other, when their nexthops are the same?

    The all seeing eye sees everything...

  • AuroraZeroAuroraZero ModeratorHosting Provider

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:

    @AuroraZero said:

    @terrorgen said:
    Somehow I have to set the next hop router's address to "onlink" for the routes to install, which is weird. The next hop router is reachable anyways without setting this.

    Lmfao wth man?

    IKR?

    How did that even work?

    Might need someone who truly understands the Linux kernel network stack to explain it to us.

    What I am saying is it should not have even partially worked.

    Someone here should know I hope because it may not even be replacable.

    Oh, yes, very true. Why does the kernel choose to accept some prefixes and deny some other, when their nexthops are the same?

    No idea man and I have not heard of one acting that strange before. Is it a stock kernel?

    Free Hosting at YetiNode | Cryptid Security | URL Shortener | LaunchVPS | ExtraVM | Host-C | In the Node, or Out of the Loop?

Sign In or Register to comment.

© LowEndSpirit 2024

ipv6 ready