Hosthatch Stockholm IPv6 down

bjobjo OG
edited August 2023 in Outages

Since one hour, Smokeping was complaing my Storage VPS is down. Rebooted it, IPv4 is up again, but IPv6 still down. Interestingly the controlpanel shows 2 /64 prefixes for the server now, bot none of them work. Even the default gw fe80::1 is not pingable.

Tagged:
«13

Comments

  • dosaidosai OG

    Same here.

  • MumblyMumbly
    edited August 2023

    Same here.

    @bjo said: Interestingly the controlpanel shows 2 /64 prefixes for the server now

    That too.
    Old GleSYS AB /64 and new Prinode AB /64 subnet.

  • nyamenknyamenk

    confirmed. 25min downtime according to hetrixtools. ping google from putty no response, but nextcloud is normal

    Sultan Muda - Amazon Store

  • bjobjo OG
    edited August 2023

    Old /64 seems up again OBE.

  • MumblyMumbly

    Not here. Still down.

    What does it mean "OBE"?

  • TheDPTheDP OG
    edited August 2023

    @Mumbly said: What does it mean "OBE"?

    The upstream maybe - Obenetwork AB.

    EDIT: Or maybe not, since that would be Portlane/GleSYS :smiley:

    For domain registrations, create an account at Dynadot (ref) and spend $9.99 within 48 hours to receive $5 DynaDollars!
    Looking for cost-effective Managed/Anycast/DDoS-Protected/Geo DNS Services? Try ClouDNS (aff).

  • bjobjo OG
    edited August 2023

    The upstream maybe - Obenetwork AB.

    EDIT: Or maybe not, since that would be Portlane/GleSYS :smiley:

    It's both?

    From Netcup:

      3.|-- ae13-1386.RT.EQX.FKT.DE.r  0.0%    10    3.5   3.6   3.4   4.3   0.2
  4.|-- RT.TC1.STO.SE.retn.net     0.0%    10   22.8  23.6  22.7  29.3   2.0
  5.|-- GW-ObeNetwork.retn.net     0.0%    10   33.2  33.1  33.0  33.5   0.2
  6.|-- ae1-19.jhv-bgc.obe.net     0.0%    10   34.3  34.4  34.2  34.7   0.2
  7.|-- eth1-49-11.hgr-vbdc.obe.n  0.0%    10   35.1  35.2  35.1  35.5   0.1
  8.|-- eth1-52-demarc.hgr-vbdc.o  0.0%    10   37.2  59.2  35.0 127.4  35.6
  9.|-- 2a00:1a28:1157:xyz     0.0%    10   34.6  34.8  34.6  35.2   0.2

    And from my FTTH:

      8.|-- ddf-b3-link.ip.twelve99.n  0.0%    10    7.0   7.3   6.7   8.6   0.5
  9.|-- ddf-b4-v6.ip.twelve99.net  0.0%    10   12.8   8.6   7.8  12.8   1.5
 10.|-- adm-b10-v6.ip.twelve99.ne 10.0%    10   10.7  10.8  10.5  11.4   0.3
 11.|-- portlane-ic316381-adm-b2. 30.0%    10   10.7  11.1  10.7  12.1   0.5
 12.|-- be-4.cr1.mal4.se.portlane 30.0%    10   20.0  20.2  20.0  20.8   0.3
 13.|-- be-5.cr2.sto1.se.portlane 50.0%    10   26.6  26.7  26.6  26.9   0.1
 14.|-- be-1.pe3.sto1.se.portlane 40.0%    10   26.6  26.9  26.6  27.2   0.2
 15.|-- 2a00:1a28:1:3091::2        0.0%    10  434.7 161.3  32.8 566.7 193.0
 16.|-- 2a00:1a28:1157:xyz     0.0%    10   32.4  32.7  32.4  33.0   0.2

    And the "new" address from Netcup:

      5.|-- 2a00:11c0:47:1:47::212     0.0%    10   22.3  22.4  22.3  22.4   0.1
  6.|-- xe-0-1-1-amsix.ams-evo.ob  0.0%    10   23.2  23.6  23.1  25.8   0.8
  7.|-- ae2-10.mma-vg4.obe.net     0.0%    10   35.5  35.7  35.5  35.9   0.1
  8.|-- ae4-5.jhv-bgc.obe.net      0.0%    10   45.6  45.8  45.6  45.9   0.1
  9.|-- eth1-49-11.hgr-vbdc.obe.n  0.0%    10   46.6  46.6  46.4  47.0   0.2
 10.|-- eth1-52-demarc.hgr-vbdc.o  0.0%    10  453.9 183.0  50.3 453.9 123.5
 11.|-- 2a0e:dc0:2:b81:xyz         0.0%    10   46.1  46.1  45.7  46.5   0.2

    And from FTTH (really strange as my ISP (AS50629) also peers at Amsix:

      8.|-- ddf-b3-link.ip.twelve99.n  0.0%    10    7.2   7.4   7.0   8.0   0.3
  9.|-- ffm-bb1-v6.ip.twelve99.ne 20.0%    10   11.1  11.8  10.8  17.2   2.2
 10.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0
 11.|-- level3-ic319606-ffm-b11.i 80.0%    10   10.6  11.0  10.6  11.3   0.4
 12.|-- 2001:1900:2::3:102         0.0%    10   58.6  38.2  32.5  58.6   8.7
 13.|-- as3356-1.arn.as54104.net   0.0%    10   33.0  32.9  32.4  33.5   0.3
 14.|-- ae2-19.kis-kg.obe.net      0.0%    10   38.6  38.8  38.3  39.1   0.2
 15.|-- ae1-17.bmm-sk1.obe.net     0.0%    10   38.6  38.9  38.5  39.8   0.4
 16.|-- ae1-19.jhv-bgc.obe.net     0.0%    10   39.2  39.4  38.8  42.7   1.2
 17.|-- eth1-49-11.hgr-vbdc.obe.n  0.0%    10   40.2  39.9  39.4  40.8   0.4
 18.|-- eth1-52-demarc.hgr-vbdc.o  0.0%    10   57.4  66.8  41.0 125.6  32.0
 19.|-- 2a0e:dc0:2:b81b::xyz         0.0%    10   39.7  39.6  39.0  41.3   0.7
    Thanked by (1)TheDP
  • TheDPTheDP OG

    Not sure if this is related, but sharing since it's Stockholm.

    Dear Customer,

    We want to inform you of a scheduled maintenance that affects your active service(s) listed below.

    Maintenance start: 2023-08-22 21:30 CEST
    Maintenance end: 2023-08-22 23:30 CEST
    Expected impact: 45-90 minutes of downtime and reboot

    Services affected:
    panopticon (176.126.x.x) - Stockholm

    During this maintenance, we will safely shut down your VM, perform the maintenance, and boot it back online. You should see improved network performance after this has been completed.

    Sincerely,
    HostHatch Support Team

    Thanked by (1)FrankZ

    For domain registrations, create an account at Dynadot (ref) and spend $9.99 within 48 hours to receive $5 DynaDollars!
    Looking for cost-effective Managed/Anycast/DDoS-Protected/Geo DNS Services? Try ClouDNS (aff).

  • MumblyMumbly
    edited August 2023

    It is related. When network came back after the maintenance, IPv6 didn't work anymore.

    btw. nice vps name.

    Thanked by (1)TheDP
  • bjobjo OG

    @Mumbly said:
    It is related. When network came back after the maintenance, IPv6 didn't work anymore.

    Well, it's still 21.08.23 CEST, not 22.08. The maintenance according to the announcement should be tomorrow.

  • MumblyMumbly
    edited August 2023

    ah, crap... you're correct

  • bjobjo OG

    And the outage began around 15:30 CEST or so, maybe @hosthatch srambled the dates and timezones?

  • TheDPTheDP OG

    @bjo said:
    And the outage began around 15:30 CEST or so, maybe @hosthatch srambled the dates and timezones?

    Or this upcoming maintenance is to address the issue(s).

    For domain registrations, create an account at Dynadot (ref) and spend $9.99 within 48 hours to receive $5 DynaDollars!
    Looking for cost-effective Managed/Anycast/DDoS-Protected/Geo DNS Services? Try ClouDNS (aff).

  • bjobjo OG

    I did not have any issues before.

  • dosaidosai OG

    Ipv6 working for me now.

  • MumblyMumbly

    @bjo said: Old /64 seems up again OBE.

    @dosai said: Ipv6 working for me now.

    Anyone tryed new one too?
    My old is still down and I am wondering if I should switch to the new one.

  • bjobjo OG

    For me both are working, but the gateway is from the old prefix.

    Thanked by (1)Mumbly
  • bjobjo OG

    Now down again since 22.12h CEST, within the announced timeslot.

  • bjobjo OG
    edited August 2023

    "boot it back online" did not happen. Started it via control panel, IPv4 works, IPv6 are broken. IPv6-gateway is still 2a00:1a28:1157::1.

    Edit: Both up again.

  • MumblyMumbly
    edited August 2023

    In my case IPv6 never came back since first outage from 2 days ago.
    Before I opened ticket I've redone everything from the beginning. No luck.

    Can you post ip -6 route output please (blank out part of your subnet).

    ::1 dev lo proto kernel metric 256 pref medium
2a00:1a28:1157:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::1 dev eth0 proto static metric 1024 pref medium
  • dosaidosai OG

    @Mumbly said:

    @bjo said: Old /64 seems up again OBE.

    @dosai said: Ipv6 working for me now.

    Anyone tryed new one too?
    My old is still down and I am wondering if I should switch to the new one.

    I didn't configure network, it auto switched to the new ipv6.

    Thanked by (1)Mumbly
  • MumblyMumbly
    edited August 2023

    @dosai said:
    I didn't configure network, it auto switched to the new ipv6.

    Can you post "ip -6 route" output please (blank out part of your subnet).

  • bjobjo OG

    @Mumbly said:

    ::1 dev lo proto kernel metric 256 pref medium
2a00:1a28:1157:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::1 dev eth0 proto static metric 1024 pref medium

    2a00:1a28:1157:yxz::/64 dev eth0  metric 256 
fe80::/64 dev eth0  metric 256 
default via fe80::1 dev eth0  metric 1024 
multicast ff00::/8 dev eth0  metric 256

    Got a reply to my ticket from the first outage which asked me to reconfigure according to https://docs.hosthatch.com/networking/#ipv6-configuration

    Does fe80::1 reply on your box?

    Thanked by (1)Mumbly
  • bjobjo OG

    @dosai said:
    I didn't configure network, it auto switched to the new ipv6.

    How? The docs says

    We do not support automatic (SLAAC) or dynamic (DHCPv6) IPv6 assignment
  • MumblyMumbly
    edited August 2023

    @bjo said: Got a reply to my ticket from the first outage which asked me to reconfigure according to https://docs.hosthatch.com/networking/#ipv6-configuration

    Does fe80::1 reply on your box?

    Yes, that's my IPv6 setup on every HH vps (and it worked also on .se until this outage from 2days ago)
    Reply?

    "If you choose to enable IPv6 when you first create your server, it will automatically be configured via cloud-init. However, manual configuration is needed if you choose to enable it after your server has been created, or if you install your OS via ISO."

    I tryed it also with clean reinstall from the template. It show initial ::1 IP in interface set by template, but still no luck.

  • bjobjo OG

    I've deleted the old one now and activated the new one from 2a0e:dc0:2::/48

      1.|-- 2a00:1a28:1157::1          0.0%    10   18.1  19.8   4.4  72.7  19.6
  2.|-- fdca:f5d5:c3aa:cb2d::1     0.0%    10    8.0  14.5   3.4  58.5  16.4
  3.|-- be-4-706.pe3.sto1.se.port 10.0%    10    1.6   1.6   1.5   1.7   0.1
  4.|-- be-6.cr2.sto2.se.portlane  0.0%    10    2.4   2.0   1.9   2.4   0.1
  5.|-- 2001:4860:1:1::85e         0.0%    10    2.1   2.1   2.0   2.3   0.1
  6.|-- 2a00:1450:8110::1          0.0%    10    1.6   1.5   1.4   1.6   0.1
  7.|-- 2001:4860:0:1::5924        0.0%    10    1.5   1.5   1.4   1.7   0.1
  8.|-- 2001:4860:0:1344::5        0.0%    10    1.3   1.8   1.3   4.4   1.0
  9.|-- 2001:4860:0:134b::1        0.0%    10    2.7   2.7   2.4   3.4   0.3
 10.|-- 2001:4860:0:1::5123        0.0%    10    1.8   1.9   1.8   2.1   0.1
 11.|-- arn11s10-in-x03.1e100.net  0.0%    10    1.5   1.6   1.5   1.7   0.1
  • MumblyMumbly

    I tryed both, no luck and I am out of ideas as everything seems in order at my end.
    I will reinstall it once again via ISO (that's template now) once I come home and this is it ...at the mercy of their support.

    root@box:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 31.220.xx.xxx  netmask 255.255.255.0  broadcast 31.220.15.255
        inet6 2a00:1a28:1157:xxxx::1  prefixlen 64  scopeid 0x0<global>
        inet6 2a00:1a28:1157:xxxx:3:3:3:3  prefixlen 64  scopeid 0x0<global>
        inet6 2a00:1a28:1157:xxxx:2:2:2:2  prefixlen 64  scopeid 0x0<global>
        inet6 2a00:1a28:1157:xxxx:1:1:1:1  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::220:56ff:fe09:c87f  prefixlen 64  scopeid 0x20<link>
        ether 00:20:56:09:c8:7f  txqueuelen 1000  (Ethernet)
        RX packets 218  bytes 18916 (18.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 175  bytes 19774 (19.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 15  bytes 1452 (1.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15  bytes 1452 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@box:~# curl ipv6.google.com
curl: (7) Failed to connect to ipv6.google.com port 80: No route to host

root@box:~# ip -6 route
::1 dev lo proto kernel metric 256 pref medium
2a00:1a28:1157:xxxx::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::1 dev eth0 metric 1024 onlink pref medium
  • bjobjo OG

    I don't see the new prefix in your output.

  • MumblyMumbly

    Because it makes no difference. I tryed all combinations.

    root@box:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 31.220.xxx.xxx  netmask 255.255.255.0  broadcast 31.220.15.255
        inet6 2a0e:dc0:2:xxxx::2  prefixlen 64  scopeid 0x0<global>
        inet6 2a00:1a28:1157:xxxx::1  prefixlen 64  scopeid 0x0<global>
        inet6 2a0e:dc0:2:xxxx::1  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::220:56ff:fe09:c87f  prefixlen 64  scopeid 0x20<link>
        inet6 2a00:1a28:1157:xxxx:2:2:2:2  prefixlen 64  scopeid 0x0<global>
        inet6 2a00:1a28:1157:xxxx:1:1:1:1  prefixlen 64  scopeid 0x0<global>
        ether 00:20:56:09:c8:7f  txqueuelen 1000  (Ethernet)
        RX packets 286  bytes 24726 (24.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 201  bytes 26519 (25.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 15  bytes 1436 (1.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15  bytes 1436 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@box:~# curl ipv6.google.com
curl: (7) Failed to connect to ipv6.google.com port 80: No route to host
  • TheDPTheDP OG

    I just tried and both, old and new subnet works.

    Thanked by (1)bjo

    For domain registrations, create an account at Dynadot (ref) and spend $9.99 within 48 hours to receive $5 DynaDollars!
    Looking for cost-effective Managed/Anycast/DDoS-Protected/Geo DNS Services? Try ClouDNS (aff).

Sign In or Register to comment.