IPv6 adventures

Network Engineering Jan 25, 2025

Inspired by Apalrd, I tried to migrate my homelab to IPv6. It took me weeks to migrate (almost) everything I setup and build (with IPv4) over the last years. Being network related, changing one component meant changing many connected devices and services as well. It was a mess, but I got everything, well almost everything, over to IPv6.

I plan on describing everything in detail another time. It will take some time to complete a full description, so for now I want to document some of my adventures here, so hopefully it will help somebody along.

GUA - ULA

Globally Unique Addresses are created using a prefix you get from the ISP. In my case the ISP (KPN, NL) uses DHCPv6 to supply me a /48 prefix. My experience with this provider is that, although they don't promise a static address, I only ever got a new address after I moved to a new house. Reading on their forums, I got the impression that other people had the same experience with IPv6 prefixes. However, this is not guaranteed. So if the prefix would change, my machines and services wouldn't be able to communicate anymore; the ip-address has changed.

The, or at least a, solution is to use Unique Local Addresses for all internal traffic. At first this was a hassle in pfSense, but in the end it works fine. It would be nicer if pfSense could route based on the suffix alone, but that does not seem to be possible. It would make the network cleaner as now almost every device gets two addresses.

Another disadvantage could be that a ULA doesn't connect to the internet. So updating a proxmox host with only a ULA is not possible. I got around this by having an apt proxy inside my network, which does have a GUA.

Hardware

Configuring most hardware was a easy. Even a printer from 2010 got an address using SLAAC. It connects through IPv4 by default, but after changing the address to IPv6, it works fine.

IPMI on my HP server was also no issue.

However, getting IPv6 on switches... I didn't expect my TP-Link TL-SG108E to have IPv6, so yeah. But it was a bit of a bummer I couldn't figure out how to give the Brocade 6610 an IPv6 address. As I only use them as layer 2 switches it is of no real concern though.

Also my Aruba WiFi can't be accessed through IPv6. They pass it along just fine too.

Proxmox

This was a bit of a hassle. I use the SDN functionality and also have my hosts (and web-ui) on the an SDN vlan. This is not a documented setup, so it is always a headache to get new things working. At first I doubted if I wanted to get an address using SLAAC, but I opted for a manual address instead. Mainly because I got this working and didn't feel like pushing it further. It is also easier to memorize.

So eventually this is what I had in /etc/network/interfaces for the management interface, where the webui (pveproxy) binds to.

iface mgmt
        address 10.33.10.117/24
        gateway 10.33.10.1
iface mgmt
        address fdc3:40c9:680f:10::117/64
        gateway fdc3:40c9:680f:10::1
        accept_ra 2
        autoconf 1

My SAN network was added too with it's own route:

iface san
        address fdc3:40c9:680f:80::117/64
        post-up ip -6 route add fdc3:40c9:680f:80::/64 dev san src fdc3:40c9:680f:80::117 table rt6san
        post-up ip -6 route add default via fdc3:40c9:680f:80::1 dev san table rt6san
        post-up ip -6 rule add from fdc3:40c9:680f:80::117/128 table rt6san
        post-up ip -6 rule add to fdc3:40c9:680f:80::117/128 table rt6san

This however didn't work for the webui, which would only open on the IPv4 address. A fix/workaround is to change the IPv4 address in /etc/hosts to a IPv6 address and restart the pveproxy service.

127.0.0.1 localhost.localdomain localhost
fdc3:40c9:680f:10::117/64 jpl-proxmox7.local.lan jpl-proxmox7
::ffff:10.33.10.117 jpl-proxmox7.local.lan jpl-proxmox7

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

After I got everything over to IPv6 I removed the IPv4 address from /etc/network/interfaces and ::ffff:10.33.10.117 jpl-proxmox7.local.lan jpl-proxmox7 from /etc/hosts.

Don't forget to update the Corosync settings too.

Update ring0_addr for all nodes
[Optionally] Update ring1_addr for all nodes
Increment config_version
Set ip_version to ipv6

...
  node {
    name: jpl-proxmox7
    nodeid: 3
    quorum_votes: 1
    ring0_addr: fdc3:40c9:680f:10::117
    ring1_addr: fdc3:40c9:680f:80::117
  }
...

totem {
  ...
  config_version: 8
  interface {
    linknumber: 0
    knet_link_priority: 255
  }
  interface {
    linknumber: 1
    knet_link_priority: 200
  }
  ip_version: ipv6
  ...
}

Restart corosync and proxmox cluster: systemctl restart corosync pve-cluster

AWS

For business purposes I use AWS to host our application and services. You would expect a large cloud provider to be fully IPv6 ready, but...no. Not all service support IPv6, see their list: AWS services that support IPv6.

Cloudfront custom origins

The biggest one for me is Cloudfront not supporting IPv6 custom origins. So although I now have IPv6 addresses on all my servers, I can't use those with Cloudfront as I described here and here.

Another reason for me to start learning how to use IPv6 was to eliminate costs for public IPv4 addresses used with our ec2 instances. To connect to those instances for administration, but also as origins for Cloudfront. But AWS doesn't support IPv6 origins. However, apparently there is now another supported solution: Amazon CloudFront VPC origins.

ECR

Another one is the container repository. You cannot download or upload images over IPv6. Downloading on an EC2 over IPv4 is fine. Uploading is more annoying as I just had our Teamcity servers and agents configured with IPv6 only.

AWS CLI / SDK dual stack

Through the years I also created multiple scripts to help me do things on AWS more easily. This run through the CLI or Node/Javascript SDK. Most stil work, but the one I that saved me most work didn't. I couldn't connect to the cloudfront API.

CLI:

Could not connect to the endpoint URL: "https://cloudfront.eu-central-1.api.aws/2020-05-31/distribution"

SDK:

Error: getaddrinfo ENOTFOUND cloudfront.eu-central-1.api.aws
    at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:120:26) {
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'cloudfront.eu-central-1.api.aws',
  '$metadata': { attempts: 1, totalRetryDelay: 0 }
}

I don't understand what's going wrong here, maybe it is similar to this bug report, but the workaround for me seems to be to not set use_dual_stack_endpoint = true in my config file as is described in the docs. Once commented out, I can use the CLI and run my scripts.

Microsoft

SQL server

According to the documentation, SQL Server should listen on both IPv4 and IPv6. However, this is not (always) the case, at least on Ubuntu. As I run SQL Server on Ubuntu I lack the tooling described in all documentation, forum post, etc. to try all the proposed fixes.

On my machines, SQL Server only listened on IPv4, not IPv6. Port 1433 is only listening on 0.0.0.0.

jodibooks@jodiSQLServer:~$ sudo netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:1433            0.0.0.0:*               LISTEN      3917/sqlservr       
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      403/master          
tcp        0      0 127.0.0.1:1434          0.0.0.0:*               LISTEN      3917/sqlservr       
tcp        0      0 127.0.0.1:1431          0.0.0.0:*               LISTEN      3917/sqlservr       
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      97/systemd-resolved 
tcp6       0      0 ::1:1431                :::*                    LISTEN      3917/sqlservr       
tcp6       0      0 ::1:1434                :::*                    LISTEN      3917/sqlservr       
tcp6       0      0 ::1:25                  :::*                    LISTEN      403/master          
tcp6       0      0 :::22                   :::*                    LISTEN      1/init

By running these commands I could switch it to :: so it would listen on IPv6.

sudo systemctl stop mssql-server.service
sudo MSSQL_IP_ADDRESS=:: /opt/mssql/bin/mssql-conf setup

jodibooks@jodiSQLServer:~$ sudo netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      403/master          
tcp        0      0 127.0.0.1:1434          0.0.0.0:*               LISTEN      869/sqlservr        
tcp        0      0 127.0.0.1:1431          0.0.0.0:*               LISTEN      869/sqlservr        
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      97/systemd-resolved 
tcp6       0      0 ::1:1431                :::*                    LISTEN      869/sqlservr        
tcp6       0      0 ::1:1434                :::*                    LISTEN      869/sqlservr        
tcp6       0      0 ::1:25                  :::*                    LISTEN      403/master          
tcp6       0      0 :::1433                 :::*                    LISTEN      869/sqlservr        
tcp6       0      0 :::22                   :::*                    LISTEN      1/init

This however disables listening on 0.0.0.0.

Weirdly enough, on our production EC2 on AWS running dual-stack works fine on both ip's:

ubuntu@ip-172-31-43-50:~$ sudo ss -ltn
State           Recv-Q          Send-Q                   Local Address:Port                     Peer Address:Port          Process          
LISTEN          0               128                          127.0.0.1:1431                          0.0.0.0:*                              
LISTEN          0               100                          127.0.0.1:25                            0.0.0.0:*                              
LISTEN          0               128                            0.0.0.0:1433                          0.0.0.0:*                              
LISTEN          0               4096                     127.0.0.53%lo:53                            0.0.0.0:*                              
LISTEN          0               4096                         127.0.0.1:8125                          0.0.0.0:*                              
LISTEN          0               128                            0.0.0.0:22                            0.0.0.0:*                              
LISTEN          0               100                              [::1]:25                               [::]:*                              
LISTEN          0               128                              [::1]:1431                             [::]:*                              
LISTEN          0               128                                  *:1433                                *:*                              
LISTEN          0               128                               [::]:22                               [::]:*

Also, I noticed that when installing SQL server on a clean Ubuntu 22.04 install (official AMI when creating an EC2) it needed to download a dependency from a IPv4 only server!

Windows ULA privacy extensions

SLAAC uses that mac to generate a unique IPv6 address. This could be used to track you online and to know which network adapter you use and thus make it easier to exploit known bugs. To mitigate this risk, Windows and Ubuntu desktop use privacy extensions to hide the mac address of your network adapter. And by generating a new address every 2 hours, tracking is also much harder.

This is all nice and well, but it makes it almost impossible to configure ACL's based on IP addresses. In my case I want to only allow certain machines to hop vlans. Chief among them is my own Ubuntu desktop. Luckily Linux has a way to configure a route so a fixed address is used internally and a privacy address externally.

I created file /etc/NetworkManager/dispatcher.d/99-ipv6-add-default-route-ula.

ULA prefix example: fd00:1234:5678::/48
Subnet / vlan: 20 --> first hop gateway (my router address on vlan 20): fd00:1234:5678:20::1
PC ULA address: fd00:1234:5678:20:1234:5678:90ab:cdef

#!/bin/bash

#interface=$1
#event=$2

#https://unix.stackexchange.com/questions/320943/add-default-route-without-gateway-nmcli
[[ "${1}" -ne "br0" ]] && exit

if [[ "${2}" -eq "up" ]]; then 
#https://superuser.com/questions/638024/use-ipv6-temporary-address-only-with-remote-peers
  ip route add fd00:1234:5678::/48 \
                  via fd00:1234:5678:20::1 \
                  dev br0 \
                  src fd00:1234:5678:20:1234:5678:90ab:cdef \
                  metric 128
fi

💡

Sources:
- https://unix.stackexchange.com/questions/320943/add-default-route-without-gateway-nmcli
- https://superuser.com/questions/638024/use-ipv6-temporary-address-only-with-remote-peers

This is not possible in Windows. You can either turn of privacy extensions for all addresses or leave it on. It is also not possible to do some clever routing as described above. In my case that meant that I cannot create an ACL in pfSense that only allows certain pc's to make remote desktop connections to Windows Servers. I can only create an ACL that allows the full subnet to do that. Not ideal at all.

Various apps

Docker

There are a lot of post that have all kinds of (experimental) ways of enabling IPv6 on docker. However in the end it was really simple. In 2025 you don't have to add anything in /etc/docker/daemon.json. You don't have to specify a subnet, prefix or ip-range or create a custom network. The only thing I needed to do was add enable_ipv6: true to my network in my docker compose files. For example for my Wiki of this blog:

networks:
  joeplaa-net:
    name: joeplaa-net
    enable_ipv6: true

Git

I use Bitbucket and Github with SSH connections. Bitbucket works fine, but Github is really slow this way. Github doesn't seem to be fully ready for IPv6, so the as a workaround I added this to my ~/.ssh/config file to force it to use IPv4:

Host github.com
    AddressFamily inet

WordPress

Yes, it's 2025 and still WordPress is only available over IPv4.

TrueNAS

The apps in TrueNAS 24.10 doesn't support IPv6. There is an open feature request to have this implemented. This means my Nextcloud, Jellyfin and Nexus repositories have to be accessed through IPv4. For now this is not a real issue as Jellyfin and Nexus are behind an Nginx reverse proxy to which the clients connect using IPv6. Nginx used IPv4 to connect to the apps. Nextcloud comes in through HAProxy, but basically the same happens.

NFS

IPv6 in TrueNAS works fine for all my Samba and iSCSI shares. However, I cannot get NFS to work. This is not a TrueNAS issue, but is a nfs-client issue. If I use the IPv6 address I get an error Address family for hostname not supported. A proposed solution was to use a hostname instead, but then I get this error: No address associated with hostname. Pinging does work, so the hostname is actually properly resolved.

Umami Analytics

The Next.js server only listens on IPv4:

$ node server.js
   ▲ Next.js 15.0.4
   - Local:        http://localhost:3000
   - Network:      http://0.0.0.0:3000

 ✓ Starting...

LibreNMS

ProtonVPN

Promises were made, but so far I can't find a server which supports this. I'm also not sure how this would exactly work. Do I get an address from the ProtonVPN pool that makes it look as though the requests come from Proton, not me? Does that address change often enough? Or will there be some NAT'ing done to hide my activity behind other users?

Did you just roll out IPv6?
by u/xmvu in ProtonVPN

IPv6 support has been expanded to more servers!
by u/Proton_Team in ProtonVPN

We’re testing IPv6 on our servers, and we need your help
by u/protonvpn in ProtonVPN

Recommended for you

Network Engineering

IPv6 adventures 3: IPv6 only hosts

12 days ago • 5 min read

More Nexus repositories

15 days ago • 6 min read

Network Engineering

IPv6 adventures part 2: more roadblocks

16 days ago • 2 min read

IPv6 adventures 3: IPv6 only hosts

More Nexus repositories

IPv6 adventures part 2: more roadblocks

Updating all Ubuntu packages

IPv6 adventures

GUA - ULA

Hardware

Proxmox

AWS

Cloudfront custom origins

ECR

AWS CLI / SDK dual stack

Microsoft

SQL server

Windows ULA privacy extensions

Various apps

Docker

Git

WordPress

TrueNAS

NFS

Umami Analytics

LibreNMS

ProtonVPN

Tags

Joep van de Laarschot

Recommended for you

IPv6 adventures 3: IPv6 only hosts

More Nexus repositories

IPv6 adventures part 2: more roadblocks