Documentation

Server 3.x

Source Routing#

NOTE: this is a WORK IN PROGRESS!

There are a number of situations to do source routing or policy routing.

  1. You already have a dedicated NAT router, i.e. CGNAT (“Carrier Grade NAT”);
  2. You have a (layer 2) connection to the target location from your VPN box where the VPN traffic needs to be sent over, i.e. when the VPN server is located outside the network where the traffic needs to go.

These require that you do not send the traffic from the VPN clients over the VPN server’s default gateway.

We created a physical test setup similar to what you see below.

Test Setup#

                           Internet
                               ^
                               |
                          .--------.
                          | Router |
                          '--------'
                               ^ 192.168.178.1
                               |
     192.168.178.10            |
      .--------.          .--------.
      | Client |--------->| Switch |<-------------------------.
      '--------'          '--------'                          |
  VPN IP: 10.10.10.2           ^                              |
                               |                              |
                               |                              |
          192.168.178.2 (eth0) |                192.168.178.3 |
                        .------------.                     .-----.
                        | VPN Server |-------------------->| NAT |
                        '------------'                     '-----'
                10.10.10.1     192.168.1.100      192.168.1.1
                (wg0/tun0)     (eth1)

Assumptions#

  1. Your VPN clients get IP addresses assigned from the 10.10.10.0/24 and fd00:4242:4242:4242::/64 pools, the VPN server has 10.10.10.1 and fd00:4242:4242:4242::1 on the tun0 or wg0 device;
  2. A network connection between the VPN box and the NAT router exists through another interface, e.g. eth1:
    • the VPN box has the IP addresses 192.168.1.100 and fd00:1010:1010:1010::100 on this network;
    • the remote NAT router has the IP addresses 192.168.1.1 and fd00:1010:1010:1010::1 on this network;
  3. You installed the VPN server using the deployment script deploy_${DIST}.sh.
  4. The network where you route your client traffic over has static routes back to your VPN server:
    • There is an IPv4 static route for 10.10.10.0/24 via 192.168.1.100;
    • There is an IPv6 static route for fd00:4242:4242:4242::/64 via fd00:1010:1010:1010::100;

Configuration#

We’ll add some routing rules manually in order to test whether we have the correct configuration. The number 5000 is an arbitrary number we pick not yet defined in /etc/iproute2/rt_tables:

$ sudo ip -4 rule add to 10.10.10.0/24 lookup main
$ sudo ip -4 rule add from 10.10.10.0/24 lookup 5000
$ sudo ip -6 rule add to fd00:4242:4242:4242::/64 lookup main
$ sudo ip -6 rule add from fd00:4242:4242:4242::/64 lookup 5000

The to rules are needed to make sure traffic between VPN clients uses the main table so traffic between VPN clients remains possible, if this is allowed by the firewall.

Next, we’ll add the routes:

$ sudo ip -4 ro add default via 192.168.1.1 table 5000
$ sudo ip -6 ro add default via fd00:1010:1010:1010::1 table 5000

WireGuard#

The above does NOT seem to work with WireGuard, i.e. then you can’t reach the other devices connected to the VPN, for that to work the following approach is needed:

$ sudo ip -4 rule add from 10.10.10.0/24 lookup 5000
$ sudo ip -4 ro add default via 192.168.1.1 table 5000
$ sudo ip -4 ro add 10.10.10.0/24 dev wg0 table 5000
$ sudo ip -6 rule add from fd00:4242:4242:4242::/64 lookup 5000
$ sudo ip -6 ro add default via fd00:1010:1010:1010::1 table 5000
$ sudo ip -6 ro add fd00:4242:4242:4242::/64 dev wg0 table 5000

We’ll have to investigate how to use the same policy routing rules that work for both OpenVPN and WireGuard. As you may have noticed, we specified the wg0 interface in the above lines. As OpenVPN can have multiple processes, it would become quite cumbersome to make that work…

Permanent Configuration#

The above instructions are meant for testing, now let’s make them permanent.

Enterprise Linux / Fedora#

NOTE: this does not seem to work anymore in EL 9 / Fedora. We are working on a solution.

Install the required dependency:

$ sudo dnf -y install NetworkManager-dispatcher-routing-rules

Make the rules and routes permanent:

# echo 'to 10.10.10.0/24 lookup main' >/etc/sysconfig/network-scripts/rule-eth1
# echo 'from 10.10.10.0/24 lookup 5000' >/etc/sysconfig/network-scripts/rule-eth1
# echo 'to fd00:4242:4242:4242::/64 lookup main' >/etc/sysconfig/network-scripts/rule6-eth1
# echo 'from fd00:4242:4242:4242::/64 lookup 5000' >/etc/sysconfig/network-scripts/rule6-eth1
# echo 'default via 192.168.1.1 table 5000' > /etc/sysconfig/network-scripts/route-eth1
# echo 'default via fd00:1010:1010:1010::1 table 5000' > /etc/sysconfig/network-scripts/route6-eth1

Ubuntu#

NOTE: Configuring policy routing on Ubuntu has been contributed by a community member and has not been tested the eduVPN team.

We’ll need to add the routing in the Netplan configuration file /etc/netplan/my-network.yaml, e.g.:

network:
  version: 2
  ethernets:
    eth0:
      addresses:
        - 192.168.178.2/24
      nameservers:
        addresses:
          - 9.9.9.9
          - 2620:fe::9
        search:
          - example.org
      routes:
        - to: 0.0.0.0/0
          via: 192.168.178.1
      routing-policy:
        - from: 10.10.10.0/24
          table: 200
        - from: fd00:4242:4242:4242::/64
          table: 200
    eth1:
      addresses:
        - 192.168.1.100/24
        - fd00:1010:1010:1010::100/64
      routes:
        - to: default
          via: 192.168.1.1
          table: 200
        - to: default
          via: fd00:1010:1010:1010::1
          table: 200

You can find more information in the official documentation here.

In order to apply the changes, run:

$ netplan generate 
$ netplan apply

Troubleshooting#

It is smart to reboot your system to see if all comes up as expected:

$ ip -4 rule show table 5000
32765:  from 10.10.10.0/24 lookup 5000
$ ip -4 ro show table 5000
default via 192.168.1.1 dev eth1 
$ ip -6 rule show table 5000
32765:  from fd00:4242:4242:4242::/64 lookup 5000
$ ip -6 ro show table 5000
default via fd00:1010:1010:1010::1 dev eth1 metric 1024 pref medium

Firewall#

See the firewall documentation on how to update your firewall as needed.

systemd-networkd#

It can be quite nice, but it would be better if it actually worked.

NAT box#

# cat /etc/systemd/network/lan0.network 
[Match]
Name=enp8s0

[Network]
Address=192.168.1.1/24
Address=fd00:1010:1010:1010::1/64
IPv6AcceptRA=no

[Route]
Destination=10.10.10.0/24
Gateway=192.168.1.100

[Route]
Destination=fd00:4242:4242:4242::/64
Gateway=fd00:1010:1010:1010::100

# cat /etc/systemd/network/wan0.network 
[Match]
Name=enp1s0

[Network]
DHCP=ipv4
IPv6AcceptRA=yes
IPMasquerade=both

The IP masquerading does NOT work, something with the firewall is not okay. Manual firewall with NAT does work. Needs investigating, but out of scope for this particular document. Just mentioning it here for completeness.

VPN Box#

$ cat /etc/systemd/network/lan0.network 
[Match]
Name=enp5s0

[Network]
Address=192.168.1.100/24
Address=fd00:1010:1010:1010::100/64
IPv6AcceptRA=no

[Route]
Table=5000
Destination=0.0.0.0/0
Gateway=192.168.1.1

[Route]
Table=5000
Destination=::/0
Gateway=fd00:1010:1010:1010::1

[RoutingPolicyRule]
From=10.10.10.0/24
Table=5000

[RoutingPolicyRule]
From=fd00:4242:4242:4242::/64
Table=5000

#[RoutingPolicyRule]
#To=fd00:4242:4242:4242::/64
#Table=main

The problem is that from the VPN client you can’t ping the gateway (10.10.10.1) which breaks online detection in the VPN client.

So we need something that prevents the traffic TO 10.10.10.0/24 to go to the gateway. I am not sure how to cleanly do this except manually as shown above with the ip rules that adds a route to the table 5000 with dev wg0.

Last Modified: 2024-09-05

Source | History