Documentation

Server 2.x

Version 2.x of eduVPN is End of Life (EOL). Switch to "Server 3.x" Documentation!

Traceable NAT#

When using NAT, it is hard to identify the user behind an action, because every uses has the same outgoing IP address. A solution to this could be Netflow, but that requires Netflow to be installed on every eduVPN host, which can be a hassle.

Instead, this document proposes a solution with rudimentary implementation, where every internal IPv4 address is mapped to a limited range of port numbers for outgoing traffic. With this, any user can be traced in two ways:

  1. Incident reports will typically include the source port number used
  2. If Netflow is used outside the eduVPN host, this will log the port number used by the eduVPN server

How it works#

The administrator must run a script that will generate iptables rules for the VPN node. This can be done at the same time that vpn-maint-apply-changes is run. Afterwards, the firewall must be reloaded, which would cause users’ connections inside the tunnel to be reset, so the script only restarts the firewall when it’s necessary.

Every time the script is run, it will check if the current iptables rules in /etc/iptables/rules.v4 are identical to the desired rules, if they are, the script does nothing. If they are not, it will update the rules in the file and restart the firewall. When it does this, it will write the rules as a new file and keep the old file so that IP/port mappings can always be retrieved. The old firewall files have a UNIX timestamp in their filename, so when you must investigate an incident, you can find how the firewall was configured at the time of that incident, and which port number was used by which internal IP address.

Set up#

This is only tested on Debian, and probably won’t work on CentOS/Fedora.

Copy the vpn-generate-natfw.php file in this repository.

install -m755 vpn-generate-natfw.php /usr/bin/vpn-generate-natfw

Now you must generate the iptables configuration and save it. If you want to test before you write, run

vpn-generate-natfw -n

If you are happy with the result, you can write it by running without arguments

vpn-generate-natfw

If you want to see what other options are available

vpn-generate-natfw --help

That’s all! If you have any procedure where you run vpn-server-node-server-config or vpn-maint-apply-changes automatically, you might consider running vpn-generate-natfw at the same time. Never run vpn-generate-natfw with -f (force) from cron/systemd timers!

“Ephemeral Ports”#

$ sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768    60999

These are the ports used by the OS itself to give to services that open connections. So you want your VPN client port range to stay out of this range.

There’s also net.ipv4.ip_local_reserved_ports which won’t be used.

These options also apply to IPv6.

Sources: * https://superuser.com/questions/1118735/how-are-source-ports-determined-and-how-can-i-force-it-to-use-a-specific-port * https://unix.stackexchange.com/questions/279157/ipv6-ephemeral-port-range

Tracing an incident#

In order to trace a user, you need three bits of information

1. Find the firewall configuration that was active when the incident happened.#

You need to have the date in UNIX timestamp format. Run

date +%s -d '2020/12/26 13:37:42'

You will get a number, such as 1608986262

2. Find the matching firewall configuration#

Get a list of all firewall configurations

ls -1 /etc/iptables/

You will see multiple files in the format rules.v4.1608946501 with different numbers. You must find the highest number that is still lower than the UNIX timestamp.

3. Find the internal IP address#

Display the firewall rules

less /etc/iptables/rules.v4.1608946501

You will see a lot of lines such as

-A POSTROUTING -s 192.168.101.2 -p tcp -j SNAT --to-source 192.0.2.1:65280-65535
-A POSTROUTING -s 192.168.101.2 -p udp -j SNAT --to-source 192.0.2.1:65280-65535
-A POSTROUTING -s 192.168.101.3 -p tcp -j SNAT --to-source 192.0.2.1:65024-65279
-A POSTROUTING -s 192.168.101.3 -p udp -j SNAT --to-source 192.0.2.1:65024-65279
-A POSTROUTING -s 192.168.101.4 -p tcp -j SNAT --to-source 192.0.2.1:64768-65023
-A POSTROUTING -s 192.168.101.4 -p udp -j SNAT --to-source 192.0.2.1:64768-65023
-A POSTROUTING -s 192.168.101.5 -p tcp -j SNAT --to-source 192.0.2.1:64512-64767
-A POSTROUTING -s 192.168.101.5 -p udp -j SNAT --to-source 192.0.2.1:64512-64767
-A POSTROUTING -s 192.168.101.6 -p tcp -j SNAT --to-source 192.0.2.1:64256-64511
-A POSTROUTING -s 192.168.101.6 -p udp -j SNAT --to-source 192.0.2.1:64256-64511

Port ranges are shown at the end of the line. Scroll with the arrow keys or PgUp/PgDown and find the line that matches your source port number. You have now found the internal IPv4 address that was involved in the incident. This internal IP address (on the left) can be matched to a user through the eduVPN portal (webinterface).

Example Script#

vpn-generate-natfw.php:

#!/usr/bin/php
<?php

// conf, faster/better to manually set the IPv4 address
$publicIp = trim(shell_exec('ip addr | grep \'state UP\' -A2 | tail -n1 | awk \'{print $2}\' | cut -f1  -d\'/\''));
$maxPort = 65535;
$minPort = 1280;

/*
 * eduVPN - End-user friendly VPN.
 *
 * Copyright: 2021, Jørn Åne de Jong, Uninett AS
 * SPDX-License-Identifier: BSD-3-Clause
 *
 * Some parts are copied from the eduVPN project by The Commons Conservancy
 * These parts are AGPL-3 licensed.  They are clearly marked with // BEGIN snippet and // END snippet
 */

if ($argc > 1 && (in_array('-h', $argv) || in_array('--help', $argv))) {
    fwrite(STDERR, $argv[0] . ' [-f] [-n] [-v] [-w filename] [-c command]' . PHP_EOL);
    fwrite(STDERR, "\t-f\tForce overwriting existing file (not needed if existing file is a symlink)" . PHP_EOL);
    fwrite(STDERR, "\t-n\tDry run (output file to stdout)" . PHP_EOL);
    fwrite(STDERR, "\t-v\tVerbose (also output if nothing changed)" . PHP_EOL);
    fwrite(STDERR, "\t-w filename\tWrite to filename, default /etc/iptables/rules.v4" . PHP_EOL);
    fwrite(STDERR, "\t-c command\tCommand to run if a new file was written, default service netfilter-persistent restart" . PHP_EOL);
    exit;
}

require_once '/usr/share/php/LC/Node/autoload.php';
$baseDir = '/usr/share/vpn-server-node';

use LC\Common\Config;
use LC\Common\HttpClient\CurlHttpClient;
use LC\Common\HttpClient\ServerClient;
use LC\Common\ProfileConfig;
use LC\Node\OpenVpn;

class IP extends LC\Node\IP {
    function getAllHosts() {
        // Works only for IPv4 but that's okay,
        // we don't support NATv6.
        // I would call $this->requireIPv4() but it's private.
        $counter = $this->getNumberOfHosts() - 1;
        $addr = ip2long($this->getFirstHost());
        while( $counter-- ) {
            yield long2ip( ++$addr );
        }
    }
    function split($networkCount) {
        // Make sure we return instances of ourself, not the parent
        return array_map(function($ip) {
                return new self($ip->__toString());
            }, parent::split($networkCount));
    }
}

$networks = [];
try {
// BEGIN snippet copy from /usr/bin/vpn-server-node-server-config
    $configFile = sprintf('%s/config/config.php', $baseDir);
    $config = Config::fromFile($configFile);

    $vpnUser = $config->requireString('vpnUser', 'openvpn');
    $vpnGroup = $config->requireString('vpnGroup', 'openvpn');

    $vpnConfigDir = sprintf('%s/openvpn-config', $baseDir);
    $serverClient = new ServerClient(
            new CurlHttpClient($config->requireString('apiUser'), $config->requireString('apiPass')),
            $config->requireString('apiUri')
        );

    $profileIdDeployList = $config->requireArray('profileList', []);
// END snippet copy from /usr/bin/vpn-server-node-server-config

// BEGIN snippet copy from /usr/share/php/LC/Node/OpenVpn.php
    $profileList = $serverClient->getRequireArray('profile_list');
    // filter out the profiles we do not want on this node
    if (0 !== \count($profileIdDeployList)) {
        foreach (array_keys($profileList) as $profileId) {
            if (!\in_array($profileId, $profileIdDeployList, true)) {
                unset($profileList[$profileId]);
            }
        }
    }

    foreach($profileList as $profileConfigData) {
        $profileConfig = new ProfileConfig(new Config($profileConfigData));
        $processCount = \count($profileConfig->vpnProtoPorts());
        $range = new IP($profileConfig->range());
        $splitRange = $range->split($processCount);
        $networks = array_merge($networks, $splitRange);
    }
// END snippet copy from /usr/share/php/LC/Node/OpenVpn.php
} catch (Exception $e) {
    fwrite(STDERR, 'Error: ' . $e->getMessage() . PHP_EOL);
    exit (1);
}

$addresses = 0;
$privateNetworks = [];
foreach($networks as $network) {
    if (substr($network, 0, 3) === '10.' // 10/8
        // 172.16/12
        || substr($network, 0, 7) === '172.16.'
        || substr($network, 0, 7) === '172.17.'
        || substr($network, 0, 7) === '172.18.'
        || substr($network, 0, 7) === '172.19.'
        || substr($network, 0, 7) === '172.20.'
        || substr($network, 0, 7) === '172.21.'
        || substr($network, 0, 7) === '172.22.'
        || substr($network, 0, 7) === '172.23.'
        || substr($network, 0, 7) === '172.24.'
        || substr($network, 0, 7) === '172.25.'
        || substr($network, 0, 7) === '172.26.'
        || substr($network, 0, 7) === '172.27.'
        || substr($network, 0, 7) === '172.28.'
        || substr($network, 0, 7) === '172.29.'
        || substr($network, 0, 7) === '172.30.'
        || substr($network, 0, 7) === '172.31.'
        // 192.168/16
        || substr($network, 0, 8) === '192.168.'
        // 100.64/10
        || substr($network, 0, 7) === '100.64.'
        || substr($network, 0, 7) === '100.65.'
        || substr($network, 0, 7) === '100.66.'
        || substr($network, 0, 7) === '100.67.'
        || substr($network, 0, 7) === '100.68.'
        || substr($network, 0, 7) === '100.69.'
        || substr($network, 0, 5) === '100.7'
        || substr($network, 0, 5) === '100.8'
        || substr($network, 0, 5) === '100.9'
        || (substr($network, 0, 6) === '100.10' && substr($network, 0, 7) !== '100.10.')
        || (substr($network, 0, 6) === '100.11' && substr($network, 0, 7) !== '100.11.')
        || substr($network, 0, 7) === '100.120.'
        || substr($network, 0, 7) === '100.121.'
        || substr($network, 0, 7) === '100.122.'
        || substr($network, 0, 7) === '100.123.'
        || substr($network, 0, 7) === '100.124.'
        || substr($network, 0, 7) === '100.125.'
        || substr($network, 0, 7) === '100.126.'
        || substr($network, 0, 7) === '100.127.'
    ) {
        // -1 to remove the gw address
        $addresses += $network->getNumberOfHosts() - 1;
        $privateNetworks[] = $network;
    }
}
if (!$privateNetworks) {
    fwrite(STDERR, 'Error: There are no private networks in ' . implode(', ', $networks) . PHP_EOL);
    exit (1);
}

$portsPerClient = floor(($maxPort-$minPort)/$addresses);
$candidate = 1;
for($candidate = 1; $portsPerClient > $candidate; $candidate = $candidate << 1) {};
$portsPerClient = $candidate >> 1;

// Buffer output, so we don't write if there's an error
$output = '';
$output .= '# Generated by vpn-generate-natfw on ' . date('D M d H:i:s Y') . PHP_EOL;
$output .= "# NAT ports per client: $portsPerClient" . PHP_EOL;
$output .= "*nat" . PHP_EOL;
$output .= ":PREROUTING ACCEPT [0:0]" . PHP_EOL;
$output .= ":INPUT ACCEPT [0:0]" . PHP_EOL;
$output .= ":POSTROUTING ACCEPT [0:0]" . PHP_EOL;
$output .= ":OUTPUT ACCEPT [0:0]" . PHP_EOL;

$port = $maxPort + 1;
$output .= "-A POSTROUTING -p icmp -j SNAT --to-source $publicIp" . PHP_EOL;
foreach($privateNetworks as $network) foreach($network->getAllHosts() as $ip) {
    $max = $port - 1;
    $min = $port - $portsPerClient;
    $port = $min;
    $output .= "-A POSTROUTING -s $ip -p tcp -j SNAT --to-source $publicIp:$min-$max" . PHP_EOL;
    $output .= "-A POSTROUTING -s $ip -p udp -j SNAT --to-source $publicIp:$min-$max" . PHP_EOL;
}
$output .= "COMMIT" . PHP_EOL;
$output .= "*filter" . PHP_EOL;
$output .= ":INPUT ACCEPT [0:0]" . PHP_EOL;
$output .= ":FORWARD ACCEPT [0:0]" . PHP_EOL;
$output .= ":OUTPUT ACCEPT [0:0]" . PHP_EOL;
$output .= "COMMIT" . PHP_EOL;

if (in_array('-n', $argv)) {
    $file = null;
} else {
    $file = '/etc/iptables/rules.v4';
}
$command = 'service netfilter-persistent restart';
for($i = 0; $i < sizeof($argv); $i++) {
    if ($argv[$i] === '-w') {
        $file = $argv[++$i];
        if ($file === null) {
            fwrite(STDERR, 'Error: Invalid file specified with -w' . PHP_EOL);
        }
    }
    if ($argv[$i] === '-c') {
        $command = $argv[++$i];
        if ($command === null) {
            fwrite(STDERR, 'Error: Invalid command specified with -c' . PHP_EOL);
        }
    }
}

if (null === $file) {
    echo $output;
} else {
    $original = @file_get_contents($file);
    if (preg_replace('/# Generated by vpn-generate-natfw on .*/', '', $original)
        === preg_replace('/# Generated by vpn-generate-natfw on .*/', '', $output)) {
        if ($argc > 1 && in_array('-v', $argv)) {
            fwrite(STDERR, 'Success: file is unchanged: ' . $file . PHP_EOL);
        }
    } else {
        if (!in_array('-f', $argv) && file_exists($file) && !is_link($file)) {
            fwrite(STDERR, 'Error: Output file already exists ' . $file . PHP_EOL);
            fwrite(STDERR, 'Hint: Use -f one time to overwrite it, or simply delete it' . $file . PHP_EOL);
            exit (1);
        }
        $writeFile = $file . '.' . time();
        if (file_put_contents($writeFile, $output)) {
            fwrite(STDERR, 'Success: Wrote new ruleset to ' . $writeFile . PHP_EOL);
            @unlink($file);
            if (symlink($writeFile, $file)) {
                fwrite(STDERR, 'Success: Updated symlink ' . $file . PHP_EOL);
                passthru($command, $return);
                if ($return) {
                    fwrite(STDERR, 'Failure: ' . $command . PHP_EOL);
                } else {
                    fwrite(STDERR, 'Success: ' . $command . PHP_EOL);
                }
                exit ($return);
            }
        } else {
            fwrite(STDERR, 'Error: Unable to write to output file ' . $writeFile . PHP_EOL);
            exit (1);
        }
    }
}

Last Modified: 2023-09-14

Source | History