Source Port selection on Linux results allways in even ports

Does this have any effect?

Well sort of i would say, so lets have a look at LAGs first. Most networks i know do utilize some sort of Link Aggregation (LAG), like LACP.

How does LACP work?

LACP do calulate a hash out of a comination of MAC, IP addresses and ports. Maybe this is described as xor-l3, xor-l3l4.

This means the hash is build from a combination out of this values with XOR. The result will be used to get the remainder (modulo) based on the amount of links.

So this will give you the link to use.

So why all even source ports can be an issue?

In some cases you will only utilize only halve of the links in an LAG.

This would mean in the example above with an 4 port LAG.

Linux

( 10.10.4.253 ^ 10.255.93.77 ^ 47806 ^ 5201 ) % 4 = 3
( 10.10.4.253 ^ 10.255.93.77 ^ 47816 ^ 5201 ) % 4 = 1
( 10.10.4.253 ^ 10.255.93.77 ^ 47824 ^ 5201 ) % 4 = 1
( 10.10.4.253 ^ 10.255.93.77 ^ 47836 ^ 5201 ) % 4 = 1
( 10.10.4.253 ^ 10.255.93.77 ^ 47852 ^ 5201 ) % 4 = 1
( 10.10.4.253 ^ 10.255.93.77 ^ 47862 ^ 5201 ) % 4 = 3
( 10.10.4.253 ^ 10.255.93.77 ^ 47864 ^ 5201 ) % 4 = 1
( 10.10.4.253 ^ 10.255.93.77 ^ 47874 ^ 5201 ) % 4 = 3
( 10.10.4.253 ^ 10.255.93.77 ^ 47882 ^ 5201 ) % 4 = 3
( 10.10.4.253 ^ 10.255.93.77 ^ 47886 ^ 5201 ) % 4 = 3

As we can see linux would only use links 1 and 3 of the lag group, ports 0 and 2 will not be used.

With FreeBSD on the ohter hand this would look like:

( 10.10.4.251 ^ 10.255.93.77 ^ 54819 ^ 5201 ) % 4 = 0
( 10.10.4.251 ^ 10.255.93.77 ^ 24383 ^ 5201 ) % 4 = 0
( 10.10.4.251 ^ 10.255.93.77 ^ 48869 ^ 5201 ) % 4 = 2
( 10.10.4.251 ^ 10.255.93.77 ^ 40010 ^ 5201 ) % 4 = 1
( 10.10.4.251 ^ 10.255.93.77 ^ 55178 ^ 5201 ) % 4 = 1
( 10.10.4.251 ^ 10.255.93.77 ^ 37904 ^ 5201 ) % 4 = 3
( 10.10.4.251 ^ 10.255.93.77 ^ 39569 ^ 5201 ) % 4 = 2
( 10.10.4.251 ^ 10.255.93.77 ^ 45923 ^ 5201 ) % 4 = 0
( 10.10.4.251 ^ 10.255.93.77 ^ 35810 ^ 5201 ) % 4 = 1
( 10.10.4.251 ^ 10.255.93.77 ^ 22573 ^ 5201 ) % 4 = 2

As you can see all 4 links will be used

Does this make a difference? Yes certainly it does make a diffrence.

Is it realy important, this depends on your particular network setup. Haveing “few” hosts in the network or a sub optimal layout in ip addressing this can impact the overall performance.

What can we do?

Since we do not contoll the service (destination) ports in a data center we can provicde provide more destination addresses, like AWS S3 does.

Whan you are talking with Public AWS S3 Buckets, you will get a few out of many addresses belonging to the S3 Infrastructure.

Where does this come from?

The not compelling answer on stackexchange and the kernel commit.

It’s to reduce contention between connect() and bind() (appeared in Linux 4.2; Jessie has 3.16 and Stretch has 4.9):

commit 07f4c90062f8fc7c8c26f8f95324cbe8fa3145a5
Author: Eric Dumazet 
Date:   Sun May 24 14:49:35 2015 -0700

    tcp/dccp: try to not exhaust ip_local_port_range in connect()

    A long standing problem on busy servers is the tiny available TCP port
    range (/proc/sys/net/ipv4/ip_local_port_range) and the default
    sequential allocation of source ports in connect() system call.

    If a host is having a lot of active TCP sessions, chances are
    very high that all ports are in use by at least one flow,
    and subsequent bind(0) attempts fail, or have to scan a big portion of
    space to find a slot.

    In this patch, I changed the starting point in __inet_hash_connect()
    so that we try to favor even [1] ports, leaving odd ports for bind()
    users.

    We still perform a sequential search, so there is no guarantee, but
    if connect() targets are very different, end result is we leave
    more ports available to bind(), and we spread them all over the range,
    lowering time for both connect() and bind() to find a slot.

    This strategy only works well if /proc/sys/net/ipv4/ip_local_port_range
    is even, ie if start/end values have different parity.

    Therefore, default /proc/sys/net/ipv4/ip_local_port_range was changed to
    32768 - 60999 (instead of 32768 - 61000)

    There is no change on security aspects here, only some poor hashing
    schemes could be eventually impacted by this change.

    [1] : The odd/even property depends on ip_local_port_range values parity

You may also want to see the followup commit 1580ab63fc9a03593072cc5656167a75c4f1d173.

https://unix.stackexchange.com/questions/379533/debian-stretch-source-tcp-port-is-always-even

In the first place the patch looks like a compelling solution for this problem.

As explained with the LAG groups this can lead to issues later in the infrastructure.

It might be that also other forms of load balancing techniques are affected.

Calculation in Python

We can do the calculation, somewhat our self.

#!/usr/bin/env python

import ipaddress

def mod(HASH,LINKS):
  return HASH % LINKS

def lag_ports(SRCIPS,DESTIPS,SPORTS,DPORT,LINKS=2,COMMENT=None):
  if COMMENT:
    print(f"{COMMENT}")

  for SIP in SRCIPS:
    SINT = int(ipaddress.ip_address(SIP))
    for DIP in DESTIPS:
      DINT = int(ipaddress.ip_address(DIP))
      for SPORT in SPORTS:
        HASH_SPORT_DPORT=SINT ^ DINT ^ SPORT ^ DPORT
        MOD = mod(HASH_SPORT_DPORT,LINKS)
        print(f"( {SIP} ^ {DIP} ^ {SPORT} ^ {DPORT} ) % {LINKS} = {MOD}")
  print(f"\n")


SRCIPS=['10.10.4.253']
SPORTS=[47806, 47816 , 47824 , 47836 , 47852 , 47862 , 47864 , 47874 , 47882 , 47886 ]
DESTIPS=['10.255.93.77']
DPORT=5201
lag_ports(SRCIPS,DESTIPS,SPORTS,DPORT,LINKS=4,COMMENT="Linux")

SRCIPS=['10.10.4.251']
SPORTS=[54819 , 24383 , 48869 , 40010 , 55178 , 37904 , 39569 , 45923 , 35810 , 22573 ]
DESTIPS=['10.255.93.77']
DPORT=5201
lag_ports(SRCIPS,DESTIPS,SPORTS,DPORT,LINKS=4,COMMENT="FreeBSD")

Cisco

cisco#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-mac

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source MAC address
  IPv4: Source MAC address
  IPv6: Source MAC address

Configuration

cisco#port-channel load-balance src-dst-ip
cisco(config)#end

Available modes depend on the platfrom in this case WS-C3560CX-8PT-S

cisco(config)#port-channel load-balance ?
  dst-ip       Dst IP Addr
  dst-mac      Dst Mac Addr
  src-dst-ip   Src XOR Dst IP Addr
  src-dst-mac  Src XOR Dst Mac Addr
  src-ip       Src IP Addr
  src-mac      Src Mac Addr

After the Change

cisco#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination IP address
  IPv6: Source XOR Destination IP address