Shared Local Ports

Onload User Guide (UG1586)

Document ID
UG1586
Release Date
2023-07-31
Revision
1.2 English

The shared local ports feature improves the performance of TCP active-opens. It:

  • reduces the cost of both blocking and non-blocking connect() calls
  • reduces the latency to establish new connections
  • enables scaling to large numbers of active-open connections
  • reduces the cost of closing these connections.

These improvements are achieved by sharing a set of local port numbers amongst active-open sockets, which saves the cost and scaling limits associated with installing packet steering filters for each active-open socket. Shared local ports are only used when the local port is not explicitly assigned by the application.

To enable shared local ports, set the EF_TCP_SHARED_LOCAL_PORTS option to ≥1. The value set gives the initial number of local ports to allocate when the Onload stack is created. More shared local ports are allocated on demand as needed up to the maximum given by EF_TCP_SHARED_LOCAL_PORTS_MAX.

Additional configuration options were added in Onload 201805:

  • When EF_TCP_SHARED_LOCAL_PORTS_NO_FALLBACK is set, connecting TCP sockets will use ports only from the TCP shared local port pool (unless explicitly bound).

    If all shared local ports are in use, the connect() call will fail.

  • When EF_TCP_SHARED_LOCAL_PORTS_PER_IP is set, ports reserved for the pool of shared local ports will be reserved per local IP address on demand.

    This helps avoid exhaustion of the ephemeral port range.

  • When EF_TCP_SHARED_LOCAL_PORTS_REUSE_FAST is set, shared local ports can be reused immediately when the previous socket using that port has reached the CLOSED state, even if it did so via LAST-ACK.

    This allows the pool of shared local ports to be recycled more rapidly.

Further configuration options were added in Onload 201811:

  • EF_TCP_SHARED_LOCAL_PORTS_PER_IP_MAX sets the maximum size of the pool of local shared ports for a given local IP address.

    When used with scalable RSS mode this setting limits the total number within the cluster.

  • EF_TCP_SHARED_LOCAL_PORTS_STEP controls the number of ports allocated when expanding the pool of shared local ports.

    This can be used to fine tune the responsiveness of the pool.