Packet Loss on the Transmission Path

Onload User Guide (UG1586)

Document ID
Release Date
1.2 English

Occasionally Onload needs to send a packet, which would normally be accelerated, via the kernel. This occurs when there is no destination address entry in the ARP table or to prevent an ARP table entry from becoming stale.

By default, the Linux sysctl, unres_qlen, will enqueue three packets per unresolved address when waiting for an ARP reply, and on a server subject to a very high UDP or TCP traffic load this can result in packet loss on the transmit path and packets being discarded.

The unres_qlen value can be identified using the following command:

sysctl -a | grep unres_qlen
net.ipv4.neigh.eth2.unres_qlen = 3
net.ipv4.neigh.eth0.unres_qlen = 3
net.ipv4.neigh.lo.unres_qlen = 3
net.ipv4.neigh.default.unres_qlen = 3

Changes to the queue lengths can be made permanent in the /etc/sysctl.conf file. It is recommended to set the unres_qlen value to at least 50.

If packet discards are suspected, this extremely rare condition can be indicated by the cp_defer counter produced by the onload_stackdump lots command on UDP sockets or from the unresolved_discards counter in the Linux /proc/net/stat arp_cache file.