Mixed Adapters Sharing a Broadcast Domain

Problem 1

Traffic from S1 to S2 can be delivered through either of the interfaces on S2, irrespective of the IP address used. This means that if one interface is accelerated by Onload and the other is not, you might or might not get acceleration.

To resolve the situation (for the current session) issue the following command:

echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore

or to resolve it permanently add the following line to the /etc/sysctl.conf file:

net.ipv4.conf.all.arp_ignore = 1

and run the sysctl command for this be effective.

sysctl -p

These commands ensure that an interface will only respond to an ARP request when the IP address matches its own. Refer to the Linux documentation Linux/Documentation/networking/ip-sysctl.txt for further details.

Problem 2

A more serious problem arises if one interface on S2 carries Onload accelerated TCP connections and another interface on the same host and same broadcast domain is non-Solarflare:

A TCP packet received on the non-Solarflare interface can result in accelerated TCP connections being reset by the kernel stack and therefore appear to the application as if TCP connections are being dropped/terminated at random.

To prevent this situation the Solarflare and non-Solarflare interfaces should not be configured in the same broadcast domain. The solution described for Problem 1 above can reduce the frequency of Problem 2, but does not eliminate it.

TCP packets can be directed to the wrong interface because:

• the originator S1 needs to refresh its ARP table for the destination IP address - so sends an ARP request and subsequently directs TCP packets to the non-Solarflare interface
• a switch within the broadcast domain broadcasts the TCP packets to all interfaces.