Onload and Linux KVM

This feature is supported on Solarflare SFN8000 and X2 series adapters.

Onload includes support to accelerate applications running within Linux VMs on a KVM host. Each physical interface on the adapter can be exposed to the host as up to 16 PCIe physical functions (PF) and up to 240 virtual functions (VF). The adapter also supports up to 2048 MSI-X interrupts.

This support requires a VF (or PF) to be exposed directly into the Linux VM – KVM call this network configuration Network hostdev. Onload provides user-level access to the adapter via the VF in exactly the same way as is achieved on a non-virtualized Linux install. Firmware on the Solarflare SFN8000 and X2 series adapter configures layer 2 switching capability that supports the transport of network packets between PCI physical functions and virtual functions. This feature supports the transport of network traffic between Onload applications running in different virtual machines. This allows traffic to be replicated across multiple functions and traffic transmitted from one VM can be received on another VM.

The following figure illustrates Onload deployed into the Linux KVM Network Hostdev architecture which exposes Virtual Functions (VF) directly to the VM guest. This configuration allows the Onload data path to fully bypass the host operating system and provides maximum acceleration for network traffic.

Figure 1. Onload and Network Hostdev Configuration

To deploy Onload in a Linux KVM:

• As detailed in the SRIOV chapter of the Solarflare Server Adapter User Guide (SF-103837-CD):
  • Install the Solarflare NET driver version 4.4.1.1017 (or later)
  • Ensure the adapter is using firmware version 4.4.2.1011 (or later)
  • Run sfboot to select the full-feature firmware variant, set the switch-mode and identify the required number of VFs:

    # sfboot firmware-variant=full-feature switch-mode=sriov vf-count=4

  • Reboot the server, so the Linux KVM host can enumerate the VFs.
• Follow the instructions in Solarflare Server Adapter User Guide (SF-103837-CD) section KVM Libvirt network hostdev - Configuration to:
  • Create a VM
  • Configure the VFs
  • Unbind VFs from the host
  • Pass VFs to the VM.

  Example virsh command line and XML file configuration instructions are provided.

• Install Onload in the VM as in a non-virtualized host - see Building and Installing from a Tarball.
• Set the sfc driver module option num_vis to create the number of virtual interfaces. A VI is needed for each Onload stack created on a VF. Driver module options should be set in a user created file (for example sfc.conf) in the /etc/modprobe.d directory.

  options sfc num_vis=<NUM>

  Note: When using Onload with multiple virtual functions (VF) it is necessary to set the Onload module option oof_all_ports_required to zero. See Module Options for details.

The Solarflare Server Adapter User Guide is available from the NIC Software and Drivers web page.