When deciding the number of packet buffers required by an Onload stack consideration should be given to the resource needs of the stack to ensure that the available packet buffers can be shared efficiently between all Onload stacks.
-
Example 1:
If we consider a hypothetical case of a single host:
- which employs multiple Onload stacks, for example 10
- each stack has multiple sockets, for example 6
- and each socket uses many packet buffers, for example 2000
This would require a total of 120,000 packet buffers
-
Example 2:
If on a stack the TCP receive queue is 1 MB and the MSS value is 1472 bytes, this would require at least 700 packet buffers - (and a greater number if segments smaller that the MSS were received).
-
Example 3:
A UDP receive queue of 500 received datagrams, each 200 bytes long, has a total payload of approximately 100 KB. However, it will consume approximately 1 MB. This is because each packet buffer is 2048 bytes, even though the packet data is less than 2048 bytes.
The examples above use only approximate calculated values. The onload_stackdump command provides accurate measurements of packet buffer allocation and usage.
Consideration should be given to packet buffer allocation to ensure that each stack is allocated the buffers it will require rather than a ‘one size fits all’ approach.
When using the Buffer Table Mode the system is limited to 120,000 packet buffers - these are allocated symmetrically across all Solarflare interfaces.