Monitoring Receive and Transmit Packet Buffers

Onload User Guide (UG1586)

Document ID
UG1586
Release Date
2023-07-31
Revision
1.2 English
onload_stackdump packets
# onload_stackdump packets
ci_netif_pkt_dump_all: id=1
  pkt_sets: pkt_size=2048 set_size=1024 max=32 alloc=2
  pkt_set[0]: free=544
  pkt_set[1]: free=437 current
  pkt_bufs: max=32768 alloc=2048 free=981 async=0
  pkt_bufs: rx=1067 rx_ring=1001 rx_queued=2 pressure_pool=64
  pkt_bufs: tx=0 tx_ring=0 tx_oflow=0
  pkt_bufs: in_loopback=0 in_sock=0
    1003: 0x200 Rx
   n_zero_refs=1045 n_freepkts=981 estimated_free_nonb=64
   free_nonb=0 nonb_pkt_pool=ffffffffffffffff

The onload_stackdump packets command can be useful to review packet buffer allocation, use and reuse within a monitored process.

The example above identifies that the process has a maximum of 32768 buffers (each of 2048 bytes) available. From this pool 2048 buffers have been allocated and 981 from that allocation are currently free for reuse - that means they can be pushed onto the receive or transmit ring buffers ready to accept new incoming/outgoing data.

On the receive side of the stack, 1067 packet buffers have been allocated, 1001 have been pushed to the receive ring - and are available for incoming packets, and 2 are currently in the receive queue for the application to process.

On the transmit side of the stack, zero buffers are currently allocated or being used. The remaining values are calculations based on the packet buffer values.

Using the EF_PREFAULT_PACKETS environment variable, packets can be pre-allocated to the user-process when an Onload stack is created. This can reduce latency jitter and improve Onload performance - for further details see Reducing Jitter from Page Faults.