C2H Stream Packet Type - 4.0 English

QDMA Subsystem for PCI Express Product Guide (PG302)

Document ID
PG302
Release Date
2022-05-20
Version
4.0 English

The following are some of the different C2H stream packets.

Regular Packet

The regular C2H packet has both the data packet and Completion (CMPT) packet. They are a one-to-one match.

The regular C2H data packet can be multiple beats.

  • s_axis_c2h_ctrl_qid = C2H descriptor queue ID.
  • s_axis_c2h_ctrl_len = length of the packet.
  • s_axis_c2h_mty = empty byte should be set in last beat.
  • s_axis_c2h_ctrl_has_cmpt = 1'b1. This data packet has a corresponding CMPT packet.

The regular C2H CMPT packet is one beat.

  • s_axis_c2h_cmpt_ctrl_qid = Completion queue ID of the packet. This can be different from the C2H descriptor QID.
  • s_axis_c2h_cmpt_ctrl_cmpt_type = HAS_PLD. This completion packet has a corresponding data packet.
  • s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id = This completion packet has to wait for the data packet with this ID to be sent before the CMPT packet can be sent.

When the user application sends the data packet, it must count the packet ID for each packet. The first data packet has a packet ID of 1, and it increments for each data packets.

For the regular C2H packet, the data packet and the completion packet is a one-to-one match. Therefore, the number of data packets with s_axis_c2h_ctrl_has_cmpt as 1'b1 should be equal to the number of CMPT packet with s_axis_c2h_cmpt_ctrl_cmpt_type as HAS_PLD.

The QDMA Subsystem for PCIe has a shallow completion input FIFO of depth 2. For better performance, add FIFO for completion input as shown in the diagram below. Depth and width of the FIFO depends on the use case. Width is dependent on the largest CMPT size for the application, and depth is dependent on performance needs. For best performance for 64 Byte CMPT, a depth of 512 is recommended.

When the user application sends the data payload, it counts every packet. The first packet starts with a pkt_pld_id of 1. The second packet has a pkt_pld_id of 2, and so on. It is a 16-bits counter once the count reaches 16'hffff it wraps around to 0 and count forward.

The user application defines the CMPT type.

  • If the s_axis_c2h_cmpt_ctrl_cmpt_type is HAS_PLD, the CMPT has a corresponding data payload. The user application must place pkt_pld_id of that packet in the s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id field. The DMA will only send out this CMPT after it sends out the corresponding data payload packet.
  • If the s_axis_c2h_cmpt_ctrl_cmpt_type is NO_PLD_NO_WAIT, the CMPT does not have any data payload, and it does not need to wait for payload. Then the DMA will send out this CMPT.
  • If the s_axis_c2h_cmpt_ctrl_cmpt_type is NO_PLD_BUT_WAIT, the CMPT does not have a corresponding data payload packet. The CMPT must wait for a particular data payload packet before the CMPT is sent out. Therefore, the user application must place the pld_pkt_id of that particular data payload into the s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id field. The DMA will not send out the CMPT until the data payload with that pld_pkt_id is sent out.
Figure 1. CMPT Input FIFO

Immediate Data Packet

The user application can have a packet that only writes to the Completion Ring without having a corresponding data packet transfer to the host. This type of packet is called immediate data packet. For the immediate data packet, the QDMA will not send the data payload, but it will write to the CMPT Queue. The immediate packet does not consume a descriptor.

For the immediate data packet, the user application only sends the CMPT packet to the DMA, and it does not send the data packet.

The following is the setting of the immediate completion packet. There is no corresponding data packet.

In some applications, the immediate completion packet does not need to wait for any data packet. But in some applications, it might still need to wait for the data payload packet. When the completion type is NO_PLD_NO_WAIT, the completion packet can be sent out without waiting for any data packet. When the completion type is NO_PLD_BUT_WAIT, the completion packet must specify the data packet ID that it needs to wait for.

  • s_axis_c2h_cmpt_user_cmpt_type = NO_PLD_NO_WAIT or NO_PLD_BUT_WAIT.
  • s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id = Do not increment packet count.

Marker Packet

The C2H Stream Engine of the QDMA provides a way for the user application to insert a marker into the QDMA along with a C2H packet. This marker then propagates through the C2H Engine pipeline and comes out on the Queue status port interface. The marker is inserted by setting the marker bit in the C2H Stream packet. The marker response is indicated by the QDMA to the user application by setting the qsts_out_op[7:0] = 0x0 (CMPT Marker response) bits on the Queue status ports. For a marker packet, QDMA does not send out a payload packet but still writes to the Completion Ring. Not all marker responses are generated because of a corresponding marker request. The QDMA some times generates marker responses when it encounters exceptional events. See the following section for details about when QDMA internally generates marker responses.

The primary purpose of giving the user application the ability of sending in a marker into QDMA is to determine when all the traffic prior to the marker has been flushed. This can be used in the shut down sequence in the user application. Although not a requirement, the marker can be sent by the user application with the user_trig bit set when sending in the marker into QDMA. This allows the QDMA to generate an interrupt and truly ensures that all traffic prior to the marker is flushed out. The QDMA Completion Engine takes the following actions when it receives a marker from the user application:

The DMA also has an option not to send completion information to completion ring during a Marker packet. Port s_axis_c2h_cmpt_ctrl_no_wrb_marker can be set during a marker packet. This option disables any write to completion ring when that Marker packet is generated. When this signal is set for a Maker packet, the DMA has no data or completion transfers, but will respond with maker response on qsts_out_op[7:0].

  • Sends the Completion that came along with the marker to the C2H Stream Completion Ring.
  • Sends lower 24bits of completion data to the Queue status data port qsts_out_data[26:3].
  • Generates Status Descriptor if enabled (if user_trig was set when maker was inserted).
  • Generates an Interrupt if enabled and not outstanding.
  • Sends the marker response. If an Interrupt was not sent due to it being enabled but outstanding, the retry marker_req bit in the marker response is set to inform the user that an Interrupt could not be sent for this marker request. See the Queue status ports interface description for details of these fields.

The marker packet has both the data packet and CMPT packet. They are one-to-one match.

The following is the setting of the data packet with marker:

  • 1 beat of data
  • s_axis_c2h_ctrl_marker = 1'b1
  • s_axis_c2h_ctrl_len = data width (for example, 64 if data width is 512 bits)
  • s_axis_c2h_mty = 0
  • s_axis_c2h_ctrl_has_cmpt = 1'b1
The following is the setting of the CMPT packet with marker:
  • 1 beat of CMPT packet
  • s_axis_c2h_cmpt_ctrl_marker = 1'b1
  • s_axis_c2h_cmpt_ctrl_cmpt_type = HAS_PLD
  • s_axis_c2h_cmpt_ctrl_wait_pld_pkt_id = This completion packet has to wait for the data payload packet with this ID to be sent before the CMPT packet is sent.
  • s_axis_c2h_cmpt_ctrl_no_wrb_marker = 1'b0. If this is set to 1'b1then no CMPT packet.

The immediate data packet and the marker packet do not consume the descriptor; instead, they write to the C2H Completion Ring. The software needs to size the C2H Completion Ring large enough to accommodate the outstanding immediate packets and the marker packets.

When a marker request is received by the DMA and the completion ring is full, the marker response will be sent out. However, because the completion ring is full, the completion entry will be dropped and the queue will be invalidated.

Zero Length Packet

The length of the data packet can be zero. On the input, the user needs to send one beat of data. The zero length packet consumes the descriptor. The QDMA will send out 1DW payload data.

The following is the setting of the zero length packet:

  • 1 beat of data
  • s_axis_c2h_ctrl_len = 0
  • s_axis_c2h_mty = 0
Note: Zero Byte packets are not supported in Internal mode and Cache bypass mode. The QDMA may hang if zero byte packets are dropped due to not available descriptor. Zero Byte Packets are supported in Simple bypass mode.

Disable Completion Packet

The user application can disable the completion for a specific packet. The QDMA provides direct memory access (DMA) to the payload, but does not write to the C2H Completion Ring. The user application only sends the data packet to the DMA, and does not send the CMPT packet.

The following is the setting of the disable completion packet:

  • s_axis_c2h_ctrl_has_cmpt = 1'b0