Cascade Stall Analysis - 2022.2 English

AI Engine Tools and Flows User Guide (UG1076)

Document ID
UG1076
Release Date
2022-10-19
Version
2022.2 English
From Performance Metrics analysis, you can identify the cascade stall that needs to be analyzed.
  1. Choose Trace view.
  2. Choose Stalls view and select Cascade Stalls from the drop-down list.
  3. The filter button on the upper-right can be used to select the signals in the Trace view. It is helpful to show all cascade stalls in a window to better understand how they occur in a timeline because cascade stalls are usually correlated to each other in the cascade chain. Click on the filter button, de-select All at the top and select all kernels of interest.
  4. Select Cascade stall to highlight the stall in Trace view.
    Figure 1. Cascade Stall in Trace View

    Each cascade stall has the following information.

    Stall ID
    The stream stall is named CS_<NUM>. The earlier the stall happens, the smaller the number. The number is unique across all types of stall.
    Stalled Tile
    The AI Engine tile where the stalled kernel is located.
    Stalled Kernel
    The kernel that is stalled. It is named <Kernel_function_name>.<Schedule_ID>.<Graph_instance_name>. Sometimes it is shown as _main and then cross-probe is required to find the real kernel function.
    Start (ns)
    The start time that the stall happens
    Duration (ns)
    The duration of the stall.
    PC
    Program counter when the stall happens.
    Stalled Port
    Which port of the stalled kernel is stalled.
    Related Stalls
    Other stalls that might cause the stall.
    Full Destination Port
    The port that the stalled kernel cannot write into because it is full.
    Empty Source Port
    The port that the stalled kernel cannot read from because it is empty.

    Explore the cascade stalls in the Trace view and see how they are related.

  5. It is possible that other types of stalls cause the cascade stall. Explore the other types of stalls to analyze why that happened. Related stalls in the Stalls view show which stall causes the cascade stall.
  6. Viewing the stalls happen in Graph view can help identify the cause of the stall. Select Graph view.
  7. Select Stalls view and then select Cascade Stalls from the drop-down list.
  8. Click a path in the Stalls view and the stall path is shown in red in Graph view.
  9. Exploring the cascade stall and its related stalls, can help in finding a hint about the cause of the stall. Clicking Related Stalls can also show the related stall in red. For example, by clicking SS_16 in Related Stalls, you can see the path in red, which gives a hint about where the stall starts.
    Figure 2. Cascade Stall in Graph View

The following table lists some possible scenarios that cause cascade stall and possible solutions.

Table 1. Cascade Stall Scenarios and Solutions
Source Destination Stall type Possible solution
Cascade Stream Cascade Stream Cascade Stall
  • Adjust cascade stream read/write instructions in kernel execution cycles to match between source and destination kernels
  • Analyze other sources that cause the cascade stall
Stream + Cascade Streams Stream + Cascade Streams Cascade Stall
  • Adjust instructions to match between different streams
  • Analyze other sources that cause the cascade stall