Profiling for AI Engine - 2023.2 English

AI Engine Tools and Flows User Guide (UG1076)

Document ID
UG1076
Release Date
2023-12-04
Version
2023.2 English
The following tables list the pre-defined metric set configurations available for AI Engine, in order of priority by which they are assigned to the available counters. In the xrt.ini file all these metric names should be in lower case and assigned to one of the following metric selectors:
  • tile_based_aie_metrics
  • graph_based_aie_metrics
Table 1. heat_map Metric
Metric Name Event ID Description
Active Time 28 Time AI Engine was active since it was enabled.
Stall Time 22 Time AI Engine was stalled. This stall includes AI Engine memory, stream, cascade, and lock stalls.
Vector Instruction Time 37 Time AI Engine spent executing instructions in the vector processor.
Cumulative Instruction Time 32 Time AI Engine spent executing load/store, stream get/put, lock acquire/release instructions.
Active Utilization Derived Time AI Engine is actively executing instructions and not stalling. The percentage is relative to active time.
Vector Instruction Utilization Derived Time AI Engine is executing vector instructions. The percentage is relative to active utilization time (active - stalls).
These indicators help you understand the efficiency of the kernels that are implemented in the AI Engines. You can compare stall time with active time to determine if there is a data communication issue for each AI Engine.
Table 2. stalls Metric
Metric Name Event ID Description
Memory Stall Time 23 Time the AI Engine was not active due to a memory stall.
Stream Stall Time 24 Time the AI Engine was not active due to a stream stall.
Lock Stall Time 26 Time the AI Engine was in a lock stall.
Cascade Stall Time 25 Time the AI Engine was in a cascade stall.
A stall in an AI Engine can occur in various situations:
  • A memory stall happens when multiple accesses to the same memory bank are requested from one core, multiple cores, and/or DMAs.
  • Stream stalls occur when data production and consumption on a stream do not have the same rate, leading to input stream starvation or output stream overflow.
  • A cascade stall is generated when the cascade writer does not have the same rate as the cascade reader.
  • A lock stall happens if the window data producer does not have the same iteration rate as the window consumer.
Table 3. execution Metric
Metric Name Event ID Description
Vector Instruction Time 37 Time spent by the AI Engine on vector instructions: vector processor instruction and vector data load/store
Load Instruction Time 38 Time spent by the AI Engine on load instructions (move data from memory to registers)
Store Instruction Time 39 Time spent by the AI Engine on store instructions (move data from registers to memory)
Cumulative Instruction Time 32 Time spent by the AI Engine on memory and stream accesses and lock acquire/release
All these indicators allow you to estimate the efficiency of your kernel. To increase efficiency, you should optimize data access, favor vector instructions over scalar instructions, and use 128-bit access to streams whenever possible.
Table 4. floating_point Metric
Metric Name Event ID Description
Floating-Point Overflow Exception 50 Number of floating-point overflow exceptions generated by AI Engine
Floating-Point Underflow Exception 51 Number of floating-point underflow exceptions generated by AI Engine
Floating-Point Invalid Exception 52 Number of floating-point Invalid exceptions generated by AI Engine
Floating-point Divide by Zero Exception 53 Number of floating-point divide by zero exceptions generated by AI Engine
Floating-point exceptions lead to erroneous results. You might have to recode your floating-point algorithm if you get too many exceptions, or even a single in a critical area of the code.
Table 5. aie_trace Metric
Metric Name Event ID Description
AI Engine Trace Word Count 75 Amount of AI Engine trace produced.
AI Engine Trace Stall Count 76 Amount of AI Engine trace back-pressure events produced.
Memory Module Trace Word Count 79 Amount of Memory Module trace produced.
Memory Module Trace Stall Count 80 Amount of Memory Module trace back-pressure events produced.
These metrics, particularly the stall counts, help in defining the right number of streams to transmit AI Engine and Memory Module events to the Programmable Logic.
Table 6. write_throughputs Metric
Metric Name Event ID Description
Active Time 28 Time AI Engine was active since it was enabled.
Stream Write Instruction Time 41 Time AI Engine spent executing write instructions on data streams.
Cascade Write Instruction Time 43 Time AI Engine spent executing write instructions on the cascade stream.
Stall Time 22 Time AI Engine was stalled. This stall includes AI Engine memory, stream, cascade, and lock stalls.
Stream Write Throughput (MB/s) Derived Write throughput of Stream Ports in MB/s
Cascade Write Throughput (MB/s) Derived Write throughput of Cascade Ports in MB/s
These metrics are useful to evaluate the overall output write throughput of the system.
Table 7. read_throughputs Metric
Metric Name Event ID Description
Active Time 28 Time AI Engine was active since it was enabled.
Stream Read Instruction Time 40 Time AI Engine spent executing read instructions on data streams.
Cascade Read Instruction Time 42 Time AI Engine spent executing read instructions on the cascade stream.
Stall Time 22 Time AI Engine was stalled. This stall includes AI Engine memory, stream, cascade, and lock stalls.
Stream Read Throughput (MB/s) Derived Read throughput of Stream Ports in MB/s
Cascade Read Throughput (MB/s) Derived Read throughput of Cascade Ports in MB/s

These metrics are useful to evaluate the overall output read throughput of the system.