Computation Bit Width Growth - 2.4 English

Video Processing Subsystem Product Guide (PG231)

Document ID
Release Date
2.4 English

Full precision (DATA_WIDTH+16+log2(N Taps ) bits) is maintained during the horizontal and/or vertical FIR convolution operation.

FIR filter outputs are rounded to DATA_WIDTH bits by adding half an output LSB in the full precision domain prior to truncation. Clipping and clamping of the output data prevents overflows and underflows. Data is clipped and clamped at 2 DATA_WIDTH -1 and 0.