Xilinx data compression library is an open-sourced performance-optimized Vitis library written in C++ for accelerating data compression applications on Xilinx Accelerator cards in a variety of use cases. The library covers two levels of acceleration: the module level and the pre-defined kernel level, and evolve to offer the third level as pure software APIs working with pre-defined hardware overlays.
- L1: Module level, it provides optimized hardware implementation of the core LZ based and data compression algorithm specific modules .
- L2: Kernel level, this section calls compression/decompression kernel which internally uses the optimized hardware modules to showcase various kernel demos.
- L3: The software API level will wrap the details of offloading acceleration with prebuilt binary (overlay) and allow users to accelerate data compression tasks on Alveo cards without hardware development.
It is designed as a specialized compression engine, multiple of which can run concurrently on the same Xilinx accelerator card to meet the high-throughput requirements of your algorithms. This reduces the bandwidth consumption and the overall infrastructure costs, on-premise or in the cloud.
GZIP compression kernel takes the raw data as input and compresses the data in block based fashion and writes the output to global memory. LZ77 is a byte based compression scheme. The resulting output from this kernel is represented in packet form of 32bit length <Literal, Match Length, Distance>.It also generates output of literal and distance frequencies for dynamic Huffman tree generation. The output generated by this kernel is referred by TreeGen and Huffman Kernel.