XIR Based Flow for DPUCAHX8H - 1.4 English

Caffe

For caffe, vai_q_caffe is supposed to generate a PROTOTXT (deploy.prototxt) and a MODEL (deploy.caffemodel). Make sure you specify “-keep_fixed_neuron” option for vai_q_caffe which is essential for DPUCAHX8H compiler. Then the following command is almost everything you need to do to get the compiled xmodel.

vai_c_caffe -p /PATH/TO/deploy.prototxt -c /PATH/TO/deploy.caffemodel -a /PATH/TO/arch/DPUCAHX8H/PLATFORM/arch.json -o /OUTPUTPATH -n netname}

The compiler will create three files in OUTPUTPATH directory. ‘netname_org.xmodel’ is the pre-compiled xmodel which is generated by compiler frontend. ‘netname.xmodel’ is the compiled xmodel which contains instructions and other necessary information. ‘meta.json’ is for runtime.

See Model Deployment Overview for more information on deploying the network on DPU with those files.

TensorFlow

For TensorFlow, vai_q_tensorflow is supposed to generate a pb file (quantize_eval_model.pb). Notice that there are two pb files generated by vai_q_tensorflow. The quantize_eval_model.pb file is the proper one for DPUCAHX8H compiler, which is different from DPUCZDX8G. The compilation command is similar.

vai_c_tensorflow -f /PATH/TO/quantize_eval_model.pb -a /PATH/TO/arch/DPUCAHX8H/PLATFORM/arch.json -o /OUTPUTPATH -n netname}

And the outputs will be as same as Caffe.

Pytorch

For Pytorch, the quantizer NNDCT will output the quantized model in XIR format directly. Use vai_c_xir to compile it.

vai_c_xir -i /PATH/TO/quantized.xmodel -a /PATH/TO/arch/DPUCAHX8H/PLATFORM/arch.json -o /OUTPUTPATH -n netname}

And the outputs will be as same as Caffe and Tensorflow.

Currently Supported Operators

Operators listed above are commonly used in CNN models, and DPU can support many configurations of these operators.

Operators below are primitively defined in different deep learning frameworks. The compiler can automatically parse these operators and distribute them to DPU or CPU. These operators are partially supported by the tools, and they are listed here for your reference.