For TensorFlow Model - 1.3 English

TensorFlow framework supports very flexible pre-processing during model training, such as using BGR or RGB color space for input images. Therefore, the pre-optimized routines dpuSetInputImage() and dpuSetInputImageWithScale() cannot be used directly while deploying TensorFlow models. Instead, you need to implement a pre-processing code.

The following code snippet shows an example to load an image into DPU input tensor for TensorFlow model. Note that the image color space fed into the DPU input tensor should be the same with the format used during model training. With data[j*image.rows*3+k*3+2-i], the image is fed into DPU in RGB color space. The process of image.at<Vec3b>(j,k)[i])/255.0 - 0.5)*2 * scale is specific to the model being deployed. It should be changed accordingly for the actual model used.

void setInputImage(DPUTask *task, const string& inNode, const cv::Mat& image) {
  DPUTensor* in = dpuGetInputTensor(task, inNode);
  float scale = dpuGetTensorScale(in);
  int width = dpuGetTensorWidth(in);
  int height = dpuGetTensorHeight(in);
  int size = dpuGetTensorSize(in);
  int8_t* data = dpuGetTensorAddress(in);

  for(int i = 0; i < 3; ++i) {
    for(int j = 0; j < image.rows; ++j) {
      for(int k = 0; k < image.cols; ++k) {
	       data[j*image.rows*3+k*3+2-i] = 
          (float(image.at<Vec3b>(j,k)[i])/255.0 - 0.5)*2 * scale;
      }
     }
   }
}

Python is very popularly used for TensorFlow model training. With Vitis AI advanced Python APIs, you can reuse the pre-processing and post-processing Python codes during the training phase. This can help to speed up the workflow of model deployment on the DPU for quick evaluation purpose. After that it can be transformed into C++ code for better performance to meet the production requirements. The DNNDK sample miniResNet provides a reference to deploy TensorFlow miniResNet model with Python.