TensorFlow Version - 1.1 English

Vitis AI User Guide (UG1414)

Document ID
UG1414
Release Date
2020-03-23
Version
1.1 English
Use the following the steps to run vai_q_tensorflow.
  1. Prepare floating-point frozen model and dataset.
    Table 1. Input Files for vai_q_tensorflow
    No. Name Description
    1 frozen_graph Frozen Resnet-50 model.
    2 calib_images

    Before launching quantization for ResNet-50, prepare the calibration dataset. You can download 100 to 1000 images of ImageNet dataset from http://academictorrents.com/collection/imagenet-2012 or http://www.image-net.org/download.php

    and then change the calibration dataset path in the input_fn.

    3 input_fn A python function to read images in calibration dataset and complete the pre-processing instructions.

    Input files for vai_q_tensorflow is shown in the above table. The frozen model can be downloaded from Xilinx modelzoo (https://github.com/Xilinx/Vitis-AI/tree/master/AI-Model-Zoo). Scripts to evaluate the models can also be found in modelzoo.

    input_fn is a python function to read images in calibration dataset and complete the pre-processing instructions. An example of input_fn.py is shown below. A class named Data_loader is implemented to load image and do pre-processing. Here the pre-processing includes center_crop and mean extraction. calib_image_list is an image list file for calibration and calib_image_dir is the directory containing the calibration image files. Function calib_input is the required input function for quantizer.

    import tensorflow as tf
    import os
    
    _R_MEAN = 123.68
    _G_MEAN = 116.78
    _B_MEAN = 103.94
    
    class Data_loader(object):
      def __init__(self, out_height, out_width, smallest_side=256):
        self._sess = tf.Session()
        self._out_height = out_height
        self._out_width = out_width
        self._smallest_side = smallest_side
    
        self._decode_jpeg_data = tf.placeholder(dtype=tf.string)
        self._decode_jpeg = tf.image.decode_jpeg(self._decode_jpeg_data, channels=3)
    
        self._image_pl = tf.placeholder(tf.float32, shape=(None, None, 3))
        self._resized_image = self._aspect_preserving_resize(self._image_pl, self._smallest_side)
    
      def _center_crop(self, image):
        image_height, image_width = image.shape[:2]
        offset_height = (image_height - self._out_height) // 2
        offset_width = (image_width - self._out_width) // 2
        image = image[offset_height:offset_height + self._out_height,
                      offset_width:offset_width + self._out_width, :]
        return image
    
      def _smallest_size_at_least(self, height, width, smallest_side):
        """Computes new shape with the smallest side equal to `smallest_side`.
    
        Computes new shape with the smallest side equal to `smallest_side` while
        preserving the original aspect ratio.
    
        Args:
          height: an int32 scalar tensor indicating the current height.
          width: an int32 scalar tensor indicating the current width.
          smallest_side: A python integer or scalar `Tensor` indicating the size of
            the smallest side after resize.
    
        Returns:
          new_height: an int32 scalar tensor indicating the new height.
          new_width: and int32 scalar tensor indicating the new width.
        """
        smallest_side = tf.convert_to_tensor(smallest_side, dtype=tf.int32)
    
        height = tf.to_float(height)
        width = tf.to_float(width)
        smallest_side = tf.to_float(smallest_side)
    
        scale = tf.cond(tf.greater(height, width), lambda: smallest_side / width, lambda: smallest_side / height)
        new_height = tf.to_int32(tf.rint(height * scale))
        new_width = tf.to_int32(tf.rint(width * scale))
        return new_height, new_width
    
      def _aspect_preserving_resize(self, image, smallest_side):
        """Resize images preserving the original aspect ratio.
    
        Args:
          image: A 3-D image `Tensor`.
          smallest_side: A python integer or scalar `Tensor` indicating the size of
            the smallest side after resize.
    
        Returns:
          resized_image: A 3-D tensor containing the resized image.
        """
        smallest_side = tf.convert_to_tensor(smallest_side, dtype=tf.int32)
        shape = tf.shape(image)
        height = shape[0]
        width = shape[1]
        new_height, new_width = self._smallest_size_at_least(height, width, smallest_side)
        image = tf.expand_dims(image, 0)
        resized_image = tf.image.resize_bilinear(image, [new_height, new_width], align_corners=False)
        resized_image = tf.squeeze(resized_image)
        #resized_image.set_shape([None, None, 3])
        return resized_image
    
      def preprocess(self, image):
        assert image is not None, "image cannot be None"
        resized_image = self._sess.run(self._resized_image, feed_dict={self._image_pl: image})
        image_crop = self._center_crop(resized_image)
        image = image_crop - [_R_MEAN, _G_MEAN, _B_MEAN]
        return image
    
      def load_image(self, img_path):
        assert os.path.exists(img_path), img_path + ' doesnot exists!'
        image_data = tf.gfile.GFile(img_path, 'rb').read()
        image = self._sess.run(self._decode_jpeg, feed_dict={self._decode_jpeg_data: image_data})
        assert len(image.shape) == 3
        assert image.shape[-1] == 3
        return image
    
    calib_image_dir = "./imagenet_images/"
    calib_image_list = "./imagenet_calib.txt"
    calib_batch_size = 50
    input_height = 224
    input_width = 224
    
    def calib_input(iter):
      images = []
      data_loader = Data_loader(input_height, input_width)
      line = open(calib_image_list).readlines()
      for index in range(0, calib_batch_size):
        curline = line[iter * calib_batch_size + index]
        calib_image_name = curline.strip()
        filename = os.path.join(calib_image_dir, calib_image_name)
        image = data_loader.load_image(filename)
        image = data_loader.preprocess(image)
        images.append(image.tolist())
      return {"input": images}
    
    The calibration image list file calib_image_list looks like this:
    ILSVRC2012_val_00000001.JPEG
    ILSVRC2012_val_00000002.JPEG
    ILSVRC2012_val_00000003.JPEG
    ILSVRC2012_val_00000004.JPEG
    ...
  2. Activate Tensorflow running environment.
    conda activate vitis-ai-tensorflow
  3. Run vai_q_tensorflow to quantize the TensorFlow frozen models.
    vai_q_tensorflow quantize \
      --input_frozen_graph resnet_v1_50_inference.pb \
      --input_nodes input \
      --input_shapes ?,224,224,3 \
      --output_nodes resnet_v1_50/predictions/Reshape_1 \
      --input_fn input_fn.calib_input \
      --method 1 \
      --gpu 0 \
      --calib_iter 20 \
      --output_dir ./quantize_results \
    

    Here --input_fn is set to be "input_fn.calib_input". input_fn is the name of python script and calib_input is the function name in input_fn.py. The script may take several minutes to finish. Once the quantization is done, quantization summary will display like this:

    INFO: Checking Float Graph...
    INFO: Float Graph Check Done.
    2020-03-07 06:46:35.567522: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:538] Convert mean node resnet_v1_50/pool5 to AvgPool
    2020-03-07 06:46:35.572301: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:628] Scale output of avg_pool node resnet_v1_50/pool5 to simulate DPU.
    INFO: Calibrating for 20 iterations...
    100% (20 of 20) |#####################################################################################################| Elapsed Time: 0:21:11 Time:  0:21:11
    INFO: Calibration Done.
    INFO: Generating Deploy Model...
    [DEPLOY WARNING] Node resnet_v1_50/predictions/Reshape_1(Type: Reshape) is not quantized and cannot be deployed to DPU,because it has unquantized input node: resnet_v1_50/predictions/Softmax. Please deploy it on CPU.
    INFO: Deploy Model Generated.
    ********************* Quantization Summary *********************
    INFO: Output:
      quantize_eval_model: ./quantize_results/quantize_eval_model.pb
      deploy_model: ./quantize_results/deploy_model.pb
    

    Two files will be generated in quantize_results directory. The deploy_model.pb could be fed to VAI compiler for the following compilation processes targeting hardware platform DPUv2. The quantize_eval_model.pb can be used for model evaluation and dump on GPU or CPU. It is also the input file for compilation processes targeting hardware platform DPUv3.