Quantizing Using the vai_q_tensorflow2 API - 3.5 English

Vitis AI User Guide (UG1414)

Document ID
UG1414
Release Date
2023-06-29
Version
3.5 English

The following codes show how to perform post-training quantization with vai_q_tensorflow2 API. You can find a full example here.


model = tf.keras.models.load_model('float_model.h5')
from tensorflow_model_optimization.quantization.keras import vitis_quantize
quantizer = vitis_quantize.VitisQuantizer(model)
quantized_model = quantizer.quantize_model(calib_dataset=calib_dataset, 
                                           calib_steps=100, 
                                           calib_batch_size=10,
                                           **kwargs) 
calib_dataset
calib_dataset is used as a representative calibration dataset for calibration. You can use full or part of the eval_dataset, train_dataset, or other datasets.
calib_steps
calib_steps is the total number of steps for calibration. It has a default value of None. If calib_dataset is a tf.data dataset, generator, or keras.utils.Sequence instance and steps is None, calibration will run until the dataset is exhausted. This argument is not supported with array inputs.
calib_batch_size
calib_batch_size is the number of samples per batch for calibration. If the "calib_dataset" is in the form of a dataset, generator, or keras.utils.Sequence instances, the batch size is controlled by the dataset itself. If the calib_dataset is in the form of a numpy.array object, the default batch size is 32.
input_shape
list(int) or list(list(int)) or tuple(int) or dictionary(int),  contains the input shape for each input layer. Use default shape info in the input layers if not set. Use list of shapes for multiple inputs, for example input_shape=[1, 224, 224, 3] or input_shape=[[None, 224, 224, 3], [None, 64, 1]]. All dimensions should have concrete values, and batch_size dimension should be None or int. If the input shape of the model is variable like [None, None, None, 3], you need to specify a shape like [None, 224, 224, 3] to generate the final quantized model.
**kwargs
dict of the user-defined configurations of quantize strategy. It will override the default built-in quantize strategy. For example, setting bias_bit=16 will let the tool to quantize all the biases with 16bit quantizers. See the vai_q_tensorflow2 Usage section for more information of the user-defined configurations.