VGG-16 - 1.1 English

AI Optimizer User Guide (UG1333)

Document ID
UG1333
Release Date
2020-07-07
Version
1.1 English

This sample demonstrates how to run vai_p_tensorflow on real-world models. VGG (https://arxiv.org/abs/1409.1557) is a network for large-scale image recognition. This sample uses a pre-trained VGG-16 model from TensorFlow-Slim image classification model library.

Download Tensorflow-Slim’s repository and its pre-trained VGG16 model:

$ git clone https://github.com/tensorflow/models.git 
$ cd models/research/slim
# mkdir models/vgg16 && cd models/vgg16
$ wget http://download.tensorflow.org/models/vgg_16_2016_08_28.tar.gz
$ tar xzvf vgg_16_2016_08_28.tar.gz

Use the following instructions to prepare the ImageNet dataset:

https://github.com/tensorflow/models/blob/master/research/inception/README.md#getting-started

Prepare graph evaluation script, named vgg16_eval.py:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import math
import tensorflow as tf

from tensorflow.python.summary import summary
from tensorflow.python.training import monitored_session
from tensorflow.python.training import saver as tf_saver

from datasets import dataset_factory
from nets import nets_factory
from preprocessing import preprocessing_factory

slim = tf.contrib.slim

dataset_name='imagenet'
dataset_split_name='validation'
dataset_dir='/dataset/imagenet/tf_records'
model_name='vgg_16'
labels_offset=1
batch_size=100
num_preprocessing_threads=4

def model_fn():
  tf.logging.set_verbosity(tf.logging.INFO)

  tf_global_step = slim.get_or_create_global_step()

  ######################
  # Select the dataset #
  ######################
  dataset = dataset_factory.get_dataset(dataset_name,
                                            dataset_split_name,
                                            dataset_dir)

####################
  # Select the model #
  ####################
  network_fn = nets_factory.get_network_fn(
      model_name,
      num_classes=(dataset.num_classes - labels_offset),
      is_training=False)

  ##############################################################
  # Create a dataset provider that loads data from the dataset #
  ##############################################################
  provider = slim.dataset_data_provider.DatasetDataProvider(
      dataset,
      shuffle=False,
      common_queue_capacity=2 * batch_size,
      common_queue_min=batch_size)
  [image, label] = provider.get(['image', 'label'])
  label -= labels_offset

  #####################################
  # Select the preprocessing function #
  #####################################
  preprocessing_name = model_name
  image_preprocessing_fn = preprocessing_factory.get_preprocessing(
      preprocessing_name,
      is_training=False)

  eval_image_size = network_fn.default_image_size

  image = image_preprocessing_fn(image, eval_image_size, eval_image_size)

  images, labels = tf.train.batch(
      [image, label],
      batch_size=batch_size,
      num_threads=num_preprocessing_threads,
      capacity=5 * batch_size)

####################
  # Define the model # 
  ####################
  logits, _ = network_fn(images)

  variables_to_restore = slim.get_variables_to_restore()

  predictions = tf.argmax(logits, 1)
  org_labels = labels
  labels = tf.squeeze(labels)

  eval_metric_ops = {
      'top-1': slim.metrics.streaming_accuracy(predictions, labels),
      'top-5': slim.metrics.streaming_recall_at_k(logits, org_labels, 5)
  }
  return eval_metric_ops

Modify models/research/slim/export_inference_graph.py to output a readable text file instead of binary file.

+ from google.protobuf import text_format

-   with gfile.GFile(FLAGS.output_file, 'wb') as f:
-     f.write(graph_def.SerializeToString())
+   with gfile.GFile(FLAGS.output_file, 'w') as f:
+     f.write(text_format.MessageToString(graph_def))

Export an inference graph:

python export_inference_graph.py \
    --model_name=vgg_16 \
    --output_file=vgg_16_inf_graph.pbtxt \
    --dataset_dir=/opt/dataset/tf_records

Run model analysis:

vai_p_tensorflow \
  --action=ana \
  --input_graph=vgg_16_inf_graph.pbtxt \
  --input_ckpt=vgg_16.ckpt \
  --eval_fn_path=vgg_16_eval.py \
  --target=top-5 \
  --max_num_batches=500 \
  --workspace=/home/deephi/models/research/slim/models/vgg16 \
  --exclude="vgg_16/fc6/Conv2D, vgg_16/fc7/Conv2D, vgg_16/fc8/Conv2D" \
--output_nodes="vgg_16/fc8/squeezed"

In vgg_16_eval.py, we defined a variable named batch_size with initial value of 100, and there are 50,000 examples in the validation set of ImageNet, so the max_num_steps is set to 500 to ensure that all the examples in the validation set are tested in evaluation.

Important: The nodes with vgg_16/fc prefix affect the number of output labels of the network, so you need to exclude these codes to prevent shape mismatch with the dataset.

Run model pruning:

vai_p_tensorflow \
  --action=prune \
  --input_graph=vgg_16_inf_graph.pbtxt \
  --input_ckpt=vgg_16.ckpt \
  --output_graph=sparse_graph.pbtxt \
  --output_ckpt=sparse.ckpt \
  --workspace=/home/deephi/models/research/slim/models/vgg16 \
  --sparsity=0.15 \
  --exclude="vgg_16/fc6/Conv2D, vgg_16/fc7/Conv2D, vgg_16/fc8/Conv2D" \
  --output_nodes="vgg_16/fc8/squeezed"

Open models/research/slim/train_image_classifier.py and insert the following line into the beginning of main() function:

def main():
+  tf.set_pruning_mode()

Fine-tune the pruned model:

python train_image_classifier.py \
    --model_name=vgg_16 \
    --train_dir=./models/vgg16/ft \
    --dataset_name=imagenet \
    --dataset_dir=/opt/dataset/tf_records \
    --dataset_split_name=train \
    --checkpoint_path=./models/vgg16/sparse.ckpt \
    --labels_offset=0 \
    --save_interval_secs=600 \
    --batch_size=32 \
    --num_clones=4 \
    --weight_decay=5e-4 \
    --optimizer=adam \
    --learning_rate=1e-2 \
    --learning_rate_decay_type=polynomial \
    --decay_steps=200000 \
--max_number_of_steps=200000

Get a dense checkpoint and freeze graph:

vai_p_tensorflow \
--action=transform \
--input_ckpt=./models/vgg16/ft/model.ckpt-200000 \
--output_ckpt=./models/vgg16/dense.ckpt

freeze_graph.py \
--input_graph=./models/vgg16/sparse_graph.pbtxt \
--input_checkpoint=./models/vgg16/dense.ckpt \
--input_binary=false  \
--output_graph=./models/vgg16/vgg16_pruned.pb \
--output_node_names=”vgg_16/fc8/squeezed"

As the process of quantization are almost the same, only Resnet_v1_50 quantization is included in this user guide.For more information, see here.