One-step pruning implements the EagleEye 1 algorithm. It introduces a strong positive correlation between different pruned models and their corresponding fine-tuned accuracy by a simple yet efficient evaluation component, that is the adaptive batch normalization. It enables you to get the subnetwork with the highest potential accuracy without actually fine-tuning the models. In short, the one-step pruning method searches for a bunch of subnetworks (i.e., generated pruned models) that meet the required model size, and applies an evaluation for selecting the most potential one from them. The selected subnetwork is then retained to recover the accuracy.
The pruning steps are as follows:
- Search for subnetworks that meet the required pruning ratio.
- Select a potential network from a bunch of subnetworks with an evaluation component.
- Fine-tune the pruned model.
Figure 1. One-step Pruning Workflow
Note:
- Bailin Li et al., EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning, arXiv:2007.02491