Abstract
In this paper, we propose a post-training pruning framework that jointly optimizes layerwise pruning to minimize model output distortion. Through theoretical and empirical analysis, we discover an important additivity property of output distortion from pruning weights/channels in DNNs. Leveraging this property, we reformulate pruning optimization as a combinatorial problem and solve it with dynamic programming, achieving linear time complexity and making the algorithm very fast on CPUs. Furthermore, we optimize additivity-derived distortions using Hessian-based Taylor approximation to enhance pruning efficiency, accompanied by fine-grained complexity reduction techniques. Our method is evaluated on various DNN architectures, including CNNs, ViTs, and object detectors, and on vision tasks such as image classification on CIFAR-10 and ImageNet, and 3D object detection and various datasets. We achieve SoTA with significant FLOPs reductions without accuracy loss. Specifically, on CIFAR-10, we achieve up to <inline-formula><tex-math notation="LaTeX">27.9\times</tex-math> <mml:math><mml:mrow><mml:mn>27</mml:mn><mml:mo>.</mml:mo><mml:mn>9</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq1-3586418.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">29.2\times</tex-math> <mml:math><mml:mrow><mml:mn>29</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq2-3586418.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">14.9\times</tex-math> <mml:math><mml:mrow><mml:mn>14</mml:mn><mml:mo>.</mml:mo><mml:mn>9</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq3-3586418.gif"/> </inline-formula> FLOPs reductions on ResNet-32, VGG-16, and DenseNet-121, respectively. On ImageNet, we observe no accuracy loss with <inline-formula><tex-math notation="LaTeX">1.69\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>69</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq4-3586418.gif"/> </inline-formula> and <inline-formula><tex-math notation="LaTeX">2\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq5-3586418.gif"/> </inline-formula> FLOPs reductions on ResNet-50 and DeiT-Base, respectively. For 3D object detection, we achieve <inline-formula><tex-math notation="LaTeX">\mathbf {3.89}\times, \mathbf {3.72}\times</tex-math> <mml:math><mml:mrow><mml:mn mathvariant="bold">3</mml:mn><mml:mo>.</mml:mo><mml:mn mathvariant="bold">89</mml:mn><mml:mo>×</mml:mo><mml:mo>,</mml:mo><mml:mn mathvariant="bold">3</mml:mn><mml:mo>.</mml:mo><mml:mn mathvariant="bold">72</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq6-3586418.gif"/> </inline-formula> FLOPs reductions on CenterPoint and PVRCNN models. These results demonstrate the effectiveness and practicality of our approach for improving model performance through layer-adaptive weight pruning.