Document Type

Thesis - Open Access

Award Date

2020

Degree Name

Master of Science (MS)

Department / School

Electrical Engineering and Computer Science

First Advisor

Kwanghee Won

Keywords

Accuracy Ensured Cmpression, CNN Pruning, Deep Neural Network, Deep Reinforcement Learning, Epoch based reward, Kernel pruning

Abstract

Apart from the accuracy, the size of convolutional neural networks (CNN) models is another principal factor for facilitating the deployment of models on memory, power and budget constrained devices. However, conventional model compression techniques require human experts to setup parameters to explore the design space which is suboptimal and time consuming. Various pruning techniques are implemented to gain compression, trading off speed and accuracy. Given a CNN model [11], we propose an automated deep reinforcement learning [9] based model compression technique that can effectively turned off kernels on each layer by observing its significance on decision making. By observing accuracy, compression ratio and convergence rate, our model can automatically re-activate (turned on) the healthiest(fittest) kernels to train it again which greatly ameliorate the model compression quality. Experimented results on MNIST dataset [7], the proposed method reduces the size of convolution layers for VGG-like model [9] up to 60% with 0.5% increase in test accuracy within less than a half the number of initial amount of training (speed-up up to 2.5×), state-of-the-art results of dropping 80% kernels (86% parameters compressed) with increase in accuracy by 0.14%. Further dropping 84% kernels (94% parameters compressed) with the drop of test accuracy 0.40%. The first proposed Auto-AEC (Accuracy-Ensured Compression) model can compress the network by preserving original accuracy or increase in accuracy of the model, whereas, the second proposed Auto-CECA (Compression-Ensured Considering the Accuracy) model can compress to the maximum by preserving original accuracy or minimal drop of accuracy. Based on experiments, further analyzed effectiveness of kernels on different layers based on how proposed model explores & exploits in various stages of training.

Library of Congress Subject Headings

Neural networks (Computer science)
Reinforcement learning.
Machine learning.

Format

application/pdf

Number of Pages

50

Publisher

South Dakota State University

Share

COinS
 

Rights Statement

In Copyright