Abstract
The Binarized Neural Network (BNN) is a type of neural network that employs bi nary weights and activations rather of the common 32-bit floating-point weights. They use smaller models and enable effective inference on mobile or embedded devices with limited power and processing resources. Nevertheless, binarization results in lower entropy feature maps and gradient vanishing, which leads to a loss in accuracy when compared to real value networks. Previous research has expanded the number of chan nels, used several binary bases, and re-scaled factors to address these issues, however those approaches significantly increases the algorithm’s time and space complexity. We introduce a novel approach for BNN application on embedded systems with multi scale neural network topology in this thesis, from two optimization perspectives: hard ware structure and BNN topology, that retains more low-level information throughout the feed-forward process with few operations. Experiments on the CIFAR-10 dataset indicate that our method outperforms a number of current binary designs in terms of efficiency and accuracy. Additionally, we implemented our BNN models on an embed ded architecture with a 4.4W power consumption using our hardware accelerator