Abstract
Image processing techniques have been indispensable parts of mobile image applications. Current image processing algorithms, including Deep Neural Networks, have achieved state-of-the-art performance on various image processing tasks. However, these advanced algorithms come with high computation, memory and energy cost, which makes it difficult to implement on a mobile device. There are mainly two general directions for image processing techniques: (1) the conven-tional approaches based on hand-craft features, and assumptions made by human observations;(2) the deep learning approaches based on the deep model that can fit probability distribution ac-curately under the training with large amounts of data. The design of computation and memory efficient algorithms for both approaches have different considerations. For the conventional approaches, the computation complexity comes from the calculation of man-made features, which requires the features to be both informative and easy to compute. For the deep learn-ing approaches, the computation complexity is because of the layers and structures of the deep models, which requires the design of the deep model to be compact. In order to reduce the computation and memory cost for both conventional and deep learning approaches, we propose different approaches to address the complexity issues: (1) for conventional approaches, we aim at reducing both the input dimension and also the computation complexity of the hand-crafted features; (2) for deep learning approaches, we reduce the size of the DNN structures, and also the computation cost of the layer-wise operations. In this work, we propose several image processing algorithms for various applications, and how to make this algorithms computation and memory efficient for the mobile platform. For conventional image processing approaches, our focus is on image segmentation, as it is an im-portant while time-consuming part of the image processing tasks. Instead of processing all pixels in an image, we reduce the number of pixels to be processed during image segmentation, and efficiently reduce the computation and memory cost of the segmentation algorithm, while keeping the same level of segmentation accuracy. For deep learning approaches, we focus on the parameter quantization problems. Parameter quantization is a simple yet efficient method for reducing the computation and memory cost of the deep model. Instead of assigning the same bit-width for all layers in a deep model, we adaptively assign the optimal bit-width com-binations for different layers in a deep model. In this way, we achieve minimal model size and computation cost while keeping the similar level of accuracy. The contributions of our approaches include: (1) memory and computation efficient image segmentation frameworks to reduce computational cost; (2) adaptive quantization approach to reduce deep model size; (3) image processing frameworks for real-life applications; and (4) computation and memory efficient approaches to reduce the computational cost on a mobile platform. The future works include: (1) computation and memory efficient deep neural net-works architecture with hardware support; (2) combine the proposed computation and memory efficient deep neural networks architecture with other complementary DNN compression tech-niques.