| 3/2021 - 1 | View TOC | « Previous Article | Next Article » |
Robust 2-bit Quantization of Weights in Neural Network Modeled by Laplacian DistributionPERIC, Z.   , DENIC, B. , DINCIC, M. , NIKOLIC, J. |
Extra paper information in    |
| Click to see author's profile in SCOPUS, IEEE Xplore, Web of Science |
Download PDF  (1,299 KB) | Citation | Downloads: 1,369 | Views: 2,150 |
Author keywords
image classification, neural networks, quantization, signal to noise ratio, source coding
References keywords
neural(18), networks(14), learning(7), information(7), quantization(6), processing(6), systems(5), machine(5), deep(5), convolutional(5)
Blue keywords are present in both the references section and the paper title.
About this article
Date of Publication: 2021-08-31
Volume 21, Issue 3, Year 2021, On page(s): 3 - 10
ISSN: 1582-7445, e-ISSN: 1844-7600
Digital Object Identifier: 10.4316/AECE.2021.03001
Web of Science Accession Number: 000691632000001
SCOPUS ID: 85114815185
Abstract
Significant efforts are constantly involved in finding manners to decrease the number of bits required for quantization of neural network parameters. Although in addition to compression, in neural networks, the application of quantizer models that are robust to changes in the variance of input data is of great importance, to the best of authors knowledge, this topic has not been sufficiently researched so far. For that reason, in this paper we give preference to logarithmic companding scalar quantizer, which has shown the best robustness in high quality quantization of speech signals, modelled similarly as weights in neural networks, by Laplacian distribution. We explore its performance by performing the exact and asymptotic analysis for low resolution scenario with 2-bit quantization, where we draw firm conclusions about the usability of the exact performance analysis and design of our quantizer. Moreover, we provide a manner to increase the robustness of the quantizer we propose by involving additional adaptation of the key parameter. Theoretical and experimental results obtained by applying our quantizer in processing of neural network weights are very good matched, and, for that reason, we can expect that our proposal will find a way to practical implementation. |
| References | | | Cited By |
Web of Science® Times Cited: 5 [View]
View record in Web of Science® [View]
View Related Records® [View]
Updated today
SCOPUS® Times Cited: 5
View record in SCOPUS® [Free preview]
View citations in SCOPUS® [Free preview]
Updated today
[1] Two Novel Non-Uniform Quantizers with Application in Post-Training Quantization, Perić, Zoran, Aleksić, Danijela, Nikolić, Jelena, Tomić, Stefan, Mathematics, ISSN 2227-7390, Issue 19, Volume 10, 2022.
Digital Object Identifier: 10.3390/math10193435 [CrossRef]
[2] Iterative Algorithm for Parameterization of Two-Region Piecewise Uniform Quantizer for the Laplacian Source, Nikolić, Jelena, Aleksić, Danijela, Perić, Zoran, Dinčić, Milan, Mathematics, ISSN 2227-7390, Issue 23, Volume 9, 2021.
Digital Object Identifier: 10.3390/math9233091 [CrossRef]
[3] Performance of Post-Training Two-Bits Uniform and Layer-Wise Uniform Quantization for MNIST Dataset from the Perspective of Support Region Choice, Tomić, Stefan, Nikolić, Jelena, Perić, Zoran, Aleksić, Danijela, Gao, Hao, Mathematical Problems in Engineering, ISSN 1563-5147, Issue , 2022.
Digital Object Identifier: 10.1155/2022/1463094 [CrossRef]
[4] Whether the Support Region of Three-Bit Uniform Quantizer Has a Strong Impact on Post-Training Quantization for MNIST Dataset?, Nikolić, Jelena, Perić, Zoran, Aleksić, Danijela, Tomić, Stefan, Jovanović, Aleksandra, Entropy, ISSN 1099-4300, Issue 12, Volume 23, 2021.
Digital Object Identifier: 10.3390/e23121699 [CrossRef]
Updated today
Disclaimer: All information displayed above was retrieved by using remote connections to respective databases. For the best user experience, we update all data by using background processes, and use caches in order to reduce the load on the servers we retrieve the information from. As we have no control on the availability of the database servers and sometimes the Internet connectivity may be affected, we do not guarantee the information is correct or complete. For the most accurate data, please always consult the database sites directly. Some external links require authentication or an institutional subscription.
Web of Science® is a registered trademark of Clarivate Analytics, Scopus® is a registered trademark of Elsevier B.V., other product names, company names, brand names, trademarks and logos are the property of their respective owners.
Copyright ©2001-2024
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania
All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.
Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.
Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania
All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.
Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.
Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.
