Click to open the HelpDesk interface
AECE - Front page banner

Menu:


FACTS & FIGURES

JCR Impact Factor: 1.221
JCR 5-Year IF: 0.961
SCOPUS CiteScore: 2.5
Issues per year: 4
Current issue: Aug 2021
Next issue: Nov 2021
Avg review time: 88 days


PUBLISHER

Stefan cel Mare
University of Suceava
Faculty of Electrical Engineering and
Computer Science
13, Universitatii Street
Suceava - 720229
ROMANIA

Print ISSN: 1582-7445
Online ISSN: 1844-7600
WorldCat: 643243560
doi: 10.4316/AECE


TRAFFIC STATS

1,748,439 unique visits
579,678 downloads
Since November 1, 2009



Robots online now
PetalBot


SJR SCImago RANK

SCImago Journal & Country Rank




TEXT LINKS

Anycast DNS Hosting
MOST RECENT ISSUES

 Volume 21 (2021)
 
     »   Issue 3 / 2021
 
     »   Issue 2 / 2021
 
     »   Issue 1 / 2021
 
 
 Volume 20 (2020)
 
     »   Issue 4 / 2020
 
     »   Issue 3 / 2020
 
     »   Issue 2 / 2020
 
     »   Issue 1 / 2020
 
 
 Volume 19 (2019)
 
     »   Issue 4 / 2019
 
     »   Issue 3 / 2019
 
     »   Issue 2 / 2019
 
     »   Issue 1 / 2019
 
 
 Volume 18 (2018)
 
     »   Issue 4 / 2018
 
     »   Issue 3 / 2018
 
     »   Issue 2 / 2018
 
     »   Issue 1 / 2018
 
 
 Volume 17 (2017)
 
     »   Issue 4 / 2017
 
     »   Issue 3 / 2017
 
     »   Issue 2 / 2017
 
     »   Issue 1 / 2017
 
 
  View all issues  








LATEST NEWS

2021-Jun-30
Clarivate Analytics published the InCites Journal Citations Report for 2020. The InCites JCR Impact Factor of Advances in Electrical and Computer Engineering is 1.221 (1.053 without Journal self-cites), and the InCites JCR 5-Year Impact Factor is 0.961.

2021-Jun-06
SCOPUS published the CiteScore for 2020, computed by using an improved methodology, counting the citations received in 2017-2020 and dividing the sum by the number of papers published in the same time frame. The CiteScore of Advances in Electrical and Computer Engineering in 2020 is 2.5, better than all our previous results.

2021-Apr-15
Release of the v3 version of AECE Journal website. We moved to a new server and implemented the latest cryptographic protocols to assure better compatibility with the most recent browsers. Our website accepts now only TLS 1.2 and TLS 1.3 secure connections.

2020-Jun-29
Clarivate Analytics published the InCites Journal Citations Report for 2019. The InCites JCR Impact Factor of Advances in Electrical and Computer Engineering is 1.102 (1.023 without Journal self-cites), and the InCites JCR 5-Year Impact Factor is 0.734.

2020-Jun-11
Starting on the 15th of June 2020 we wiil introduce a new policy for reviewers. Reviewers who provide timely and substantial comments will receive a discount voucher entitling them to an APC reduction. Vouchers (worth of 25 EUR or 50 EUR, depending on the review quality) will be assigned to reviewers after the final decision of the reviewed paper is given. Vouchers issued to specific individuals are not transferable.

Read More »


    
 

  3/2021 - 1
View TOC | « Previous Article | Next Article »

Robust 2-bit Quantization of Weights in Neural Network Modeled by Laplacian Distribution

PERIC, Z. See more information about PERIC, Z. on SCOPUS See more information about PERIC, Z. on IEEExplore See more information about PERIC, Z. on Web of Science, DENIC, B. See more information about  DENIC, B. on SCOPUS See more information about  DENIC, B. on SCOPUS See more information about DENIC, B. on Web of Science, DINCIC, M. See more information about  DINCIC, M. on SCOPUS See more information about  DINCIC, M. on SCOPUS See more information about DINCIC, M. on Web of Science, NIKOLIC, J. See more information about NIKOLIC, J. on SCOPUS See more information about NIKOLIC, J. on SCOPUS See more information about NIKOLIC, J. on Web of Science
 
View the paper record and citations in View the paper record and citations in Google Scholar
Click to see author's profile in See more information about the author on SCOPUS SCOPUS, See more information about the author on IEEE Xplore IEEE Xplore, See more information about the author on Web of Science Web of Science

Download PDF pdficon (1,299 KB) | Citation | Downloads: 331 | Views: 236

Author keywords
image classification, neural networks, quantization, signal to noise ratio, source coding

References keywords
neural(18), networks(14), learning(7), information(7), quantization(6), processing(6), systems(5), machine(5), deep(5), convolutional(5)
Blue keywords are present in both the references section and the paper title.

About this article
Date of Publication: 2021-08-31
Volume 21, Issue 3, Year 2021, On page(s): 3 - 10
ISSN: 1582-7445, e-ISSN: 1844-7600
Digital Object Identifier: 10.4316/AECE.2021.03001
Web of Science Accession Number: 000691632000001
SCOPUS ID: 85114815185

Abstract
Quick view
Full text preview
Significant efforts are constantly involved in finding manners to decrease the number of bits required for quantization of neural network parameters. Although in addition to compression, in neural networks, the application of quantizer models that are robust to changes in the variance of input data is of great importance, to the best of authors knowledge, this topic has not been sufficiently researched so far. For that reason, in this paper we give preference to logarithmic companding scalar quantizer, which has shown the best robustness in high quality quantization of speech signals, modelled similarly as weights in neural networks, by Laplacian distribution. We explore its performance by performing the exact and asymptotic analysis for low resolution scenario with 2-bit quantization, where we draw firm conclusions about the usability of the exact performance analysis and design of our quantizer. Moreover, we provide a manner to increase the robustness of the quantizer we propose by involving additional adaptation of the key parameter. Theoretical and experimental results obtained by applying our quantizer in processing of neural network weights are very good matched, and, for that reason, we can expect that our proposal will find a way to practical implementation.


References | Cited By  «-- Click to see who has cited this paper

[1] A. Zhang, Z. C. Lipton, M. Li, A. J. Smola, Dive into Deep Learning. Amazon Science, 2020.

[2] Z. Nagy. Artificial Intelligence and Machine Learning Fundamentals: Develop Real-World Applications Powered by the Latest AI Advances. Packt Publishing, 2018.

[3] A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. of the International Conference on Neural Information Processing Systems, Harrahs and Harveys, Lake Tahoe, NV, USA, 2012, pp. 1097-1105

[4] G. Mukhtar, S. Farhan, "Convolutional neural network based prediction of conversion from mild cognitive impairment to Alzheimer's disease: A technique using hippocampus extracted from MRI", Advances in Electrical and Computer Engineering, vol. 20, no. 2, pp. 113-122, 2020.
[CrossRef] [Full Text] [Web of Science Times Cited 1] [SCOPUS Times Cited 2]


[5] S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," in Proc. of the Conference on Advances in neural information processing systems (NeurIPS), Montreal, Canada, 2015, pp. 91-99.

[6] A. Conneau, H. Schwenk, L. Barrault, Y. Lecun, "Very deep convolutional networks for text classification," arXiv preprint arXiv: 1606.01781, 2016.

[7] V. Delic, Z. Peric, M. Secujski, N. Jakovljevic, J. Nikolic, et. al, "Speech technology progress based on new machine learning paradigm," Computational Intelligence and Neuroscience, vol. 2019, Article ID 4368036, 19 pages, 2019.
[CrossRef] [Web of Science Times Cited 9] [SCOPUS Times Cited 17]


[8] A. Albu, R. E. Precup, T. A. Teban, "Results and challenges of artificial neural networks used for decision-making and control in medical applications," Facta Universitatis Series: Mechanical Engineering, vol. 17, no. 3, pp. 285-308, 2019.
[CrossRef] [Web of Science Times Cited 32] [SCOPUS Times Cited 37]


[9] M. U. Ahmed, S. Brickman, A. Dengg,N. Fasth, M. Mihajlovic, et. al., "A machine learning approach to classify pedestrians' events based on IMU and GPS," International Journal of Artificial Intelligence, vol. 17, no 2, pp. 154-167, 2019.

[10] U. L. Yuhana, N. Z. Fanani, E. M. Yuniarno, S. Rochimah, L. T. Koczy, et.al., "Combining fuzzy signature and rough sets approach for predicting the minimum passing level of competency achievement," International Journal of Artificial Intelligence, vol. 18, no. 1, pp. 237-249, 2020.

[11] Y. Sheng, H. Ma, W. Xia, "A Pointer neural network, for the vehicle routing problem with task priority and limited resources," Information Technology and Control, vol. 49, no.2, pp. 237-248, 2020.
[CrossRef] [Web of Science Times Cited 1] [SCOPUS Times Cited 1]


[12] Y. Li, Y. Bao, W. Chen, "Fixed-sign binary neural network: An efficient design of neural network for Internet-of-Things devices," IEEE Access, vol. 8, pp. 164858-164863, 2018.
[CrossRef] [Web of Science Times Cited 4] [SCOPUS Times Cited 5]


[13] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, Y. Bengio, "Quantized neural networks: Training neural networks with low precision weights and activations," Journal of Machine Learning Research, vol. 18, no. 1, pp. 6869-6898, 2017.
[CrossRef]


[14] D. Lin, S. Talathi, S. Annapureddy, "Fixed point quantization of deep convolutional networks," in Proc. of the 33rd International Conference on Machine Learning, New York, NY, USA, 2016.

[15] R. Banner, Y. Nahshan, E. Hoffer, D. Soudry, "ACIQ: Analytical Clipping for Integer Quantization of Neural Networks," arXiv preprint arXiv: 1810.05723, 2018.

[16] L. Enderich, F. Timm, W. Burgard, "SYMOG: Learning symmetric mixture of Gaussian modes for improved fixed-point quantization," Neurocomputing, vol. 416, pp. 310-315, 2020.
[CrossRef] [Web of Science Times Cited 3] [SCOPUS Times Cited 4]


[17] A. Nannarelli, "Variable precision 16-bit floating-point vector unit for embedded processors," in Proc. of IEEE 27th Symposium on Computer Arithmetic, (ARITH 2020), Portland, OR, USA, 2020.

[18] R. Banner, I. Hubara, E. Hoffer, D. Soudry, "Scalable methods for 8-bit training of neural networks," in Proc. of the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, Canada, 2018.

[19] R. Banner, Y. Nahshan, D. Soudry, "Post training 4-bit quantization of convolutional networks for rapid-deployment," in Proc. of the 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, 2019.

[20] J. Choi, S. Venkataramani, V. Srinivasan, K. Gopalakrishnan, Z. Wang, et. al, "Accurate and efficient 2-bit quantized neural networks," in Proc. of the 2nd SysML Conference, Stanford, CA, USA, 2019.

[21] L. Deng, P. Jiao, J. Pei, Z. Wu, G. Li, "GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework," Neural Networks, vol. 100, pp. 49-58, 2018.
[CrossRef] [Web of Science Times Cited 41] [SCOPUS Times Cited 47]


[22] H. Qina, R. Gonga, X. Liu, X. Baie, J. Songc, et. al, "Binary neural networks: A survey," Pattern Recognition, vol. 105, Article ID: 107281, 2020.
[CrossRef] [Web of Science Times Cited 32] [SCOPUS Times Cited 47]


[23] Z. Peric, B. Denic, M. Savic, V. Despotovic, "Design and analysis of binary scalar quantizer of Laplacian source with applications," Information, vol. 11, 18 pages, 2020.
[CrossRef] [Web of Science Times Cited 5] [SCOPUS Times Cited 5]


[24] N. S. Jayant, P. Noll, Digital Coding of Waveforms: Principles and Applications to Speech and Video. New Jersey, Prentice Hall, Chapter 4, pp. 115-188, 1984.

[25] S. Na, "Asymptotic formulas for mismatched fixed-rate minimum MSE Laplacian Quantizers," IEEE Signal Processing Letters, vol. 15, pp. 13-16, 2008.
[CrossRef] [Web of Science Times Cited 28] [SCOPUS Times Cited 31]


[26] Z. Peric, G. Petkovic, B. Denic, A. Stanimirovic, V. Despotovic, et al., "Gaussian source coding using a simple switched quantization algorithm and variable length codewords," Advances in Electrical and Computer Engineering, vol. 20, no. 4, pp. 11-18, 2020.
[CrossRef] [Full Text] [Web of Science Times Cited 1] [SCOPUS Times Cited 1]


[27] S. Tomic, Z. Peric, M. Tancic, J. Nikolic, "Backward adaptive and quasi-logarithmic quantizer for sub-band coding of audio," Information Technology and Control, vol. 47, no. 1, pp. 131-139, 2018.
[CrossRef] [Web of Science Times Cited 7] [SCOPUS Times Cited 18]


[28] M. Dincic, Z. Peric, D. Denic, Z. Stamenkovic, "Design of robust quantizers for low-bit analog-to-digital converters for Gaussian source," Journal of Circuits, Systems and Computers, vol. 28, no. supp01, 1940002, 2019.
[CrossRef] [Web of Science Times Cited 4] [SCOPUS Times Cited 4]


[29] Z. Peric, J. Nikolic, D. Aleksic, A. Peric, "Symmetric quantile quantizer parameterization for the Laplacian source: Qualification for contemporary quantization solutions," Mathematical Problems in Engineering, vol. 2021, Article ID 6647135, 12 pages, 2021.
[CrossRef] [Web of Science Times Cited 1] [SCOPUS Times Cited 1]


[30] S. Gazor, W. Zhang, "Speech probability distribution," IEEE Signal Processing Letters, vol. 10, pp. 204-207, 2003.
[CrossRef] [Web of Science Times Cited 190] [SCOPUS Times Cited 234]


[31] Y. LeCun, C. Cortez, C. Burges, "The MNIST Handwritten Digit Database," available online: yann.lecun.co.



References Weight

Web of Science® Citations for all references: 359 TCR
SCOPUS® Citations for all references: 454 TCR

Web of Science® Average Citations per reference: 11 ACR
SCOPUS® Average Citations per reference: 14 ACR

TCR = Total Citations for References / ACR = Average Citations per Reference

We introduced in 2010 - for the first time in scientific publishing, the term "References Weight", as a quantitative indication of the quality ... Read more

Citations for references updated on 2021-10-13 19:17 in 106 seconds.




Note1: Web of Science® is a registered trademark of Clarivate Analytics.
Note2: SCOPUS® is a registered trademark of Elsevier B.V.
Disclaimer: All queries to the respective databases were made by using the DOI record of every reference (where available). Due to technical problems beyond our control, the information is not always accurate. Please use the CrossRef link to visit the respective publisher site.

Copyright ©2001-2021
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania


All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.

Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.

Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.




Website loading speed and performance optimization powered by: