• Journal of Semiconductors
  • Vol. 41, Issue 2, 022404 (2020)
Chunyou Su1, Sheng Zhou2, Liang Feng1, and Wei Zhang1
Author Affiliations
  • 1Department of Electronics and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China
  • 2Department of Computer Science Engineering, Hong Kong University of Science and Technology, Hong Kong, China
  • show less
    DOI: 10.1088/1674-4926/41/2/022404 Cite this Article
    Chunyou Su, Sheng Zhou, Liang Feng, Wei Zhang. Towards high performance low bitwidth training for deep neural networks[J]. Journal of Semiconductors, 2020, 41(2): 022404 Copy Citation Text show less
    References

    [1] O Russakovsky, J Deng, H Su et al. Imagenet large scale visual recognition challenge. Int J Comput Vision, 115, 211(2015).

    [2] A Krizhevsky, I Sutskever, G E Hinton. Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst, 1097(2012).

    [3] K He, X Zhang, S Ren et al. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770(2016).

    [4] S Han, J Pool, J Tran et al. Learning both weights and connections for efficient neural network. Adv Neural Inform Process Syst, 1135(2015).

    [5]

    [6]

    [7] H Li, S De, Z Xu et al. Training quantized nets: A deeper understanding. Adv Neural Inform Process Syst, 5811(2017).

    [8]

    [9]

    [10]

    [11]

    [12]

    [13]

    [14] R Banner, I Hubara, E Hoffer et al. Scalable methods for 8-bit training of neural networks. Adv Neural Inform Process Syst, 5145(2018).

    [15] I Hubara, M Courbariaux, D Soudry et al. Quantized neural networks: Training neural networks with low precision weights and activations. J Mach Learning Res, 18, 6869(2017).

    [16] S Gupta, A Agrawal, K Gopalakrishnan et al. Deep learning with limited numerical precision. International Conference on Machine Learning, 1737(2015).

    [17] Sa C De, M Feldman, C Ré et al. Understanding and optimizing asynchronous low-precision stochastic gradient descent. ACM SIGARCH Computer Architecture News, 45, 461(2017).

    [18]

    [19]

    [20]

    [21]

    [22]

    [23]

    [24] M Courbariaux, Y Bengio, J P David. Binaryconnect: Training deep neural networks with binary weights during propagations. Adv Neural Inform Process Syst, 3123(2015).

    [25] I Hubara, M Courbariaux, D Soudry et al. Binarized neural networks. Adv Neural Inform Process Syst, 4107(2016).

    [26] M Rastegari, V Ordonez, J Redmon et al. Xnor-net: Imagenet classification using binary convolutional neural networks. European Conference on Computer Vision, 525(2016).

    [27]

    [28]

    Chunyou Su, Sheng Zhou, Liang Feng, Wei Zhang. Towards high performance low bitwidth training for deep neural networks[J]. Journal of Semiconductors, 2020, 41(2): 022404
    Download Citation