• Journal of Semiconductors
  • Vol. 41, Issue 2, 022404 (2020)
Chunyou Su1, Sheng Zhou2, Liang Feng1, and Wei Zhang1
Author Affiliations
  • 1Department of Electronics and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China
  • 2Department of Computer Science Engineering, Hong Kong University of Science and Technology, Hong Kong, China
  • show less
    DOI: 10.1088/1674-4926/41/2/022404 Cite this Article
    Chunyou Su, Sheng Zhou, Liang Feng, Wei Zhang. Towards high performance low bitwidth training for deep neural networks[J]. Journal of Semiconductors, 2020, 41(2): 022404 Copy Citation Text show less
    NR simulation.
    Fig. 1. NR simulation.
    SR simulation.
    Fig. 2. SR simulation.
    Execution modules.
    Fig. 3. Execution modules.
    Whole design structure.
    Fig. 4. Whole design structure.
    Module structure example.
    Fig. 5. Module structure example.
    Random number generator.
    Fig. 6. Random number generator.
    Model8-bit model (SR)8-bit model (NR)Acc. Drop
    AlexNet54.34%52.46%1.88%
    ResNet-1865.96%65.72%0.24%
    Table 1. Top-1 accuracy of 8-bit AlexNet and ResNet18, SR versus NR.
    ModelFull8-bit modelAcc. Drop
    ResNet-2092.24%92.12%0.12%
    ResNet-5694.14%93.75%0.39%
    Table 2. Top-1 accuracy on CIFAR-10 dataset.
    ModelFull8-bit modelAcc. Drop
    AlexNet(DoReFa[14]) 55.9%53.0%2.9%
    AlexNet54.76%54.34%0.42%
    ResNet-5075.46%74.14%1.32%
    Inception V376.95%75.03%1.92%
    Table 3. Top-1 accuracy on ImageNet dataset.
    ParameterBRAMDSPFFLUT
    Used238610434213564233
    Percentage5%8%18%47%
    Table 4. Resource usage of FPGA prototyping.
    Chunyou Su, Sheng Zhou, Liang Feng, Wei Zhang. Towards high performance low bitwidth training for deep neural networks[J]. Journal of Semiconductors, 2020, 41(2): 022404
    Download Citation