• Infrared and Laser Engineering
  • Vol. 51, Issue 3, 20210253 (2022)
Shengjie Du, Xiaofen Jia, Yourui Huang, Yongcun Guo, and Baiting Zhao
Author Affiliations
  • School of Electrical and Information Engineering, Anhui University of Science and Technolog, Huainan 232000, China
  • show less
    DOI: 10.3788/IRLA20210253 Cite this Article
    Shengjie Du, Xiaofen Jia, Yourui Huang, Yongcun Guo, Baiting Zhao. High efficient activation function design for CNN model image classification task[J]. Infrared and Laser Engineering, 2022, 51(3): 20210253 Copy Citation Text show less

    Abstract

    Activation Functions (AF) play a very important role in learning and fitting complex function models of convolutional neural networks. In order to enable neural networks to complete various learning tasks better and faster, a new efficient activation function EReLU was designed in this paper. By introducing the natural logarithm function, EReLU effectively alleviated the problems of neuronal "necrosis" and gradient dispersion. Through the analysis of the activation function and its derivative function in the feedforward and feedback process of the mathematical model of the EReLU function exploration and design, the specific design of the EReLU function was determined through test, and finally the effect of improving the accuracy and accelerating training was achieved; Subsequently, EReLU was tested on different networks and data sets, and the results show that compared with ReLU and its improved function, the accuracy of EReLU is improved by 0.12%-6.61%, and the training efficiency is improved by 1.02%-6.52%, which strongly proved the superiority of EReLU function in accelerating training and improving accuracy.
    Shengjie Du, Xiaofen Jia, Yourui Huang, Yongcun Guo, Baiting Zhao. High efficient activation function design for CNN model image classification task[J]. Infrared and Laser Engineering, 2022, 51(3): 20210253
    Download Citation