High efficient activation function design for CNN model image classification task

Shengjie Du; Xiaofen Jia; Yourui Huang; Yongcun Guo; Baiting Zhao

doi:10.3788/IRLA20210253

[1] M P Hassell, J H Lawton, J R Beddington. Sigmoid functional responses by invertebrate predators and parasitoids. The Journal of Animal Ecology, 249-262(1977).

[2] Kalman B L, Kwasny S C. Why tanh: Choosing a sigmoidal function[C]Proceedings 1992IJCNN International Joint Conference on Neural wks. IEEE, 1992, 4: 578581.

[3] A Krizhevsky, I Sutskever, G E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60, 84-90(2017).

[4] Dubey A K, Jain V. Comparative study of convolution neural wk’s relu leakyrelu activation functions[M]Applications of Computing, Automation Wireless Systems in Electrical Engineering, 2019: 873880.

[5] He K, Zhang X, Ren S, et al. Delving deep into rectifiers: Surpassing humanlevel perfmance on image classification[C]Proceedings of the IEEE International Conference on Computer Vision. 2015: 10261034.

[6] D A Clevert, T Unterthiner, S Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv, 1511.07289(2015).

[7] Shi Qi. Research verification of image classification optimization algithm based on convolutional neural wk[D]. Beijing: Beijing Jiaotong University, 2017. (in Chinese)

[8] Hongxia Wang, Jiaqi Zhou, Chenghao Gu, et al. Design of activation functions in convolutional neural networks for image classification. Journal of Zhejiang University (Engineering Science), 53, 1363-1373(2019).

[9] Zhang X, Zou Y, Shi W. Dilated convolution neural wk with LeakyReLU f environmental sound classification[C]2017 22nd International Conference on Digital Signal Processing (DSP). IEEE, 2017: 15.

[10] Xu L, Choy C, Li Y W. Deep sparse rectifier neural wks f speech denoising[C]2016 IEEE International Wkshop on Acoustic Signal Enhancement (IWAENC). IEEE, 2016: 15.

[11] Jun Liu. Analysis of algorithm design and algorithm time complexity. Computer Knowledge and Technology, 2, 878-879(2008).

[12] X Liu, Y Zhang, F Bao. Kernel-blending connection approximated by a neural network for image classification. Computational Visual Media, 6, 467-476(2020).

[13] Carrier P L, Courville A, Mirza M, et al. Challenges in representation learning: A rept on three machine learning contests[C]International Conference on Neural Infmation Processing, 2013: 117124.

[14] H Wang, J Zhou, C Gu, et al. Design of activation function in CNN for image classification. Journal of ZheJiang University (Engineering Science), 53, 1363-1373(2019).