Owing to the continuing decrement in the pixels of the images, the signal output of the digital imaging sensor is increasingly sensitive to photon noise, making the photon noise the main source of noise in the digital image sensor. To address this issue, an image denoising algorithm based on asymmetric convolutional neural networks is proposed herein. To enhance the generalization ability of the model, the network framework is divided into two parts: noise evaluation network and denoising network. To reduce the semantic gap between the network feature mapping in the encoder and the decoder, the skip connection in the denoising network is improved to make the features more similar in semantics to facilitate task optimization. From the qualitative and quantitative aspects of comparative experiments, the experimental results show that the proposed network model exhibits better denoising performance.