The sintering flame image has fine-grained local flame state feature information and complex global flame state feature information. However, the traditional convolutional neural network is often more sensitive to local features, and it is difficult to extract the global feature information of the flame state, which restricts the expression ability of sintering flame features, resulting in low accuracy in the classification and recognition of the sintering flame state. In response to such problems, a dual-stream network feature fusion classification method based on CNN-Transformer was proposed, which includes two modules: convolutional neural networks (CNN) flow and Transformer flow. Firstly, the CNN block and the Transformer block were designed in parallel. The CNN stream extracts the local feature information of the RGB image of the sintering flame, and the Transformer stream extracts the global feature information of the GRAY image of the sintering flame. Then, the local feature information and the global feature information of the sintering flame state extracted by the dual-stream network was fused using the cascade interactive feature fusion method. Finally, the softmax classifier was used to achieve the classification of sintering flame states. The experimental results show that the flame classification accuracy can reach 96.20%, which is 6%~8% higher than that of the traditional convolutional neural network.