Hyperspectral image classification has gained widespread attention in recent years, particularly with the significant advances in the application of Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs). While CNNs handle pixel information in small, regular regions, GNNs excel at capturing features from irregular, superpixel regions. To combine the strengths of both, a novel hyperspectral classification network named the SpectraFuse GAT and Bipolar Self-Attention Fusion Network (SGBF) is proposed, which integrates Bipolar Self-Attention CNN and SpectraFuse GAT for high-quality hyperspectral classification. In the GNN component, the SpectraFuse GAT (SGAT) is introduced and the Spectral Refinement Module (SRM) is developed to enhance the spectral information extraction capability. In the CNN component, we incorporate the Bipolar Self-Attention mechanism (BSA) to capture spatial-spectral information. The experimental results demonstrate that SGBF performs exceptionally well across multiple datasets. On the Indian Pines dataset, SGBF achieved a classification accuracy of 91.59%, which is 13.2% higher than CNN and 12.23% higher than GNN methods. On the PaviaU dataset, the accuracy reached 98.54%, surpassing the current best method by 2.37%. These results validate the superiority and robustness of SGBF in hyperspectral image classification.