A method for retinal vessel segmentation is proposed based on a fully convolutional neural network with multi-scale feature fusion, which does not need hand-crafted features or specific post-processing. The architecture of skip connection is utilized, which combines the high-level semantic information with the low-level features. Residual block has been introduced to help learn details and texture features. The multi-scale spatial pyramid pooling module is built by atrous convolutions with different atrous rates to further enlarge the receptive fields and fully combine the context information. The class-balanced loss function is applied to solve the problem of imbalanced distribution of samples. The experimental results show that in the two datasets of digital retinal images for vessel extraction (DRIVE) and structured analysis of the retina (STARE), the accuracies are 95.46% and 96.84%, the sensitivities are 80.53% and 82.99%, the specificities are 97.67% and 97.94%, and the areas under receiver operating characteristic (ROC) curve are 97.71% and 98.17%, respectively. The proposed method is superior to the other existing methods.