When the Centernet algorithm in an anchor-free algorithm is used to identify and detect fish, the low-level feature information is easily lost, resulting in a decrease in recognition accuracy and efficiency. Therefore, a fish recognition and detection algorithm based on the Feature fusion Module and Loss function optimization of Centernet (FML-Centernet) algorithm is proposed herein. The feature fusion module is incorporated into the Centernet algorithm’s network structure to fuse the low- and high-level feature information, generate a more complete feature map, and improve the recognition and detection accuracy. By adjusting the loss ratio of positive and negative samples, the loss function of the network model is optimized and the recognition and detection efficiency of the overall model is improved. The effectiveness of the proposed algorithm is verified using the Pascalvoc dataset, and the performance of the network structure is analyzed. The optimized network structure is then compared with different models using a large number of target datasets and labeling dataset information. The experimental results show that the average recognition accuracy (AP50) of the FML-Centernet algorithm can exceed 85% and the average detection time is less than 100 ms. The proposed algorithm not only has high recognition and detection accuracy but also improves the recognition and detection efficiency.