A real-time detection algorithm F-YOLO for the feeding behavior of lightweight fish based on the fish swarm's texture, shape, and density characteristics is proposed to realize accurate feeding in fishery breeding based on the traditional detection algorithm. The initial backbone feature extraction network CSPDarkNet53 of the YOLOv4 algorithm is replaced by MobileNetV3, which significantly enhances the real-time detection performance of the network and the detection performance of small fish targets at the cost of a slight reduction in detection accuracy; channel pruning, and knowledge distillation are performed on the convolution layer of the network structure to compress the model and reduce the number of floating-point operations (FLOPs) and the amount of calculation; using optimized K-means clustering and DIoU loss function with global non-maximum suppression to determine the anchor frame, the problem of missing anchor frame caused by mutual occlusion of fish bodies are solved. The experimental results reveal that the model size of the suggested F-YOLO algorithm, the average recognition time of each image, the accuracy, the FLOPs, and detection speed in the embedded device are 13.7 MB, 50 ms, 99.13%, 1.64×10¹⁰, and 33 frame/s, respectively, which can provide theoretical guidance for the actual fishery breeding.