Considering the limitation of manual and contact monitoring of cattle’s daily behavior, this paper proposes a recognition method of the cattle daily behavior based on the spatial relationship of feature parts. First, YOLOv5(You only look once, v5) target detection model is used to locate the position of the cattle feature parts in the image, and the spatial relation vector of the cattle feature parts is constructed based on the position information of the feature parts. Thereafter, the fully connected neural network is used to classify the spatial relation vector to recognize the cattle’s standing, lying, and feeding behaviors. Finally, the method’s feasibility is demonstrated by counting the duration of each behavior in a video. The experimental results show that the method has a high recognition accuracy for the standing, lying, and feeding behaviors of cattle, and the relative error of each behavior duration in the statistical video is low, meeting the needs of daily behavior monitoring of cattle.