Aiming at the problem that small masses and occluded masses are easy to be missed in breast cancer diagnosis based on deep learning, an improved YOLOv3 algorithm for breast mass detection is proposed. First, a bottom-up path is added into the feature fusion module, and the cascading and cross-layer connections are adopted to make full use of the underlying feature information to improve the recognition accuracy of small masses. Second, to filter out more accurate prediction bounding boxes and avoid missed detection of masses that occlude each other, the distance intersection over union (DIoU) is introduced in soft non-maximum suppression (Soft-NMS) algorithm to suppress the redundant prediction bounding boxes. The experimental results demonstrate that the proposed breast mass detection algorithm has high accuracy and speed in detecting small masses and occluded masses, mean average precision (mAP@0.5) reaches 96.1%, which is 1.8 percentage point higher than that of YOLOv3, and the detection time of each mammogram target image is only 28 ms.