Drone or unmanned aerial vehicle (UAV) technology has undergone significant changes. The technology allows UAV to carry out a wide range of tasks with an increasing level of sophistication, since drones can cover a large area with cameras. Meanwhile, the increasing number of computer vision applications utilizing deep learning provides a unique insight into such applications. The primary target in UAV-based detection applications is humans, yet aerial recordings are not included in the massive datasets used to train object detectors, which makes it necessary to gather the model data from such platforms. You only look once (YOLO) version 4, RetinaNet, faster region-based convolutional neural network (R-CNN), and cascade R-CNN are several well-known detectors that have been studied in the past using a variety of datasets to replicate rescue scenes. Here, we used the search and rescue (SAR) dataset to train the you only look once version 5 (YOLOv5) algorithm to validate its speed, accuracy, and low false detection rate. In comparison to YOLOv4 and R-CNN, the highest mean average accuracy of 96.9% is obtained by YOLOv5. For comparison, experimental findings utilizing the SAR and the human rescue imaging database on land (HERIDAL) datasets are presented. The results show that the YOLOv5-based approach is the most successful human detection model for SAR missions.