Precise object detection and recognition plays an important role in information-based warfare. Due to panoramic vision sensors’ large field of view (LFOV), they are gradually applied to security and military areas. In this paper, first, the difficulties and challenges in the development of object detection and recognition in LFOV are presented from three aspects: camera imaging model, image imaging quality, and asymmetry of object. Then, based on whether the distortion correction preprocessing is carried out or not, the object detection and recognition algorithms in LFOV are classified into two categories: distortion correction based algorithms and original LFOV image based algorithms. These two kinds of algorithms are comprehensively combed and summarized. Finally, the paper analyzes the unity and difference of various algorithms for object detection and recognition in LFOV and discusses their future development trend.