The defocus blur problem of catadioptric omnidirectional imaging, which is caused by aperture and mirror curvature, becomes more severe along with introducing high-resolution sensors and large aperture. One design of catadioptric omnidirectional imaging system for defocus deblurring is proposed. The defocus blur of catadioptric omnidirectional imaging is analyzed in theory, and the relation between point spread function of omnidirectional image and scene′s spatial points or the position of virtual features is established. Then, the defocus blur kernel of omnidirectional image is spatially invariant when image plane is moved in a particular pattern during image integration. A deconvolution algorithm is used for the defocus blur omnidirectional image restoration to obtain all sharp omnidirectional images. The method is effective for catadioptric omnidirectional image defocus deblurring, and has an important meaning in improving catadioptric omnidirectional imaging quality and promoting applications in related fields.