Conventional wide-field three-dimensional (3D) measurement is usually implemented by measuring system consisting of multiple sensors. Difficulties, including sensor synchronization and unified measuring coordinate system of multiple sensors, lead to the incompatibility between high accuracy and high speed. An omnidirectional stereo vision sensor consisting of a single camera and two rectangular pyramid mirrors is designed to realize real-time 3D measurement of objects in wider field. The rectangular pyramid mirrors are symmetrically placed, with their top opposite to each other, and a high resolution industrial camera is set at the head of the under rectangular pyramid. The rectangular pyramid mirrors are reflecting and imaging as four pairs of virtual cameras. It is equivalent to the traditional binocular vision sensor in which the counterpart points are obtained by two cameras. Thus synchronization measurement in horizontal four direction is realized by four pairs of virtual cameras. Some disadvantages of traditional binocular vision sensor are overcame, such as big size，narrow field of view, asynchronous image collection etc. Moreover, constant perspective projection of images is ensured and distortion in images reflected by curved mirror is avoided, which makes the later work much easier.