We present a three-dimensional (3D) shape measurement technique based on parallel four-color channels (red, green, blue and infrared) fringe projection, which only needs one-time measurement to reconstruct the 3D surface shape of an object. The proposed technique overcomes the shortcomings of slow measurement speed caused by multi-frame fringe images projection in the traditional fringe projection, and possesses high accuracy. Two-step phase shift, Fourier transform with the optimum three-frequency method are combined to calculate the absolute phase point-by-point independently and recover 3D shape measurement of complex objects, such as discontinuities, from the deformed composite color fringe pattern and infrared fringe pattern captured simultaneously. A beam splitter has been used to realize coaxial design and to build the projecting transformation between the visible light projector and the infrared light projectors, so that they build up the accurate pixel-to-pixel correspondence. Several experiments on static and dynamic objects are performed to obtain the 3D shape, and experimental results show that the proposed method is suitable for fast 3D shape measurement.