In this study, a fast calculation method of the computer generated integrated color rainbow holography based on the light field image is proposed, and the color three-dimensional (3D) display of the hologram is realized through optical experiments. First, the hologram plane is divided into several continuous unit line hologram planes, and the 3D coordinates of the unit line hologram planes projection points are calculated according to the vertex coordinates of each unit line hologram plane and the virtual slit vertex coordinates. Then, using the projection point as a virtual pinhole, the 3D object is projected through this pinhole; the light field image on the plane of the unit line hologram and the phase of the spherical waves converging at the virtual pinhole are used as the object light amplitude and phase in the computational hologram, and the reference light code is used to obtain the unit line hologram. Finally, all unit line holograms are combined to form a color rainbow hologram. The experimental results show that only takes 43 min to realize a hologram with a size of 50 mm×50 mm and a resolution of 157232 pixel×157232 pixel using the method, which has broad application prospects in the fields of holographic packaging and 3D advertising.