A three-dimensional (3D) object reconstruction technique that uses pure-phase computer-generated holograms (CGHs) and a phase-only spatial light modulator (SLM) is proposed. The full parallax CGHs are generated by the point source method and the wave-oriented method without paraxial approximation. Different from conventional CGHs, the pure-phase information on the hologram plane is loaded on the SLM to reconstruct the 3D diffusive objects without considering the reference wave. This technique is more efficient in its utilization of the space-bandwidth product of the SLMs. Numerical simulations and experiments are performed, and the results show that our proposed method can reconstruct 3D diffusive objects successfully.