Three dimensional deformation field of the dumbbell shaped specimen surface is obtained by color digital holographic system. The young′s modulus of TC4 titanium alloy is calculated by the three dimensional deformation field in the linear zone of the specimen, which is 90.79 GPa. Moreover, the young′s modulus of the material is measured as 99.05 GPa by the strain measuring instrument. The experimental results is approximate to the results of traditional strain measuring instrument. It indicates that the method of measuring Young′s modulus by color digital holography is feasible and effective. The method has the characteristics of real-time recording, full-field, non-contact and so on. It can adapt to the real-time detection in the process of material service under high temperature environment.