As the imaging performance of an optical system is related to its transversal magnification, this paper proposes a transversal magnification measurement scheme based on wavefront error metrology for the optical system. On the basis of that the locations of image plane and object plane satisfy the Newtons and Gausss formulas in a given optical system, the defocus in wavefront error was used to monitor the movement of image point. According to the minor movement of object point, the longitudinal magnification could be calculated by the differential of Newtons and Gausss formulas, and the transversal magnification was obtained finally. In order to validate the scheme, a model of transversal magnification measurement was established, and the criterions of the object point shift and the initial defocus were summarized. Then, the influences of geometric tolerance and positioning accuracy of the image point on transversal magnification were analyzed. Finally, an experimental platform based on a point diffraction interferometer was established for measuring the wavefront error of a reduced projection system, and the transversal magnification of the system was measured. It shows that the difference between theoretical and measured results is lower than 0.24%, which verifies the feasibility and veracity of this scheme.