Computer aided alignment (CAA) plays an important role in the assembly of optical systems. However, most proposed CAA methods are based on wavefront aberrations, which need the aid of an additional high precision wavefront sensor (WFS). Based on the ellipticity distribution of stellar image in different fields of view (FOVs), a misalignments calculation method for aligning optical systems efficiently without WFSs was proposed, which only needed an image sensor such as CCD/CMOS. Based on the nodal aberration theory, the theoretical basis of the proposed method was introduced and the ellipticity distribution of stellar image under the effect of misalignments and FOVs was analyzed, which revealed the essence of the proposed method. The ellipticity distribution could be expressed as a function of misalignments, just like the wavefront aberrations. Based on those, the process of misalignments calculation for optical systems was established and it turned to be a multi-objective optimization problem which could be solved by optimization algorithms. The method was tested in Hilbert telescope, which was a typical two-mirror optical system, and four lateral misalignments were used to be calculated according to the ellipticity distribution in three FOVs. The simulation results show that the precision of misalignments calculation can reach up to micron level and verify the correctness of the proposed method, which can promote the application of CAA and provide new insights for non-WFS optical alignment.