Detecting a decenter or tilt is a key problem in aligning an off-axis telescope system. Therefore, this paper proposes a second-order sensitivity matrix method. When the ranges of decenter and tilt are from -3 mm to 3 mm and from -3 mrad to 3 mrad, respectively, the relation curves between the Zernike coefficient and misalignment are drawn. All their goodness of fit is larger than 0.99. Based on these curves, two sensitivity matrixes are solved and verified using simulation experiments. The results are as follows: The root mean squares (RMSs) of the x-axis decenter and y-axis decenter are 0.0023 mm and 0.0043 mm, respectively, and the RMSs of x-axis tilt and y-axis tilt are 0.0177 mrad and 0.0031 mrad, respectively. The simulation results show that this method is more precise than the traditional sensitivity matrix method.