In order to reduce the influence of aerodynamic thermal radiation generated during the reentry process of hypersonic vehicle on the performance of infrared detection window, from the perspective of trajectory optimization, a trajectory optimization algorithm of hypersonic vehicle based on improved whale optimization algorithm was proposed with the total infrared radiation of reentry flight stagnation point as the objective function. Firstly, the Whale optimization algorithm was improved by Tent chaotic map and control factor chord change. The position directivity of the algorithm was improved when the position was updated, and the global search ability of the algorithm was enhanced. At the same time, the reentry trajectory optimization problem was transformed into the parameter optimization problem of the control profile, and a one-time reversal strategy of the inclination angle was proposed. The Planck formula was used to calculate the infrared radiation of the stagnation point, and the objective function was designed. The resistance acceleration was used to enter the corridor to deal with the path constraint. The penalty function method was used to combine the terminal constraint with the objective function. Finally, the improved Whale optimization algorithm was used to optimize the parameters of the designed control profile to obtain the optimal solution of the objective function. The simulation results showed that the improved whale algorithm can effectively complete the reentry trajectory optimization task with the minimum total infrared radiation, and has strong global search ability and good robustness.