Aiming at the problem of UAVs autonomous decision-making in close air combat, an autonomous guidance method for UAV based on Proximal Policy Optimization (PPO) algorithm is proposed.The rewards are reshaped, such as distance, angle, speed and mission constraint, a three-degree-of-freedom model of UAV is established, and the state and action of reinforcement learning are constructed on the velocity coordinate system.The simulation training is carried out on the model of PPO algorithm combined with the fully connected neural network(standard PPO algorithm) and the PPO algorithm combined with the long short-term memory network (improved PPO algorithm) respecitively.According to the training results, it can be proved that, compared with the standard PPO algorithm, the improved PPO algorithm proposed in this paper can handle the UAV autonomous guidance tasks that are highly correlated with time series more effectively.