In order to better solve the routing, modulation format and spectrum allocation (RMSA) problems of elastic optical networks(EON), and further reduce the network blocking rate, an RMSA algorithm based on deep reinforcement learning(DRL) is proposed. This algorithm will consider two indicators, resource occupancy and spectral adjacency, which affect RMSA decision making in reward design, to encourage agents to prioritize selecting paths with low resource occupancy and high spectral adjacency to establish optical paths, and compare the performance of this algorithm with other algorithms in different networks.The simulation results show that compared with several typical DRL algorithms, the proposed algorithm has a lower network blocking rate.