In gravitational wave telescopes, the energy of the collected space target light signals is dwarfed by the energy of stray light, necessitating robust stray light suppression for reliable telescope operation. Due to the inherent unpredictability of scattered light and the intricate nature of opto-mechanical systems, the formulation of stray light suppression strategies often involves complex mathematical modeling, substantial expertise, and iterative simulations. This paper introduces a Reinforcement Learning-based approach to devise the stray light suppression scheme within a Monte Carlo ray tracing environment, specifically for space gravitational wave telescope systems. Our empirical findings confirm the efficacy of this methodology in generating effective stray light suppression strategies, yielding favorable suppression performance. This study contributes a novel, efficient, and adaptable solution to the stray light challenges faced in space gravitational wave detection as well as other high-precision optical systems, thereby holding extensive applicative promise.