The exercise recommendation system is emerging as a promising application in online learning scenarios, providing personalized recommendations to assist students with explicit learning directions. Existing solutions generally follow a collaborative filtering paradigm, while the implicit connections between students (exercises) have been largely ignored. In this study, we aim to propose an exercise recommendation paradigm that can reveal the latent connections between student-student (exercise-exercise). Specifically, a new framework was proposed, namely personalized exercise recommendation with student and exercise portraits (PERP). It consists of three sequential and interdependent modules: Collaborative student exercise graph (CSEG) construction, joint random walk, and recommendation list optimization. Technically, CSEG is created as a unified heterogeneous graph with students’ response behaviors and student (exercise) relationships. Then, a joint random walk to take full advantage of the spectral properties of nearly uncoupled Markov chains is performed on CSEG, which allows for full exploration of both similar exercises that students have finished and connections between students (exercises) with similar portraits. Finally, we propose to optimize the recommendation list to obtain different exercise suggestions. After analyses of two public datasets, the results demonstrated that PERP can satisfy novelty, accuracy, and diversity.