Under the influence of rapid urbanization and industrialization, the problems of "rural diseases" faced by rural areas, especially the deep poverty areas are prominent, affecting rural sustainable development. There is an urgent need to better understand the process and characteristics of rural regional system change, identify restricting factors of rural development, and propose the suitable paths of rural revitalization for promoting rural sustainable development. This study took Yangyuan County as an example, investigated rural central agglomeration and growth from the perspective of land-use change, and revealed the evolution characteristics of rural human-land relationship in deep poverty areas. The results show that: 1) With the continuous reduction of arable land, the expansion intensity of villages is higher than that of township government seats and the county seat. 2) The agglomeration degree of production factors in the county seat, towns, and villages is imbalanced, resulting in a declining central agglomeration from the areas near the county seat to township government seats, and to remote rural villages. 3) Affected by the factors of traffic, location, and natural environment, areas near the county seat, better-located townships, and areas near transportation arteries are more likely to gather production factors and become high-value areas for rural growth. 4) The problem of rural resident population outflow is serious in deep poverty areas, causing rural hollowing with a phenomenon of declining population- increasing land. The article emphasized that the economic development level and urban-rural connection should be improved to enhance the county seat's ability to facilitate development in the countryside, and ruralization should be promoted alongside urbanization. Besides, it is necessary to implement comprehensive land consolidation in hollowed villages and cultivate rural social capital to promote endogenous rural development.