The spatio-temporal distribution characteristics and change of mountain populations are of great significance for coordinating human-land relationship and implementing the rural revitalization strategy in the process of urbanization. Based on the data of three agricultural censuses in Luoyang City, this study selected 318 administrative villages in Song County as the research objects, and used a GIS spatial analysis tool and geographically weighted regression to analyze the spatial pattern of population distribution and its influencing factors. The results show that: 1) The population of Song County is extremely uneven in spatial distribution. The high density areas are clustered near the main traffic lines and in areas with relatively flat terrain, and over time, they changed from point to spread and strip distribution. 2) On the whole, the population growth is bounded by the high-speed Luoluan Highway, and shows a pattern of rapid growth to the northwest and slow growth to the southeast. The high-speed growth areas are mainly concentrated in the county seat and a few townships around the northwestern part of the Luoluan Highway. 3) The population growth centers are also located in the towns around the county seat and the Luhun Reservoir, and the towns in the southern mountainous areas are expected to form new population agglomeration areas. 4) The population distribution pattern of the county is the result of the interaction of natural conditions, geographical location, and public services. Among them, the accessibility time of county seat has the greatest impact on population distribution, followed by topography. The impact of arable land area and accessibility time of middle school are relatively small, but the impact of different factors has spatial and temporal heterogeneity. The above conclusions imply that mountainous counties can take into account the differences in natural conditions and resource endowments, and make use of urbanization layout, infrastructure construction, and public services adjustments to promote the optimization of population distribution.