The detectable star magnitude limit, the number of navigation stars in the field of view, the size and weight of the optical system, and etc. are important indicators of optical detection system for all-time star sensor. And the improvement of these indicators is often contradictory and restrictive, which leads to the lack of clear optimization objectives for the parameter design of the optical system. In order to improve the comprehensive performance of the all-time star sensor, the parameter optimization of the optical system is studied in this paper.According to the response characteristics of the short-wave infrared detector, a signal to noise ratio model based on image gray-scale information is established. Based on this model, a parameter optimization scheme of the optical system for the star sensor is proposed aiming at two different navigation modes of tracking and strapdown. The optimization goal of tracking mode is the number of navigation stars in the celestial sphere. The maximum focal length is firstly determined according to the control accuracy of optical axis direction. Then, detectable star magnitude limit with different focal length and diameter is calculated (Fig.2) using the sky background radiation obtained by MODTRAN (Fig.1). A relatively small diameter is chosen to balance the detection ability and the optical system size. After slightly adjusting the cut-off wavelength of the detection band (Fig.3), the focal length is finally determined according to the requirements for detection capability (Fig.4). If the average number of navigation stars in FOV (field of view) is less than 3 for any focal length and diameter, only tracking mode can work (Fig.5). Otherwise, strapdown mode is selected for certain working altitude (Fig.6), and the average number of navigation stars in FOV is the optimization goal. Its change with focal length and diameter is calculated. An appropriate focal length and a relatively smaller diameter are determined when the number of navigation stars meets the requirements and its mean value is greater than 3 (Fig.7). The cut-off wavelength of the detection band is slightly adjusted to improve the average navigation star number in FOV (Fig.8).The parameters of InGaAs sensor from Sofradir are selected as typical values for simulation, based on which the optical system parameters of the sea-level tracking mode and the high-altitude strapdown mode are designed respectively. Working at sea level, the optimal aperture for daytime star detection is 60 mm, and the cut-off wavelength is between 1.45 μm and 1.6 μm. The focal length is selected according to the requirements of the detectable star magnitude limit. While the optimization results for 20 km is aperture of 125 mm, focal length of 400 mm, and cut-off wavelength between 1.4 μm and 1.5 μm. Besides, the corresponding conversion altitude of the all-time star sensor between tracking and strapdown navigation mode is around 20 km.The prototype system (Fig.9) has a diameter of 60 mm, a focal length of 515 mm, and a cut-off wavelength of 1.48 μm, which is used to carry out daytime star tracking experiments on the ground. Stars brighter than H-band –0.1 magnitude (Fig.10-11) can be detected around 10:00 am at Hunan, Changsha (altitude of 69 m), which shows that the system has the capability of daytime star detection near the ground. The experiment verifies the validity of the simulation results, indicating that the parameter optimization theory proposed in this paper has high reference value for the optical system design of all-time star sensor.