According to the characteristics of the all-fiber lidar, the transmitting and the receiving optical systems are designed and optimized. Aiming at the characteristics of the laser Gauss transmission, the extended target and the optical fiber transceiver of the all-fiber lidar optical system, the transmitting antenna gain and the target reflection cross-section in the lidar equation are modified, and the fiber reception efficiency is introduced. The modified transmitting antenna gain is inversely proportional to the square of the target spot size. The radar reflection cross-section of the diffuse reflection target depends on the target reflectivity and the laser incidence angle. The relationship between the echo power and the core diameter of the reception fiber is explored in experiment, and the optimum fiber core diameter is obtained. The modified lidar equation can calculate the laser echo power from the optical fiber transceiver accurately and provide precise theoretical support for subsequent signal processing.