Semiconductor detector is widely used in energy dispersive X-ray fluorescence measurementsdue to its excellent performance. In this paper, Si-PIN and CdTe semiconductor detectors were studied, performances of the two detectors were compared in material properties, detection efficiency, energy resolution and other aspects. Focused on the performance of the detectors influenced by the thickness of detector sensitive area, energy of incident X-ray, shaping time of post-stage circuit, and analyzed the differences of energy spectrum caused by escape peaks and hole trailing. Aiming at the problem of incomplete hole collection in detector, a digital multi-channel analyzer (DMCA) based on FPGA with rise-time discriminator was designed, it could reduce the influence of hole trailing effectively and improve energy resolution. The experimentation results indicate that the detection efficiency of Si-PIN and CdTe is roughly equal when energy is below 15 keV while CdTe has much higher detection efficiency than Si-PIN when energy is above 15 keV. The optimum forming time of the Si-PIN detector is about 10 μs, and the CdTe detector is about 2.6 μs, so the CdTe detector is more suitable for the high count rate condition. Si-PIN detector has better energy resolution than CdTe detector for different energy incident X-ray. CdTe detector has obvious hole tailing effect and the energy resolution of CdTe detector is significantly improved by using DMCA with rise-time discrimination.