Aiming at the problems of the traditional fractal detection algorithm in Fractional Fourier Transform (FRFT) domain in sea cluttersuch as difficulty in determining the optimal order and unstable performancean improved fast detection algorithm of small targets on sea surface based on fractal transform domain is proposed.The new algorithm uses the fast estimation algorithm to quickly determine the optimal order of the target echo in FRFT transformand after FRFT transformthe energy is accumulated and the signal-to-clutter ratio of the echo is improved.On this basisthe Improved Multifractal Detrended Fluctuation Analysis (IMFDFA) algorithm is used to study the multifractal of sea clutter.The target detection is carried out on the measured data of IPIX radar.The experimental results show that the optimal order optimization algorithm proposed in the new algorithm has stable performance.Compared with the traditional fractal algorithmthe performance of the IMFDFA algorithm is obviously improved and more stablewhich is beneficial to the application of the algorithm in detecting small targets on sea surface to a certain extent.