Aiming at the Track-Before-Detect (TBD) problem of near space hypersonic target, a kind of track-before-detect algorithm based on Random Hough Transform (RHT) is proposed. By making error analysis to target measurement data, the time-radial distance plane random Hough transform is used to convert measurement to the parameter space. And the selection criterion of parameter space permissible error is obtained by calculation and analysis. Then the target measurement can be accumulated effectively. The algorithm validity is verified by simulation, and the algorithm performance is compared with the algorithm based on Standard Hough Transform (SHT). The simulation results show that, compared with the standard Hough transform, the algorithm in this paper can effectively reduce the algorithm running time and the false track number, and realize effective detection of near space hypersonic target.