To reduce the computational complexity and time consumption of intraframe prediction in high-efficiency video coding (HEVC), an improved algorithm for intraframe prediction mode fast selecting in HEVC based on the size of the prediction units (PUs) is proposed. For the maximum size of PUs, the statistical probability is used to hierarchically set the candidate modes into the rough mode decision (RMD) process. For other PUs, candidate modes are set by extracting the texture direction in two different ways into the RMD process. We use the pixel gradient for the 32×32 and 16×16 PUs and use pixel value deviation for the 8×8 and 4×4 PUs to extract the texture direction; therefore, fewer modes are selected to calculate and reduce the time taken by the RMD process. Experimental results show that the proposed algorithm reduces the encoding time by approximately 32.2% on average with only a 0.86% increase in code rate in comparison with HM16.9. In compared with the existing algorithms, the proposed algorithm further reduces the coding time and produces better coding quality.