An adaptive object tracking algorithm based on hierarchical convolution features is proposed to solve the problems of variable scale, rotation and occlusion in object tracking. The hierarchical convolution features are extracted using the convolution neural network, the response maps of convolution features are obtained by the correlation filtering algorithm, and the weighted fusion response is employed to estimate the location of object. An edge detection algorithm is used to realize the scale adaptive tracking. Peak-side-ratio is used to judge the object confidence and solve the problem of template updating under occlusion. The proposed algorithm is tested in the OTB2013 database. The overall success rate and the precision of the proposed algorithm is 0.618 and 0.861, respectively. In the case of object scale variation, rotation and occlusion, the proposed algorithm can accurately and reliably track the object.