In some opto-electrical imaging system for extended target, because the target is lack of local detail, using complex detector and high-dimensional descriptor not only brings the problem of weak discrimination of description, but also results in high sources occupation and slow calculation, lead to a hard process in real-time. A specialized algorithm is proposed with features from accelerated segment test (FAST) as the detector and binary robust independent elementary feature (BRIEF) as the descriptor, to solve the problems efficiently. In the meantime, because the BRIEF is sensitive to the changes of target, a process named major orientation constraint is employed which can improve the stability of feature recognition. Experiments are done and this new algorithm is compared to two widely used algorithms, speed-up robust feature (SURF) and scale invariant feature transform (SIFT). The result shows that our algorithm performs 5 times of SURF and 17 times of SIFT in speed, meanwhile the recognition accuracy is comparable to that of SURF. Hence this algorithm achieves fast object detection and stable tracking without decline of recognition accuracy.