In terms of the existing drift problems in the process of extended object tracking caused by rotation, scale transformation and the partial region of the object leaving the field of view, an extended object tracking method is proposed based on feature point matching and generalized Hough transform. Two adjacent frames are processed by ORB feature, and the initial matching feature points are obtained, then the feature points with high matching accuracy are selected as basic feature points by random sampling and consistent processing. These initial feature points are used to carry out generalized Hough transform, and a relative position fixed tracking point is extracted to track the object. Test results show that the proposed method can keep the tracking point relative position fixed for the extended object with rotation, scale transformation and the partial region of the object leaving the field of view.