The motion estimation based on genetic algorithm has better global optimization ability, but its higher complexity enhances the cost for computation and storage and further increases the encoding time. On the other hand, the traditional motion estimation based on genetic algorithm usually adopts lesser genetic iterations, which reduce the searching precision. In order to solve the defects of longer search time and lower accuracy in traditional algorithms, this paper proposed a hybrid algorithm based on genetic search and pattern matching. According to the statistical properties of the motion vector and existing motion vector predicting methods, three termination strategies were designed in the present algorithm, and the matching algorithm was also adopted to optimize the genetic search process. Experiment results show that the proposed algorithm can greatly reduce the search points and coding time while attaining good quality in coding process.