Abstract: Voice conversion algorithm based on Gaussian mixture model (GMM) may result in the over-smoothing of spectral envelop and the damage of speech feature.By analyzing the relationship between covariance's accuracy and over-smoothed phenomena,this paper proposes an adaptive GMM conversion algorithm based on spectral shift,which uses the weighted average algorithm to predict the converted spectral shift.Both the proposed spectral shift and the GMM are adopted to realize the appropriate converted spectral sequence.Moreover,the spectral shift proportion and GMM correlation are adaptively adjusted by using the spectral parameter.The experiment results show that the proposed algorithm can effectively alleviate the over-smoothing and improve the clearness naturalness and intelligibility of converted voice.