Abstract: Emotional state has an important influence on people's work and life. Electroencephalogram (EEG) signals, as central neurophysiological signals, are closely related to human emotions. This paper uses the public EEG dataset SEED to undergo the emotion recognition research for automatically extracting the abstract features of the EEG samples and avoiding the process of artificial feature selection and dimensionality reduction. Firstly, the wavelet packet transform (WPT) is utilized to perform six-level wavelet packet decomposition on the EEG signals of the subjects and extract the features of the signals and form two-dimensional structure samples. Since the structural depth has an essential effect on the classification ability of the network, we take the depth of the convolutional neural network as the main line to establish six CNN classification models with different depths. Finally, ensemble convolutional neural networks are constructed by means of the voting method and the weighted averaging method, respectively, such that the classification accuracy can be improved. The experimental results show that the CNN with the appropriate depth has excellent classification ability, and the network structure that is too shallow or too deep is not conducive to the improvement of classification performance. The ensemble model constructed by the weighted method can obtain the best classification accuracy and minimum variance among all the classifiers, which illustrates its reliable and stable classification ability. The proposed research scheme in this paper achieves an average classification accuracy of 93.12% for the three emotional categories, which can meet the needs of emotion recognition.