Hybrid Fuzzy Neural Network based on Error Distribution Analysis for Time-series Prediction
-
摘要: 针对时间序列预测问题,提出了一种基于误差分布分析的混杂模糊神经网络预测模型。首先提出了一种混杂模糊神经网络结构,将原单一输出层替换为由一个全连接层和非线性激活函数混合的组合网络,用于学习组合隶属度层的输出。然后,提出了基于误差分布的损失函数,使得更新参数的过程中既考虑了误差的大小又考虑了期望的误差分布。根据新的模型结构和新的损失函数,梯度下降过程中,预测误差小而出现概率较高或误差大而出现率低的两类样本将获得较少的训练梯度贡献,而处于中间的样本在训练过程中获得更新增益,通过证明表明本文提出的方法可以获得更均匀和稳定的预测输出。最后通过两个仿真实验验证了本文提出的方法的有效性和准确性。Abstract: This paper considers a hybrid fuzzy neural networks (FNN) for time-series prediction based on error distribution analysis. Firstly, a new hybrid FNN (HFNN) structure is established, where the last two layers is replaced by a combination of a full connection layer and nonlinear activation function. Thus, more parameters can be updated in training process to guarantee the prediction accuracy. Secondly, a novel attention loss function is proposed to make a sample with a certain error distribution get more gains in training process. Based on rule analysis with probability density function, it is seen that the proposed method can provide a more uniform and stable predicted output. The prediction errors of HFNN converge to a compact set. Finally, two benchmark problems are applied to demonstrate the hybrid model performance on time series prediction. The comparisons with other prediction models have verified the efficiency and accuracy of the proposed HFNN model.
-
表 1 FNN, LSTM和HFNN等非线性系统建模实验结果
Table 1. Experimental results off FNN, LSTM and HFNN for nonlinear system modeling
MSE mean var HFNN 1.5870×10−5 2.5497×10−5 FNN 3.3466×10−5 5.7975×10−5 LSTM 2.6356×10−5 3.1110×10−5 BPNN 2.8839×10−5 3.2695×10−5 ARIMA 2.8840×10−5 6.6095×10−5 DBN+FNN[16] 2.8839×10−5 2.6355×10−5 -
[1] MAMDANI E H, ASSILIAN S. An experiment in linguistic synthesis with a fuzzy logic controller[J]. International Journal of Man-Machine Studies, 1975, 7(1): 1-13. doi: 10.1016/S0020-7373(75)80002-2 [2] LARSEN P M. Industrial applications of fuzzy logic control[J], International Journal of Man-Machine Studies, 1980, 12(1): 3–10. [3] TONG S C, WANG T, LI Y M, et al. A combined back stepping and stochastic small-gain approach to robust adaptive fuzzy output feed-back control[J]. IEEE Transactions of Fuzzy System, 2013, 21(2): 314-327. doi: 10.1109/TFUZZ.2012.2213260 [4] XIE X P, YUE D, PARK J H, et al. Relaxed fuzzy observer design of discrete-time nonlinear systems via two effective technical measures[J]. IEEE Transactions of Fuzzy System, 2018, 26(5): 2833-2845. doi: 10.1109/TFUZZ.2018.2791983 [5] XIE X P, YUE D, PENG C. Relaxed real-time scheduling stabilization of discrete-time takagi–sugeno fuzzy systems via an alterable-weights-based ranking switching mechanism[J]. IEEE Transactions of Fuzzy System, 2018, 26(6): 3808-3819. doi: 10.1109/TFUZZ.2018.2849701 [6] ZHANG T F, MA F M, YUE D, et al. Intervaltype-2 fuzzy local enhancement based rough k-means clustering considering imbalanced clusters[J]. IEEE Transactions of Fuzzy System, 2020, 28(9): 1925-1939. doi: 10.1109/TFUZZ.2019.2924402 [7] WU X L, HAN H G, LIU Z, et al. Data-knowledge-based fuzzy neural network for nonlinear system identification[J]. IEEE Transactions of Fuzzy System, 2020, 28(9): 2209-2221. doi: 10.1109/TFUZZ.2019.2931870 [8] LIU Y J, GAO Y, TONG S C, et al. Fuzzy approximation-based adaptive back stepping optimal control for a class of nonlinear discrete-time systems with dead-zone[J]. IEEE Transactions of Fuzzy System, 2016, 24(1): 16-28. doi: 10.1109/TFUZZ.2015.2418000 [9] CHEN Y C, TENG C C. A model reference control structure using a fuzzy neural network[J]. Fuzzy Sets and Systems, 1995, 73(3): 291-312. doi: 10.1016/0165-0114(94)00319-3 [10] TANG J J, LIU F, ZOU Y J, et al. An improved fuzzy neural network for traffic speed prediction considering periodic characteristic[J]. IEEE Transactions of Intelligent Transportation Systems, 2017, 18(9): 2340-2350. doi: 10.1109/TITS.2016.2643005 [11] BARBOUNIS T, THEOCHARIS J B. A locally recurrent fuzzy neural network with application to the wind speed prediction using spatial correlation[J]. Neurocomputing, 2007, 70(7-9): 1525-1542. doi: 10.1016/j.neucom.2006.01.032 [12] PAN Y, ER M J, LI X, et al. Machine health condition prediction via online dynamic fuzzy neural networks[J]. Engineering Applications of Artificial Intelligence, 2014, 35: 105-113. doi: 10.1016/j.engappai.2014.05.015 [13] LIU Y, LIN Y, WU S, et al. Brain dynamics in predicting driving fatigue using a recurrent self-evolving fuzzy neural network[J]. IEEE Transactions of Neural Networks and Learning Systems, 2016, 27(2): 347-360. doi: 10.1109/TNNLS.2015.2496330 [14] YILMAZ S, OYSAL Y. Fuzzy wavelet neural network models for prediction and identification of dynamical systems[J]. IEEE Transactions of Neural Networks and Learning Systems, 2010, 21(10): 1599-1609. doi: 10.1109/TNN.2010.2066285 [15] HANNAN M A, GEE C T, JAVADI M S. Automatic vehicle classification using fast neural network and classical neural network for traffic monitoring[J]. Turkish Journal of Electrical Engineering & Computer Sciences, 2015, 23(1): 2031-2042. [16] WANG G, JIA Q S, QIAO J, et al. A sparse deep belief network with efficient fuzzy learning framework[J]. Neural Networks, 2020, 121: 430-440. doi: 10.1016/j.neunet.2019.09.035 [17] YIN H B, WONG S C, XU J, et al. Urban traffic flow prediction using a fuzzy-neural approach[J]. Transportation Research PartC:Emerging Technologies, 2002, 10(2): 85-98. doi: 10.1016/S0968-090X(01)00004-3 [18] JUANG C F, LIN C T. An online self-constructing neural fuzzy inference network and its applications[J]. IEEE Transactions of Fuzzy System, 1998, 6(1): 12-32. doi: 10.1109/91.660805 [19] GOODFELLOW I, BENGIO Y, COURVILLE A, et al. Deep Learning[M]. Cambridge: MIT press, 2016. [20] ZHANG R, TAO J. A nonlinear fuzzy neural network modeling approach using an improved genetic algorithm[J]. IEEE Transactions of Industrial Electronics, 2018, 65(7): 5882-5892. doi: 10.1109/TIE.2017.2777415 [21] ZHOU H, YANG C, LIU W, et al. A sliding-window t-s fuzzy neural network model for prediction of silicon content in hot metal[J], IFAC-Papers OnLine, 2017, 50(1): 14988–14991. [22] KONEˇCNÝ J, LIU J, RICHTÁRIK P, et al. Mini-batch semi-stochastic gradient descent in the proximal setting[J]. IEEE Journal of Selected Topics in Signal Process, 2016, 10(2): 242-255. doi: 10.1109/JSTSP.2015.2505682 [23] MANDIC D P. A generalized normalized gradient descent algorithm[J]. IEEE Signal Processing Letters, 2004, 11(2): 115-118. doi: 10.1109/LSP.2003.821649 -