MTransE: Mirrored Translation Embedding Model
-
摘要: 知识图广泛应用于许多人工智能(AI)任务。然而，现有知识图通常是不完整的，需对知识图进行补全或链接预测。本文通过对知识图中的实体和关系进行嵌入来预测知识图的缺失环节的问题。引入镜像空间的概念，使得模型具有学习对称/反对称模式的能力。在新的空间模型中，关系仍然被建模为平移，而实体被建模为具有镜像点的点。最后，提出的MTransE模型将镜像空间的概念应用到TransE上，并在四个广泛使用的数据集上进行实验，实验结果表明，该方法能减少参数的规模，并提高在4个广泛使用的知识补全数据集上的性能。Abstract: Knowledge graphs are useful for many artificial intelligence (AI) tasks. However, knowledge graphs often lack of complete facts. In this paper, we study the problem of predicting missing links by learning embeddings of entities and relations in graph knowledge. We introduce a mirror space translation method to learning the symmetric/antisymmetric patterns. Relations are still modelled as translations in our new space, while entities are modelled as points that have mirror points. Within this space, translation-based models gain the ability to model symmetry/antisymmetry relations. Our proposed model MTransE applies the concept of mirrored space to TransE, with experiments on four well-known datasets, shows the performance over other baseline models.
-
图 1 对称关系在二维平面上的嵌入示意图
Figure 1. A diagram of the embedding of symmetric relations in a two-dimensional space
图 2
$ [-8\pi ,8\pi ] $ 中的距离函数图像Figure 2. Distance functions in
$ [-8\mathrm{\pi },8\mathrm{\pi }] $ 表 1 各算法打分函数及其复杂度
Table 1. The scoring functions of algorithms and their complexities
Model Scoring Function Parameters ${\mathcal{O} }_{\mathit{{\rm{t}}}\mathit{{\rm{i}}}\mathit{{\rm{m}}}\mathit{{\rm{e}}} }$ TransE $\left\|\left(\boldsymbol{h}+\boldsymbol{r}\right)-\boldsymbol{t}\right\|$ $\boldsymbol{h},\boldsymbol{r},\boldsymbol{t}\in {\mathbb{R} }^{k}$ $ \mathcal{O}\left(k\right) $ HolE $< \mathbf{r},\boldsymbol{h}*\boldsymbol{t} >$ $\boldsymbol{h},\boldsymbol{r},\boldsymbol{t}\in {\mathbb{R} }^{k}$ $\mathcal{O}\left(k\;\mathrm{lg}\;k\right)$ DistMult $< \boldsymbol{r},\boldsymbol{h},\boldsymbol{t} >$ $\boldsymbol{h},\boldsymbol{r},\boldsymbol{t}\in {\mathbb{R} }^{k}$ $ \mathcal{O}\left(k\right) $ ComplEx $Re( < \boldsymbol{r},\boldsymbol{h},\stackrel{-}{\boldsymbol{t} } > )$ $\boldsymbol{h},\boldsymbol{r},\boldsymbol{t}\in {\mathbb{C} }^{k}$ $ \mathcal{O}\left(k\right) $ MtransE $\left\|\boldsymbol{h}+\boldsymbol{r}-{\boldsymbol{t} }^{\mathit{*} }\right\|$ $\boldsymbol{h},\boldsymbol{r},\boldsymbol{t},{\boldsymbol{t} }^{\mathit{*} }\in {\mathbb{R} }^{k},$
$\left(\boldsymbol{t}-{\boldsymbol{t} }^{\mathit{*} }\right)mod\;b={\boldsymbol{0} }$$ \mathcal{O}\left(k\right) $ 
下载: 导出CSV
表 2 实验数据集
Table 2. Datasets
Dataset #entity #relation #training #validation #test FB15k 14951 1345 483142 50000 59071 WN18 40943 18 141442 5000 5000 FB15k-237 14541 237 272115 17535 20466 WN18RR 40943 11 86835 3034 3134
下载: 导出CSV
表 3 数据集FB15K上的实验结果
Table 3. Results on dataset FB15K
Model MR MRR Hit@1 Hit@3 Hit@10 TransE - 0.463 0.297 0.578 0.749 HolE - 0.524 0.402 0.613 0.739 DistMult 42 0.798 - - 0.893 MTransE 42 0.8 0.752 0.83 0.885
下载: 导出CSV
表 4 数据集FB15K-237上的实验结果
Table 4. Results on dataset FB15K-237
Model MR MRR Hit@1 Hit@3 Hit@10 TransE 357 0.294 - - 0.465 ComplEx 339 0.247 0.158 0.275 0.428 DistMult 254 0.241 0.155 0.263 0.419 MTransE 247 0.316 0.226 0.349 0.5
下载: 导出CSV
表 5 数据集WN18上的实验结果
Table 5. Results on dataset WN18
Model MR MRR Hit@1 Hit@3 Hit@10 TransE - 0.495 0.113 0.888 0.943 HolE - 0.938 0.93 0.945 0.949 DistMult 655 0.797 - - 0.946 MTransE 352 0.949 0.945 0.951 0.957
下载: 导出CSV
表 6 数据集WN18RR上的实验结果
Table 6. Results on dataset WN18
Model MR MRR Hit@1 Hit@3 Hit@10 TransE 3384 0.226 - - 0.501 ComplEx 5261 0.44 0.41 0.46 0.51 DistMult 5110 0.43 0.39 0.44 0.49 MTransE 5183 0.454 0.424 0.463 0.514
下载: 导出CSV
表 7 RotatE和MTransE在FB15K上的实验结果
Table 7. Results of RotatE and MTransE on FB15K
Model MR MRR Hit@1 Hit@3 Hit@10 RotatE 40 0.797 0.746 0.83 0.884 MTransE 42 0.8 0.752 0.83 0.885
下载: 导出CSV
表 8 RotatE和MtransE在WN18上的实验结果
Table 8. Results of RotatE and MTransE on WN18
Model MR MRR Hit@1 Hit@3 Hit@10 RotatE 309 0.949 0.944 0.952 0.959 MTransE 352 0.949 0.945 0.951 0.957
下载: 导出CSV
-
[1] HAO Y, ZHANG Y, LIU K, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C]//ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). Vancouver, Canada: Association for Computational Linguistics, 2017: 221–231. DOI: 10.18653/v1/P17-1021. [2] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating Embeddings for Modeling Multi-relational Data[C]//Advances in Neural Information Processing Systems. Lake Tahoe, Nevada, USA: Neural Information Processing Systems, 2013. [3] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: A collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data. New York, United States: Association for Computing Machinery, 2008: 1247–1249. DOI: 10.1145/1376616.1376746. [4] SUCHANEK F, KASNECI G, WEIKUM G, et al. Yago : A Core of Semantic Knowledge[C]//Proceedings of the 16th International Conference on World Wide Web. New York, United States: Association for Computing Machinery, 2007: 697–706. [5] MILLER G A. WordNet: A Lexical Database for English[J]. Communications of the ACM, 1995, 38(11): 39-41. doi: 10.1145/219717.219748 [6] SUN Z, DENG Z H, NIE J Y, et al. Rotate: Knowledge graph embedding by relational rotation in complex space[C]//7th International Conference on Learning Representations, ICLR 2019. New Orleans, LA, USA: OpenReview. net, 2019: 1–18. [7] WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, California USA: AAAI Press, 2014: 1112–1119. [8] LIN Y, LIU Z, LUAN H, et al. Modeling relation paths for representation learning of knowledge bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: Association for Computational Linguistics, 2015: 705–714. DOI: 10.18653/v1/d15-1082. [9] JI G, HE S, XU L, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing, China: Association for Computational Linguistics, 2015: 687–696. DOI: 10.3115/v1/p15-1067. [10] YANG B, YIH W, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases[C]//3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. San Diego, CA, USA: OpenReview. net, 2015: 1–12. [11] NICKEL M, ROSASCO L, POGGIO T. Holographic embeddings of knowledge graphs[C]//30th AAAI Conference on Artificial Intelligence, AAAI 2016. Phoenix, Arizona USA: AAAI Press, 2016: 1955–1961. [12] TROUILLON T, NICKEL M. Complex and Holographic Embeddings of Knowledge Graphs: A Comparison[J/OL]. 2017(2). . http://arxiv.org/abs/1707.01475. [13] KAZEMI S M, POOLE D. Simple embedding for link prediction in knowledge graphs[C]//NIPS’18: Proceedings of the 32nd International Conference on Neural Information Processing Systems. NY, United States: Curran Associates Inc, 2018: 4284–4295. [14] SOCHER R, CHEN D, MANNING C D, et al. Reasoning with neural tensor networks for knowledge base completion[C]//NIPS’13: Proceedings of the 26th International Conference on Neural Information Processing Systems. NY, United States: Curran Associates Inc, 2013: 926–934. [15] WANG K, LIU Y, XU X, et al. Knowledge Graph Embedding with Entity Neighbors and Deep Memory Network[J/OL]. 2018. . http://arxiv.org/abs/1808.03752. [16] TOUTANOVA K, CHEN D. Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. Beijing, China: Association for Computational Linguistics, 2015: 57–66. DOI: 10.18653/v1/w15-4007. [17] ZHANG S, TAY Y, YAO L, et al. Quaternion Knowledge Graph Embeddings[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. NY, United States: Curran Associates Inc, 2019. DOI: 10.1007/978-3-319-77525-8_284. -
点击查看大图
计量
- 文章访问数: 12
- HTML全文浏览量: 7
- PDF下载量: 1
- 被引次数: 0
