TY - GEN
T1 - FIRE
T2 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022
AU - Du, Xin
AU - Tanaka-Ishii, Kumiko
N1 - Funding Information:
This work was supported by JST, CREST Grant Number JPMJCR2114, Japan, and by JSPS, KAK-ENHI Grant Numbers JP20K20492 and JP21J11781.
Publisher Copyright:
© 2022 Neural information processing systems foundation. All rights reserved.
PY - 2022
Y1 - 2022
N2 - State-of-the-art word embeddings presume a linear vector space, but this approach does not easily incorporate the nonlinearity that is necessary to represent polysemy. We thus propose a novel semantic FIeld REepresentation, called FIRE, which is a D-dimensional field in which every word is represented as a set of its locations and a nonlinear function covering the field. The strength of a word* relation to another word at a certain location is measured as the function value at that location. With FIRE, compositionality is represented via functional additivity, whereas polysemy is represented via the set of points and the function* multimodality. By implementing FIRE for English and comparing it with previous representation methods via word and sentence similarity tasks, we show that FIRE produces comparable or even better results. In an evaluation of polysemy to predict the number of word senses, FIRE greatly outperformed BERT and Word2vec, providing evidence of how FIRE represents polysemy. The code is available at https://github.com/kduxin/firelang.
AB - State-of-the-art word embeddings presume a linear vector space, but this approach does not easily incorporate the nonlinearity that is necessary to represent polysemy. We thus propose a novel semantic FIeld REepresentation, called FIRE, which is a D-dimensional field in which every word is represented as a set of its locations and a nonlinear function covering the field. The strength of a word* relation to another word at a certain location is measured as the function value at that location. With FIRE, compositionality is represented via functional additivity, whereas polysemy is represented via the set of points and the function* multimodality. By implementing FIRE for English and comparing it with previous representation methods via word and sentence similarity tasks, we show that FIRE produces comparable or even better results. In an evaluation of polysemy to predict the number of word senses, FIRE greatly outperformed BERT and Word2vec, providing evidence of how FIRE represents polysemy. The code is available at https://github.com/kduxin/firelang.
UR - http://www.scopus.com/inward/record.url?scp=85163185130&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85163185130&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85163185130
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 35 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022
A2 - Koyejo, S.
A2 - Mohamed, S.
A2 - Agarwal, A.
A2 - Belgrave, D.
A2 - Cho, K.
A2 - Oh, A.
PB - Neural information processing systems foundation
Y2 - 28 November 2022 through 9 December 2022
ER -