Two Approaches to Factor Time into Word and Entity Representations Learned from Text


In this talk, Prof. Matteo Palmonari (unimib) and I have introduced two methods to account for time into distributional representations. This talk is based on two of our papers: “Training Temporal Word Embeddings with a Compass” (AAAI) and “Towards Encoding Time in Text-Based Entity Embeddings” (ISWC).


Vector representations of words, also known as word embeddings, are now used in a variety of downstream applications related to natural language processing and knowledge representation. These representations are usually learned from text corpora and account for word meaning based on distributional semantics, according to which similar words appear in similar contexts. The very same principles can be also applied to learn representations of entities and ontology types that capture their intuitive meaning using a data-driven and sub-symbolic approach. Time is a crucial factor when dealing with distributional models of language and knowledge. For example, tracking word meaning shift and entity evolution can have several applications and time may sneak into similarity as computed with these models in way that may be difficult to control. In this presentation we discuss two novel approaches to factor time into word and knowledge representations learned from text: explicit, with representations of temporal references (e.g., years, days, etc.), and implicit, with time-dependent representations of words and entities (e.g., amazon_1975 vs. amazon_2012). Finally, being this an emerging field of research, we will discuss several open topics in this research domain.