A representation learning method is considered stable if it consistently generates similar representation of the given data across multiple runs. Word Embedding Methods (WEMs) are a class of representation learning methods that generate dense vector representation for each word in the given text data. The central idea of this paper is to explore the stability measurement of WEMs using intrinsic evaluation based on word similarity. We experiment with three popular WEMs: Word2Vec, GloVe, and fastText. For stability measurement, we investigate the effect of five parameters involved in training these models. We perform experiments using four real-world datasets from different domains: Wikipedia, News, Song lyrics, and European parliament proceedings. We also observe the effect of WEM stability on two downstream tasks: Clustering and Fairness evaluation. Our experiments indicate that amongst the three WEMs, fastText is the most stable, followed by GloVe and Word2Vec.

NLP, word embedding, stability evaluation

This direct conversion preserves the complete paper as high-resolution, page-body visual assets. This is intentional for fidelity: the source has dense two-column prose, equations, comparison tables, charts, captions, footnotes, and bibliography layouts whose lossless semantic reconstruction cannot be guaranteed from the PDF text layer alone. Running conference headers and printed folios have been excluded. The source-page plates below preserve the original reading order and every scholarly element, including the ending references.