A Collection of Swedish Diachronic Word Embedding Models Trained on Historical Newspaper Data

Simon Hengchen; Nina Tahmasebi

doi:10.5334/johd.22

Abstract

This paper describes the creation of several word embedding models based on a large collection of diachronic Swedish newspaper material available through Språkbanken Text, the Swedish language bank. This data was produced in the context of Språkbanken Text’s continued mission to collaborate with humanities and natural language processing (NLP) researchers and to provide freely available language resources, for the development of state-of-the-art NLP methods and tools.

References

1Adesam, Y., Dannélls, D., & Tahmasebi, N. (2019). Exploring the quality of the digital historical newspaper archive KubHist. In Proceedings of the 2019 DHN conference (pp. 9–17).
Back to article
2Antoniak, M., & Mimno, D. (2018). Evaluating the stability of embedding-based word similarities. Transactions of the Association for Computational Linguistics, 6, 107–119. DOI: 10.1162/tacl_a_00008
Back to article
3Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135–146. DOI: 10.1162/tacl_a_00051
Back to article
4Borin, L., Forsberg, M., Hammarstedt, M., Rosén, D., Schäfer, R., & Schumacher, A. (2016). Sparv: Språkbanken’s corpus annotation pipeline infrastructure. In The Sixth Swedish Language Technology Conference (SLTC), Umeå University (pp. 17–18).
Back to article
5Borin, L., Forsberg, M., & Roxendal, J. (2012). Korp — the corpus infrastructure of Språkbanken. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12) (pp. 474–478). Istanbul, Turkey: European Language Resources Association (ELRA). URL: http://www.lrec-conf.org/proceedings/lrec2012/pdf/248_Paper.pdf
Back to article
6Budts, S. (2020). A connectionist approach to analogy. On the modal meaning of periphrastic do in Early Modern English. Corpus Linguistics and Linguistic Theory, 1(ahead-of-print). DOI: 10.1515/cllt-2019-0080
Back to article
7Cook, P., & Stevenson, S. (2010). Automatically identifying changes in the semantic orientation of words. In N. C. C. Chair, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, & D. Tapias (Eds.), Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10). Valletta, Malta: European Language Resources Association (ELRA).
Back to article
8Dubossarsky, H., Hengchen, S., Tahmasebi, N., & Schlechtweg, D. (2019). Time-out: Temporal referencing for robust modeling of lexical semantic change. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 457–470). Florence, Italy: Association for Computational Linguistics. DOI: 10.18653/v1/P19-1044
Back to article
9Dubossarsky, H., Weinshall, D., & Grossman, E. (2017). Outta control: Laws of semantic change and inherent biases inword representation models. In Proceedings of the 2017 Conference onEmpirical Methods in Natural Language Processing (pp. 1136–1145). Copenhagen, Denmark: Association for Computational Linguistics. DOI: 10.18653/v1/D17-1118
Back to article
10Duong, Q., Hämäläinen, M., & Hengchen, S. (2020). An unsupervised method for OCR post-correction and spelling normalisation for Finnish. arXiv preprint arXiv:2011.03502.
Back to article
11Hengchen, S., Ros, R., Marjanen, J., & Tolonen, M. (2021a). A data-driven approach to studying changing vocabularies in historical newspaper collections. Digital Scholarship in the Humanities.
Back to article
12Hengchen, S., Tahmasebi, N., Schlechtweg, D., & Dubossarsky, H. (2021b). Challenges for computational lexical semantic change. In N. Tahmasebi, L. Borin, A. Jatowt, Y. Xu, & S. Hengchen (Eds.), Computational Approaches to Semantic Change, Language Variation, chap. 11. Berlin: Language Science Press.
Back to article
13Hill, M. J., & Hengchen, S. (2019). Quantifying the impact of dirty OCR on historical text analysis: Eighteenth Century Collections Online as a case study. Digital Scholarship in the Humanities, 34(4), 825–843. DOI: 10.1093/llc/fqz024
Back to article
14Hämäläinen, M., & Hengchen, S. (2019). From the paft to the fiiture: a fully automatic NMT and word embeddings method for OCR post-correction. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019) (pp. 431–436). DOI: 10.26615/978-954-452-056-4_051
Back to article
15Kaiser, J., Schlechtweg, D., Papay, S., & Schulte im Walde, S. (2020). IMS at SemEval-2020 Task 1: How low can you go? Dimensionality in Lexical Semantic Change Detection. In Proceedings of the 14th International Workshop on Semantic Evaluation. Barcelona, Spain: Association for Computational Linguistics.
Back to article
16Kim, Y., Chiu, Y.-I., Hanaki, K., Hegde, D., & Petrov, S. (2014). Temporal analysis of language through neural language models. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science (pp. 61–65). DOI: 10.3115/v1/W14-2517
Back to article
17Kulkarni, V., Al-Rfou, R., Perozzi, B., & Skiena, S. (2015). Statistically significant detection of linguistic change. In Proceedings of the 24th International Conference on World Wide Web (pp. 625–635). International World Wide Web Conferences Steering Committee. DOI: 10.1145/2736277.2741627
Back to article
18Kutuzov, A., Øvrelid, L., Szymanski, T., & Velldal, E. (2018). Diachronic word embeddings and semantic shifts: a survey. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 1384–1397).
Back to article
19Marjanen, J., Kurunmäki, J., Pivovarova, L., & Zosa, E. (2020). The expansion of isms, 1820–1917: Data-driven analysis of political language in digitized newspaper collections. Journal of Data Mining and Digital Humanities.
Back to article
20McGillivray, B., Alex, B., Ames, S., Armstrong, G., Beavan, D., Ciula, A., Colavizza, G., Cummings, J., De Roure, D., Farquhar, A., et al. (2020). The challenges and prospects of the intersection of humanities and data science: A white paper from The Alan Turing Institute.
Back to article
21Michel, J.-B., Shen, Y. K., Aiden, A. P., Veres, A., Gray, M. K., Pickett, J. P., Hoiberg, D., Clancy, D., Norvig, P., Orwant, J., et al. (2011). Quantitative analysis of culture using millions of digitized books. Science, 331(6014), 176–182. DOI: 10.1126/science.1199644
Back to article
22Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
Back to article
23Oliphant, T. E. (2006). A guide to NumPy, vol. 1. USA: Trelgol Publishing.
Back to article
24Řehůřek, R., & Sojka, P. (2010). Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks (pp. 45–50). Valletta, Malta: ELRA. http://is.muni.cz/publication/884893/en
Back to article
25Rosenfeld, A., & Erk, K. (2018). Deep neural models of semantic shift. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 474–484). New Orleans, Louisiana. DOI: 10.18653/v1/N18-1044
Back to article
26Schlechtweg, D., Hätty, A., del Tredici, M., & Schulte im Walde, S. (2019). A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 732–746). Florence, Italy: Association for Computational Linguistics. DOI: 10.18653/v1/P19-1072
Back to article
27Schlechtweg, D., McGillivray, B., Hengchen, S., Dubossarsky, H., & Tahmasebi, N. (2020). SemEval-2020 task 1: Unsupervised Lexical Semantic Change Detection. In Proceedings of the 14th International Workshop on Semantic Evaluation. Barcelona, Spain: Association for Computational Linguistics.
Back to article
28Schlechtweg, D., & Schulte im Walde, S. (2020). Simulating Lexical Semantic Change from Sense-Annotated Data. In A. Ravignani, C. Barbieri, M. Martins, M. Flaherty, Y. Jadoul, E. Lattenkamp, H. Little, K. Mudd, & T. Verhoef (Eds.), The Evolution of Language: Proceedings of the 13th International Conference (EvoLang13). DOI: 10.17617/2.3190925
Back to article
29Shoemark, P., Liza, F. F., Nguyen, D., Hale, S., & McGillivray, B. (2019). Room to Glo: A systematic comparison of semantic change detection approaches with word embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 66–76). Hong Kong, China: Association for Computational Linguistics. DOI: 10.18653/v1/D19-1007
Back to article
30Språkbanken. (2019). The Kubhist Corpus, v2. Department of Swedish, University of Gothenburg. URL: https://spraakbanken.gu.se/korp/?mode=kubhist, version downloaded in 2019.
Back to article
31Tahmasebi, N., Borin, L., & Jatowt, A. (2018). Survey of computational approaches to lexical semantic change. arXiv preprint arXiv:1811.06278.
Back to article
32Tripodi, R., Warglien, M., Levis Sullam, S., & Paci, D. (2019). Tracing antisemitic language through diachronic embedding projections: France 1789–1914. In Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change (pp. 115–125). Florence, Italy: Association for Computational Linguistics. DOI: 10.18653/v1/W19-4715
Back to article
33van Eijnatten, J., & Ros, R. (2019). The eurocentric fallacy. A digital approach to the rise of modernity, civilization and Europe. International Journal for History, Culture and Modernity, 7. DOI: 10.18352/hcm.580
Back to article
34van Strien, D., Beelen, K., Ardanuy, M. C., Hosseini, K., McGillivray, B., & Colavizza, G. (2020). Assessing the impact of OCR quality on downstream NLP tasks. In ICAART, 1, 484–496. DOI: 10.5220/0009169004840496
Back to article

A Collection of Swedish Diachronic Word Embedding Models Trained on Historical Newspaper Data

Abstract

Paradigm

My account