Have a personal or library account? Click to login
Word pattern prediction using Big Data frameworks Cover
By: Bence Szabari and  Attila Kiss  
Open Access
|Jul 2020

Abstract

Using software applications or services, which provide word or even word pattern recommendation service has become part of our lives. Those services appear in many form in our daily basis, just think of our smartphones keyboard, or Google search suggestions and this list can be continued. With the help of these tools, we can not only find the suitable word that fits into our sentence, but we can also express ourselves in a much more nuanced, diverse way. To achieve this kind of recommendation service, we use an algorithm which is capable to recommend word by word pattern queries. Word pattern queries, can be expressed as a combination of words, part-of-speech (POS) tags and wild card words. Since there are a lot of possible patterns and sentences, we use Big Data frameworks to handle this large amount of data. In this paper, we compared two popular framework Hadoop and Spark with the proposed algorithm and recommend some enhancement to gain faster word pattern generation.

Language: English
Page range: 51 - 69
Submitted on: Jan 31, 2020
|
Accepted on: Feb 28, 2020
|
Published on: Jul 16, 2020
In partnership with: Paradigm Publishing Services
Publication frequency: 2 issues per year

© 2020 Bence Szabari, Attila Kiss, published by Sapientia Hungarian University of Transylvania
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.