Big data analysis is significant and widely used in many real-world applications in different fields. With the increasing volumes of unstructured data and the multiplicity of their sources, such as the internet, social media, and other large data sources, using a single computer processor may not be able to deal with such a large amount of data. By contrast, the need for new technologies that can deal with large and unstructured data is in high demand, such as non-traditional databases NoSQL, given that traditional databases cannot efficiently deal with such data (Anber et al., 2016).
Recently, predictive analyses of big data gained increasing attention in the computer science research society. The analysis of big data sets employs statistical methods or machine learning models to predict future results or unknown outcomes (Brown et al., 2015). Moreover, data mining techniques are used to search unstructured data based on current and historical data to predict future or unknown information on behavior, direction, and activity of these data. Probabilistic analyses require many experts to construct predictive models for the prediction process (Jayanthi et al., 2017). After the emergence of social media such as Facebook and Twitter, big data analysis and prediction techniques became one of the most important elements for identifying human behavior and predicting unknown aspects such as location, gender, and nationality. Social media data analysis is important for scientists, researchers, and governments (Tufekci, 2014) to understand and analyze many daily issues.
Social networking, characterized by its great potential and openness to data sharing, is a main platform for exchanging information and ideas. The availability of large data through social communication greatly impacted the occurrence of qualitative breakthrough in the study and analysis of the methodology of human behavior, which requires careful studies on strengths and weaknesses (Tufekci, 2014). Many cases of big data analyses require real-time data analyses with comprehensive understanding of the impress factors for some cases, such as influence, reach, and relationship information. To perform correct and accurate analysis of social media, the data require a clear and accurate understanding of the facts they contain (Gentry, 2015). In most cases, we want to know the connections between social data and another event or obtain motivation results from social data analytics to predict some events. A number of outstanding articles in this area include Twitter Mood Predicts, The Personality Predicting, and The Present with Google Trends.
Twitter is one of the largest social media that enables researchers to verify their hypotheses, provide them access to enormous data, which they could use to build realistic applications (Liu, 2013). Twitter allows subscribers to socialize via text messages with a length of 140 characters. The Twitter application contains various types of data, namely, static and dynamic data, such as profile data and Twitter messages, respectively. Tweets can be images, videos, text messages, and more. Twitter has recently been used as a source of information for predicting and/or monitoring real-world results (Atefeh and Khreich, 2015). The large information provided by Twitter, such as Twitter messages, user profile information, and number of followers in the network, plays an important role in data analysis.
In this study, a new efficient and robust algorithm is proposed to predict the country location of Twitter users. The new proposed algorithm uses the available public information of Twitter users and that of their friends and followers. No GPS information is used in this study given that many Twitter users disable this feature due to privacy concerns. To test our proposed algorithm, a big data set is downloaded and gathered from the Twitter site using its API and KNIME software. Results show that our algorithm is efficient and robust against wrongly written and meaningless information by many Twitter users.
The rest of this paper is organized as follows. The second section presents a brief literature review on the prediction techniques from social media with a focus on Twitter. The third section explains the data gathering methods and data statistics. Our proposed algorithm for predicting the home location of Twitter users is explained in the fourth section. Results and discussions are provided in the fifth section. The sixth section concludes the paper and discusses future work.
Related works
In literature, there are many algorithms and methods proposed to predict several human and social patterns and information from social media sites. In this section, we will briefly review some of these algorithms.
Huang et al. (2014) discussed the challenge of the using rich personality traits to identify the nationality of the Twitter users from their profiles. Authors made two constraints when querying the application programming interface (APIs), that they either explicitly stated in their profile that they are located in Qatar or have at least one geo-tagged tweet originating from Qatar. The best performance of Gradient Boosted Tree was achieved with a number of trees equal to 300 and the best overall accuracy is 83%. This algorithm gives good results but the authors applied it only on one country and needs to be further examined in other country locations.
Another new algorithm for predicting Twitter user location is proposed in the study of Chang et al. (2012). In the present research, the authors aimed to improve the quality of predicting the home location of a Twitter user using probability frameworks. The authors proposed a new approach to estimate the spatial word usage probability with Gaussian mixture models. Furthermore, unsupervised measurements are used to rank local words which effectively remove noises that harm prediction. The authors showed that their approach can achieve a comparable and enhanced performance to the state-of-the-art in some cases, using less than 250 local words selected by the proposed method. In this paper, the authors utilized 3183 local words selected through supervised classification based on 11,004 hand-labeled ground truths. Using only 250 selected local words, the proposed approach can predict the home locations of selected Twitter users within an area of 100 miles with an accuracy of 0.499 or 509.3 miles of average error distance at best.
Mcgee et al. (2013) proposed a novel network-based approach (Friendly Location System) for location estimation in social media that integrates evidence of socialite strength between users to improve location estimation based on an examination of over 100 million geo encoded Tweets and 73 million Twitter user profiles. Several identified factors include the number of followers and how users interact which could reveal the distance between a pair of users. The system reduces the average error distance for 80% of Twitter users from 40 miles to 21 miles using only the information from the user’s friends and their friends. In this study, fivefold cross validation is used on the target users to evaluate the system and the Friendly Location System against 249,584 target users. The basic Friendly Location System predicts location within 25 miles or 63.9% of the time, whereas the basic system has an average error distance of 21.4.
In the study of Tsujioka et al. (2016), a study estimated the location of a person according to the Tweets of a Twitter user, and this approach is used to develop tourism. The estimation method for generating decision trees is based on machine learning, and it aims to classify Tokushima and non-Tokushima Twitter users. The accuracy of the estimation method is at 60%, and the method is based on analyzing Twitter messages with the distinction between local citizens and tourists. The proposed method generates a decision tree using machine learning, and four algorithms are used for the decision tree production method, namely, Forest algorithm, C4.5, NBTree, and REPTree, and the correct classification of Tweets for these algorithms are 82.58, 80.95, 78.89, and 66.35, respectively.
Culotta et al. (2015) showed that Twitter follower information is a strong data source for performing demographic inference. From six days’ worth of data (December 6–12, 2013), authors randomly sampled 1000 profiles, categorized them by analyzing the profile, Tweets and profile image for each user and categorized 770 Twitter profiles into one of four ethnicities (Asian, African–American, Hispanic, and Caucasian). The authors fit a regression model to predict the demographics using information about the followers of each website in Twitter. The approach outperforms a fully supervised method for gender classification, and it is competitive for ethnicity classification. The authors found that the identities of only 10 randomly chosen followed accounts per user are sufficient to achieve 90% of the accuracy obtained using 200 followed accounts.
In the study of Hecht et al. (2011), Twitter user profiles are reviewed for user behavior relative to the location field. A total of 34% of users do not provide their true location information, in which the user should write the place where he or she is organically located. The manner in which data from Twitter is collected, as well as the characterization work and effects and machine learning work are explained in the present study. This work is descriptive and does not address causal explanations for the natural behavior of users. A total of 62 million Tweets are collected and set in English using Ling Pipe in text Classification 3 and two-stage combination of Google Language. Explicit user behaviors are shown by implicit position sharing behavior. User and country statuses are estimated with a classifier of user’s Tweets. The methodology behind the creation of training sets for the classifier is highlighted, and how we divided a subset of these data for validation purposes is explained.
A web-based demo in the study of Kong et al. (2014) presents the large-scale user location estimation system (SPOT), which showcases different location estimating models on real-world data sets. The demo shows three different location estimation algorithms: friend-based, social closeness-based, and energy and local social coefficient-based. The first algorithm is a baseline, and the other two new algorithms utilize social closeness information traditionally treated as binary friendship. The two algorithms are based on the premise that friends are different and that close friends can help estimate location better. The demo shows that all three algorithms benefit from confidence-based iteration method, and provides two data sets: Twitter (148,860 located users) and Gowalla (99,563 located users).
The SPOT system uses several location estimating models on different data sets, and demonstrates both estimation accuracy and selected samples to help users compare these models under various settings and observe the estimation process clearly. We present statistics such as the average error distance and average number of friends.
In the studies of Zhang et al. (2011), Bothos et al. (2010), Tumasjan et al. (2010), Golbeck et al. (2011), De Choudhury et al. (2013), Lazarus et al. (2011), Volkova and Yarowsky (2014), many other methods and studies have predicted or determined other features such as predicting the stock market and future events, estimate election results, user information and personalities from their public data like personality traits and privacy concerns, determine diseases including depression and flu trends and identify gender.
Data gathering
The data gathering procedures and steps are described in this section. Moreover, a set of figures and tables are included to present our collected data and their properties in detail.
Kenime
Once the public data sets of Twitter and other social media websites were restricted, collecting the suitable and necessary data set from the internet became one of the most important issues in the big data research community. The collection of a big data set with convenient properties is a significant step to correctly predict and extract unknown information in any prediction system. The use of free and available public information grabbers takes a long time and waste great efforts in cleaning and managing the collected data. Therefore, the utilization of Twitter application programming interface (API) is preferred (■). Twitter API is a package (■) developed by Twitter to allow developers and companies to interact with data including Tweets, user information, and several other attributes. To use Twitter API easily, a scripting language such as Perl or Python must be used to send data request to Twitter servers and then receive and gather the returned data in the required shape. Different public packages or tools are used to simplify the data gathering process. In this research, a quick and easy to use software is employed to connect to the Twitter API request the necessary information and gather them through KNIME software (■). This software is built for rapid and intuitive access to advanced data science, which helps organizations and companies make effective and fast data collection and analysis. The KNIME Analytics Platform, which is designed for discovering potential hidden data through data mining or predicting new futures, is one of the leading open solutions for data-driven problems. The KNIME software includes many important and widely used features such as data gathering and analysis, feature extraction, pattern classification, and clustering techniques. The software contains several nodes that can be connected to one another to perform sequential tasks on a data set, in which each task starts only after finishing the previous one. Figure 1 shows a part of the KNIME project from our implemented workspace that uses numerous nodes to search in Twitter public data, collect the results and then handle it using different ‘Row Filter’ nodes.
Figure 1:
A series of KNIME nodes that used in our data gathering and analyzing processes.
Data set collection
The Twitter data set is collected in this paper, which includes the Twitter information of 129,234 users using the tools described in the previous subsection. The collected information for each Twitter user includes id, user name, location, numbers of friends and followers, time zone, description, and languages. These data are gathered with different convenient keywords for each target country. We targeted five countries to ensure a diverse data set. Table 1 shows the countries and words used in each to gather random users.
Table 1.
The used words in each country.
| Country | Sample keyword for search | ||||
|---|---|---|---|---|---|
| USA | ‘USA’, ‘health’ | ||||
| Spain | ‘Spain’, ‘moda’ | ||||
| Turkey | ‘Turkey’, ‘moda’ | ||||
| France | ‘France’, ‘paris’ | ||||
| Saudi Arabia | ‘ ’, ‘ ’ |
| Country | Sample keywords | ||||
|---|---|---|---|---|---|
| USA | USA – Miami – Los Angeles – California – Chicago – Houston | ||||
| France | France – Landau –Melnibone – Bordeaux – Tours – Lyon – Paris – Nice | ||||
| Saudi Arabia | Saudi Arabia – Dammam – – – – -
| ||||
| Turkey | Turkey – Istanbul – Izmir – Samsun – Adana – Antalya – Ankara | ||||
| Spain | Spain – Barcelona – Madrid – Agitando – Granada – Barna |
| Country | FrLoc | FLLoc | FrLG | FLLG | Sum |
|---|---|---|---|---|---|
| Turkey | 0.4 | 0.3 | 0.5 | 0.4 | 1.6 |
| USA | 0.2 | 0.3 | 0.3 | 0.4 | 1.2 |
| Spain | 0.1 | 0.2 | 0.2 | 0.1 | 0.6 |
| Country | Accuracy | |
|---|---|---|
| USA | 90% | |
| Turkey | 98% | |
| Spain | 94% | |
| Saudi Arabia | 86% | |
| France | 96% |
| Algorithm | No. of country | Accuracy |
|---|---|---|
| Huang et al. (2014) | 1 | 83.8% |
| Culotta et al. (2015) | 1 | 90% |
| Abbas et al. (2017) | 4 | 90% |
| Proposed | 5 | 92.8% |