Have a personal or library account? Click to login

A Topic Detection Method Based on Word-attention Networks

Journal of Data and Information Science
Volume 6 (2021): Issue 4 (November 2021)
By:
Open Access
|Aug 2021

Figures & Tables

Figure 1

The statistics of the PNAS papers’ in three major disciplines. There are 90.44% of papers belonging to major disciplines, namely Biological Sciences, Physical Sciences, and Social Sciences. Panel (a) shows the annual number of papers of each discipline. Panel (b) shows the fraction of papers of Biophysics in Biological Sciences.
The statistics of the PNAS papers’ in three major disciplines. There are 90.44% of papers belonging to major disciplines, namely Biological Sciences, Physical Sciences, and Social Sciences. Panel (a) shows the annual number of papers of each discipline. Panel (b) shows the fraction of papers of Biophysics in Biological Sciences.

Figure 2

The illustration of the proposed method. The method consists of two steps: constructing word branches by the Transformer; using methods of community detection to partition the network that are formed by connecting word branches. The topics here are in the form of subgraph.
The illustration of the proposed method. The method consists of two steps: constructing word branches by the Transformer; using methods of community detection to partition the network that are formed by connecting word branches. The topics here are in the form of subgraph.

Figure 3

The performances of using the random walk method to detect node communities. The word networks are treated as directed ones. The step is the parameter of the random walk method, the length of random walks to perform. Panels show the performances of the method with the step 3 (blue dot lines), 4 (red dot lines) and 5 (orange dot lines).
The performances of using the random walk method to detect node communities. The word networks are treated as directed ones. The step is the parameter of the random walk method, the length of random walks to perform. Panels show the performances of the method with the step 3 (blue dot lines), 4 (red dot lines) and 5 (orange dot lines).

Figure 4

The performances of using the louvain method to detect node communities. The word networks are treated as undirected ones. The resolution is the parameter of the louvain method. Panels show the performances of the method with the resolution 0.5 (orange dot lines), 0.7 (red dot lines), and 1.0 (blue dot lines).
The performances of using the louvain method to detect node communities. The word networks are treated as undirected ones. The resolution is the parameter of the louvain method. Panels show the performances of the method with the resolution 0.5 (orange dot lines), 0.7 (red dot lines), and 1.0 (blue dot lines).

Figure 5

The performances of using the louvain method to detect edge communities. The louvain method is used to obtain overlapped communities. Panels show the performances of this method with resolution 0.5 (orange dot lines), 0.7 (red dot lines), and 1.0 (blue dot lines).
The performances of using the louvain method to detect edge communities. The louvain method is used to obtain overlapped communities. Panels show the performances of this method with resolution 0.5 (orange dot lines), 0.7 (red dot lines), and 1.0 (blue dot lines).

Figure 6

The comparisons between the LDA and our method. Panels (a–d) show the average values of assessment indexes, and Panels (e–h) show the standard deviations for running the LDA (red dot lines) and the provided method using the louvain method to detect node communities (resolution 0.7, blue dot lines).
The comparisons between the LDA and our method. Panels (a–d) show the average values of assessment indexes, and Panels (e–h) show the standard deviations for running the LDA (red dot lines) and the provided method using the louvain method to detect node communities (resolution 0.7, blue dot lines).

Figure 7

The performances of the LDA with the same number of topics as that of our method. Panels show the performances of the case with the number of communities of tokens (blue dot lines) and the performances with the number of communities of edges (red dot lines). These communities are detected by the louvain method.
The performances of the LDA with the same number of topics as that of our method. Panels show the performances of the case with the number of communities of tokens (blue dot lines) and the performances with the number of communities of edges (red dot lines). These communities are detected by the louvain method.

Figure 8

The evolution of the tokens is emphasized by the proposed method using the louvain method. Panels show the word clouds of the top five tokens in the top five topics according to tokens’ betweenness centrality in the word network.
The evolution of the tokens is emphasized by the proposed method using the louvain method. Panels show the word clouds of the top five tokens in the top five topics according to tokens’ betweenness centrality in the word network.

Figure 9

The evolution of the tokens emphasized by the LDA. Panels show the word clouds of the top five tokens in the top five topics according to the summation of a token's weight overall topics. The number of topics is five. The index n here is the number of tokens emphasized by running the LDA ten times.
The evolution of the tokens emphasized by the LDA. Panels show the word clouds of the top five tokens in the top five topics according to the summation of a token's weight overall topics. The number of topics is five. The index n here is the number of tokens emphasized by running the LDA ten times.

Figure 10

The difference on emphasized tokens. Panel (a) shows the tokens emphasized by both the LDA and our method. The set of tokens emphasized by the provided method includes that emphasized by the LDA. Panel (b) shows the tokens only emphasized by our method.
The difference on emphasized tokens. Panel (a) shows the tokens emphasized by both the LDA and our method. The set of tokens emphasized by the provided method includes that emphasized by the LDA. Panel (b) shows the tokens only emphasized by our method.

j_jdis-2021-0032_tab_004

Algorithm 3. Constructing word branches.
Input:
titles of papers;
parameter m;
Output:
word branches.
1:for each preprocessed title a0, ..., an do
2:  let b01 = a0;
3:  for i from 1 to n do
4:    predict bi1, ..., bim ranked according to the probability given by the Transformer;
5:    if i > 1 then
6:      generate directed edges from b(i−1)1 to bi1, ..., bim;
7:      use b01, ..., bi1 to predict next tokens by the Transformer;
8:    end if
9:  end for
10:end for

j_jdis-2021-0032_tab_005

Algorithm 4. Cropping word branches.
Input:
word branches;
Output:
cropped word branches.
1:calculate the token-paper matrix (fij)N×M;
2:calculate the tf-idf matrix (wij)N×M;
3:for i from 1 to M do
4:  rank tokens according to {wi1, ..., wiN};
5:  if token bi1 is not in top x% then
6:    crop the tokens of {bi1, ..., bim};
7:    crop the edges connecting those tokens;
8:  else
9:    crop the tokens of {bi2, ..., bim} that are not in top x%;
10:    crop the edges connecting those tokens.
11:  end if
12:end for

j_jdis-2021-0032_tab_002

Algorithm 1. Data preprocessing.
Input:
titles and abstracts of papers;
Output:
list of token lists.
1:stem words using the PorterStemmer of NLTK2;
2:remove stopwords using the stopword corpus of NLTK;
3:remove the words that appear in less than x = 6 papers’ abstract and title.

j_jdis-2021-0032_tab_003

Algorithm 2. The architecture of the Transformer used here.
1:model=get model(token num=max(len(source token dict), len(target token dict)), embed dim=32, encoder num=4, decoder num=4, head num=4, hidden dim=32, dropout rate=0.05, use same embed=False,)
2:model.compile(‘adam’, ‘sparse categorical crossentropy’)
3:model.fit(x=[np.array(encode input∗30), np.array(decode input∗30)], y=np.array(decode output∗30), epochs=5, batch size=32,)

The information of the dataset dblp_

Timeabcdef
19992,4753,27495,0210.112.3710.998
20002,3803,34793,9100.1012.3950.998
20012,4553,477108,9540.1082.3550.999
20022,8123,710117,2690.0942.2721.0
20032,6563,592115,0190.12.3120.999
20042,9553,919138,4510.1012.2990.999
20053,1314,084154,0410.0992.2750.999
20063,2484,260166,6140.12.2890.999
20073,4194,368184,4200.1022.2790.999
20083,4084,436184,8810.1042.3041.0
20093,6584,609212,7710.0982.2181.0
20103,6394,668221,0900.12.2040.999
20113,4624,688220,0200.1112.2280.999
20123,6214,875209,5170.1142.281.0
20133,5934,846231,9590.0962.1891.0
20143,3344,679210,0990.0962.2081.0
DOI: https://doi.org/10.2478/jdis-2021-0032 | Journal eISSN: 2543-683X | Journal ISSN: 2096-157X
Journal RSS Feed
Language: English
Page range: 139 - 163
Submitted on: Jun 19, 2021
Accepted on: Jul 23, 2021
Published on: Aug 18, 2021
Published by: Chinese Academy of Sciences, National Science Library
In partnership with: Paradigm Publishing Services
Publication frequency: 4 times per year
Keywords:
Scientific topics,
Text analysis,
Deep learning
Related subjects:
Computer sciences,
Information technology,
Project management,
Databases and data mining

© 2021 Zheng Xie, published by Chinese Academy of Sciences, National Science Library
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Previous articleVolume 6 (2021): Issue 4 (November 2021)