Have a personal or library account? Click to login
Are We in Agreement? Benchmarking and Reliability Issues between Social Network Analytic Programs Cover

Are We in Agreement? Benchmarking and Reliability Issues between Social Network Analytic Programs

Connections
Volume 37 (2017): Issue 1-2 (January 2017)
By: Philip J. Murphy,  Karen T. Cuenco and  YuFei Wang  
Open Access
|Jun 2018

Figures & Tables

Figure 1

Scatterplot matrix comparing closeness centrality output for a large, two-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal.
Scatterplot matrix comparing closeness centrality output for a large, two-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal.

Figure 3

A variety of solutions are possible when analyzing two-mode networks in UCINET. Top Row: Scatterplots of UCINET’s degree (r = 0.3713) and closeness (r = -0.0134) output using the two-mode centrality procedure, compared with other analytic packages. All other packages performed identically. Bottom Row: When transformed into a bipartite network format, UCINET calculates as for a one-mode network, and results are analogous to other packages. Closeness centrality for the bipartite aspect was calculated using Freeman normalization in UCINET.
A variety of solutions are possible when analyzing two-mode networks in UCINET. Top Row: Scatterplots of UCINET’s degree (r = 0.3713) and closeness (r = -0.0134) output using the two-mode centrality procedure, compared with other analytic packages. All other packages performed identically. Bottom Row: When transformed into a bipartite network format, UCINET calculates as for a one-mode network, and results are analogous to other packages. Closeness centrality for the bipartite aspect was calculated using Freeman normalization in UCINET.

Figure 2

Scatterplot matrix comparing output for closeness centrality in a small, one-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal. Note that the sna package for R does not produce measures between disconnected components, resulting in correlation values listed as “NA”.
Scatterplot matrix comparing output for closeness centrality in a small, one-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal. Note that the sna package for R does not produce measures between disconnected components, resulting in correlation values listed as “NA”.

Figure 4

Scatterplot matrix comparing degree centrality output for a small, one-mode network containing loops. Pearson’s correlation coefficients between programs are provided above the diagonal.
Scatterplot matrix comparing degree centrality output for a small, one-mode network containing loops. Pearson’s correlation coefficients between programs are provided above the diagonal.

Figure 5

Scatterplot matrix comparing eigenvector centrality output for a large, two-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal. Pajek output for this plot was calculated using “important vertices”, a two-mode generalization of hubs and authorities.
Scatterplot matrix comparing eigenvector centrality output for a large, two-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal. Pajek output for this plot was calculated using “important vertices”, a two-mode generalization of hubs and authorities.

Figure 6

Scatterplot matrix of eigenvector centrality output for a small, one-mode network with loops. Pearson’s correlation coefficients between programs are provided above the diagonal. Note, initial calculations in sna – shown above – were run using the default argument (diag=FALSE). For additional variation, consult the text above.
Scatterplot matrix of eigenvector centrality output for a small, one-mode network with loops. Pearson’s correlation coefficients between programs are provided above the diagonal. Note, initial calculations in sna – shown above – were run using the default argument (diag=FALSE). For additional variation, consult the text above.

Figure 7

Scatterplot matrix comparing eigenvector centrality output for a moderately large, one-mode network containing loops and disconnected components. Pearson’s correlation coefficients between programs are provided above the diagonal.
Scatterplot matrix comparing eigenvector centrality output for a moderately large, one-mode network containing loops and disconnected components. Pearson’s correlation coefficients between programs are provided above the diagonal.

Figure 8

Scatterplot matrix comparing betweenness centrality output for a large, one-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal.
Scatterplot matrix comparing betweenness centrality output for a large, one-mode network. Pearson’s correlation coefficients between programs are provided above the diagonal.

Figure 9

Scatterplot matrices comparing degree and eigenvector output for a two-mode network. Neither network contains loops or disconnected components. Pearson’s correlation coefficients between programs are provided above the diagonal.
Scatterplot matrices comparing degree and eigenvector output for a two-mode network. Neither network contains loops or disconnected components. Pearson’s correlation coefficients between programs are provided above the diagonal.

Analytic interfaces used in this study

VersionSource
UCINET6.564 http://www.analytictech.com/
Pajek4.03 http://pajek.imfm.si/
ORA-NetScenes3.0.9.9.20 http://www.casos.cs.cmu.edu/projects/ora/
Gephi0.8.2 http://gephi.org/
sna2.3-2 http://www.statnet.org/
igraph0.7.1 http://igraph.org/

Data used for reliability comparisons

DataNodes in Main ComponentNodes in Smaller ComponentsNumber of LoopsMax. Number of NodesAverage Degree
Small One-mode2966353.5
Large One-mode18761126020003.0
Small Two-mode(10, 21)4NA(10, 25)3.7
Large Two-mode(300, 1815)185NA(300, 1700)3.7

Output scaling

DegreeClosenessBetweennessEigenvector
GephiRawNormalized AverageRawScaled (max=1)
PajekRawNormalizedNormalizedNormalized
UCINETRawAverageRawNormalized
ORANormalizedNormalizedNormalizedNormalized
snaRawNormalizedRawNormalized
igraphRawNormalizedRawScaled (max=1)

Consistency of output by centrality type and network conditions

No Disconnected Components & No LoopsDisconnected ComponentsLoopsDisconnected Components Loops
1 Mode2 Mode1 Mode2 Mode1 Mode2 Mode1 Mode2 Mode
Betweenness CentralityHighHighHighHighHighNAHighNA
Degree CentralityHighMediumHighMediumLowNALowNA
Eigenvector CentralityMediumMediumMediumLowMediumNALowNA
Closeness CentralityMediumMediumLowLowMediumNALowNA
DOI: https://doi.org/10.21307/connections-2017-002 | Journal eISSN: 2816-4245 | Journal ISSN: 0226-1766
Journal RSS Feed
Language: English
Page range: 23 - 44
Published on: Jun 4, 2018
Published by: International Network for Social Network Analysis (INSNA)
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year
Related subjects:
Social sciences,
Social sciences, other

© 2018 Philip J. Murphy, Karen T. Cuenco, YuFei Wang, published by International Network for Social Network Analysis (INSNA)
This work is licensed under the Creative Commons Attribution 4.0 License.

Previous articleVolume 37 (2017): Issue 1-2 (January 2017)Next article