Shared Subscribe Hyper Simulation Optimization (SUBHSO) Algorithm for Clustering Big Data – Using Big Databases of Iran Electricity Market

Mesbaholdin Salami; Farzad Movahedi Sobhani; Mohammad Sadegh Ghazizadeh

doi:10.2478/acss-2019-0007

Abstract

Many real world problems have big data, including recorded fields and/or attributes. In such cases, data mining requires dimension reduction techniques because there are serious challenges facing conventional clustering methods in dealing with big data. The subspace selection method is one of the most important dimension reduction techniques. In such methods, a selected set of subspaces is substituted for the general dataset of the problem and clustering is done using this set. This article introduces the Shared Subscribe Hyper Simulation Optimization (SUBHSO) algorithm to introduce the optimized cluster centres to a set of subspaces. SUBHSO uses an optimization loop for modifying and optimizing the coordinates of the cluster centres with the particle swarm optimization (PSO) and the fitness function calculation using the Monte Carlo simulation. The case study on the big data of Iran electricity market (IEM) has shown the improvement of the defined fitness function, which represents the cluster cohesion and separation relative to other dimension reduction algorithms.

References

[1] H. Chen, and Z. Mao, “Study on the failure probability of occupant evacuation with the method of Monte Carlo sampling,” Procedia Engineering, vol. 211, 2018, pp. 55–62. https://doi.org/10.1016/j.proeng.2017.12.13710.1016/j.proeng.2017.12.137
Search in Google Scholar Back to article
[2] T. G. Penkova, “Principal component analysis and cluster analysis for evaluating the natural andanthropogenic territory safety,” Procedia Computer Science, vol. 112, 2017, pp. 99–108. https://doi.org/10.1016/j.procs.2017.08.17910.1016/j.procs.2017.08.179
Search in Google Scholar Back to article
[3] E. Vera, D. Lucio, L. A. F. Fernandes, and L. Velho, “Hough transform for real-time plane detection in depth images,” Pattern Recognition Letters, vol. 103, 2018, pp. 8–15. https://doi.org/10.1016/j.patrec.2017.12.02710.1016/j.patrec.2017.12.027
Search in Google Scholar Back to article
[4] M. H. Yang, J. H. Li, and B. X. Liu, “Fractal analysis on the cluster network in metallic liquid and glass,” Journal of Alloys and Compounds, vol. 757, 2018, pp. 228–232. https://doi.org/10.1016/j.jallcom.2018.05.06910.1016/j.jallcom.2018.05.069
Search in Google Scholar Back to article
[5] T. Cui, F. Caravelli, and C. Ududec, “Correlations and clustering in wholesale electricity markets,” Physica A: Statistical Mechanics and its Applications, vol. 492, 2018, pp. 1507–1522. https://doi.org/10.1016/j.physa.2017.11.07710.1016/j.physa.2017.11.077
Search in Google Scholar Back to article
[6] G. Zhu, J. Wang, and H. Lu, “Clustering based ensemble correlation tracking,” Computer Vision and Image Understanding, vol. 153, 2016, pp. 55–63. https://doi.org/10.1016/j.cviu.2016.05.00610.1016/j.cviu.2016.05.006
Search in Google Scholar Back to article
[7] S. Chormunge, and S. Jena, “Correlation based feature selection with clustering for high dimensional data,” Journal of Electrical Systems and Information Technology, vol. 5, no. 3, 2018, pp. 542–549. https://doi.org/10.1016/j.jesit.2017.06.00410.1016/j.jesit.2017.06.004
Search in Google Scholar Back to article
[8] K. Fujiwara, M. Kano, and S. Hasebe, “Development of correlation-based clustering method and its application to software sensing,” Chemometrics and Intelligent Laboratory Systems, vol. 101, no. 2, 2010, pp. 130–138. https://doi.org/10.1016/j.chemolab.2010.02.00610.1016/j.chemolab.2010.02.006
Search in Google Scholar Back to article
[9] R. Veroneze, A. Banerjee, and F. J. von Zuben, “Enumerating all maximal biclusters in numerical datasets,” Information Sciences, vol. 379, 2017, pp. 288–309. https://doi.org/10.1016/j.ins.2016.10.02910.1016/j.ins.2016.10.029
Search in Google Scholar Back to article
[10] S. Chen, J. Liu, and T. Zeng, “Measuring the quality of linear patterns inbiclusters,” Methods, vol. 83, 2015, pp. 18–27. https://doi.org/10.1016/j.ymeth.2015.04.00510.1016/j.ymeth.2015.04.00525890245
Search in Google Scholar Back to article
[11] G. F. de Sousa Filho, L. dos A. F. Cabral, L. S. Ochi, and F. Protti, “Hybrid metaheuristic for bicluster editing problem,” Electronic Notes in Discrete Mathematics, vol. 39, 2012, pp. 35–42. https://doi.org/10.1016/j.endm.2012.10.00610.1016/j.endm.2012.10.006
Search in Google Scholar Back to article
[12] M. Wang, X. Shang, X. Li, W. Liu, and Z. Li, “Efficient mining differential co-expression biclusters in microarray datasets,” Gene, vol. 518, no. 1, 2013, pp. 59–69. https://doi.org/10.1016/j.gene.2012.11.08510.1016/j.gene.2012.11.08523276708
Search in Google Scholar Back to article
[13] Y. Lee, J. Lee, and C. H. Jun, “Stability-based validation of bicluster solutions,” Pattern Recognition, vol. 44, no. 2, 2011, pp. 252–264. https://doi.org/10.1016/j.patcog.2010.08.02910.1016/j.patcog.2010.08.029
Search in Google Scholar Back to article
[14] F. Divina, B. Pontes, R. Giráldez, and J. S. Aguilar-Ruiz, “An effective measure for assessing the quality of biclusters,” Computers in Biology and Medicine, vol. 42, no. 2, 2012, pp. 245–256. https://doi.org/10.1016/j.compbiomed.2011.11.01510.1016/j.compbiomed.2011.11.01522196882
Search in Google Scholar Back to article
[15] C. C. Aggarwal, J. L. Wolf, P. S. Yu, C. Procopiuc, and J. S. Park, “Fast algorithms for projected clustering,” Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, SIGMOD, ACM, New York, NY, USA, 1999, pp. 61–72. https://doi.org/10.1145/304181.30418810.1145/304181.304188
Search in Google Scholar Back to article
[16] G. Moise, J. Sander, and M. Ester, “Robust projected clustering,” Knowledge and Information Systems, vol. 14, no. 3, 2008, pp. 273–298. https://doi.org/10.1007/s10115-007-0090-610.1007/s10115-007-0090-6
Search in Google Scholar Back to article
[17] G. Gan, and J. Wu, “A convergence theorem for the fuzzy subspace clustering (fsc) algorithm,” Pattern Recognition, vol. 6, no. 2, 2008, pp. 1939–1947. https://doi.org/10.1016/j.patcog.2007.11.01110.1016/j.patcog.2007.11.011
Search in Google Scholar Back to article
[18] Z. Deng, K. S. Choi, F. L. Chung, and S. Wang, “Enhanced soft subspace clustering integrating within-cluster and between-cluster information,” Pattern Recognition, vol. 43, no. 3, 2010, pp. 767–781. https://doi.org/10.1016/j.patcog.2009.09.01010.1016/j.patcog.2009.09.010
Search in Google Scholar Back to article
[19] X. Chen, Y. Ye, X. Xu, and J. Z. Huang, “A feature group weighting method for subspace clustering of high-dimensional data,” Pattern Recognition, vol. 45, no. 1, 2012, pp. 434–446. https://doi.org/10.1016/j.patcog.2011.06.00410.1016/j.patcog.2011.06.004
Search in Google Scholar Back to article
[20] D. S. Modha, and W. S. Spangler, “Feature weighting in k-means clustering,” Machine Learning, vol. 52, no. 3, 2003, pp. 217–237. https://doi.org/10.1023/A:102401660952810.1023/A:1024016609528
Search in Google Scholar Back to article
[21] C. Domeniconi, D. Gunopulos, S. Ma, B. Yan, M. Al-Razgan, and D. Papadopoulos, “Locally adaptive metrics for clustering high dimensional data,” Data Mining and Knowledge Discovery, vol. 14, no. 1, 2007, pp. 63–97. https://doi.org/10.1007/s10618-006-0060-810.1007/s10618-006-0060-8
Search in Google Scholar Back to article
[22] Y. Zhu, K. M. Ting, and M. J. Carman, “Grouping points by shared subspaces for effective subspace clustering,” Pattern Recognition, vol. 83, 2018, pp. 230–244. https://doi.org/10.1016/j.patcog.2018.05.02710.1016/j.patcog.2018.05.027
Search in Google Scholar Back to article
[23] H. Chen, W. Wang, and X. Feng, “Structured sparse subspace clustering with within-cluster grouping,” Pattern Recognition, vol. 83, 2018, pp. 107–118. https://doi.org/10.1016/j.patcog.2018.05.02010.1016/j.patcog.2018.05.020
Search in Google Scholar Back to article
[24] W. Zhu, J. Lu, and J. Zhou, “Nonlinear subspace clustering for image clustering,” Pattern Recognition Letters, vol. 107, 2018, pp. 131–136. https://doi.org/10.1016/j.patrec.2017.08.02310.1016/j.patrec.2017.08.023
Search in Google Scholar Back to article
[25] X. Wang, Z. Lei, X. Guo, C. Zhang, H. Shi, and S. Z. Li, “Multi-view subspace clustering with intactness-aware similarity,” Pattern Recognition, vol. 6, no. 2, 2018, pp. 50–63. https://doi.org/10.1016/j.patcog.2018.09.00910.1016/j.patcog.2018.09.009
Search in Google Scholar Back to article
[26] Y. Chen, and Z. Yi, “Locality-constrained least squares regression for subspace clustering,” Knowledge-Based Systems, vol. 163, 2019, pp. 51–56. https://doi.org/10.1016/j.knosys.2018.08.01410.1016/j.knosys.2018.08.014
Search in Google Scholar Back to article
[27] Ł. Struski, J. Tabor, and P. Spurek, “Lossy compression approach to subspace clustering,” Information Sciences, vol. 435, 2018, pp. 161–183. https://doi.org/10.1016/j.ins.2017.12.05610.1016/j.ins.2017.12.056
Search in Google Scholar Back to article
[28] D. L. Davies, and D. W. Bouldin, “A Cluster Separation Measure,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-1, no. 2, 1979, pp. 224–227. https://doi.org/10.1109/TPAMI.1979.476690910.1109/TPAMI.1979.4766909
Search in Google Scholar Back to article
[29] N. Amjady, F. Keynia, and H. Zareipour, “Wind power prediction by a new forecast engine composed of modified hybrid neural network and enhanced particle swarm optimization,” Sustainable Energy, vol. 2, no. 3, 2011, pp. 265–276. https://doi.org/10.1109/TSTE.2011.211468010.1109/TSTE.2011.2114680
Search in Google Scholar Back to article
[30] T. P. Latchoumi, K. Balamurugan, K. Dinesh, and T. P. Ezhilarasi, “Particle swarm optimization approach for waterjet cavitation peening,” Measurement, vol. 141, 2019, pp. 184–189. https://doi.org/10.1016/j.measurement.2019.04.04010.1016/j.measurement.2019.04.040
Search in Google Scholar Back to article
[31] F. Korner-Nievergelt, T. Roth, S. von Felten, J. Guélat, B. Almasi, and P. Korner-Nievergelt, “Chapter 12: Markov chain Monte Carlo simulation,” in Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and STAN, Academic Press, 2015, pp. 197–212. https://doi.org/10.1016/B978-0-12-801370-0.00012-510.1016/B978-0-12-801370-0.00012-5
Search in Google Scholar Back to article
[32] IGMC. [Online] Available from: https://www.igmc.ir
Search in Google Scholar Back to article

Shared Subscribe Hyper Simulation Optimization (SUBHSO) Algorithm for Clustering Big Data – Using Big Databases of Iran Electricity Market

Abstract

Paradigm

My account