Have a personal or library account? Click to login
Efficient Stratified Sampling Graphing Method for Mass Data Cover

Efficient Stratified Sampling Graphing Method for Mass Data

Open Access
|Nov 2019

Abstract

Sequentially linking data during polyline graphing of mass data (millions of points or more) generally results in poor graphing efficiency. Numerous curves are buried behind each pixel and cannot be displayed due to resolution limits of the width of the X-axis. Herein, a new efficient stratified sampling graphing method is proposed. The test results demonstrated that: (1) The full dataset is divided into 2X subsets, where X is the width of the X-axis in pixels, and the maximum and minimum values of the data in each subset are respectively calculated and linked in order of appearance. This method yields 4X sampled data graphs that are highly consistent with the full dataset graphs. (2) When the dataset is divided into 2X, 4X, 6X, 8X, or more subsets (progressively increasing by even multiples), the similarity gradually increases. The average similarities can reach approximately 99.24% and 99.93% in the 2X and 50X subsets, respectively. We think that 2X is the optimal subset allocation, which can achieve a high similarity, but also achieve the fastest sampling speed. (3) Compared with the speed of full dataset graphing, the overall speed of the “single-thread sampling + graphing” is increased by approximately 70 times, and that of the “threadPool sampling + graphing” was enhanced by approximately 200 times. The method employs the minimum amount of sampled data to obtain the full dataset graph that users expect to see, thereby significantly improving graphing speeds of mass data.

Language: English
Submitted on: Jul 16, 2019
|
Accepted on: Oct 31, 2019
|
Published on: Nov 13, 2019
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2019 Jianjun Wang, Yingang Zhao, Jun Chen, Suqing Zhang, Xudong Zhao, Yufei He, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.