OutlierFlag: A Tool for Scientific Data Quality Control by Outlier Data Flagging

Shuai Huang; Yaqiang Wang; Yuanli Xie; Peng Zhao; Johannes Lüers

doi:10.5334/jors.90

OutlierFlag: A Tool for Scientific Data Quality Control by Outlier Data Flagging

Journal of Open Research Software

Volume 4 (2016): Issue 1

By: Shuai Huang , Yaqiang Wang, Yuanli Xie, Peng Zhao and Johannes Lüers

Open Access

|May 2016

Abstract

Scientific datasets collected by instruments usually include outliers which have to be flagged in data quality control process. OutlierFlag was developed to make this process accurate and simple by providing a suitable outlier data flagging algorithm and a user friendly GUI. The algorithm consists of three steps performed one by one: limitation check, error check and standard deviation check. Several parameters are configurable so the algorithm can be used for various datasets. OutlierFlag is an open source software written in Java and the MeteoInfo library was used for data plotting function.

References

Ren Z The quality control of surface monthly climate data in China Journal of Applied Meteorological Science 2007 18 4 516 523 (In Chinese)
Search in Google Scholar Back to article
Yang P Liu W Zhong J Yang J Evaluating the quality of temperature measured at automatic weather stations in Beijing Journal of Applied Meteorological Science 2011 22 6 706 715 (In Chinese)
Search in Google Scholar Back to article
Liu X Ren Z Progress in quality control of surface meteorological data Meteorological Science and Technology 2005 33 3 109 203 (In Chinese)
Search in Google Scholar Back to article
Eischeid J K Baker C B Karl T R Diaz H F The quality control of long-term climatological data using objective data analysis Journal of Applied Meteorology 1995 34 2787 2795 10.1175/1520-0450(1995)034<;2787:TQCOLT>2.0.CO;2
Open DOI Search in Google Scholar Back to article
Zhao P Lüers J Foken T GaFiR: a gap-filling package for ecosystem-atmosphere carbon dioxide flux and evapotranspiration data 2014 University of Bayreuth, Dept. of Micrometeorology 59 1614-8916 19 Work Report
Search in Google Scholar Back to article
Komsta Lukasz outliers: Tests for outliers 2011 R package version 0.14 http://CRAN.R-project.org/package=outliers
Search in Google Scholar Back to article
Dixon W J Analysis of extreme values The Annals of Mathematical Statistics 1950 21 4 488 506 10.1214/aoms/1177729747
Open DOI Search in Google Scholar Back to article
Dixon W J Ratios involving extreme values The Annals of Mathematical Statistics 1951 22 1 68 78 10.1214/aoms/1177729693
Open DOI Search in Google Scholar Back to article
Grubbs F E Sample Criteria for testing outlying observations The Annals of Mathematical Statistics 1950 21 1 27 58
Search in Google Scholar Back to article
Wang Y Q MeteoInfo: GIS software for meteorological data visualization and analysis Meteorological Applications 2014 21 2 360 368
Search in Google Scholar Back to article
Wang Y Q Zhang X Y Sun J Y Zhang X C Che H Z Li Y Spatial and temporal variations of the concentrations of PM10, PM2.5 and PM1 in China Atmospheric Chemistry and Physics 2015 15 13585 13598 10.5194/acp-15-13585-2015
Open DOI Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.5334/jors.90 | Journal eISSN: 2049-9647

Journal RSS Feed

Language: English

Published on: May 31, 2016

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

data quality control,

observational data,

outlier data,

Java,

Sofware development

© 2016 Shuai Huang, Yaqiang Wang, Yuanli Xie, Peng Zhao, Johannes Lüers, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 4 (2016): Issue 1