Please use this identifier to cite or link to this item:
Title: Exploring the Quality of Dynamic Open Government Data Using Statistical and Machine Learning Methods
Authors: Karamanou, Areti
Brimos, Petros
Kalampokis, Evangelos
Tarabanis, Konstantinos
Type: Article
Subjects: FRASCATI::Natural sciences::Computer and information sciences
Keywords: open government data
dynamic government data
high-valuable data
real-time data
traffic data
data quality
isolation forest
eXplainable artificial intelligence
Issue Date: 2022
Source: Sensors
Volume: 22
Issue: 24
First Page: 9684
Abstract: Dynamic data (including environmental, traffic, and sensor data) were recently recognized as an important part of Open Government Data (OGD). Although these data are of vital importance in the development of data intelligence applications, such as business applications that exploit traffic data to predict traffic demand, they are prone to data quality errors produced by, e.g., failures of sensors and network faults. This paper explores the quality of Dynamic Open Government Data. To that end, a single case is studied using traffic data from the official Greek OGD portal. The portal uses an Application Programming Interface (API), which is essential for effective dynamic data dissemination. Our research approach includes assessing data quality using statistical and machine learning methods to detect missing values and anomalies. Traffic flow-speed correlation analysis, seasonal-trend decomposition, and unsupervised isolation Forest (iForest) are used to detect anomalies. iForest anomalies are classified as sensor faults and unusual traffic conditions. The iForest algorithm is also trained on additional features, and the model is explained using explainable artificial intelligence. There are 20.16% missing traffic observations, and 50% of the sensors have 15.5% to 33.43% missing values. The average percent of anomalies per sensor is 71.1%, with only a few sensors having less than 10% anomalies. Seasonal-trend decomposition detected 12.6% anomalies in the data of these sensors, and iForest 11.6%, with very few overlaps. To the authors’ knowledge, this is the first time a study has explored the quality of dynamic OGD.
ISSN: 1424-8220
Other Identifiers: 10.3390/s22249684
Appears in Collections:Department of Business Administration

Files in This Item:
File Description SizeFormat 
sensors-22-09684-v2.pdf2,01 MBAdobe PDFThumbnail

This item is licensed under a Creative Commons License Creative Commons