Advanced Metering Data Quality & CompletionAdvanced Metering Data Quality & CompletionAdvanced Metering Data Quality & Completion
Data completion approach that ensured full regulation compliance across the globe

,

data quality, data completion
Karel Novák

The solution provided by BigHub helped our large customer to fully comply with local regulation and to completely mitigate the numerous business issues connected with huge data gaps.

Karel Novák
Karel Novák
Director of Services, EMEA, GoodData s.r.o.
Project background

Client product provides analytics to its customers, specifically using energy usage data from energy meters aka AMI data. AMI data can be messy and incomplete preventing trust and adoption of the analytics product.

Our approach

Identify data issues and replace with estimated data records to ‘complete’ the dataset.​ With increased trust in the data, customer can launch a product to the remaining clients.

For the purpose of data completion, our solution leveraged the power of similarity modeling based on big data processing in combination with state-of-art recurrent neural network.

Benefits of our approach

More reliable source of data for business activities
First thorough and exact data quality assessment that allows customer to apply additional corrective measures
Full compliance with local regulation reduces risk of disputes and penalties
Advanced Metering Data Quality & CompletionAdvanced Metering Data Quality & CompletionAdvanced Metering Data Quality & Completion
Identification of Gaps

Thorough data exploration

  • Detailed data profiling, found continuous gaps in very large data sets
  • Find the symptoms in the dataset
  • Several heatmaps were produced to indicate the data issues types and distribution
Completion via LSTM

Precise approach for wider gaps

  • LSTM with Attention Mechanism proved itself to perform a very precise forecasting capabilities
  • Suitable for wider gaps that require more complex models
  • Requires to be trained per segment on GPU enabled HW
Next Solution
+420 720 732 783Vaclavske namesti 802/56, 110 00 Nove Mesto
ID 05388066
VATCZ05388066
Privacy Policy