Data Validation – Empowering Utility Business Systems’ Effectiveness!

‘Utilities, need accurate data to get accurate results In fact, business transformation is built on a foundation of accurate data and data validation plays a vital role to meet one’s business objectives!

Every Utility constantly strives for ‘operational excellence’ with the aspiration to transform entire business performance directly impacting their operational costs, growth, and revenues. In order to achieve this objective, companies must take relevant action to safeguard their GIS and Asset management data. Many utilities go through multiple checks at each stage of the process to ensure that they meet the predefined business rules and comply with the industry’s regulatory requirements. In fact, data needs to go through hundreds of checks for various parameters and it is not feasible to do this manually.

In such a scenario, data validation is of crucial importance and can be achieved by using efficient tools and methods. This approach helps utilities ensure that their systems function the way you expect them, in order to meet business objectives.

In our engagements, I am often asked by clients if I was in their position, how I would validate data?    To make sure I was giving them an answer I believed in, this past weekend I did exactly that; I validated data just like I was working for a utility. I set up a scenario based on two of the most common use cases from distribution and transmission side of electric and gas domain. I chose a solution that I was familiar with, VaidateX to do the work. Here is what I found.”

We conducted an in-depth study of different data quality parameters & business rules requirements, and subsequently developed robust processes and tools that can be used for;

a) Asset Data Life cycle management for Data Quality Assessment and

b) Data Gap Analysis to identify remediation processes before data Migration to Utility Network

This Data Validation framework ‘’ValidateX’’ is a standalone product for Electrical / Gas / Telecom clients who maintain their final data sets in ESRI or ArcFM compatible formats.

This application is a flexible, configurable and easily adoptable solution to audit the network data in meeting industry standards & business needs.

It enables you to manage metadata and help you with customized reporting through its business rules driven data validation and built-in data profiling tools.

Without such tools, it wouldn’t have been feasible to manually identify all the errors resulting in high chances of missing many issues and data gaps. The application ensure network compliance as per the business needs and standards.

The Process

Data validation process involves running data through many levels of assessments as explained below.

a) Core GIS Data Quality Assessment:

First Data validation process involves running data through all kinds of Core GIS Quality Assessments as explained in the below sections. The application has a provision to select the GDB that needs to go through Quality Assessment. It lists out feature classes that are available within the GDB. One needs to select the type of checks and relevant feature class of the GDB that needs to undergo those checks.

b) Core Domain Data Quality Assessment

The second Data validation process involves passing data through all kinds of domain related checks. Currently this application caters to Electric / Gas domain checks and later it will be catering to other domains such as Telecommunications. User has an option to select the type of domain before proceeding on checks.

With the domain again there are two types of Data validations 1) Mandatory validations and 2) Spatial validations. These are customizable and business rules can be predefined as per the client requirements.

I. Mandatory Validations

Take care of individual feature class and mandatory attribute field’s values as per the data model

II. Spatial Validations:

Take care of Domain Related Checks

Now that I understand how valuable this is, I started to think about other places it could be used such as electric and gas distribution specifically, as presented in below scenario; 

Gas Distribution

• Transmission mains should not be connected directly to Distribution Main and it should be connected with regulator station
• The Gas main Construction Status and its connected fitting Construction status should be same.
• Diameter of the Fitting and diameter of the connected Gas main should be same
• Material of the fitting and material of connected Gas Main should be same
• Abandon gas main should not be connected to the active gas main pipe
• T fitting should be connected to three Gas Main features 
• Gas Main should have a break at fittings like Reducer, coupling, Tee and Elbow
• Only one pipe should be connected to the End Cap fitting
• If the direction of Gas Pipe changes then a fitting should be present
• Gas main with two different sizes should be connected with reducer fitting

Electric Distribution

• UG asset connecting to OH asset should have a Riser feature at the connecting point
• All over head conductors (Primary and Secondary) should have a start pole and end pole relation ship
• All UG conductors (Primary and Secondary) should have a start structure and end structure relationship
• Based on the phase information, number of units linked to it should match (Example, a 3 single Phase Asset should have 3 units)
• All Services should be relationship with Secondary conductor or Transformer and also with the service point (Customer point)
• All service point (Customer point) should be connected to the service conductor
• Street Lights should be always fed with secondary conductors
• Every Transformer should have parent as Primary conductor (Or Bus bar) and child as Secondary conductor
• For the switches or Fuse, if the status is OPEN, then the Feeder ID2 must contain value
• Feeder id of all point features (Electrical Assets with closed status) should match with its connected conductor

If you’d be interested to learn more, please contact me rahulkrishna.p@rmsi.com or Robert Brook robert.brook@rmsi.com