A hidden goldmine: 3 ways utilities can transform historical data into strategic advantage

The story of innovation can be oversimplified into binary terms, black and white, this or that. There’s a misconception that to move ahead we must leave the past behind, and that progress requires all new things, writes Kaitlyn Albertoli, chief executive and co-founder of Buzz Solutions.

That’s the pitch from a lot of software and artificial intelligence (AI)/generative AI (GenAI) vendors nowadays, and it’s led companies to believe that in order to have a successful data analysis tool they need to work with new data that are collected in a certain way.

But in this rush towards the future, companies can overlook the value of their historical data (i.e. data that might already exist), and misconstrue it as “old” or “outdated” information instead of what it really is – the foundation of their entire data infrastructure, and a wealth of historical knowledge.  

Unlocking the value of historical data

For years, the utilities sector has been beset by rising costs, ageing infrastructure, worsening environmental/security threats, and a surging demand for electricity. Moving on from the past does not mean running from it, and while these challenges have been difficult, they have also led to much-needed reforms, including a renewed focus on energy efficiency and decarbonisation.

We learn from our actions, and by analysing historical data like meter readings and price fluctuations over time, companies have been able to identify consumption patterns, better understand asset lifecycles, improve forecasting, and ensure regulatory compliance. 

Historical data isn’t just numbers, it’s knowledge, and knowledge can be a valuable commodity when it’s in short supply. According to recent estimates, almost half of the current utility workforce is due for imminent retirement, meaning the industry is poised to lose decades of field crew intelligence and experience in the field. Preserving that knowledge is critical for continued operations and continuity in maintenance.

With so many tenured line workers retiring, it’s especially important to harness and retain their knowledge to reduce ongoing labour gaps in the field, which puts immense amounts of stress on existing crews, especially as workloads continue to mount. Leveraging innovative capabilities, such as an AI-powered analysis tool, enables these field crews to focus on decision-making, rather than data analysis. 

A few considerations

Before utilities can use historical data more effectively, they first must grapple with certain inherent challenges of working with data in their industry. By now, utilities understand that data is critical for making informed, strategic decisions about their operations and the grid, but traditional processes are not optimized for storing and managing that data, and the siloed nature of utility software systems makes it tough to turn data into actionable insights, whether it’s actionable today or stored for future maintenance.

What’s happening is that as data flows into a utility without a proper storage or management system, that data ends up getting held in massive data lakes, like big murky reservoirs of information. One central repository would be challenging enough to wade through, but there is also a tendency among utilities for different departments or functions to operate in isolation from each other, with limited data sharing and collaboration.

The resulting silos create a lack of holistic understanding of overall systems, and leads to teams collecting duplicate datasets, what’s known as data redundancy. And for smaller or rural utilities that don’t have large budgets or extensive teams to help deduplicate and analyse all this existing data, teams waste a ton of time trying to clean and organize it to make sense of what they have.

By adopting cloud-native technologies that can store, manage, and analyse massive amounts of data at lower costs, utilities can de-silo their data and facilitate increased collaboration and data sharing between different functions. This way, data will be better stored and utilized across operations, allowing utilities to gain real-time visibility into day-to-day functions, while also building a foundation for robust, long-term system planning. 

Three ways to transform historical data into strategic assets

Once utilities ensure their systems align with their specific data storage, access, and analysis needs, they can start to look at how their existing data can work harder for them. Here are three ways that historical data can be used to improve future performance.

1. Productivity

The reality is that much of the utility industry today is still manual, relying on human workers to sort through stacks of disparate records for simple inspection information or analyse tens of thousands of raw dataset images to monitor asset health. Not only does this waste valuable worker time, but it can also have a discouraging impact on younger talent who could add value elsewhere on more high-priority projects.

According to Accenture, 38% of working hours in utilities can be either automated or augmented by GenAI, delivering a potential 25% productivity uplift and creating an additional $334 billion in value. To be clear, this does not mean that automation will replace these workers, but rather it unlocks workload capacity for utilities who are short on labour, where every resource counts. 

The value of leveraging historical data to make more informed present-day and even future decisions is a type of innovation that is transforming the industry. Rather than discarding all those original records and dataset images, this historical data can be fed into these AI-powered tools to give utilities a much better picture of their operations. 

Key takeaway: When evaluating new technology, be sure to inquire how existing historical data will be incorporated. Think about that data like (a lot) of spare change lying around the house or car – individually, it’s not worth much, but combined it creates wealth. 

2. Asset management 

According to McKinsey, asset management can account for 20%-30% of a utility’s operating expenses and 15%-20% of its capital expenditures. Yet as recently as 5-10 years ago, very little of this data was digitized. Now, utilities are on an exponential curve with the amount of data they’re collecting, and they’re trying to figure out how to make sense of all that information. That’s where historical data can play a huge role.

For example, imagine a utility that wants to build an automated drone program to monitor various assets. If that utility doesn’t have its historical data organized in an efficient data management system, these programs tend to stall and often have a harder time scaling. So, instead of generating impact, the drones just sit on the shelf because the utility can’t analyse new data coming in or compare it to historical data to glean any meaningful insights.

Without a seamless data ingestion, mapping, and management tool, the data is cumbersome to store and more difficult to retrieve for future use. As a result, many of these data sets are only used one time for drafting an inspection report, rather than serving as a reusable reference point for change detection between inspections. When investing in inspecting thousands of structures per year with a visual inspection program, the ROI can be compounded with the use of easily accessible and referenceable historical data for asset tracking.

Key takeaway: In many cases, utilities already have the historical data they need, often stored away on hard drives in the back of someone’s desk. Maximising available data is a critical first step before investing in any new technology.

3. Compliance reporting & audits

Utilities are required to track compliance in multiple parts of the business to ensure infrastructure safety and reliability. Inspections are a core aspect of maintaining compliance, utilities are mandated to inspect portions (if not all) of their service territory each year.

To date, many of the necessary inspections have been conducted by workers either walking the lines or climbing poles (which is dangerous and expensive), and the reporting has been manual (which is time-consuming and prone to human error).

Now, remote drone inspections armed with AI can process millions of visuals from an entire energy system quickly, from any source, to deliver accurate asset insights. Asset images can be analysed in hours or days rather than months, and allow maintenance groups to prioritize repairs, reduce the potential of failure, and remain in compliance.

Regulators aren’t messing around, and they’ve made it clear that utilities will be held liable, regardless of their own data capabilities. For example, as of March 2024, California utility PG&E had recorded aggregate liabilities of $1.125 billion, $400 million, $1.6 billion, and $100 million for claims in connection with wildfires in their territories between 2019-2022. 

Key takeaway: Historical data can simplify compliance reporting and audits, reduce penalties, and improve transparency with regulators. Whether internally or with strategic partners, utilities should invest in building a roadmap for utilizing historical data effectively.

Past is prologue 

As utilities continue their data-collection journey, adopting new technology doesn’t have to mean starting from scratch. Historical data is an important part of company history and contains valuable information that can prop up what comes next. Meaning, that despite our bold new era of innovation, the past isn’t something to forget, it’s something to use. 

About the author

Kaitlyn Albertoli is the CEO and co-founder of Buzz Solutions. Prior to founding Buzz Solutions with her co-founder Vik Chaudhry in 2017, Kaitlyn ran a non-profit focused on sustainable food with 60 people overseeing the needs of 300 people. She was previously a wealth management analyst at JP Morgan Chase. Kaitlyn was named to Forbes 30 under 30 in 2021. She studied international relations and finance at Stanford University, where she received her B.A.