Active grid management starts before the digital twin

The electricity industry is entering a new era of network operation.

As the penetration of distributed energy resources (DERs), electric vehicles, heat pumps, and community energy schemes continues to accelerate, distribution networks are being asked to do something they were never originally designed for: actively manage power flows, voltage profiles, and flexibility services in real time.

For many network operators, the long-term vision is clear. Future grids will be managed through highly accurate digital twins, physics-based optimisation engines, and autonomous control systems capable of continuously balancing technical performance, operational costs, and customer needs.

At SMPnet, we firmly believe that physics-based optimisation represents the golden standard for grid operation and control.

However, there is a practical challenge that many utilities face today:

What happens when the network model needed to run these optimisation engines doesn't exist?

In an ideal world, every distribution network would have:

• Complete and accurate network models
• Detailed asset information
• Up-to-date connectivity data
• Comprehensive telemetry
• Real-time visibility across all voltage levels

In reality, this is often not the case.  Network models and operational visibility are generally more complete and accurate at transmission level, with data quality, completeness, and observability progressively declining towards medium- and particularly low-voltage networks.

Many low-voltage networks were designed decades ago and have evolved continuously over time. Asset replacements, network reconfigurations, customer connections, and the growing deployment of DERs can gradually create discrepancies between documented models and actual network conditions.

In some cases, network models are incomplete. In others, they are outdated. And in some areas, particularly at low voltage, detailed network models may simply not exist. Yet the operational challenges remain. Voltage violations still occur. Network constraints still need to be managed. Energy resources and flexibility still need to be coordinated.

Customers and regulators continue to expect networks to accommodate increasing levels of electrification and renewable generation.

Waiting for perfect information and data models is not an option.

Starting with the data we already have

Within a UK DSO programme, SMPnet has been exploring how data-driven methods can support active network management where traditional model-based approaches are not yet feasible.

Instead of relying on detailed electrical network models, the approach begins with the information that is already available:

• Historical voltage and feeder measurements across secondary substation and feeders
• Smart meter data from LV customer
• Asset nameplate data (e.g. transformer ratings, voltage ratios)

Rather than relying on a detailed electrical network model, the methodology analyses historical operational data to infer how the network responds to changing operating conditions. By examining historical voltage measurements alongside variations in active power flows, demand, distributed generation, and flexibility dispatch, it becomes possible to estimate voltage sensitivity relationships that describe how changes in power injections influence local voltage levels.

The resulting framework enables network operators to address the key challenges outlined below, which align with Ofgem's ED3 requirements. The second column identifies the SMPnet solution that addresses each of these requirements.

Most importantly, it enables utilities tobegin actively managing low-voltage networks today rather than waiting forcomplete digital representations to become available.

Data Driven Optimisation in LV Grids – Insights from a practical deployment

We have already seen very promising results from the Northern Powergrid Community DSO trial. Before deploying the methodology operationally, an offline sensitivity analysis was performed using historical voltage measurements, smart meter data, and flexibility information to derive the relationships between changes in power flows and local voltage behaviour. This analysis enabled the estimation of voltage sensitivity factors, allowing the potential impact of flexibility actions to be quantified without relying on a detailed network model.

As illustrated in Figure 1 below, the offline analysis evaluated multiple voltage thresholds and estimated how targeted flexibility dispatch could reduce voltage excursions under different operating conditions. This allowed the potential operational benefits of the Omega suite’s data-driven voltage optimisation approach to be assessed and validated across a range of network scenarios before deployment.

The encouraging results from this offline assessment provided confidence that data-driven optimisation can deliver meaningful operational value even where detailed network models are unavailable, paving the way for subsequent real-world implementation and validation within the trial.

Following the successful offline validation, the methodology was deployed as part of the Northern Powergrid Community DSO trial. The results shown below are derived from real operational grid data collected during live network operation. By combining measured customer voltages, battery dispatch records, and operational telemetry, the Omega suite was able to detect emerging voltage issues, estimate the flexibility required to mitigate them, and coordinate targeted flexibility dispatch in real time.

The example shown illustrates an actual overvoltage event on the Way 4 feeder. Using the data-driven voltage sensitivity relationships derived during the offline analysis, the platform identified the developing voltage excursion and initiated a flexibility dispatch from participating battery assets. The measured battery response reduced the local voltage by approximately 0.86 V, preventing the voltage from exceeding the statutory limit. This demonstrates how the data-driven methodology can move beyond offline analysis to deliver measurable operational benefits on a live distribution network, even in the absence of a complete network model.

Understanding the limitations

Like any engineering methodology, data-driven optimisation has its strengths and limitations. Therefore, it is important to acknowledge what data-driven approaches cannot do. A data-driven model is fundamentally different from a physics-based model.

Physics-based optimisation provides:

• Explicit representation of electrical parameters
• Detailed power flow and optimal power flow calculations
• Predictive capability across a wider range of operating conditions

Data-driven approaches, by contrast, infer behaviour from observations rather than first principles.

As a result, their performance depends heavily on:

• Data quality
• Historical operating conditions
• Forecast accuracy
• Measurement availability

Recent trials have demonstrated this clearly. In several cases, forecasting successfully identified low-loading scenarios and prevented unnecessary flexibility procurement, reducing operational costs while maintaining network performance.

However, sudden and unexpected demand spikes occasionally occurred that were not reflected in historical patterns.

These events highlight an important limitation of purely data-driven methods: they can only learn from what they have previously observed.

Data quality and availability remain important challenges for any data-driven approach. Unlike physics-based models, which rely primarily on network parameters and electrical equations, data-driven methods are heavily dependent on the quality of the underlying measurements. In practice, operational datasets can contain missing values, non-numerical entries, incorrect measurements, and other inconsistencies that require careful validation and cleansing. Furthermore, many of the data sources available today are not truly real-time. Smart meter measurements are often provided as 30-minute aggregated values, while some operational platforms can introduce delays of several hours before data becomes available. These limitations can reduce visibility into rapidly changing network conditions and occasionally result in unexpected events not being captured in advance. While the trial results have demonstrated that valuable optimisation can still be achieved under these conditions, they also highlight why enhanced observability, improved data quality, and ultimately physics-informed models remain important elements of the industry's long-term journey towards autonomous grid management.

This is precisely why physics-informed optimisation remains the long-term objective.

Data-driven today, physics-informed tomorrow

Data-driven optimisation should not be viewed as a replacement for physics-based approaches, but rather as a practical stepping stone towards them.

When detailed network models are unavailable, data-driven methods allow network operators to establish visibility, begin active voltage management, and coordinate flexibility using the information already available. As data quality, smart meter coverage, and operational visibility improve, these methods can progressively evolve into network model reconstruction, physics-informed optimisation, digital twins, and ultimately autonomous grid management.

Physics-informed optimisation remains the north star, but active grid management does not need to wait for the perfect model...

It can start today.

Interested in learning how SMPnet is helping utilities unlock active grid management using both data-driven and physics-informed approaches?

Get in touch to learn more about the Omega suite and our real-world deployments across flexibility coordination, voltage optimisation, and autonomous grid management.

‍