Back to News

How to focus energy on insights

23 September 2019

The power cut on 9 August that affected 5% of demand in Great Britain left nearly a million homes and businesses across England and Wales without electricity, and spread chaos across the transport network.

Could real-time data and insights have made a difference on the day or even helped to avert the power cut? We think they possibly could. We will go on to explain why not only real-time data, but also designing systems to promote real-time insights, enables organisations to make better decisions, faster.

But first, let’s take a look at what happened during the power cut.

What happened on 9 August?

The power cut was caused by several near-simultaneous events which pushed the grid’s operational procedures and protection systems beyond their limits.

Within a second of the lightning strike on the grid between Eaton Socon and Wymondley, an unexpected effect in the control system at Hornsea One wind farm led to a large reduction in output and, milliseconds later, the steam turbine at Little Barford CCGT power plant went offline. Despite the large drop in output, the system then started to recover. It was a second incident at Little Barford – the disconnection from the grid of one of its gas turbines – that caused the second and decisive fall in grid output and frequency, triggering the Low Frequency Demand Disconnection.

For the UK’s National Grid, recovery to a stable operating state following this type of event still requires a mix of automated and manual processes, better suited to a time before today’s grid network of smaller, distributed power generation.

And while the grid’s 1,000MW reserves were maxed out by the 1,300MW loss of power, at the time of the incident there was 4,000MW of unused generation in reserve; if output from this could have been increased within the minute between the two incidents at Little Barford, it’s possible that the load shedding could have been avoided.

All of which points to a need for more automation in the grid’s systems, and access to real-time data to inform rapid, expert decision-making. But is that enough?

The difference between real-time data and real-time insights

At Scott Logic, we have a global, blue-chip client base including most of the major investment banks – an industry that has turbo-charged the evolution of real-time networks and technology. In high-frequency trading it’s now not just about real-time, but how real-time, down to the microsecond.

Even away from the dizzying frontiers of high-frequency trading, real-time cloud-based data flows have become of crucial importance to financial services businesses in supporting professionals to make good decisions rapidly.

But while real-time data feeds can now present experts in any industry with a wealth of live data, this is not enough to generate insights or inform the best decisions. A figure out of context is useless; presented in the context of other relevant information, there’s a much higher chance that the data can be used to inform the right decision.

For example, a drop in power of 1,300MW might be interpreted as acceptable or terrible; in the context of available reserves of 1,000MW, it’s terrible; in the additional context of 4,000MW of unused generation in reserve, it could be acceptable – if a decision is made quickly enough to increase output from that unused generation.

At the opposite extreme, an overabundance of difficult-to-interpret data is equally useless. A badly designed system requires large amounts of slow, effortful thinking in the user just to interpret the data; a well-designed system eases the cognitive load in interpreting the data, enabling the user to focus their energy on using that data to make good decisions.

To illustrate this, let’s take a look at this Gridwatch site. The data is difficult to read, let alone interpret, and the gauges at the top of the screen are devoid of context. If you are interested in looking at power output from coal this week compared with last week, the tiny charts require you to delve down into minute detail. If you want to go back further in time, there’s a separate chart for each time period.

In contrast, now let’s look at the Powerwatch site, designed by Scott Logic. Here, there are simple, easily readable charts for demand, transfers and production. In each case, context is provided – on every chart, the live and most recent data is compared with the data for an earlier period, allowing the user to identify instantly where anything is out of the ordinary. At the top right, the user can quickly and easily toggle the time periods.

To design in this way, making it as easy as possible for people to draw meaningful conclusions and make sound decisions is called designing for insights.

Examples of designing for insights

We have been supporting our clients for years in financial services and the public sector to design for insights.

NEX Regulatory Reporting, a leading transaction reporting provider, was embarking on a major redesign of its platform, aiming to introduce a next-generation interface allowing users to process large volumes of data and deliver final reports to regulators with a high level of transparency. By designing for insights, we helped them to manage those large volumes of data but, crucially, to focus the user interface on flagging exceptions in the transaction process. In an ideal world, all of the post-trade process flows along smoothly; for NEX’s users, it’s critical for them to be able to see any impediments in that flow in real-time and to resolve them rapidly. You can read more in our NEX Regulatory Reporting case study.

The Foreign and Commonwealth Office’s (FCO) Open Source Unit aims to increase the use of open data to help FCO deliver across a raft of policy priorities. In an era of ‘fake news’, it’s vital for FCO analysts to have fast, secure access to accurate open data sources, with tools that allow them to derive actionable insights. Having established a secure data platform and ingestion pipeline that could cope with the variety of data sources and formats needed, we then worked with the FCO team to design for insights – producing with them a simple, intuitive but powerful user interface, including a component to support advanced Boolean logic while retaining simplicity of search.

We are experts in cloud and data engineering, and have a wealth of experience in enabling our clients to harness and process vast amounts of data. But it’s through our additional expertise in designing for insights that we help our clients to make the very best use of that data. And through our incremental approach, our clients gain early and continuing value from our work together – for example, we established the FCO’s data platform from start to finish in just five months.

If you want to find out more about how we can work together to achieve your goals around data and insights, contact Patrick Bishop: And if you are at the Energy Trading Operations and Technology Summit in October, Patrick and colleagues would be delighted to meet you to discuss how we can support your business.