How Government can use data to inform decision-making

How can we empower government departments to use their data to inform decision-making? How are decisions currently made, and how can this be improved? As decision-making can be a laborious process, especially at the policy level, some common factors to consider when collating public sector data to improve service delivery and design services that better serve citizens. 

I’m a Principal Consultant at Scott Logic, and until recently, I was a senior civil servant in the Cabinet Office’s Government Digital Service. At Scott Logic, we design, build and run technology for government departments to collaborate, manage data, facilitate payments, verify identification, deliver notifications, publish information and even provide emergency support services like those needed during the Covid pandemic. 

A notable project I worked on was to develop the vulnerable people service, a big cross-government data project designed to find the people who had to shield during the early stages of the crisis. This included getting supplies delivered and securing shopping slots at supermarkets, which necessitated the public and private sectors working together, informed by data, to provide an essential service.

Harnessing the power of data and interoperability 

As a former Civil Servant, I'm obsessed with value and how this relates to efficiency. Data is imperative to driving decision-making and the value that departments can achieve, as it helps support better service delivery, increasing citizen satisfaction and better outcomes. In an era when data is ever present and abundant, it is one of the most valuable assets an organisation can have. If you can harness it effectively, your data can help chart the path you want to take and base it on knowledge rather than assumptions. It also gives us the tools to measure the success of project delivery. This all works in theory, but it can be a challenge when the data is hard to reach or interpret. This was evident during the Covid crisis, as data was siloed. Initially, we had to rely more on assumptions and instincts.

Current capability in government data management 

Government departments generally use data to inform their policy and service design, but there’s room for improvement. When I worked on an identity service, we collected data about user interaction and the insights generated were all combined to help design how the service operated. We looked at how priorities should be chosen and how contracts with the private sector should work. This helped to set the policy around data and who should have access. Our data collection also informed aspects of user interaction like device use and page load speed, helping us understand where services are working well or need improvement. 

The data has helped to shape government guidance for other departments on service building and design. But at the policy level, it gets more challenging when collaborating across departmental boundaries. Cross-cutting data can be tough to reach, and this is one of government’s current focuses. At the top levels, success is judged by whether policies have the intended effect. So the question is: what do we have to change or invest in – or even work on collaboratively across departments – to be able to measure these critical things and enable data-driven decision-making across policy areas? 

Improving cross-government collaboration and capability

There can be a lack of clarity around the rules for processing data and barriers to using it.

Where government departments need to collaborate and cross departmental boundaries, success or failure of these efforts is often a question of the scale of the problem, differing incentives and legacy systems. Different teams may use alternative platforms, and managing collaboration across these can prove challenging. Organisations often see big data infrastructure as the solution to the transformation of their businesses. But many of them aren’t yet getting the value they expect from those big data investments. This is often because they don’t have the right foundations in place: the data is often hard to identify and access within organisations; data skills are often in short supply; and there is a lack of clarity around the rules relating to the data (who can access it, how can it be used). These barriers act like an anchor, slowing progress.  Organisations can end up in a cycle of re-procurement, buying new tech to fix the old tech, limiting the innovation they're trying to deliver.

Delivering value and enabling data decision-making

Accessing and integrating legacy systems can be challenging when making different data sources talk to each other within organisations and beyond. It’s a question of dealing with the sheer volume of data, understanding which parts are usable and actionable, and how to collate it. Quality can also be a challenge – you have to establish whether the data is even good enough to make decisions with. I’ve led projects where the first stopping point is sharing the data held with customers and then finding it needs updating. Permissions are another concern: who is this data being shared with, and what’s the right approach for this? 

Scott Logic has successfully worked with public sector organisations by following a simple three-step approach: locate, understand, surface. First, we locate the data in systems, and then uncover the data of value and establish the reliable source of that data. Next, we focus on understanding this data and what it enables, we ensure consistency across datasets, and we create a common terminology that is understood by both people and machines. Finally, we surface the data in a way that is consumable by anyone who needs to make decisions as part of their day job (i.e. everyone). Any organisation can adopt this approach. Some organisations jump straight to adopting AI or robotics, but until your people can reach, understand and act upon your business data, robots aren’t going to be able to do any better.

Taking a data-driven approach

Getting departments all working in this way unlocks really interesting future opportunities. Having full departmental buy-in is critical for a truly data-driven approach across government. The next logical step is to build whole-of-government services where different departments collaborate and enrich each other to empower services and policy delivery that transcend departmental boundaries. This collaboration and joint working by civil servants unlock delivering policy across borders and sectors, whether provided by local government, central government or even the private sector. To enable this, we have to ensure different systems within each business area can speak to each other, have rules that govern how this happens, and provide the proper protections. And coming back to the value of all of our work, these steps help build a bigger picture of the overall policy impact, enriched with the collated data that informs decision-making and improves service delivery along the way.

Measuring success

There are lots of ways to assess success but one interesting idea is to measure how successful we are at using data to support departmental policy delivery or a joined-up or whole-of-government data approach. To do this we can look at those complex life event scenarios like “Finding a job” or “Having a baby”.  These events tend to affect most of the population at some time and require multiple departments to deliver all of the various entitlements or processes Once we join up data and delivery across departments, these life events can be a good test of our performance as we already have lots of data on how each part of the public sector ecosystem operated when they were not interlinked. Measuring and comparing this performance helps to illustrate how the joined-up view performs and identify ongoing areas for improvement.

Sharing best practice

I’m looking forward to having open conversations about public sector data. I’m co-hosting the Data-Driven Decision-Making discussion table at October's Government Data Summit. I’ll share some practical examples of how we tackled challenges across various projects. I’ll also share actionable steps towards gathering insights from leveraged data that multiple departments within an organisation can adopt. There is enormous value in bringing senior civil servants and public sector experts together to see how challenges manifest within different organisations and create innovative solutions.