Government digital transformation hampered by data-sharing challenges, research reveals

From allocating resources in healthcare to processing passports, data has become integral to government functionality. But a new research report reveals that entrenched silos, legacy technology and resource challenges are stalling the momentum needed to unlock its full potential.

The report, Leading from the Front: Using Data and Digital Integration to Cut Costs and Improve Public Services, sheds light on the critical role data plays in modernising government operations. 

Commissioned by esynergy in collaboration with Government Transformation Magazine, it draws on insights from 100 senior government data professionals and highlights both the progress and persistent challenges in the push for digital integration in the public sector.

As well as quantitative research, it also includes in-depth insights from the UK’s most senior Civil Servants including:

• Craig Suckling, Chief Data Officer for UK Government
• Fiona James, Chief Data Officer at ONS
• Ming Tang, Chief Data and Analytics Officer for NHS England
• Jeremy Gould, Chief Digital Officer at Homes England
• Sue Bateman, Chief Data Officer at the Department for Environment, Food and Rural Affairs

Optimism amid challenges

Despite the recurring obstacles, there is significant optimism about the future of data integration in government. According to the report, 63% of civil servants rate their department’s data integration efforts at seven out of ten or higher. Additionally, 50% of public service decisions are now driven by data insights, demonstrating an increasing trust in data-informed policymaking.

However, the report underscores critical gaps that need addressing. A striking 79% of respondents cited the perceived or actual burden of data sharing as a major inhibitor of cross-departmental collaboration. Privacy concerns, inconsistent data formats, and the lack of standardised frameworks exacerbate these issues.

Recognition and resources

A recurring theme in the findings is the insufficient recognition of data’s value, both within and outside government. Only 34% of respondents believe that the external value of data is properly acknowledged, and 74% reported inadequate resources to support data-sharing initiatives.

Neil McIvor, Head of Data for Public Services at esynergy and former Chief Data Officer at the Department for Education, emphasised the importance of breaking down these barriers: “Data is at its highest quality and usefulness when used as a by-product of operational delivery. But for government to truly harness data’s potential, foundational infrastructure and interdepartmental collaboration are essential.”

Data leadership gaps 

Another key insight from the report is the limited authority and prevalence of Chief Data Officers (CDOs) within government. Only 13% of respondents indicated that a CDO oversees their department’s data and digital functions at an executive level. Instead, 47% reported that these functions fall under the Chief Operating Officer, diluting the CDO’s ability to drive transformative change.

This misalignment also extends to emerging technologies - only 14% of respondents said that AI policy falls within the remit of the CDO, while responsibilities are split across roles such as the Chief Digital and Information Officer (CDIO) and the COO.

Pathways to progress

The report offers a roadmap to overcome these challenges, advocating for a cohesive approach to data integration. McIvor highlights the need for a “common bedrock of data foundations” that prioritises interoperability and collaboration over duplication of efforts. Such an approach, coupled with investments in governance and resources, could unlock significant efficiencies and improve public service delivery.

This report offers insights for leaders across government and sets the stage for meaningful progress in digital transformation efforts. As McIvor puts it, “Government has much to do to resource and reduce burdens to data sharing, but the transformative potential of data is undeniable.”