Unlocking the full potential of public sector data with AI

According to Tom Christian, AI/ML Platform Lead at Snowflake, the potential of AI in the public sector is immense, particularly in handling and analysing the vast amounts of unstructured data that government entities deal with daily.

Structural data challenge

Public sector organisations deal with a unique and complex array of data. From healthcare records to infrastructure details, the diversity and sheer volume of this data present significant challenges. 

“The way that data is structured currently in the public sector is a problem in and of itself,” says Christian. “AI is here to help us understand what’s going on faster than before, especially in areas where data is vast and complex.” 

This data often resides in silos, making it difficult to transfer, integrate, or analyse effectively across different departments and services. He added, “There’s a misconception that data needs to be perfectly structured before AI can be applied,” highlighting the challenges that come with the traditional approaches to data management in government.

This is where AI comes into play, offering the ability to work with this data in new and transformative ways. 

Christian pointed out that AI allows for a more flexible approach, stating, “We can start generating insights and value from data as it exists now, without waiting for everything to be perfectly aligned.” 

Rather than waiting to “make all this data architecture perfect and transferable” before deploying AI, there’s a growing recognition that AI can be implemented now to start generating insights and value. He explained, “AI helps to move data from one place to another, breaking down silos and enabling a more integrated approach to public sector challenges.”

Contributing to the growth agenda

At a time when much public sector attention is focused on driving economic growth - the potential for AI extends beyond just internal government operations. There is increasing interest from the private sector in public data, particularly as businesses seek to leverage this information for their own decision-making processes. 

Christian highlighted an example from the energy sector: “Energy companies have begun to use public sector data, such as weather reports, to make more informed decisions, like optimising the tilt of solar panels for maximum efficiency. There’s an enormous amount of public data that the private sector is interested in, and AI can help unlock that value.” 

He mentioned how “businesses can use this public sector data to better inform their decisions, which in turn benefits the public sector by improving service delivery and operational efficiency.”

One of the key insights from recent discussions is the idea that AI doesn’t require a perfect foundation to start delivering value. Christian emphasised, “The traditional approach of waiting for data to be fully integrated and structured before introducing AI is being challenged.” 

Instead, the focus is shifting to a more iterative process where AI is applied early on, allowing organisations to “fail fast” and learn quickly. He stressed, “This approach not only speeds up the implementation process but also helps to uncover insights that might otherwise remain hidden in the vast sea of public sector data.”

He also noted, “AI provides the ability to rapidly iterate on existing data, even when it’s not perfectly structured. This is crucial for the public sector, where data complexity can be a significant barrier to innovation.”

Efficiency and quality

AI’s ability to analyse and interpret large volumes of data has significant implications for public sector services. 

Christian remarked, “AI enables better decision-making by providing a clearer understanding of complex datasets,” which in turn helps optimise resource allocation and improve service delivery. He added, “This is particularly relevant as the government seeks to address pressing issues such as budgetary constraints and the need to identify vulnerable populations more effectively.”

Christian concluded that AI’s role isn’t just about efficiency; it’s about enhancing the quality of public services. “By helping humans do more, rather than replacing us, AI can create a more responsive and effective government.”