Local government needs support for responsible procurement of AI

Oct 1, 2024 10:00:00 AM · 2 min read

Local authorities are being left without adequate support to procure AI in the public interest, despite a proliferation of Government guidance documents, according to new research from the Ada Lovelace Institute published today.

3:55

Researchers analysed 16 different pieces of guidance and legislation – published under the 2010-2024 Conservative Government – relevant to the procurement of AI. This included newer AI technologies, such as generative AI, as well as data-driven technologies already widely used by local authorities, such as predictive analytics.

The research finds that local government does not have access to a clear, comprehensive or consistent account of how to procure AI in the public interest. This means local authorities face significant challenges in navigating existing guidance and relevant legislation during procurement. In particular, there is insufficient clarity about how to apply concepts like fairness, how to define public benefit and how to ensure that the use of AI is transparent and understandable to affected people.

The report includes a number of practical suggestions for improving procurement of AI and data-driven systems in local government, including clearer guidance, definitions, success metrics and responsibilities, as well as implementing governance mechanisms like the Algorithmic Transparency Recording Standard, piloting impact assessments and supporting public participation.

Imogen Parker, Associate Director at the Ada Lovelace Institute said, "Procurement can and should be a key lever in ensuring that AI tools being used by local government are safe, effective, fair and in the public interest. Local authorities face the unenviable task of having to navigate unclear, overlapping and sometimes conflicting guidance.

It's essential that the procurers in the public sector are confident about the products they are buying in - and neither they nor the public are put at risk. Embedding a robust, ethical procurement process in the context of reduced budgets is of course a significant challenge. But it is important to also consider the cost of not doing this, both financially and ethically, something demonstrated all too clearly by the Post Office’s Horizon scandal."

This research lands amidst rising expectations about the potential of AI in the public sector. While there is optimism that AI could enhance public services, this will only be achieved if the public sector can ensure the adoption of new technologies are safe, effective and in the public interest.

Given many AI technologies are supplied by the private sector, the procurement process can and should play an important role in assessing the effectiveness of potential solutions, anticipating and mitigating risks, and ensuring that any deployment is proportionate, legitimate and in line with broader public sector duties.

The paper will be followed by a second publication, based on qualitative and collaborative research with procurement stakeholders from across the public and private sectors.

Anna Studman, Senior Researcher, Ada Lovelace Institute

Anna Studman, Senior Researcher at the Ada Lovelace Institute and lead author, said, "AI and data-driven systems can severely damage public trust and reduce public benefit if the predictions or outcomes they produce are discriminatory, harmful or simply ineffective. Procurement provides a vital opportunity for local authorities to interrogate suppliers on the possible societal impacts of their technologies.

Our research highlights the need for greater clarity around guidelines and responsibilities, and enforceable redress. Procurement teams need better support and clearer guidance so that they can procure AI that is effective, ethical and in the interest of people and society."

Oct 1, 2024 10:00:00 AM · 2 min read

By Ayesha

Ayesha is a journalist and conference producer at Government Transformation Magazine. With experience in UAE government, public sector consulting and a degree from Durham University in international relations and sociology, Ayesha is interested in large-scale, multi-stakeholder transformation; ethical AI and the dynamics of cross-government collaboration. To wind down, Ayesha enjoys drinking tea and being pitched stories.