How AI can be used to maximise collective knowledge

In an age of information-overload, government organisations face an increasingly complex knowledge environment. However, using AI to harness the collective knowledge of an organisation has the potential to maximise the value Civil Servants get from their data, says Mark Azzam, Chief Data Officer of Germany's Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection.

“If you ask somebody in any big organisation what is known about a certain issue, nobody can give you that answer - first they have to identify the different areas of competence and aggregate the right information from ten different reports,” Azzam explains. “This is how we work today, but it's a pretty inefficient and unnecessary way to work.”

To tackle this problem, Azzam tells Government Transformation Magazine that a new perspective is needed on how to establish a support system for management. 

His team is currently in the early phases of building a “knowledge management system” that aims to use text mining and knowledge graph technology to extract the right information from within an organisation’s unstructured files and provide it to the right person at the right time.

“We have a linked data approach to how we order all unstructured information that we have stored in different folders and file systems. This will allow the system to be able to automatically pull out all of the relevant documents and decide which pieces of information are redundant and which can add the most value in gaining the full picture to address an inquiry.” 

He adds: “We're still building up the team here at the Ministry and setting everything up to actually get to a point where we can start implementing the system, but it’s going to be a major priority this year.”

Accessing the unknown unknowns 

Where Azzam sees the system driving the most value for Civil Servants is in making relevant information promptly available in a new context. “A lot of the time, the knowledge is there, but we can’t easily access it, especially when we don’t even know what it is we should be looking for,” he says. 

Modern Natural Language processing with sentence embeddings and transformer models currently being built in AI have the capability to extract not just the relevant information but reveal “unknown unknowns,” Azzam explains. “Even if you don’t know exactly what you’re looking for, if the context of information is similar, the system has a chance to actually identify that linkage and supply the right kind of insights.” 

This can drive faster, better and more informed decisions. According to Azzam, it will be particularly helpful in extracting information across multiple disciplines and departments, where different jargon and terminology is often used by Civil Servants working on similar topics -  allowing information that would otherwise fall through that gap to be fully utilised.

Developing and implementing a knowledge management system

Azzam stresses the importance of addressing privacy, confidentiality and security from the very start of the project.

“It’s important to consider how open the organisation is to the endeavour. Across governments, tapping into data that includes personal information is a very sensitive matter. Acting responsibly within the limits of the EU privacy law and international human rights law is a matter of course for us. If there’s personalised information in these documents, you need a coping mechanism - including managing access rights, anonymising data or hardening the system. This is a difficult one, because there’s no consensus on the best way to go about it yet.”

Creating a data structure for all the existing information within an organisation is another challenge to overcome from the outset. 

"A knowledge organisation scheme", Azzam calls it. “This requires a networked approach to data handling, where you need to think about what your entities are - these can be organisations, units, topics, places or even programmes, legal norms or timestamps - then you then need to find a data model that can efficiently support both graph analysis, but also text documents.”

Building this data structure is dependent on what system an organisation already has; what data is available and what systems are being used. Afterall, organisations do not want to build everything from scratch. 

Just as important as the legal and the technical solution is how to involve future users, Azzam explains. “Our approach aims at supporting the Civil Servants within our ministry. This means the AI we implement has the role of supporting everyday work, for example, by supplying relevant information.”

It goes without saying that decision-making remains in the hands of humans. In order to successfully operationalise the technology, Azzam insists that it must follow a human-centric approach, involving colleagues from the outset and identifying an accepted user interface that supports given workflows.

The potential of digital twins  

There is a growing school of thought around the potential for digital twins to support better management within government organisations; helping to prioritise interventions and make better decisions. 

While this is not something that Azzam and his team are currently looking at, he sees potential value in an “informational digital twin” of an organisation being created as the basis for a dynamic organisational model. 

“Creating a dynamic system model that represents the organisation generates complete transparency over what is happening -  such as how many resources are allocated to which topics and in which structure, what requests are being processed and what the output is.”

With such a digital twin of an organisation, if management decides to save 10% of the budget or invest additional resources, the model could give an idea of which critical parts of the organisation might fail or could translate growth into productivity. 

Azzam points to other probable effects that could be analysed, such as changes to the organisational structure, procedures, or the impact of changes in the environment of the organisation.

However, this is still a matter of research and "rather far from practical implementation", Azzam says. “Especially simulating the complex systems dynamic is very challenging in order to generate effects like nonlinearity, emergence or phase transitions. This is a big obstacle even after solving the issue of collecting the necessary information and tapping into its content using AI.”

You can come and hear Mark Azzam speak, alongside other Civil Service data leaders, at the Government Data Summit on 30th March.