Commercial access to public sector data needs to change

I was delighted to be asked by Scottish Government to investigate and lay out the evidence for the circumstances under which the commercial sector, such as private limited businesses and other types of profit-making organisations, are allowed access to public sector data and how this process affects researchers, policymakers and wider society.  

It was a fascinating process to hear from actual and potential users of data working in industry, academia and the public sector, with those working in Scotland’s Safe Havens – also known as Trusted Research Environments. The review, which I led over the first three months of the year, included interviews with data owners and panels who support decision making about use of public sector data. In addition, the Innovative Healthcare Delivery Programme (IHDP) helped organise two workshops for private sector organisations wanting to use data. In the end, the review included views from over 40 stakeholders and is pretty comprehensive.  

So what did we learn?

Current state of play

Data is held safely and securely in Safe Havens in Scotland with all five of them adhering the to the Five Safes framework.

The five safes are:

• Safe projects: Is this use of the data appropriate, lawful, ethical and in the public interest?
• Safe people: Can the user be trusted to use it in an appropriate manner?
• Safe data: Does the data itself contain sufficient information to allow confidentiality to be breached?
• Safe settings: Does the Safe Haven limit unauthorised use or mistakes?
• Safe outputs: Is the confidentiality maintained for research outputs coming out of the Safe Haven?

This is an internationally-regarded best practice framework that goes much further than legal requirements, and should be reassuring to everyone who’s data is stored within these environment.

Future concerns and opportunities

What this review does is lay bare that within this framework, ways of working are subtly different across the Safe Havens. This makes it difficult, at times, for researchers to navigate complex projects requiring data from different places.  

Unless we get an aligned Scottish approach, we won’t be fit for purpose in the future.  

The public expect to see stringent oversight, governance, and safeguard arrangements around private sector use of public sector data, especially concerning independent scrutiny of proposed uses of data, transparency and accountability processes, and arrangements for data security and safety, consent, and confidentiality.  

However, this review suggests the precise nature of what the safeguards should be is contested, and it may be that the nature of the safeguards is less important than the fact that effective safeguards exist. 

The good news is that many of the issues should be relatively straightforward to rectify, and that building on the conversations from this review should be able to get agreement on a Scottish approach – this is beneficial to researchers and would give them clarity. 

A loss of trust in use of public sector data for public benefit would set back the ability to use data to improve wellbeing through research, so listening and engaging with the general public on how their data is used and kept secure needs to be central to any future policies.    

The industry voices I heard from, strongly support that the system in Scotland should be aligned with the rest of the UK and globally, where possible. Therefore, any aligned Scottish approach would need to be adaptable as standards emerge. 

Questions that don't have simple answers

The work identified a few really tricky issues that need some in-depth thinking to resolve. For example, questions remain about how private sector companies pay for access to Scottish data for research – how do we work out the worth of such information, and how do we share benefits that result from that access?  

Another key aspect is to consider how new technologies used by many firms would be handled. For example, using AI approaches brings new challenges, such as disclosure risk of data embedded in the models themselves. There are a number of evolving issues around deploying the models in public sector setting, like ethics, intellectual property and regulation. Again, a clear Scotland-wide approach here will be helpful. 

The review also identified several fundamental issues that need addressing in order to maximise the public value from data. In particular, ensuring there are trustworthy and efficient processes governing access to data is vital. Research Data Scotland will continue to work collaboratively with others to address these challenges, reaching a consensus amongst stakeholders and garnering public support. 

So, it is clear that work is required to develop clear aligned approaches that balance public concerns about appropriate use of their data with greater research that can improve lives. This review provides the groundwork for that work to take place and I’m keen to support that work and improve lives with data.