The Spatial Hub is an emerging resource which will provide a single point of access to local authority data in a consistent, quality-assured format. The hub is developed, operated and managed by the Spatial Information team within the Improvement Service. Whilst not yet fully functional, it has progressed from soft launch, through a series of iterations and is now becoming a reality. It has received considerable interest and there has been substantial support for the concepts and ideas behind it, but there have also been some misconceptions about what it will actually deliver.
The main aim is to make data accessible
The key intention is to open up access to the wealth of spatial information created and maintained by local government in Scotland, and do so in a way that is both sustainable and self-supporting. Local government data is generally kept hidden away in boxes and thus its potential is not being realised. Our aim is to make spatial data accessible and easy to use, so that it can start delivering value.
More often than not spatial resources are scattered across departments, reflecting the “cell based” organisational structure of local government. As a result, spatial information is often buried in silos, virtually inaccessible – even for those working internally within the organisation. Consequently, it holds little value for those at a senior level and is not being used within the decision-making process.
Whilst this may appear to be a sweeping generalisation, it is based on many years of working in this environment and was confirmed by the extensive spatial data audit undertaken across all 32 councils in 2014. It was clear from the response that people were often unaware of the full scope of data assets that their council held, which has led to much frustration and criticism from those wishing to make use of what is extremely valuable information. It also leads to the roles of those managing the information being devalued, which in these troubled financial times carries an inherent degree of risk.
Authorities are struggling to maintain their datasets
Scottish local government is facing an increasing gap between projected sources of funding and projected expenditure over the next few years. The recently announced cuts to local authority budgets are in the order of 5%, and given that education and social services are ring-fenced, support services are being disproportionally affected. This is of course where the “niche” spatial data managers work, meaning these jobs are already being reduced and any vacant posts are being left unfilled. Some authorities are already struggling to maintain datasets which are core to their decision-making process. At a time when there is a requirement to work smarter and more efficiently this is liable to pose fundamental problems, and some people have argued that failure to address this is tantamount to “organisational ignorance.”
Inspire requirements then sit perched atop this heap of woe: local authorities, like all public bodies, are obliged under the EU regulation (and Scots law) to make their spatial data available for discovery, view and download. Discovery alone is challenging when no one really knows what datasets the council actually owns, and the Scottish SDI metadata records are far from complete and likely to remain so given the above lack of resources.
As far as view and download services are concerned, a few of the better resourced authorities are able to meet this requirement but many are really struggling. Licensing this (primarily) Ordnance Survey derived data also presents problems in whether to make it available under OGL or Inspire licences. Many authorities are reluctant to make their datasets available at all, as they acknowledge their poor quality and are concerned about any potential liability, but they are being guided by the Scottish Government’s Open Data Strategy towards doing so. These conflicting influences and situational differences mean that the data published by local authorities varies significantly.
The idea for the Hub
The data challenges facing councils pose a number of problems and these are only going to worsen unless something is done about it. This is where the Spatial Hub comes in.
The idea started in 2003 under the modernising government fund, where £7.5M of funding was provided to create what has now become the One Scotland Gazetteer (OSG). Local authorities were recognised as the most effective custodians for address information through: Street Naming and Numbering, Development Management, Building Standard and Council Tax
These address registers were conflated, de-duplicated and then used to create local gazetteers called Corporate Address Gazetteers. These conformed with BS7666 and the Scottish Gazetteer conventions and were then quality assured and aggregated to create the OSG. This remains the most authoritative and up to date list of addresses available for Scotland.
The plan was to apply the same process used for the OSG to all Scottish local authority spatial information. The 2014 spatial audit had identified that each authority held about 70 spatial datasets, and given that local authorities in Scotland perform largely the same functions it was reasonable to assume that they more or less held the same datasets – even if called by different names. The intention was to create national datasets of local authority spatial information and release the significant value buried away in authority silos.
The Spatial Hub provides a facility for each authority to upload its datasets to, where it can be processed and made available for discovery, view and download. This would then ensure all 32 authorities met their requisite obligations under Inspire. After discussion with the Scottish Information Commissioner we were advised to produce a formal agreement with each authority outlining both the authority’s and the Improvement Service’s obligations. This has now been actioned.
It’s a gradual process
At the present time, access to the quality assured national datasets from local authorities via the Spatial Hub is only available to One Scotland Mapping Agreement members. However, work is currently on-going to complete the necessary licence arrangements to extend and open-up access so that all can benefit from the value of this resource.
The processes which underpin the Spatial Hub are complicated because locating the actual datasets and corresponding “custodians” is challenging. There is a need for management plans for each dataset and these require development over time to ensure that the national layers are maintained in a consistent manner.
Maintaining quality datasets requires funding
The benefits of the hub, particularly in meeting Inspire requirements, has raised the profile of the programme to senior levels, something which has helped with engagement and participation across the board. Our market research identified that many organisations – both public sector and commercial – struggle to access the local authority information they required. So, people certainly see the benefits in opening up access to good quality spatial data, rather than it festering away in silos as before.
However, there remains a lack of recognition of the costs associated with this process, and this creates an element of contention when it comes to funding. We estimate the current cost of maintaining these datasets at approximately £2 million per annum. So, even though the Hub is cloud-based and built on open source software there are still sizeable costs incurred in the process.
Many councils (but not all) are happy to make their local spatial data available on Open Government Licences. This requires agreement from Ordnance Survey but several local datasets have been made available this way. OSMA members already make a contribution to the Spatial Hub funding, as many incur considerable cost in gathering and collating local government information and will benefit directly from the Spatial Hub. Further funding is expected from businesses that can create commercial value from utilising the datasets. These organisations recognise that the Spatial Hub will save them considerable time and expense, and have indicated that when the service becomes fully available they would be willing to pay for commercial use. As such, the idea is that the data will be available under an Inspire licence to provide ‘open’ access for non-commercial purposes, then we will endeavour to recover our costs through the commercial subscriptions. Commercial organisations will retain the current option to source the data they currently access from individual councils as open data, but the Spatial Hub adds considerable value through simplifying the process and providing more consistent data.
Viability in challenging financial times
The Improvement Service board believes that this hybrid model allows all to benefit from the Spatial Hub in terms of opening up the data and improving its availability and quality. The commercial sector will reduce its costs and improve profitability, while at the same time supporting the on-going management of the core data and ensuring the programme can be sustainable.
Several members of the open data community have commented on the proposal, and whilst they praised the Spatial Hub as a means to open up access to local authority data, some questioned why it is not made available free of charge under an Open Government Licence. The simple answer to this is that there is a cost incurred in turning raw data from councils into quality data sets. This cost has to be met by someone, and if commercial organisations gain direct benefit by reducing their costs or increasing profitability then it seems perfectly reasonable that they contribute towards the underlying costs. However, if government were to provide the necessary funding, the datasets could be made available, free of charge, under an Open Government Licence.
Indeed, this appears to be the stance of the commercial organisations themselves. They realise that the Improvement Service is uniquely placed to provide this facility on a cost-effective basis, and at a time when budgets are strained across the public sector, they acknowledge that this is only viable if a sustainable approach to funding is pursued.