Databases > Sharing your data

Opening Up To Open data

By Sarah Parker, Lamplight Database Systems

Open data can enhance accountability, transparency and learning for charities and voluntary organisations. At the same time it presents a series of challenges to the sector. Technical expertise, time and fear are all factors but without the data standards to back it up the open data movement threatens to keep its insights closed.

What is open data?

Open data is anonymised data, freely available to anyone who wants to use it. Those uses are limited only by the imagination of the people asking the questions. A growing movement is advocating the adoption of open data within the voluntary sector. Sharing performance and other data can lead to greater collaboration, improved service delivery and informative bench-marking.

The Benefits

Now more than ever we are aware of the importance of revealing the inner workings of organisations and agencies. The government has stressed the transparency and accountability that comes from open data, and have published many data sets: from expenses and MP interests to hospital waiting times, or the cost of maintaining government websites under the banner of Opening Up Government.

This approach has started to extend into the voluntary sector with organisations like ecdp, an organisation run by and for disabled people in Essex, publishing their performance data, governance documents and annual reports. The publication of this data allows greater accountability to their service users, funders and other key stakeholders – “they have nothing to hide and are willing to learn”.

That process of learning is one of the primary benefits of open data. Organisations can see how their own work compares to other organisations and, as government data increases, how that compares to national averages. Sharing impact data can help identify where needs are not being met and new approaches can be assessed as a community of organisations. Such tools and shared impact approaches are being tested by projects such as the NCVO Value in Infrastructure Programme where infrastructure organisations can share impact using a common set of impact planning and monitoring tools.

Sharing government and public sector data has also lead to the development of applications based on that data which provide real benefits to individuals: a road works database, a care homes map, perhaps most famously a local area crime map. Where the data is available, people find fascinating and unexpected thing to do with it.

The Challenges

Though the potential benefits are great for the voluntary sector, the challenges it raises are significant.

Technical Skill

At present the open data movement is driven largely, though not exclusively, by specialists who relish the challenge of mining data for trends. But for the average organisation, there are two problems.

1. Data out: Once data has been captured, there are difficulties around how and where to publish that data. Most small organisations who could benefit from the collaboration and learning potential of open data would not know where to start.
2. Data in: Once that data is published how do they access and compare data from other similar organisations? Tools like Yahoo Pipes allow the aggregation and mash up of data from around the web. But actually using those tools requires a high level of technical skill.

Time

Even with the relevant expertise, the time and resources involved in preparing and publishing such data can seem prohibitive. Overcoming this hurdle will require the development of intuitive tools and an awareness of the potential benefits of open data which will make any time spent on it worthwhile.

Fear

Opening up performance is an intimidating step. The openness of the data means that failures as well as successes are shared. While this has the benefits associated with learning it also raises the danger that short-term set-backs could impact on longer-term projects in the form of cuts to funding or commissioning.

However, by sharing such data it is immediately clear that an organisation is prepared to learn and develop in response to the data they gather and the results it reveals. Funders in turn will need to be more flexible in their support.

Interpreting data

Presenting raw data opens up external interpretation; this may or may not be welcome. The sector will itself need to provide some of this interpretation and be willing to challenge inappropriate uses of the data.

The enemy within

The final challenge is perhaps the single biggest barrier to the success of open data in the voluntary sector - the data itself.

It is not uncommon for an organisation to maintain a number of databases to capture incompatible monitoring and data requirements from different funders. A classic example of this is equalities monitoring, where one funder uses one set of ethnicity categories while another uses a separate set. Resolving this may be relatively straightforward in the case of ethnicity, but when you start looking at impact measures the issues become very complex.

Bringing these disparate data sets together within a single organisation can be challenging enough before even considering open data. Incompatible monitoring categories can be further compounded by different data formats and export protocols.

There are already organisations seeking to address this problem. LinkedGov collect and clean disparate sets of government data, bringing it together into a form that will be readily searchable in an intuitive format. However, more should be done to minimise the need for such clean-up in the first place.

The need for standards

For public data to be genuinely useful, it needs some standards. Without machine readable file formats (no tables in pdfs, please!) using a common set of definitions, open data will fail to deliver robust benefits to the sector. A parallel can be seen with the internet. Without a common language and agreed standards on how to write webpages every website would need its own different browser to view it. There would be no internet.

Defining standards is a hard problem: but not impossible. We hope that the sector, government, and other partners can start to develop some solutions. Open data is still in its infancy but it is going to be incredibly significant. The voluntary sector needs to be involved from the start.