As such we are considering implementing a basic technical quality framework on data.govt.nz and would like your feedback on whether this approach would be useful, whether you are a data user, data publisher or both.
We recognise that trying to simply enforce a new standard across the system is difficult but if the basic technical requirements for finding and using data are met, then it is a great start.
For government open data to meet its full economic and social potential outside the public service, there needs to be a substantial increase in the meaningful reuse of data by the private and community sectors. Promotion of government data programs can help encourage data users to explore and experiment with available data but user research and feedback has repeatedly revealed frustration by data users in trying to find data they can actually use. We need to make it much easier for users to find data they can actually use and at the same time create additional motivation for government agencies to publish data in ways that can be readily consumed.
It is worth noting that data users are not the general public. Data users are the people, companies, universities and analysts with technical skills, motivation and capacity to innovate with government data with new applications, analysis, data visualisations and undreamt of uses of government data. Provision of front end visual tools to play with data is of less interest to serious data users but investment in improving discovery of the current supply chain, and investment in an improved data supply chain is critical for such reuse to be sustainable. To improve the data users’ experience with government data, we must first make it easier for them to easily find data they can actually use.
The Basic Technical Quality Measures is a series of technical attributes that are necessary for data users to innovate with government data. Where agencies or specific data domains have existing quality frameworks these could be a second tier quality measure for identified datasets of that agency or type, but these measures are focused on the very real needs of data users. For example, a beautifully crafted dataset that is published as a PDF is of almost no use to anybody. Whereas a less beautiful but regularly updated dataset published with a machine readable API (Application Programmable Interface) is pure gold for a developer or data visualisation expert.
This series of basic technical quality measures is kept as simple as possible and is based on the most basic needs of data users regarding format, metadata, APIs and public scoring. The four measures are a 5 star score to make it quick and easy for data users to search and visually assess the suitability of data to their purposes. Almost all measures are purposefully automated because quality measures that require human intervention are subject to mistakes, consistency and subjectivity. It will be implemented on beta.data.govt.nz for public consultation and testing with data owners and data users. It is only through active testing that such a framework can be properly tested.
Ideally it would be worth updating the data search interface with options for users based on the measures above. See mockup below.
The format data is published in is important. It can be a first barrier or enabler to the data user doing anything with the data. For example, if it is not machine readable then the data user has to transform it to a machine readable version before they can do anything clever with it. If there is no API then they need to download it and then either embed it in their application or set up their own API. These barriers can turn a data user off from even trying to use the data let alone discovering some novel or innovative use.
The Tim Berners-Lee’s (TBL) 5 star CKAN plugin (used by the UK) is a useful start however, it prioritises openness over usefulness and some of the attributes are difficult to automate meaningfully. The implementation below is a slight tweak of the plugin to test with data users. Stars are associated with format type as per the list athttps://github.com/okfn/ckan-barnet/wiki/Data-quality
Note: very few datasets are linked data and are difficult to generically automate, so manual curation of these is required.
Although there is an extended metadata schema on data.govt.nz and myriad metadata schemas for data specialisations, these metadata attributes are important for data users to identify data they want to use in the first instance. A star for meeting each of the below to give cumulative number:
If the data is available via an API it goes a long way to supporting innovative reuse. APIs are critical for serious data users to build persistent analysis, applications or visualisations on government data so to raise public confidence in using government data, we need to make it easier for data users to identify API quality. Below is a 5 star quality ranking based on latency and uptime. Where an API achieves only one of the two conditions, they drop 1 star. So a 99% uptime but 10 second latency would get 4 stars.
The public feedback on the quality of the dataset provides not only a form of peer review that data users can use to identify quality data according to other data users, but provides a check and balance for the quality measures above. If the public feedback on a dataset is consistently and markedly different from how the quality measures above represent the data, then the model will obviously need to be assessed. It is possible that in some cases the basic technical quality measures might be high but the public don't enjoy using the data for other reasons worth identifying.