U•Geo at GISRUK 2012

We will be presenting the U•Geo project findings, and the Browser app, at GISRUK 2012in Lancaster this April. This is the UK’s largest GIS research conference, and a great chance to show off what we’ve done and bolster awareness of the issues our work has highlighted. The abstract is included below, and the full paper will be published in the conference proceedings later in the year. Hope to see you there.

Unlocking the geospatial potential of survey data

National survey data presents a hugely valuable, but currently underused resource for researchers, learners and teachers using geospatial methodologies. Spatial units are the fundamental method of georeferencing survey data and the integrity of any analysis or visualisation relies on their appropriate usage. Building on the UK Data Archive’s expertise in the preservation and dissemination of social science data, we looked in-depth at ways of increasing the value of UK Data Archive services to GIS users. Our developments include a novel web application which assists in the location, interpretation and linkage of spatial unit variables in datasets.

What	GISRUK 2012
Where	Lancaster, UK
When	Apr 11, 2012 – Apr 13, 2012

Budget note

This presentation is funded from the dissemination budget of the JISC-funded U·Geo project, costed at £800 for conference registration (£265pp) and train travel for 2 people.

EDIT: Paper presented and well recieved. The slides from the presentation are available here: http://www.slideshare.net/tomensom/unlocking-the-geospatial-potential-of-survey-data .

Advertisement

Unconference @ JISC Geospatial Event 28-29 Nov 2011

Recommendations to JISC from the Unconference sessions at the JISC Geospatial event on 29 Nov 2011.

During the second Unconference session of the day, focussing on common themes running through the projects. 6 themes were identified for geospatial data:

• Create
• Manage
• Repurpose
• Analyse
• Share
• Express

We discussed theme: managing geospatial data (table 2)

We = Scott Orford and Richard Fry, WISERD Geoportal ; Tom Ensom and myself, U.Geo proejct.

Key topics we discussed: metadata, standards, INSPIRE, linking, security, disclosure

Key recommendation:  providing expertise/services on  STANDARDS for geospatial data: INSPIRE, metadata standards and standards for data security.

We discussed the issues of often complex/frustrating access to micro-level data with disclosure risk and how to manage access to such data. Solutions are needed.

Services are needed for data linkage / geo analysis for disclosive data, that provide user-frendly options for researchers. E.g. Health Informatics Wales does secure anonymised information linkage for researchers. Can re-aggregations be provided via e.g. Secure Data Service  (for non-SDS users).

Anonymised mapping techniques are needed.

Another problematic issues – how to keep track of changes to boundaries, where versioning is needed, e.g. changes to postcodes, objects, counties, etc.

Data creators need awareness of spatial standards.

Metadata need to be openly exposed.

Text mining is feasible to certain degree, then needs human intelligence = intelligent text mining.

FINAL BLOG POST: the U•Geo GeoBrowser

Unlocking spatial units in social science survey data

Primary audience: Researchers

This is the final product blog post that showcases the main tangible output being developed by the JISC-funded U•Geo project: a resource discovery tool for geospatially enabled survey data. Come and see our live demo on Monday 28 November 2011 in London, at the free jiscGeo Programme event.

What is it for?

Using social science survey data in GIS can be a pain. The complexities of spatial units – the primary means of geo-referencing this kind of data – and their ever-changing boundaries can lead to a minefield of problems and potential analysis error. The U•Geo GeoBrowser will make spatial units more accessible, transparent and usable, and will help users easily find which survey data can be linked to which boundary file. The GeoBrowser matches large-scale social survey datasets available from the UK Data Archive with digital boundary files available from EDINA.

The key ingredients in this are extensive enhanced metadata compiled by the U•Geo team, standard and time-referenced definitions for spatial units and an intuitive faceted search & browse interface, using SOLR. This interface allows for on-the-fly filtering of search results by relevant criteria.

The GeoBrowser means researchers can easily find out which spatial units are present in archived large-scale survey datasets available from the UK Data Archive and which digital boundary shape files can be used to map those spatial units to, to geo-enhance those survey data.

How does it work?

Let’s walk through an example. Our researcher Cheryl is studying the geography of happiness across different regions and localities in the UK. She would like to use various social parameters of the British Household Panel Survey (BHPS), and wants to find out which spatial variables she can use to geo-reference the dataset. As she is particularly interested in patterns over time, she needs to know this for all waves (i.e. years) of the survey.  She is keen to use a statistically sound unit of analysis, but also needs to know which data service (and license) will be required for her to access the variable she wants. Finally,  she wants to find an appropriate UK-wide shape file for her chosen spatial unit that she can map the data onto and create some nice shiny maps to impress the stakeholders.

By using the GeoBrowser she finds that three spatial units are available for the BHPS via Conditional Access (she can access one spatial unit after agreeing to click-use conditions): Local Authority Districts, Parliamentary Constituencies and Area Classification for Output Areas. These spatial units are available for all waves from 1991 until 2009 under differing access conditions. The GeoBrowsers also shows her that EDINA has the following shapefiles for England, Scotland, Wales and Northern Ireland: Local Authority District boundaries for 2001 only; Parliamentary Constituencies boundaries for 1991, 1997 and 2001; Output Areas boundaries for 2001.

A demonstration video will be added here shortly, as our developers are finalising our demo version. For now I’ve compiled some screenshots of the browser in action.

Which datasets have this spatial unit?

This is just one example of a huge array of potential problems that our GeoBrowser elegantly solves.  Using a faceted search/browse interface, the user can very easily burrow into the mass of metadata and find out exactly what they need to know.

Where is it?

The developers are busy finalising the U•Geo GeoBrower. It will go live soon at  geo.data-archive.ac.uk

What next?

The GeoBrowser provides the first model of its kind, and will feed into larger scale developments to resource discovery services at the Archive. In the future the Archive’s resource discovery will allow the discovery of mappable boundary data.

Follow the UK Data Archive’s Twitter feed to keep up to date on the latest news.

Project team

Tom Ensom (UK Data Archive)

Veerle Van den Eynden (UK Data Archive)

Anne Robertson (EDINA)

Matthew Brumpton (UK Data Archive)

With invaluable contributions from: James Crone, John Payne, Oscar Dovao, Sidharth Balakrishnan, Saadat Ali and James Reid.

Disclaimer and licensing

Please note that the U•Geo Geo-Browser is a demonstration product and is provided as is, with no guarantee as to future updates.

The license our geo-browser will be delivered under is still uncertain. However we will be making the source code available through the GitHub repository when the product is launched and sufficient documentation has been compiled.

Blog contents

DELIVERABLES

INSPIRE – DDI metadata mapping UK Data Archive
Report – What do social science geo-users want from Archive data?

All deliverables are also available via the U·Geo webpage

PROGRESS UPDATES

Progress update 1: Geospatial data review
Progress update 2: Metadata mapping
Progress update 3: Key findings summary
Progress update 4: Developing a ‘geo-browser’
Blog stats

PROJECT PLAN

Project plan 1: Who Geo? U•Geo!
Project plan 2: Benefits
Project plan 3: Risks ands successes
Project plan 4: IPR
Project plan 5: U•Geo team and users
Project plan 6: Timeline, workplan, methods
Project plan 7: Budget

FINAL PRODUCT POST

Presenting: the U•Geo Geo-Browser

COMMENT

‘Child of ten’ standard and U∙Geo
Innovative web tools
INSPIRE for the social sciences – where do we go from here?
Licensing of data and metadata
Linked data and georeferenced survey data

Blog stats

Our blog activity (as taken on 10/11/11) has spiked during the last 2 months. We’re expecting record views in November :-), thanks in part to the UK Data Archive’s new twitter account!

Month Feb  Mar  Apr  May  Jun  Jul  Aug   Sep   Oct   Nov
Views   63    65     45     52      47    59   54     128   170   62

Monthly blog stats

Topical blog stats

INSPIRE – DDI metadata mapping UK Data Archive

We have mapped the UK Data Archive catalogue metadata to INSPIRE and GEMINI 2.1 metadata specifications, as part of the development of a roadmap towards INSPIRE compliance for the Archive’s metadata schema.

The Archive’s metadata schema is based on DDI (Data Documentation Initiative) 2.1; therefore DDI-Codebook and DDI-Lifecycle elements are also included in the mapping.

This work fits into wider resource discovery and infrastructure developments at the Archive, with an aim towards implementing a DDI-Lifecycle (3.1) metadata structure.

DOWNLOAD: Mapping UK Data Archive metadata to INSPIRE and GEMINI 2.1 metadata specifications and to DDI metadata elements [XLS]

 

Citation

UK Data Archive (2011). Mapping UK Data Archive metadata to INSPIRE and GEMINI 2.1 metadata specifications and to DDI metadata elements. Colchester, UK Data Archive, University of Essex.

This output is published under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales license.

Report – What do social sciences geo users want from Archive data ?

We asked researchers that regularly use UK Data Archive data collections in geospatial applications, what they want or need to make better use of social science data for these purposes.

These researchers either use social science survey data available from the the Economic and Social Data Service (ESDS) or the Secure Data Service (SDS). We focused in on how they work with geospatial data and what they require from data providers (like the UK Data Archive) to make their research most effective.

We also drew upon additional info from two recent reports: an ESRC-commissioned report on geospatial data needs in the social sciences community [1] and a report on geospatial support services in higher education institutions [2].

Our brief report shows the suggestions and recommendations made by users, and how we took those on board in further activities of the project.

DOWNLOAD: U·GEO REPORT ON USER NEEDS [PDF]

[1] Owen, D., Green, A. & Elias, P. (2009) Review of geospatial resource needs.
[2] Sutton E., Chisholm, H. & Armitage, T. (2010) Survey of Support for Geospatial Resources within Higher and Further Education.

JISCgeo Innovation Programme event

We will be demonstrating the U·Geo GeoBrowser at the JISCgeo Innovation Programme meeting on Monday 28 November 2011, 9.00-17.30, at Ravensborough College in London.

At this open event, several new BETA products and tools centred around geospatial technology and aimed at Universities, Colleges and Schools, developed by JISC-funded geo projects will be demonstrated. We encourage students, researchers and teaching staff from all disciplines to attend and sample these new tools to further their usefulness to everyone in Academia. Come along and discover how Geo really is part of everyday work and play 🙂

Programme of the day.

Register for this free event.

Licensing of data and metadata

We want to use Creative Commons licences, but can we publish everything the U·Geo project produces under CC?

Things may be more complex than they seem at first hand.

In this project we mainly worked around ‘metadata’, in particular the metadata that the UK Data Archive holds about its archived data collections, rather than the datasets themselves that are held by the UK Data Archive.

We worked around improving those metadata from a geo point of view: to improve resource discovery for users keen to use the data collections in geo applications, to make it easier for users to be able to link survey data with boundarioes, and to harmonise metadata in line with the INSPIRE directive.

So we created some databases (or tables) of metadata.

1. A first one is the table that maps the UK Data Archive’s metadata elements to INSPIRE and Gemini2.1 metadata specifications and to DDI metadata elements.

We are publishing this table under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK licence (CC BY-NC-SA-UK), but we obviously do not own the underlying metadata elements, only the mapping.

Copyright / licenses of constituting metadata are  acknowledged as follows in the table:
– UK Data Archive catalogue metadata are copyright of the University of Essex
– DDI 3.0 metadata schema is licensed under a GNU Lesser General Public Licence (this allows redistribution and/or modification); DDI Alliance licence all their other documents under Creative Commons licenses.
– Gemini 2.1 metadata copyright is held by AGI
– INSPIRE metadata are ‘Public right’

2.Then we have the database that underpins the GeoBrowser we are developing. This contains metadata on datasets the Archive holds and the spatial units those datasets contain, as well as a structured list of spatial unit definitions.

Who owns these metadata? Much of the information has been gathered from the ESDS data catalogue and then polished and standardised (this feeds back into the catalogue). The ESDS catalogue metadata are partially provided by the individuals and organisations depositing data, partially created by the UK Data Archive. All these metadata are in the public domain, via the ESDS data catalogue. They are considered copyright of the University of Essex, but strictly speaking, hundreds of data depositors have partial copyright. The spatial unit definitions may be (partially) owned by a defining organisation such as ONS. Then user descriptions were added by the U.Geo project.

3. The code for the GeoBrowser is based on SOLR and Umbraco (the Archive website content management system). Once we have a workable GeoBrowser the code will be placed in the code reposoity https://github.com/daidp/uGeo, under a Creative Commons licence.

4. So what about the research data then, that these metadata are about? How are they licensed and who owns them? Whilst for the U·Geo project we have not used the archived datasets. But it is important for researchers who are keen to access and use the data to understand the ownership and licensing.

Data collections archived and distributed by the UK Data Archive are owned by the individuals or organisations that deposited them with the Archive. The Archive does not own them, but has the right to preserve and distribute them. A Licence Agreement signed by the depositor specifies the rights and responsibilities of the depositor and the Archive and authorises the UK Data Archive to preserve and distribute the data collection (under terms and conditions specified) and for specified use purposes. This Licence Agreement is available:  http://www.esds.ac.uk/aandp/create/licence.asp.

The datasets are then distributed under a ‘End use licence’ that users need to agree to before they can download data. This licence gives users the right to use the data according to the specified use conditions, such as an agreement to inform the UK Data Achive/ESDS of any publications; to cite the data correctly; not to share data with anyone who is not authorised to access them; not to use the data for commercial purposes, except with permission; and not to try to use the data to identify individuals, households or organisations. This End Use Licence is available at: http://www.esds.ac.uk/aandp/access/licence.asp.

INSPIRE workshop – where do we go from here?

I think any attempt at blogging the actual events of the day here are rendered somwhat redundant by Nicola Osborne’s brilliantly comprehensive live blogging of proceedings over at GECO. Thanks Nicola!

In short though – the day was a great success. Getting people excited about metadata is not necessarily easy, but there was lots of interesting discussion among a group with pretty varied backgrounds and angles on geospatial issues. This included representatives from funders (JISC, ESRC), data and geospatial service providers  (Swedish NDS, WISERD) and researchers.

Rob Berry from WISERD demo-ing their new INSPIRE compliant geoportal.

The discussion ended up extending beyond just INSPIRE and more generally into the geospatial potential of social science data. I’ve extracted what for me were some of the key, take-home-and-act-on messages from the workshop below:

• Hazy legal obligations aside – INSPIRE provides a solid basis for geospatially useful metadata. To ensure compliance is just good data management practise, and not is necessarily very difficult to accomplish for those already using a good metadata schema.
• Confidentiality issues should be carefully considered by working groups when it comes to INSPIRE data specifications – of particular importance to social sciences i.e. dealing with micro-data.
• Generally, there is insufficient institutional expertise in GIS to assist users and ensure it’s being ‘done right’. Providing data and visualisation is great, but we need a skill base in order to use these resources properly.
• Metadata based web portals/ browsers are not only cool, but useful! We need more of these for the social sciences, while carefully considering users and potential extensibility along the way.

As an aside, I was particularly interested in the difference of opinion between the data infrastructure purists and those more inclined toward innovation and ‘public science’. I emphathise with both sides of the argument, and think that in our approach to the geospatial challenge a union of the two will be critical.

Child of ten standard and U∙Geo

Various #jiscgeo communications have invited comment on the ‘child of ten standard’ as applied to our project deliverables – i.e. could a child of ten achieve anything with our product in ten minutes. It’s difficult to comment in any really compelling way on this – reasonably, I find it difficult to imagine any child of ten taking even a vague interest in metadata mapping or trying to locate geospatial variables in survey data 😉 .

To take things slightly less literally we could ask; what could a non-expert achieve with our product in 10 minutes? Well, we can confidently say that there is a very clear difference between our geo-browser product and the relatively minimal documentation that it builds upon. The interface  invites the exploration of geospatial metadata – a task that could previously have involved much frustration is now elegantly simple. Given that one of our geo-browsers key aims is to provide non-expert users with better tools for working with data, we are particularly proud of this usability.

We should be ready to give more information on our final products in the very near future!