– is it just me?

I thought I’d share with you (now that I’ve had a couple of weeks to calm down) the process of publishing some open data via – the site that aims to make public data more easily accessible.


If there’s a slightly less frustrating method of doing it, I’d very much like to know…


Day one:

Arrive at site. Look for a handy button labelled ‘Publish data’ or a useful link saying something like ‘How to publish your data on this site’, or some such like. Start to feel slightly stupid when you can’t find anything relevant.

Notice the ‘Sign up’ link, but note that signing up will only enable you to join the forum, post comments etc. No information is given about publishing data.

Try the FAQs – do you want to know what a mashup is? No. Try the search – do you want to browse through 99 unrelated documents? Not really.

Scratch head. Mutter. Give up.
Day two:

Take deep breath. Repeat all steps from day one, in the certainty that you’ve missed something really, really obvious.

Determined not to be put off again, fill in the registration form. Wait several hours for the confirmation email, retrieve it from the spam filter, then login.

View the discussion forum, in which people are saying things like:

Sorry if this is a stupid question, but how do I submit a dataset to this site?


This may sound like a silly question, but how do I add new data to a dataset I have already created?


By rummaging through the replies on the forum, find a link called “Guidance: Including data into the index”. Hang on, you might just possibly be on to something here…

The guidance is to “send your username to your departmental family’s lead” so that you can be given publisher access. Rummage a bit further and discover that the lead for local authorities is the Department for Communities and Local Government.

Send polite email to the DCLG (using the email address given on the guidance page) asking for publisher access, including your registration details, council details, information about why you need access, and your deadline for publishing data on the site. Wait.
Day three:
Continue to wait. Wait all day. Eventually follow the “failing all else” advice given on the web page you found yesterday (which you thankfully bookmarked) and forward your polite, clearly-explained message to the Public Data people at The National Archives.

Also publish frustrated message on Twitter:

which is quickly picked up on by Will Perrin from Talk About Local (and member of the Local Public Data Panel), who asks you to email him the details if he can help.

Go home at 6pm, missing the email that arrives at 6.13pm telling you that the Local Public Data Unit – Transparency Division (your lead at the DCLG) have now forwarded your email to

This email also makes a comment about your spending data (which isn’t what you enquired about), thus making clear that the person who received your message didn’t bother to read it properly.
Day four:
Email Will Perrin with the full details. Will sends your request to the Cabinet Office and asks them to help move things along.
Day five:
Spend the morning helping people to use technology in the neighbourhood that your project is based in (and which feels like a very long way away from all this publisher access gubbins).
Day six and seven:
Wait (but without much hope, it being the weekend).
Day eight:
Receive an apology from The National Archives, asking “Please could you register on and send me your username” – thus making it clear that the person who your message has eventually been forwarded to hasn’t bothered to read it properly either.

Reply with a reminder that you are already registered.

Wait a couple of minutes.
Ta – dah! You have now been granted publisher access to
You will now receive two separate guidance documents, each listing the various fields that you’ll need to fill in on the site – with two different sets of instructions in some cases. You’ll be really amused by one of the documents, which starts by telling you who to email to get publisher access.
Day nine:
The process finishes with you having finally entered your information and looking in vain for a button saying “Publish” or some such like, but finding only something called “Submit query”.
I do have a query: Is it just me, or is this whole process not very easy, accessible or transparent?
Diane Sims, Kirklees Council

Empty spaces and the “holes in the map”

One of the questions I asked when we started working on ‘Who owns my neighbourhood?’ is whether any part of a neighbourhood should be unused. I’ve been thinking about how people view local land, what value they see in it, and whether our data can contribute something useful to this relationship between people and place.


I read something once (I think it was in the RSPB magazine) about how people view “empty” spaces. The writer pointed out that a property developer might view a green field as empty, whilst a conservationist might view the same field as being full – full of life and full of interest. This came to mind when we started looking at printed maps of our pilot area, on which the open green spaces appear as gaps between the printing, white nothingness, often devoid of labels or landmarks. I wondered how local residents would respond to that kind of representation.

Of course, for local residents these are not blank spaces at all – they are places that they know well. At a series of community events, we made about 50 recordings of people commenting about places on the map. The thing people talked about most often was how much they enjoyed using local green spaces of one kind or another.

A place to play

Walking to work

That little bit…

People had their own suggestions about how these spaces could be better used and cared for. We saw enthusiasm and expertise that local service providers are probably largely unaware of at present. We also heard from concerned residents who feel worried that decisions might be taken about their neighbourhood without their knowledge. For example, one lady said of the field next to her flat: “It’s the second time in two years that I’ve seen people standing there gesticulating and talking, as though they’re discussing what’s going on there… we were told that they couldn’t build on that.”

The question of abandoned, derelict or overgrown places arose a few times. People were mostly interested in seeing something done to make better use of disused buildings and untidy land, and were generally willing to get involved in doing something to help. We saw evidence that uncertainty over ownership is sometimes what prevents people from taking positive local action. A retired lady who lives opposite an overgrown green space said she has “itchy fingers” about getting out there and tackling it, but she asked: “would we be trespassing – would we get into trouble?”

Empty houses also drew people’s attention. People told us their concerns over derelict buildings attracting anti-social behaviour, but they also made suggestions about how things could be improved.

Old cottages

Art Deco house

A second home


If there’s someone in the community who can help make such suggestions a reality – be it a resident, owner, service provider, voluntary organisation or business – how do these interested parties find each other? What part can open data play in making that happen?

Our work with land ownership data in Kirklees has unearthed empty spaces of a different nature.  We’re looking at what data we have available and we’re trying to put it all together in a way that makes sense to people. Our base data is “acquisitions” and “disposals” of land, held separately until now. Our mapping team are using these two data sets to create a combined Land Ownership layer. This involves cutting away parts of the original acquisitions layer, leaving “holes in the map”.

Land that the council have disposed of is not tracked after it’s sold, so these land parcels are gradually removed. There are yet other bits of our map missing – for example, those places which the council didn’t own to begin with. So circumstance is already nibbling away at our map, but we’re trying to make sure that things don’t get lost down the folds.

One of the confusing things about council data is that it turns out there are actually many maps. Our search has revealed extra data that we hope to add, such as parks, conservation areas and woodlands. In some cases, there is an over-abundance of data (we have two separate sets of allotments data, held by two different council services) but in other cases we might not be able to use the data at all (for example, the council currently charges people for access to data about village greens, so we might not be able to display those on our map).

Now that we’ve got into it a bit, we’ve started to learn about the complexities of land ownership itself. Although the council might own a piece of land, it way be on lease (possibly for hundreds of years), there may be tenants, covenants and all manner of other confusion. It’s our job to try and help people to understand some of these complexities whilst still offering a simple and engaging interface that helps people to share what they know.

The sharing is the important part – because it’s local people who know how to fill in the holes in the map. They know the things that you can’t see by looking, such as where a Post Office once stood, where they built bonfires as children, and what local people’s aspirations are for the future.

Listening to people reminisce and imagine about these so-called empty spaces made me think that it isn’t just a question of whether any part of a neighbourhood should be unused, but also whether any part of a neighbourhood should be forgotten.

Diane Sims, Kirklees Council

Where does a librarian sleep?

Sutton Open Library is a new way to share books and make friends in the borough.

Our project will open up the data that lives in the library services database – making it available for re-use and (hopefully) the creation of lots of clever hacks, apps and widgets.

The really exciting part of our project is that you will be able to list your books on a website and lend them to other people. You can also browse what others are offering and if you see something you like – get in touch and ask to borrow it.

The potential for this is huge – a library made of all the books that people in Sutton have on their shelves and a great excuse to meet new people, share books and talk about the things that interest them.

We are in the design stages at the moment and if you have any thoughts about our project, please come over to our blog and tell us what you think

(Answer: Between the covers!)

Some very sketchy sketches.

Diane and I sat down and tried to sketch out how Who Owns My Neighbourhood would work when someone came to the service (for some reason I find it easier to say sevice than site at the moment).

This included a long discussion about the meaning of flags and flowers as icons.

Gently Flows the Data

Matt, Mudlark’s CTO, always make pictures – which usually help technical-dullard me far more than some of the more elaborate discussions.  Here’s his sketch of the data flow for the BCD project.

And the data has begun to flow – into our developer Chris Thrope’s grateful clutches. We’ve had some very interesting chats about what we can know about each call and what we want to know.How precisely should we pinpoint a call? Should we (can we?) focus on the location where the resident lives or where they say the issue is ? These may or may not be the same thing.

We reckon that as much as showing patterns of incidents and issues, the BCD may well plot where the most active complainers are. What cliches and stereotypes we reveal or destroy remains to be seen.

How to talk about open data?

The week before last Diane and I went to three (in truth I went to two and Diane went to all three) community events in Newsome to see if we could get a sense of what people say when they talk about local issues through places.

To get things started we took a gigantic map and some blue and yellow flags made out of cocktail sticks and asked people to look at the map, orientate themselves, and then think of something positive or less positive about their neighbourhood, put a flag there and tell us why.

This was something I’d seen done, a little bit more practisedly and with slightly better flags, by some geographers, as part of a consultation process I’d been involved in. What struck me at the time was that people seemed to be really enjoying themselves and the map looked and felt like a board game, which welcomed people to take part and so to talk.

It seemed to work in the same way in Newsome, with people standing over the map for ages finding their own houses, then tracing their old routes to work or Sunday afternoon walks, and from that, being happy to talk about what they did or didn’t think positively about. This seemed a valuable confirmation in itself, that maps are a good way to get people talking.

What we were hoping to find were themes in what people were saying that we could use to pull data sets out of the council that matched what people wanted to know and do. I haven’t listened to all the interviews yet but my sense is that there are themes, not least ownership and responsibility, but that they are more or less the same as Diane identified in her original application. This isn’t so much because the citizens of Newsome are easy to read, but because when Diane isn’t working for the council, she is so active in that community anyway that she already had a good sense of what people were talking about, and the examples from the application are all very real.

What I found trickier on the day was finding a way to explain to people why we were doing what we were doing with the maps.

At first I stayed away from mentioning the open data project at all, just saying we were collecting stories about Newsome, because I thought that if I started talking about “data that the council holds” people would wonder what we were going to do with what they told us, and that would get in the way of them talking.

It was just a practical choice to avoid mentioning open data at first, because if we were going to get any unexpected insights from the map sessions people had to be comfortable saying what they needed to say, but it felt a bit unsatisfactory, and not honest, to avoid mentioning the open data part, so I tried out saying:

“the council holds all sorts of useful information and we are trying to find out what information people would want to have about Newsome.”

I got the feeling that was being met with the kind of polite agreement that meant “well that sounds like it makes sense but I don’t really know what you are talking about in practice.”

And who is to say what “useful” is, after all. Useful to whom, and what for? And it was a bit dry anyway.

So by the end of the second session I was trying out this:

“If you could look at a map, and put your finger on it, what would you want to know about that point and why?”

Andrew Wilson, Thumbprint Co-operative

Birmingham Civic Dashboard- The Genesis

Sometimes it just works.

We were musing about NESTA’s Make It Local competion, got chatting to our estimable Fazeley colleague Will Perrin of TalkaboutLocal, took the idea that came out of Will’s brilliant enthusiasm to Digital Birmingham’s event for the NESTA competition, where it got some great feedback, not least from Birmingham City Council’s Simon Whitehouse.

Dodging the obstacles put in our path by NESTA’s August deadline and nearly-everyone-involved’s summer holidays, the city ( in the shape of Simon)  and Mudlark got in an application to NESTA and in September we learned we were one of the winners.

The result will be the Birmingham Civic Dashboard, coming to a screen near you in January 2011. Simon has blogged about it here, NESTA reported it here, and New Media Age also reported it.

There will  a lot more to come as we hit our stride, but in short the BCD will present  a searchable, drillable, zoomable, tellable heat-map of all the contact calls that come into the council’s contact centre each and every day.

We hope it will offer a new way of  viewing (and, of course, interacting with) all the new conversations a big city council has with its residents. People will be able to see how residents in  different areas of the city are concerned about different (or the same) things – and where different problems or questions occur.

Our first big block of data has just arrived and we are getting our teeth into it – more soon.

Charles Hunter, Mudlark.

Photo Credit – Stef Lewandowski –