Browsed by
Category: data

UK local online #mapping is a stupid mess…

UK local online #mapping is a stupid mess…

Yuck, online mapping 1990s style

Today something is bothering me,   it has been bubbling away since I wrote this post “The Awful mess of Local Plans online” and I can’t contain it anymore.

Where’s my broadband coordinator? Cumbria CC

Cumbria as a geographic area I think has a lot of online maps and they all vary in quality and usability.   Sometimes it looks like the people who are operating them haven’t told other departments what they are doing…. to be fair they probably have but red tape has got in the way…

Lets take Sedbergh (yes I used Google Maps!) for example, if I want to see what services my local authorities’ provide on a map I can go here for Cumbria CC services but if I want to see their Public Rights of Way mapping (and the two national parks), I have to go here.  Now Cumbria also has a Historic Environment Record and their mapping is here (seriously slow & doesn’t always work).

Walk this way , Cumbria PROW map
Cumbria’s Historic Environment Record mapping is shit, no really

Now here’s the fun bit, bins, building control and libraries are dealt with by South Lakeland DC .   Ah, but if I want to see my local plan online rather than via PDF I have to go to the Yorkshire Dales online GIS here.   Oh and if I forget what Local Authority I’m in Eden DC provide some mapping for part of Sedbergh too.

Eden Web Mapping, basic but okay?
South Lakeland’s mapping just like Barrow’s and I quite like it..

All these maps provide detailed OS Mastermap level mapping.  Some are symbolised quite well, others not so well, I’ve always liked Barrow’s online GIS which South Lakeland seem to use, but that’s because it’s a bit technical and I hear open source [PDF].   I also worry when watermarks aren’t done well.

As a resident I shouldn’t have to check 5 different online maps to check out what’s happening in my area.  I certainly shouldn’t have to learn how each one operates! (look I used bold and underline I must be serious)

I can’t help wondering why no one is trying to get a national government mapping organisation whereby everyone gets the same online mapping frontend/interface but is in charge of their own data. A bit like the Planning Portal but for publishing data not just receiving it.  If people think this can’t be done, I think people working on the EU’s INSPIRE directive might say otherwise.

What I would like to see is web mapping become central not just an after thought to local and national government websites.   Yes there is a place of localisation (depending on usage, tourism, history etc…).   But a national web mapping site is needed so we can seamlessly browse geographically adjacent datasets.   This would be great not just for residents but also policy makers, politicians and professionals.  Imagine for example, seeing planning statistics and local plans for neighbouring authorities on one seamless map?   Local councillors could see how neighbouring areas with similar demographics are doing.  Think how easy consultation with neighbouring councils could be!

Enough of my musings, I’m off to make a map for a local authority in Iraq….

Related Posts

 

ESRI UK Conference – Day 2

ESRI UK Conference – Day 2

UPDATED: Various errors (spelling, grammar etc..) have been changed since the original was posted


I stayed on the Geo-Futures track throughout, don’t get me wrong there was great choice out there.   I would have been interested in seeing more, but on balance there was more of interest to me on this track.   Last year I did jump around, this year I thought I’d try staying where I was, both approaches worked for me.

The following is quick run-down of what stood out for me, it’s not a review or judgement on anyone’s presentation.   They were all good and very interesting.

  1. “The Transition to a Low Carbon economy” – Emily Martin, ESRI UK – More detail on this subject from day one.   She gave me some interesting ideas that I want to explore further, I love that GIS can help us understand and assess the effectiveness of new technologies.   Whilst giving us real monetary values and pay back times!
  2. “Games and the City”My presentation which you can find out about here.
  3. “GeoDesign: Asset management in the Public Forest Estate” – Tony Farndon, Forestry Commission – I have great respect for anyone that manages forests (call it a family thing). I was interested in their use of 3D visualisation to see what future landscapes would look like with new plantings (I have some ideas about this to…).
  4. “Data in the Public Domain: Is Anyone Ready?”Lisa Thomas, The Coal Authority – As a Durham Postgraduate Alumni, (Geographical Information for Development anyone?) I am aware of subsidence and old mine shafts (the library and much of Durham’s campus is on an old mine!) so I found this quite interesting.  Dramatic pictures aside, there was a valuable point to be made about releasing their data to the public.   As there needs to be a lot of knowledge required to understand some of the implications of the data that they hold.   Personally I think that without educating people, no one is ready for this kind of data.   Her points also linked quite nicely with Steven Feldman’s presentation. Also her interesting insight into the world of INSPIRE was an eye opener for me (being in the private sector) and now I understand why @alexrcoley couldn’t make it (too busy!).
  5. “The OS Road Map” – Dave Russell, Ordnance Survey – Good stuff from the OS (as always really), interesting to hear about where they think the money is, as well as upcoming 3D and other products.
  6. “Open Data – is it like giving a kid an AK47”Steven Feldman, Knowwhere Consulting – I did attend last year’s presentation entitled “Navigating in turbulent waters”.  I’ve not really spoken to him before this year but I certainly have heard of him!   No bad things, of course, but he is one these presenters with a style that you remember.  Personally I wouldn’t call it provocative or controversial but it comes close for some I guess.   This presentation dealt with the question (in my mind at least) of whether OpenData should be open to everyone.   In that, he meant that perhaps only professionals who use a rigorous and professional approach to analysing and publishing data should get to have a go.   I may have paraphrased it a bit, but using the police.uk fiasco as an example of how not to do GIS was a good example.   He also illustrated this with a ‘police crime map’ of where he lives showing a large number of crimes occurring right next to him.  

    But if you don’t understand the context of the data or how it is displayed (and it can’t just be some minor piece of text disclaiming the data) the information is useless. Other than for journalists! I would like to add, that whilst the data is made “anonymous” by a particularly stupid method, the data isn’t very anonymous in areas of smaller populations (or small streets).  Anyway, it was a very good presentation, and you can tell it got my brain working a bit!

  7. “ESRI UK Online services “ – Dave Bayer, ESRI UK – Well I was glad our hosts had technical issues (made me feel better about some of my presentations issues), but I’m glad to see that ESRI is not standing still on the online front.  It’s a shame they couldn’t access the server.  But I’m looking forward to the OS opendata base maps!
  8. “Using new technologies to deliver savings in the Public Sector” – Duncan Hill, Europa Technologies – Interesting look at joined up approaches and integration of cloud mapping services into peoples systems.   My company is not really big enough or doing the right jobs to benefit dramatically from this kind of approach (we’re on a per job basis), having said that someone else does manage our maps on a regular basis, thank you ESRI for including that in ArcGIS 10.
  9. “Real Time GIS” – Charles Kennelly, ESRI UK – The resident DJ (didn’t he play S-Club7 as an opener at last years conference?) also had technical issues, well if you put us far away from anyone else at the conference maybe this will happen!  That didn’t faze him as he whipped out the latest cutting edge technology, pen + paper!   Who knew you could do such things without a battery!   I honestly found it quite refreshing to see a presentation done a flip-chart.   Joking aside, he made some very interesting points about how computing power has come on so far that real-time GIS processing is a reality and that we should be thinking about it now because it is coming.   He also warned against the dangers of focusing attention on the finished mapped product as being the ‘source data’.   He suggested our attentions should be on the process to we used to get to those mapped products.

    He’s right and I think I’d like to talk to him further about this in relation to what I do….

So, I hope that was of some interest to people, it’s one of my longest blog posts but conferences always get me fired up and thinking about new approaches.   Which considering the title of this year’s conference is quite appropriate!

Let me have your feedback, if I’ve missed anyone out or have additional observations or information please leave a comment below.   I will add your thoughts to the appropriate parts of the post as well.

ESRI UK Conference – Day 1

ESRI UK Conference – Day 1

UPDATED: Various errors (spelling, grammar etc..) have been changed since the original was posted


Julie Pearce presented an interesting piece on GIS use at the Animal Health and Veterinary Laboratories Agency (it trips off the tongue doesn’t it?).   It is good to see how people are overcoming inconsistent and disparate data sets from a variety of sources/agencies.   The over-riding impression I got was that although complicated in implementation the concept is very simple.   I too wasn’t surprised by the level of farmer involvement and enthusiasm for the technology.  After all farmers are businessmen and women first and foremost!   They want, like all of us, systems that are simple, easy-to-use and cost effective.   I would have thought the main ‘fly-in-ointment’ here is a lack of provision of true (not just silly 1meg) rural broadband.   Another agency that should be lobbying government to properly fund a national fibre optic broadband network (forget urban vs rural it is a national issue!)?

Rowan Douglas from Willis Research Network tried to convince us that the insurance industry wasn’t evil!  After listening to him I’m almost inclined to agree.   It’s all about managing risk and then how society as a whole and through the wonders of insurance companies pays for things, when nature or other events interrupt our lives.   When it’s put like that you realise at the same time, that he’s correct and that somehow it doesn’t quite tie in with why our insurance companies put up our premiums without explanation.   Is their industry in need of an image makeover, I wonder?  Probably.

However his presentation did include a lot about analysis, modelling and trying to predict the impacts of disasters on people and the economy.   Some great stuff going on which I’m sure adds to the sum of human knowledge about how to cope better with ‘events’.

After that there was some really good thinking going on in the Army with a presentation from Major Andrew Williams of the Joint Aeronautical and Geospatial Organisation (JAGO).   Joined up thinking and connecting people to easy-to-use and interpret systems.  So all tiers of the Army and partners can be working from the same page.   Again it is a simple idea but difficult to implement in detail.  Portable and ruggedized servers for in the field use, using limited bandwidths.   I wondered if is there something the rural broadband campaigners can learn from here?

The ESRI UK people did what they do best with working presentations on the Crown Estate, GIS for a Low Carbon Economy (I’ll come back to this on day 2) and ArcGIS.com with the RSPB and map sharing.    Good demos I especially enjoyed the Low Carbon one as I can see this type of analysis being ever more important.

The Capability Track after lunch was a bit blurry for me as this was when I had my 10 minute presentation.  I would like to have gone to the technology track as well and learned about ArcGIS 10.1.   But listening to my fellow presenters I learned a lot and was most impressed with their presentations.  Anyway here is what I remembered please leave comments if you have anything to add or point out where I have missed something or got it wrong:

  • South Yorkshire Police – DI Gary Williams presented how they were “delivering more for less”.   Obviously, nearly everything that the police does can be located spatially.   Making it easy-to-use was imperative for them and hence the web based interface that was developed.   Now officers are able to have more information about where they are and why there are there.   Some detailed calculations of cost savings were also done as well.   It is odd to see how much a crime actually costs across the board including victim costs, but there you are.   Personally, I just want the Police to get on with it and not worry all the time about cost.   I want them to spend time doing their job correctly and not rushing through to save money, but maybe I’m in a minority…
  • Birmingham City Council – Presented by Stu Lester “Corporate Data reuse and Benefits”, okay I’m sure he doesn’t just want me to remember the Terry Gilliam stills from Monty Python right?  Fortunately I remember other things to about how bringing various data silos together can have tremendous benefits.   It’s important in a council the size of Birmingham too.   I talked to him more at the event as well and he’s got a lot of good ideas I just hope he can maintain that level of drive and enthusiasm!
  • Garsdale Design Limited (my presentation) – “The 10 Minute City” (video available here) about the potential of tools like CityEngine from Procedural to help us with master planning. I hope the video went down well click here to the article about my presentations.
  • RBSI – “Scene and Not Word”, another presentation from the insurance industry, surprisingly a lot of what they do to assess risk, claims etc may not have required a GIS system!   Fortunately they did choose GIS, and wow, do they have an awful lot of information about where we live and work!   Again most of us might not like the insurance industry until you need to claim from them (hopefully successfully).   Their GIS seems to hold lots of information and helps quickly and efficiently assess risk.   The automation of some of the processes of insuring properties can only make the job easier.   Although now all I can think of is “the computer says no” lady from Little Britain…
  • Wales and West Utilities – Talked about how from a standing start they managed to get up and running within timescales a GIS management system for their new company and network purchased off of the National Grid.  Honestly, wow! To start with no offices and rented IT, I think their achievements are pretty amazing.   Especially considering the legacy systems they will have inherited (pipes and systems).   What was encouraging was seeing how they weren’t just doing the bare minimum but looking at using the knowledge of where all their repair teams were to properly and effectively allocate them to the right place if an emergency came in.    The passing of data seemed very important to them too.   

Following on from the Capability and Technology Track we all came back to see Michael Palin who was late (I hope he wasn’t lost) so Walking with Wounded presentation came first.   I like that ESRI does this by getting someone in that is not to GIS industry focused for some inspirational presentations.   A very good and moving presentation you can find out more by visiting there site.   Next year its Everest!

Walking with the Wounded –

Then the star of the show Michael Palin came on for an ‘informal chat‘ with Richard Waite (ESRI UK Managing Director) this was good and everything you expect from Mr Palin who is now the President of the Royal Geographical Society.    However I wish he had come on first as Walking with the Wounded was a really tough act to follow.

Proof that he did make it: 

Michael Palin – silly walk = a little disappointed

There was an evening meal and awards ceremony, it was nice to talk to people informally and our table was pretty lively, I think we finished the wine….

Before the wine…

Digital Elevation Models – OS OpenData to SketchUp : A workflow

Digital Elevation Models – OS OpenData to SketchUp : A workflow

Update: Geolocating the DEM in SketchUp has been solved see at bottom of page!


So you have SketchUp and you’ve heard wondrous things about the UK’s Ordnance Survey OpenData?!    In particular you hear there maybe some contour/elevation models out there for free as well!   This quick workflow guide shows you how get that elevation model into SketchUp so you can plan horrible developments in undeserving places (I’m a planner so I know…).

Prerequisites:

  1. Windows Vista (I’m sure XP, Windows 7 etc.. all work as well)
  2. SketchUp Pro (although you can import DEMs with the free version) http://sketchup.google.com/intl/en/product/whygopro.html
  3. MicroDEM – follow the install guide to get yourself up and running http://www.usna.edu/Users/oceano/pguth/website/microdem/microdem.htm
  4. OS OpenData, in particular the Land-Form Panorama dataset, select the download option enter your details (it only requires your email address) and then wait for  523MB zip file to download https://www.ordnancesurvey.co.uk/opendatadownload/products.html
  5. Google Earth (to check your DEM is georeferenced properly)
  6. You need a basic working knowledge of MS Windows, SketchUp and some file management skills. 

Once downloaded you need to know what OS Grid square you want to import.   You can read the wiki article or I have used StreetMap and once you’ve searched for your location you can look just below the map and it says “Click here to convert coordinates” on this page LR seems to relate to the OS grid.
http://en.wikipedia.org/wiki/Ordnance_Survey_National_Grid


The Walkthrough

Double click on the ‘panorama_gb.zip’ file and navigate to this directory ‘\DTM\ASCII\data’.   Yes there are other types of data (contours as DXF and DEM as NTF) but this is what worked for me.

1 – Extract a tile from the directory name corresponding to your chosen OS tile.  In this case we’re going to use Sedbergh, Cumbria tile which is under ‘\DYM\ASCII\data\sd\sd68.asc’.  You can copy it to any directory but in this instance I tend to copy it to the ‘mapdata\DEMs’ directory created by your installation of MicroDEM (you did install it right?)

2 – Load up MicroDEM and then click File –> Open DEM now navigate to the ‘mapdata\DEMs’ directory and select the ‘sd68.asc’ file.

3 – MicroDEM will ask you to pick its projection parameters as its OS OpenData my guess is that these settings are okay and then click ‘Mercator’ instead of ‘OK’

4 – The DEM should load up and look something like this :

Nice Colours?

5 – If you want to get rid of the legend and scale bar and any grid that may appear right click on the image and select ‘Legend/marginalia’.  Uncheck the boxes and click the ‘Grid’ button and select the option ‘neither’

6 – Click ‘OK(Close’ and say ‘yes to redraw… I often get errors and warnings at this point which I ignore….
Now click on the menu heading file again and save this as a DEM and in particular a USGS ASCII one:

7 – Once it is saved close MicroDEM and open up SketchUp…  I’m assuming you will import into a fresh new SketchUp Model, so click on the File menu and select import.

8 – Choose the file type DEM (*.dem, *.ddf) and find that file you saved in MicroDEM, before you click open click on Options:

9 – Here you can see I’ve entered 20000 points to import the lower the number the less detail for this tile 20000 as suggested by Chris Fullmer’s tutorial seems good.  I suggest you experiment with this to get what you want though!  Also I’ve check ‘Generate gradient texture’ this is entirely up to you, I suggest you first try with and then without.

The DEM should be imported and the axes, click the zoom extents button to check it’s all there:

10 – Now to get rid of all those lines, double click the DEM (to edit component) and select all of the DEM (keyboard shortcut : ‘ctrl-a’).   Now right click the selected DEM and  click on ‘Soften/Smooth Edges’:

11 – As per Chris Fullmer’s suggestion slide to around 90 degrees and check both boxes (Smooth normals and Soften coplanar)

12 – Et voila! You now have a terrain model for placing your models on!

mmm smooth elevation model!

13 – One important thing to note this is not GeoReferenced.  I haven’t figured out why SketchUp doesn’t load the DEM in the correct place.  If anyone has any suggestions please tell me (via Twitter or otherwise) and I’ll add it to this tutorial.

Geo-Reference (or Geo-Locate) your DEM

14 – First you need to know where your DEM is in Latitude and Longitude you can do this by going to nearby.org Coordinate Convertor and putting in your OS tile number (in this case SD68), I suggest you select output as Coordinate Conversion only:

15 – You are now going to copy the Lat and Long coordinates into SketchUp so leave this webpage open and….

16 – In SketchUp click on the menu ‘Window’ then ‘Model Info’ and select ‘Geo-Location’

17 – Give the Country name and location something meaningful…. and copy and paste your latitude and longitude’s full number (and letter after) in the appropriate places.
Now to test it press the Preview in Google Earth button:

18 – If you’re computer is up to it you should see the DEM appear in the correct location in Google Earth, it may take a while to load though so be patient!

DEM Placed in GE

Mapping Cumbria – Designation Datasets

Mapping Cumbria – Designation Datasets

Listed Buildings Near Sedbergh

So in my travels to find GIS data for Cumbria I’ve been looking at national data sets.  One interesting site I’ve found is from English Heritage who allow the download of ‘designation datasets’.

The following datasets can be downloaded in zipped folders containing ESRI shapefile format files:

  • Listed Buildings
  • Scheduled Monuments
  • Registered Parks and Gardens
  • Registered Battlefields
  • World Heritage Sites
  • Protected Wreck Sites

Ignoring the ESRI centric nature of the data for the moment this is a surprising lot of data to release, being a map man though I’m very happy!

As you can see from the screen captures (above and below) I’ve managed to extract the dataset and make it into something usable in Google Earth as a KML file.  Of course you can equally use ArcGIS Explorer.

I’ve used this point shapefile data, excel and my knowledge of Images of England website to create clickable points map that brings up the building listing and if available the photograph.   It’s really using the Listed Buildings ID and combining it with a search term to link through to the appropriate page on Images of England.

I’m slowly building up quite a dataset of Cumbrian related material, and combine that with the Portable Antiquities Scheme live datasets you can make some quite useful maps.

That’s a lot of listed buildings in Cumbria!
Opinion: PSMA and what it means

Opinion: PSMA and what it means

Well I’ve been away from this blog for awhile, please accept my apologies, life it seems can get in the way (and I’m glad it does!).   So today’s post on the Ordnance Survey blog finally starts to answer some of the frustrating things about Geographic data in the UK.

The Ordnance Survey is a great organisation that creates what I think of as the best paper mapping product anywhere, both in terms of looks and accuracy.   Other countries I’m sure look at the UK and its quality of mapping with envy.   However there is a little stain on this mapping perfection and that is licencing and something called derived data (see bottom of this article).

Working for local planning authorities I always wanted more access to data and to access it my way.   As a planning officer Google Earth/Maps was great and if only StreetView was available to me as well….   One of my roles in one authority I was working at was to develop the front facing planning pages (as well as coordinate the back office).   Forget text searches, I wanted maps!   Asking IT whether I could do something always started with a cost implication and then a licensing implication.  It was here I learnt that some of the data that the OS relied upon for its maps was collected by Local Authorities who then had to buy back their mapping (as MasterMap), think address points, new developments, rights of way maps etc…
 
So this new post on the OS blog (What the PSMA really means) offers us all some light at the end of the tunnel, and I think regarding derived data, we may see some kind of happy compromise in regards to public use of locally held data like Rights of Way mapping, for PSMA stands for Public Sector Mapping Agreement.

“all organisations will have access to the same data meaning there will be no disparity between the datasets available to those in Central and Local Government and Health”

and

So, let’s be clear. Under the terms of the PSMA, all 800 members will be able to share Ordnance Survey data, and data derived from it, with any other organisation as part of their ‘core activity’.
This is basically the framework that exists under the current Pan Government Agreement (PGA) today.
For example, the Rural Payments Agency uses OS MasterMap to help it calculate the grants owed to farmers. As part of that job (or ‘core activity’) they can freely share that data with the farmers they are working with, even though the farmer is not a PGA member themselves.
This principle will also extend to organisations that want to publish key information online, like the location of public amenities, using a web mapping API.

Lets hope Local Authorities now act on this new agreement and encourage its residents to access their geographic data and create new ways of using it.   You never know this might be a cost saving for local authorities!

Not Sure what “derived data” is? Well here is the an extract from The Guardian that might help shed light :

Q What constitutes data ‘derived’ from Ordnance Survey data?

A Simply put, Ordnance Survey derived data is any data created using Ordnance Survey base data. For example, if you capture a polygon or a point or any other feature using any Ordnance Survey data, either in its data form or as a background context to the polygon/point/other feature capture, this would constitute derived data.

This is often an eye opener to people, Local Authorities are afraid of sharing data sets like Rights of Way maps, that they themselves create maintain and actually keep open on the ground physically!   The OS has in the past claimed ownership (and may still do) of these maps and then sold them back to the Local Authority!  All because the Authority uses an OS basemap.  

OS Blog post : http://blog.ordnancesurvey.co.uk/2010/08/what-the-psma-really-means/
Update: New post at Knowhere bloghttp://knowwhereconsulting.co.uk/how-complex-can-this-derived-data-thingmy-be/