Browsed by
Month: October 2018

Loading the OS Open Zoomstack Geopackage into ArcGISPro (with OS styling).

Loading the OS Open Zoomstack Geopackage into ArcGISPro (with OS styling).

OS Open Zoomstack in ArcGIS Pro with symbology created by the Ordnance Survey

This is a quick write up that’s related to the blog series I’m doing on custom symbology in ArcGIS Pro.  Well sort of, it just so happens to be the perfect dataset to use to use for UK based mapping projects where you don’t want to use a costly licenced dataset (maybe the forthcoming Open MasterMap may change that?).   Now, I know I normally write about 3D and CityEngine related stuff but I do love a good 2D map as well!  This post assumes a simple working knowledge of ArcGIS Pro.   I’m considering making this a video as well so you can see the entire process.

First you need to download the OS Open Zoomstack as a ‘geopackage’ don’t worry whilst ArcGIS Pro doesn’t support direct editing of a ‘geopackage’ it does support native reading of it. * there’s some discussion in the comments at the end of this post, if you’re reading on the front page click here to view

Fun fact: the GeoPackage is an ‘open format for geospatial information’

You can read about it here.

A word of warning this is a UK national dataset, so naturally the size is large (10.1 Gigabytes!).

To help as well I suggest you download the Ordnance Survey’s custom symbology for this dataset which you can get from github.

As you can see there are some instructions on what to do with style sheets if you’re an ArcMap user in that PDF linked above… well I do not use ArcMap much anymore more so I’ve pretty much made a complete transition to ArcGIS Pro so here’s what you do next.

Step 1: New ArcGIS Pro project and then ‘insert’ a new ‘map’.

Step 2: Convert the Geopackage into a File Geodatabase… as far as I am aware you don’t need an Advanced licence or FME or the Data Interoperability Extension (if I’m wrong comment below on this post) you can drag in each layer manually into a Map in ArcGIS Pro and then right-click the ‘export data’ function.  Or better yet, you can use the copy features GP tool (using the Batch function). *I’ve created a toolbox with two tools that simplifies this process for me.  I will share this as a separate blog post soon…

Step 2: Remove the prefix ‘main_’ from all the feature classes you imported into the new file geodatabase, otherwise you can’t use the lyr file on them….

Step 3: Find your “OS-Open-Zoomstack.lyr” (link to download it here) and drag it in to your Map, notice all those red “!” marks, this means it can’t find the data these symbols are linking too.   Click on one of these red “!” to fix them all.   It will ask you where the data layer is located ( in this case ‘names’).  Find the data in the new file geodatabase you created.

Step 4: Well it should all work and all those “!” should have gone and you have a nicely symbolised OS Open Zoomstack data set courtesy of the nice folks at the Ordnance Survey.

A final note this workflow unbelievably helped me find where Esri hid the ‘repair data’ function went, basically they built it into the “!“… d’oh.

Coming up in a future blog post: How we can use OS Open Zoomstack with our hand drawn custom symbology.

CREATING AND USING CUSTOM ARCGIS PRO SYMBOLOGY – PART 2: Image Preparation and Processing

CREATING AND USING CUSTOM ARCGIS PRO SYMBOLOGY – PART 2: Image Preparation and Processing

This is the second part in a series of posts on my journey to create custom symbology in ArcGIS Pro. Inspired by John Nelson this post is primarily a reminder to myself about how I did it.  Actually this whole blog is my personal notes made public (so yes you can do this differently and achieve the same results), I’ve done on more than one occasion a Google Search for a solution only to find a post I did about it ages back…. doh.

So you have the kit because you read my previous post, so now what?  Well we need to do some drawing and painting!   Then we will process those drawings by scanning them modifying them in a image editor (like GIMP or Adobe Photoshop Elements) and saving them in a nice and organised way.

The next part of this blog series will deal specifically with each symbol type (lines and points) and I will also cover watercolour swatches I’ve created.  This post will get your drawings and sketches to an image editor ready for use in ArcGIS Pro.

Inspiration doesn’t have to come from specific texts or Cartography books (but it helps)

There’s two steps to my process (you can of course do it differently!), first read some books and get some inspiration.  The I have a scrap page which I start doing test runs on symbology and lines a bit like the image below…

doodles can turn into something more digital later…

Once I’m happy with something I will add it to my grid paper I’ve created which you can download here (below).

Don’t be afraid to experiment, don’t think it’s not good enough!  Lots of people say they can’t draw, what I think they mean is they’re not confident enough to draw for other people.  Besides which some mistakes or badly drawn things might look just right in the correct context.

Some look okay, some I’m embarrassed about to be honest, I’m trying to learn not to care too deeply about what others think!

So we have a paper grid of hand drawn symbols (no you don’t need to till them up).  Notice I’ve used white paper for this the whiter and cleaner the paper the better I can edit them later.

Now I would scan them or take a photo, just be warned that take a photo you need really good light, no shadows and try and make sure them are photo’d flat so you don’t have distortions to fix later.   I recommend scanning them if you can as this will keep things nice and consistent.  As a rule I scan at 200dpi or more as a minimum, I do scan as colour even if black and white scans just so I can choose later what’s done with the image also not all pens are black!

I scan the whole A4 page (sorry people from USA we’re metric around here!), and it will look like this in Adobe Photoshop Elements.

Ready to process!

Step 1: Duplicate the layer 

Step 2: Delete the background layer

Step 3: Use the crop tool and choose your symbology

Step 4: Use the Magic Wand selection Tool (uncheck contiguous and adjust the tolerance to adjust) to select all the white space on the image.

Step 5: Go to Edit menu and select ‘Cut’ or use the keyboard shortcut ‘Ctrl-X’, now you have a small image with a transparent background! Use the crop tool again to adjust the image size to your requirements (I tend to crop just to the image, but some like to make it precisely square)

filename “draft_v1_v2_final_final_v0.png”

Step 6: Save in a folder with a nice file name but save as a PNG file which supports transparent backgrounds.

So that’s it! The next blog post in this series (coming soon) will take you through symbology a types (lines or points) and show you how we get it working in ArcGIS Pro.  I will also do a separate blog in the series for coloured scans of my watercolours swatches…

Creating and using Custom ArcGIS Pro Symbology – Part 1: Stuff I use to help me.

Creating and using Custom ArcGIS Pro Symbology – Part 1: Stuff I use to help me.

You’ll have seen on social media I’ve been ‘playing’ with techniques to create custom mapping styles.   This is a direct result of me attending EsriUK’s Perth conference and getting all inspired by John Nelson.

Firstly if you haven’t read or seen John Nelson’s blog, go look at it now (I’ll wait): adventuresinmapping.com There’s more obviously around but I’ve been using ArcGISPro for all our 3D GIS and Esri CityEngine content.  However I’ve wanted to do something more artistic, more in-depth and one that pushes my comfort zone a little.   John Nelson’s cartography using ArcGIS Pro are a master class in the art of what’s possible.

So this is the first blog post in a few and maybe even a video (yes I do that occasionally) on the lessons I’ve learnt from using ArcGIS Pro to make some unique maps that look hand drawn (and sort of are).   Can you do this in a product like QGIS?  Yes I think you probably can, can you apply some of what I write here to QGIS, I hope so!

Equipment and preparation

I’m aiming to make this repeatable and consistent, therefore I’ve done some preparation which I will share with you here.  You don’t have to purchase anything of course! I just wanted to record what I had done here.

The glorious Staedtler triplus® fineliner 334

Pens. While we will be using ArcGIS Pro we will also need to do our own drawing, and no I don’t believe you have to be very good at drawing just consistent and willing to try new things.  I’ve settled on the Staedtler triplus fineLiner 334-9 a nice pen with a good line quality. Unsure?  Go to a good pen shop and try some out, for me we’re doing symbology of lines and symbols so it needs to be crisp and good for scanning.

Winsor & Newton Cotman Sketcher’s Pocket Box set of 12… is nice to have and small!

Paints.  Well I like watercolours and I also wanted to replicate some what John Nelson has done, so I’ve chosen a simple set we got my eldest child from Winsor & Newton.

Paper. Honestly 80gsm everyday paper for simple pen work it has a nice crisp white perfect for scanning.  If I was to improve it so you didn’t see anything on the back or to stop it curling, 90gsm is better.  For watercolour work obviously you need watercolour paper (190gsm to 300gsm) just ensure any scanner can handle it, if you intend to use a scanner!

The grid. I wanted to inject some consistency in to the process and left to my own devices just sketching on a piece of paper will get messy pretty quickly so I created a series of A4 grids for the different symbols.  I’ve made this a PDF which you can download below (see resources heading below).

Scanner or Camera. I’m using an iPhone 7 camera and the dropbox app to quickly upload to my PC (you could equally use a cable or other app).  For the most part I would recommend to use a scanner this allows for clear distortion free scans/images of your drawing.   If you use a camera I find without amazing light quality when taking it you will end up doing additional processes to clean and brighten your image.  The scanner i’m using at home is an Epson Stylus Office BX610FW, I can scan directly to a memory card or via wifi straight to my PC using their Windows program.

Sedbergh-on-sea created with hand drawn elements placed using ArcGIS Pro

Software. Well I’ve been using Adobe Photoshop Elements, but you can also use GIMP (which is fantastic!).  Oh yes and ArcGIS Pro….  

I like it.

Books and inspiration. Well John Nelson and his blog I’ve already talked about but I’ve been looking at a wide variety of books and maps to see what works.  I don’t 100% want to copy (especially if newer material!) but also you don’t have to re-invent the wheel..  From my perspective I really like “Great City Maps”, but then I’m a sucker for urban mapping, I recommend finding a style your’re passionate about and trying to replicate elements of it.

Other publications are available (this is not an endorsement)

Conclusion. So that’s what I’ve been using, I haven’t finished everything yet but Part 2 of this series will look at a workflow for the various elements of a map in ArcGIS Pro you may want to replicate.  I haven’t decided whether to do one giant post about all types of symbology creation or do individual posts for points, lines, polygons etc…

RESOURCES:

Kindle Fire – ‘Kids Edition’ freeing up storage (& deleting 300+ Minecraft PE Worlds).

Kindle Fire – ‘Kids Edition’ freeing up storage (& deleting 300+ Minecraft PE Worlds).

Why is file management so hard on these things?!

Firstly a gripe, no one told me parenthood would be basically unpaid IT support.  On the one hand I want my kids to be able to use technology responsibly and be able to confidently manage their digital lives.  On the other-hand I want to tightly control their exposure initially and ensure they have the right experiences without being put off with the unsavoury aspects of say the internet.  I firmly believe that …

Computers and technology are a wonderful thing, but it is imperative we give our kids the right tools and knowledge to cope with it all.

That’s why a couple of years back we bought Kindle Fire tablets (Kids Edition) for our two eldest children.  I’ve got to be honest these devices are great and have lasted.   The appeal of a managed kid-centric/walled garden  environment seem a good compromise.  All the apps and games they want (within reason) allows them to explore their likes without the parental fear of massive bills every month.  Eventually I also bought them Minecraft Pocket Edition, which they’ve used almost consistently.  For those who don’t know Minecraft is the digital equivalent of Lego (okay I know you can get Lego computer games too).   It’s also more than that you sort of get introduced to programming because combing blocks and resources produce different effects and items.

I know some frown at this sort of thing as somehow ‘less creative’ or encourage a child’s isolation.  Firstly my kids sit next to each other talking about what they build, secondly I’ve bought a Minecraft realm where they can join up with their cousins (who don’t all live here in the UK) to build and chat, it’s the opposite of anti-social.   Furthermore take a look at where our economy is going… yes we are still building physical stuff but we are also increasingly creating digital products.  You only have to look at the rest of my blog to see I trained as a geographer and town planner only to end up creating 3D digital cities for all sorts of uses (mapping, analysis, military and entertainment.).  I firmly believe some of our kids need the skills of building digital environments.

Oh dear, I digressed didn’t I? This was a blog post about freeing up space on those damned Kindle Fires!

My method of freeing up space on the Kindle Fire (Kids Edition)

Anyone who has a kid with these devices and the Kids Edition probably knows that the hardware memory allocation wasn’t great, and kids being kids will end up installing ALL THE APPS.  Yes you can uninstall them (by long clicking an application/game icon and clicking ‘remove from device’), but sometimes that doesn’t seem enough.    

So you check memory usage and you see space taken up by applications and pictures, oh and the system.  Well you can do something about some of those applications and pictures but then there’s this huge bar representing ‘Miscellaneous‘ to be honest I’m still not sure what all of it is (probably app related essentials) but basically this bit seems to get larger and can’t be simply deleted.

You search the internet forums and lots of people have this issue, some use special clean-up applications installed (logged on as a parent) on to the kids device…. others recommend deleting games/applications cache.  Well I’ve done this stuff before but for one of my kids tablets it didn’t work.  So this is what I did:

BEFORE PROCEEDING YOU HAVE TO ACCEPT THAT THE RESPONSIBILITY IS YOURS FOR FOLLOWING WHAT I DID.  I CANNOT GUARANTEE THIS WILL WORK FOR YOU OR THAT IT WILL NOT DAMAGE YOUR DEVICE  (having said that it should be fine, but if you are unsure don’t do this). Also these instructions assume you know something about Windows, file explorer, and file management.  A suggestion for those nervous about deleting files, create an archive directory on your PC with appropriate folder names and copy files to them before deleting on device making sure your record where they had come from).

  • Switch on tablet and login as your child
  • Plug in the tablet to your Windows PC.
  • look in file explorer under “This PC” for an icon representing your Kindle (Windows seems to recognise it as a media device not as a simple USB stick) double click on it.
  • Under the Kindle device you will have two ‘drives’ Internal Storage’ and ‘SD/External Storage’, notice how the internal storage bar is red and looks full… double click on ‘Internal Storage’
  • Navigate to ‘InternalStorage/Android/data/com.amazon.venezia/files’ there maybe lots of files with file extensions ‘apk’ here these are install files for applications and games.   It seems you can delete these and it won’t hurt your tablet (they get downloaded again when you need them).
  • Navigate to ‘imageCache/com.amazon.venezia/’ lots of thumbnails here for apps my kids no longer use I deleted all here EXCEPT for the two generic thumbnail images not stored in a sub-directory.
  • Now there maybe more tips to freeing up space if you have any add a comment to this post and I will add it in here (with proper acknowledgement of course!)
Fantastic but I didn’t sign-up for file managing 300 ‘worlds’

MINECRAFT PE USERS: My kids had created 300 + Minecraft worlds over the last two years which took up a lot of storage, this doesn’t count as ‘Miscellaneous’ in your usage but I think but having to manually delete each world in the Minecraft application seems awful.

  • Navigate to “InternalStorage/games/com.mojang/mineCraftWorlds/”
  • List the weird folder names by modified date and delete all the old ones by selecting multiple items (you know click one file then ‘shift-left-click’ to select multiples)
  • If you need to check you’re deleting the correct worlds in each directory there is a text file with the Minecraft world name in it,

I hope this all helps put your comments below if you have anything to add 🙂

Highlights from the EsriUK Scotland Conference 2018 #EsriUKSC

Highlights from the EsriUK Scotland Conference 2018 #EsriUKSC

Well I always intending on writing up about the EsriUK Scotland conference held in Perth this year.  I’ve said before it’s been a great conference in a great town and this year was no exception.  The speakers were all great with some notable highlights being (this is not an exhaustive list but ones that have stuck in my mind):

EsriUK: The live traffic count demo using a camera from a mobile phone roaming Perth and some machine learning algorithms ‘hats off’ for this amazing live demo on conference wifi.   Shows how all our devices can be connected to scarily powerful cloud services to perform analysis for good… or evil. 

like Siri or Alexa but from EsriUK and for planning applications…

There was also an interesting proof of concept demo from EsriUK (called ada I think) which walked people through a planning application type scenario.

RSPB Scotland and drone use..

RSPB Scotland:  They talked about “Saving nature with drones” and actually a really inspiring use of drone technology for producing up to date high resolution terrain and imagery for habitat management.  Also great uses for bird counts too!   They produced a best practice for using drones guide which probably everyone should read which is here (not sure this is final official version but only one I could find online): Drones for GIS (PDF) Did you also know they have an opendata site? No neither did I!

Powerful and effective communication techniques from Esri StoryMaps…

South Lanarkshire Council: This presentation on “Protecting the quality of the air that we breathe” stood out for me because the presentation hosted by 3 presenters from 3 departments showed what GIS should be about.  That’s joining of information and providing a powerful communication analysis and tool.  Here was a great use of storymap to communicate serious ideas to a wide audience, here’s what can happen when departments talk and work together.  I recommend you look at their storymap here.

That’s a few users…

EsriUK Education: Back to EsriUK again but here on their ‘Education Session’ I have a renewed interest due to being an new School Governor at my local primary school.  Now I’m always of two minds about private large organisations providing free stuff to schools/education.  I can hear the criticism of the open source movement loud and clear that a large company is perhaps abusing it’s position to increase it’s market share…. I know there are free (in money terms at least) programs teachers can use, I know a cloud solution (Esri is giving ArcGIS Online accounts away for free to schools) isn’t ‘true’ GIS in that it won’t teach the intricacies and science that’s required for a better understanding of the power of GIS and associated technologies.  However I know that teachers and schools also don’t have the time and resources to manage installations, perhaps a managed solution from Esri is actually the best solution.  Afterall those who are truly interested will seek out other tools to achieve what they need.  Who among us started their journey into GIS with Esri technology and now use other software and tools to do better?  I bet many of us.  Shocking I know but sometimes all they want IS a map…

So that’s my small write up of the EsriUK Scotland conference, a smaller more intimate and frankly more comfortable conference than the larger EsriUK one in London they do in May…  Perth is a lovely location and I hope they keep it there.

Finally I need to talk about the inspiring keynote from one of the Esri cartographic legends called John Nelson…. well no, I’m saving that for the next blog post as it deserves some nice maps that I made which he inspired me to make.

A helping handle & visual cues for my rules in Esri CityEngine

A helping handle & visual cues for my rules in Esri CityEngine

Sometimes in CityEngine it can be hard to figure out what’s going on.  Whether that’s understanding scope (CityEnginers understand this can get complicated) or just simple metrics.  

Down the CityEngine rabbit hole I go again…..

I’ll often use a combination of ‘print’ and ‘report’ to give me a better understanding of my code at any given point.  What I also do is use bright colours (which have simple RGB/hex colour codes) to indicate whether a part of the code has been reached.  once I’ve confirmed it works I continue the code.

Bright colours can show you interesting metrics but also when you’ve screwed up…

Recently I’ve been working on some code where an understanding of the orientation of an model is important, not just as a world orientation but also relative to the initial shapes scope.  As is the case with most of my work in CityEngine I start to wonder, how would I go about making something more visual for me?   Thus I decided to spend some (okay probably too much!) time creating a procedural protractor.  This allows you to switch between displaying an angle relative to the shapes scope, or the world.

Any excuse to use an animated GIF…

I’ve used the Handle features in CityEngine to make interacting with the attributes associated with this rule simple.  I’m starting to use ‘handles’ in CityEngine to expose attributes for users in a friendlier way.

To sum up for me I’ve found that programming language in Esri CityEngine called Computer Generated Architecture (CGA) is easier to pick up than traditionally programming languages because it is a visual one and by that I mean you create geometries.