Sunday, June 15, 2014

Creating Map Tiles (Part 3) - Generating Tiles from a TileMill Project Using MBUtil and PowerShell

This post is a continuation of Creating Map Tiles (Part 2) - Creating a Sales Territory Boundary Set Using TileMill, where I prepared my transparent boundary set in TileMill. In this post I use PowerShell to call TielMill and MBUtil to generate the map tiles.

As I mentioned in Part 1, for much of the process of creating map tiles I relied on an excellent series of posts by Pedro Sousa on his blog, Pedro's Tech Mumblings. In Part 2 of his series, he walks you through how to export your TileMill project to a MBTiles SQLite file and then use MBUtils (a command line MapBox tool) to export the map tiles. Below, I've taken a the same approach but created a PowerShell script to automate the process.

In order to perform this process manually or using the PowerShell script below, you need to install Python with SQLite and MBUtils. Pedro walks you through this, however I performed the following steps outlined in the accepted answer to Is there any way to use mbutil on windows? posted on StackExchange.
  • Download and install ActiveState Active Python 
  • Install MBUtil by downloading and unzipping to C:\mbutil-0.2.0.
  • From a command-line navigate to C:\mbutil-0.2.0 and run " install" to install mbutils
At this point the pre-requisites for the following PowerShell script exist. 

function CreateTiles() {
        [Parameter(mandatory=$true)] [string]$project,
        [Parameter(mandatory=$true)] [string]$inputPath,
        [Parameter(mandatory=$true)] [string]$outputPath
    # delete .mbtiles, .export and .export-failed files
    remove-item  "$($inputPath)$($project).*"
    # delete directory
    $pngPath = $outputPath + $project
    if (test-path $pngPath){
       write-host "Deleting $($pngPath)... this may take awhile." 
       #remove-item $pngPath -recurse  # very slow
       $fso = New-Object -ComObject scripting.filesystemobject
       $fso.DeleteFolder($pngPath, $true)
    # Create mbtiles file. Requires TileMill.
    # changed the following so that the executable's path doesn't 
    # contain spaces (i.e. using the call (&) operator is optional)
    cd "C:\Program Files (x86)\TileMill-v0.10.1\tilemill\"
    $mbtilesFile = "$($inputPath)$($project).mbtiles"
    & ".\node.exe" ".\index.js" "export" $project $mbtilesFile "--format=mbtiles"
    # Create PNGs from mbtile file. Requires:
    # Python with SQL Lite (install ActivePython);
    # MB Util (download mbutil zip from
    python "C:\mbutil-0.2.0\mb-util" --scheme=xyz $mbtilesFile $pngPath

CreateTiles "State_And_County_Boundaries" "C:\SHPs\" "C:\MapTiles\"

The first step in the script removes the existing MBTiles files, if they exist, which will block the creation of the new file. It also deletes the existing path of previously created tiles. This is necessary because I don't want to have any old tiles mixed with the new (remember not all tiles are generated within the bounds set in the TileMill project because my map has a transparent background). Depending on the number of tiles generated in a previous run, this can take awhile.

The second step is what you can do manually from TileMill by clicking the Export button in the main toolbar and selecting MBTiles. It calls TileMills index.js file and is documented here on the MapBox site.

The final step calls the MBUtils executable to produce the map tiles in a set of folders based on the XYZ tiling scheme.

The files now just have to be copied to a web server where they can accessed by the Bing Maps API. The following, modified from a function taken from another of Pedro's blog posts (taken in turn from a blog post by Alastair Aitchison), shows how the tiles can be called from the Bing Maps API using JavaScript:

var tileSource = new MM.TileSource({ 
    uriConstructor:  function getTilePath(tile) {
        // only created custom tiles for zoom 0-12
        if (tile.levelOfDetail <= 12) {
            var x = tile.x;
            var z = tile.levelOfDetail;
            var yMax = 1 << z;
            var y = yMax - tile.y - 1;

            return "Images/Tiles/BaseMap4/" + z + "/" + x + "/" + y + ".png";

var tileLayer = new MM.TileLayer({ mercator: tileSource, opacity: 1 });

// Push the tile layer to the map

One of the modifications that I made to the above prevents the call to retrieve tiles that were not generated for certain zoom levels. This will prevent the server from having to generate a multitude of 404 error response codes as you zoom below the level for which tiles were generated. Another modification that I made to my own code (not included above) was to add a version argument to the end of the URL which is retrieved from a web service call to the database when the page initially loads. I use this argument to force the fetching of new tiles when the tiles have been replaced on the server.

The Tile Layers Used with the Bing Maps API

Using the PowerShell script above, I have been able to generate three different, fairly static boundary sets, consisting of 3 to 5 layers each (resulting in around 100,000 tiles for each set). Along with the script from Part 1, Creating Map Tiles (Part 1) - Exporting MapInfo TAB Files to ESRI SHP Files Using PowerShell and MapInfo, I have been able to automate the creation of an additional two boundary map tile layers that have to be generated monthly, due to changes in the boundary definitions.