Pimp Your Server, Part 2

Note: This is the third part in a series on building your own home-brewed map server  It is advisable to read the previous installments, found here and here.

Last time, I walked you through installing TileMill, and I promised a similar treatment for TileStream and Flex Viewer.  I am a man of my word, so here we go.  Don’t worry – this will be easy in comparison to what we’ve already accomplished.

We’ll start with TileStream, simply because we’re going to have to avail ourselves of the command line.  Once again, you can either plug a keyboard and monitor into your server or use whatever SSH client you’ve been using thus far.

Once you’re in the terminal, take control again (‘sudo su’).  For your TileStream installation, you can follow the installation instructions as presented, except for one detail:  it’s assumed we already have an application we don’t have.  Let’s correct that:

sudo apt-get install git

And then proceed with the installation (don’t forget to hit ‘enter’ after each command):

sudo apt-get install curl build-essential libssl-dev libsqlite3-0 libsqlite3-dev

git clone –b master-ndistro git://github.com/mapbox/tilestream.git

cd tilestream

./ndistro

And that’s that (TileStream, even more than TileMill, will throw up errors during the installation.  None of them should stop the process, though, so you can safely ignore them).  Like TileMill, TileStream needs to be started before it can be accessed in a browser.  Since the plan is to run the server headless, let’s set this up in Webmin in a fashion similar the one employed for TileMill.

Back to Webmin, again open the ‘Other’ menu, and this time click on ‘Custom Commands’.  We’ll create a new Custom Command and configure it as follows (substitute your name for mine as appropriate):

Save it, and you will now have a Custom Command button to use for starting TileStream (we didn’t do this for TileMill because we cannot.  The Webmin Custom Command function simply won’t accept it.  I think it has to do with the nature of the command.  I think the ‘./’ in the TileMill command confuses it).

At this point, TileStream is fully functional, but it doesn’t yet have a tileset to work with.  Using the same browser with which you just accessed Webmin, go here to download one.  Scroll down the page, pick a tileset you like and click on it to proceed to the download page (I picked World Light).  Download the file to wherever you please.  Once you have the file, go back to Webmin and open the ‘Other’ menu again.  Click on ‘Upload and Download’, then select the ‘Upload to Server’ tab.  Click on one of the buttons labeled ‘Choose File’, then browse to the tileset file you downloaded.  For ‘File or directory to upload to’, click the button and browse your way to /home/terry/tilestream/tiles (by now, you should know you’re not ‘terry’).  Click the ‘Upload’  button.

Once your tileset is finished uploading, you can point the browser to http://maps:8888 (yeah, yeah – not ‘maps’) to access TileStream.  Enjoy:

Our last order of business is Flex Viewer (otherwise known as ‘ArcGIS Viewer for Flex’).  This is the easiest of the lot, mainly because it doesn’t actually have to be installed.  Still using the same browser, go to the download page (you’ll need an ESRI Global Account.  If you don’t have one, create one), agree to the terms and download the current version (again – download it to wherever you please).  Once you have the package, use Webmin to upload it to the server.  This time you’ll want to upload the file to /var/www and you’ll want to check the ‘Yes’ button adjacent to ‘Extract archive or compressed files?’

And you’re in.  Point the browser to http://maps/flexviewer/ (you know the drill) and play with your new toy:

You can see I have customized the flex viewer.  You should do so as well (it’s designed for it, after all).  Open the file manager in Webmin (the ‘Other’ menu again) and navigate to /var/www/flexviwer.  Select config.xml, then click the ‘Edit’ button on the toolbar.  The rest is up to you.

* * * * *

So now you have a headless Ubuntu map server up and running, and the question you are probably asking yourself is:  “Do I really need all this stuff running in my server?”  The answer is, of course, ‘no’.  The point of this exercise was to learn a thing or two.  If you’ve actually been following along and have these applications running in your own machine, you are now in a good position to poke around for a while to figure out what sort of server you’d like to run.

For instance, there’s no real reason to run TileMill on a server.  TileMill doesn’t serve tiles, it fires them.  Therefore it’s probably not the best idea to be eating up your server’s resources with TileMill (and it seriously devours resources).  The server doesn’t have a use for the tiles until they’re done being fired, at which point TileStream is the tool for the job.

That said, there’s no compelling reason why you couldn’t run TileMill on your server.  If you’d rather not commit another machine to the task (and if you’re not in any kind of hurry), why not give the job to the server?  It’ll take it a while, but it will get the tiles fired (if your server is an older machine like mine, I would strongly advise you to fire your tiles in sections, then put them together later.  I suggest firing them one zoom level at a time and combining them with SQLite Compare).

Flex Viewer and the OpenGeo Suite don’t often go together, but there’s no reason why they can’t.  Flex Viewer can serve up layers delivered via WMS – there’s nothing to say GeoServer can’t provide that service.  They are, however, very different applications, with vastly different capabilities, strengths and weaknesses.  They also have a very different ‘feel’, and we should never discount the importance of aesthetics in the decision making process.

A final – and very important – consideration in the final configuration of our home server is the nature of the face it presents to the world.  In order for a server to serve, it must connect to and communicate with the world at large.  This means some kind of front end, the nature of which will influence at least some of our choices.

Which brings us neatly to the next post.  See you there.

Pimp Your Server, Part 1

Note:  This is the second part in a series on building your own home-brewed map server (I would tell you how many installments the series will entail, but I won’t pretend to have really thought this through.  There will be at least one more.  Probably two).  It assumes you have read the previous installment.  You have been warned.

Last time, I walked you through setting up your very own headless map server using only Free and Open Source Software.  Now, I’m going to show you how to trick it out with a few extra web mapping goodies.  The installation process will be easiest if you re-attach a ‘head’ to your server (i.e., a monitor and keyboard), so go ahead and do that before we begin (alternately, if you’re using PuTTY to access your headless server, you can use it for this purpose).

At the end of my last post, I showed you all a screenshot of my server running TileMill, TileStream and Flex Viewer, and I made a semi-promise to write something up about it.  So here we are.

I tend toward a masochistic approach to most undertakings in my life, and this one will not deviate from that course.  Whenever I am faced with a series of tasks that need completion, I rank them in decreasing order of difficulty and unpleasantness, and I attack them in that order.  In other words, I work from the most demanding to the least troublesome.

I originally intended to write a single post covering TileMill, TileStream and Flex Viewer, but a short way into this post I realized that I had to split it into two pieces.  The next post will cover TileStream and Flex Viewer.  This one will get you through TileMill.

TileMill can be a bear to install – not because you need catlike reflexes or forbidden knowledge or crazy computer skills – but simply because there are many steps, which translate into lots of room for error.  A quick glance at TileMill’s installation instructions may seem a bit daunting (especially if you’re new to this kind of thing):

Install build requirements:

# Mapnik dependencies 
sudo apt-get install -y g++ cpp \ 
libboost-filesystem1.42-dev \ 
libboost-iostreams1.42-dev libboost-program-options1.42-dev \ 
libboost-python1.42-dev libboost-regex1.42-dev \ 
libboost-system1.42-dev libboost-thread1.42-dev \ 
python-dev libxml2 libxml2-dev \ 
libfreetype6 libfreetype6-dev \ 
libjpeg62 libjpeg62-dev \ 
libltdl7 libltdl-dev \ 
libpng12-0 libpng12-dev \ 
libgeotiff-dev libtiff4 libtiff4-dev libtiffxx0c2 \ 
libcairo2 libcairo2-dev python-cairo python-cairo-dev \ 
libcairomm-1.0-1 libcairomm-1.0-dev \ 
ttf-unifont ttf-dejavu ttf-dejavu-core ttf-dejavu-extra \ 
subversion build-essential python-nose 

# Mapnik plugin dependencies 
sudo apt-get install libgdal1-dev python-gdal libgdal1-dev gdal-bin \ 
postgresql-8.4 postgresql-server-dev-8.4 postgresql-contrib-8.4 postgresql-8.4-postgis \ 
libsqlite3-0 libsqlite3-dev  

# TileMill dependencies 
sudo apt-get install libzip1 libzip-dev curl 

Install mapnik from source:

svn checkout -r 2638 http://svn.mapnik.org/trunk mapnik 
cd mapnik python scons/scons.py configure INPUT_PLUGINS=shape,ogr,gdal 
python scons/scons.py 
sudo python scons/scons.py install 
sudo ldconfig 

Download and unpack TileMill. Build & install:

cd tilemill ./ndistro 

It’s not as scary as it looks (the color-coding is my doing, to make it easy to differentiate things).  The only circumstance that makes this particular process difficult is that the author of these instructions assumes we know a thing or two about Linux and the command line.

Let’s start at the top, with the first ‘paragraph’, which begins: # Mapnik dependencies.  Translation:  We will now proceed to install all the little tools, utilities, accessories and such-rot that Mapnik (a necessary and desirable program) needs to function (i.e., “dependencies”).

It is assumed that we know the entire ‘paragraph’ is one command and that the forward-slashes (/) are not actually carriage returns and shouldn’t be followed by spaces.  It is also assumed that we will notice any errors that may occur during this process, know whether we need concern ourselves with them and (if so) be capable of correcting them.

Let’s see what we can do about this, shall we?  Since we’re installing this on our server and actually typing in the commands (rather than copying and pasting the whole thing), we have the luxury of slicing it up into bite-sized pieces.  This way the process becomes much less daunting, and it makes it easier for us to correct any errors that crop up along the way.

We’ll start by taking control.  Type “sudo su” (sans quotation marks), then provide your password.  Now we can proceed to install everything, choosing commands of a size we’re comfortable with.  I found that doing it one line at a time works pretty smoothly.  Two important points here:  start every command with “sudo apt-get install” (not just the first line) and don’t include the forward-slashes (unless you’re installing more than one line at a time).  I would therefore type in the first two lines like this (don’t forget to hit ‘enter’ at the end of each command):

sudo apt-get install –y g++ cpp

sudo apt-get install libboost-filesystem1.42-dev

You get the idea.  Continue along in this fashion until you have installed all the necessary dependencies for Mapnik.  I strongly recommend doing them all in one sitting.  It just makes it easier to keep track of what has and hasn’t been installed.

At this stage of the game, any errors you encounter will most likely be spelling errors.  Your computer will let you know when you mistype, usually through the expedient of informing you that it couldn’t find the package you requested.  When this occurs, just double-check your spelling (hitting the ‘up’ cursor key at the command prompt will cause the computer to repeat your last command.  You can then use the cursors to correct the error).  At certain points in the installation process, your server will inform you of disk space consumption and ask you to confirm an install (in the form of yes/no).  Hitting ‘y’ will keep the process moving along.

While packages install in your system, slews of code will fly by on your screen, far too fast to read or comprehend.  Just watch it go by and feel your Geek Cred grow.

By now you should have developed enough Dorkish confidence to have a go at # Mapnik plugin dependencies and # TileMill dependencies.  Have at it.

When you’re done, move on to installing Mapnik from source.  Each line of this section is an individual command that should be followed by ‘enter’.  The first line will throw up your first real error.  Simply paying attention to your server and following the instructions it provides will fix the problem (in case you missed it, the error occurred because you haven’t installed Subversion, an application you attempted to use by typing the command ‘svn’.  Easily fixed by typing sudo apt-get install subversion).  You can then re-type the first line and proceed onward with the installation.  When you get to the scons commands, you will learn a thing or two about patience.  Wait it out.  It will finish eventually.

Now we should be ready to do what we came here to do:  install TileMill.  Unfortunately, TileMill’s installation instructions aren’t very helpful at this point for a headless installation.  All they tell us is to “Download and unpack TileMill”.  There’s a button further up TileMill’s installation page for the purpose of the ‘download’ part of this, but it’s not very helpful for our situation.  We could use Webmin to manage this, but what the hell – let your Geek Flag fly (later on, we’ll use Webmin to install Flex Viewer, so you’ll get a chance to see the process anyway).

Our installation of Mapnik left us within the Mapnik directory, so let’s start by returning to the home directory:

cd~

Then we can download TileMill:

wget https://github.com/mapbox/tilemill/zipball/0.1.4 –no-check-certificate

Now let’s check to confirm the name of the file we need to unpack:

dir

This command will return a list of everything in your current directory (in this case, the home directory).  Amongst the files and folders listed, you should see ‘0.1.4’ (probably first).  Let’s unpack it:

unzip 0.1.4

Now we have a workable TileMill folder we can use for installation, but the folder has an unwieldy name (which, inexplicably, the installation instructions fail to address).  Check your directory again to find the name of the file you just unpacked (in my case, the folder was ‘mapbox-tilemill-4ba9aea’).  Let’s change that to something more reasonable:

mv mapbox-tilemill-4ba9aea tilemill

At long last, we can follow the last of the instructions and finish the installation:

cd tilemill

./ndistro

Watch the code flash by.  Enjoy the show.  This package is still in beta, so it will probably throw up some errors during installation.  None of them should be severe enough to interrupt the process, though.  Feel free to ignore them.

Once the installation is complete, we’ll have to start TileMill before we can use it.  This can be achieved by typing ‘./tilemill.js’in the terminal, but TileMill actually runs in a browser (and we’ll eventually need to be able to run it in a server with no head), so let’s simplify our lives and start it through Webmin.

Go to the other computer on your network through which you usually access your server (or just stay where you are, if you’ve been doing all this through PuTTY), open the browser and start Webmin.  Open the ‘Others’ page and select ‘Command Shell’.  In the box to the right of the ‘Execute Command’ button, type:

cd /home/terry/tilemill (substitute your own username for ‘terry’)

Click the ‘Execute Command’ button, then type in:

./tilemill.js

Click the button again (after you’ve gone through this process a couple of times, Webmin will remember these commands and you’ll be able to select them from a drop-down list of previous commands).

And now enjoy the fruits:  type http://maps:8889 into the location bar of your browser (again, substitute the name of your server for ‘maps’).  Gaze in awe and wonder at what you have wrought:

Take a short break and play around with the program a bit.  You’ve earned it.  When you’re done I’ll be waiting at the beginning of the next post.

Off With Its Head!

Fellow Map Dork and good Twitter friend Don Meltz has been writing a series of blog posts about his trials and tribulations while setting up a homebrewed map server on an old Dell Inspiron (here and here).  I strongly recommend giving them a read.

At the outset, Don ran his GeoSandbox on Windows XP, but recently he switched over to Ubuntu.  While I applaud this decision whole-heartedly, I thought I’d take the extra step and build my own map server on a headless Ubuntu Server box (when I say ‘headless’, I am talking about an eventual goal.  To set this all up, the computer in question will initially need to have a monitor and keyboard plugged into it, as well as an internet connection.  When the dust settles, all that need remain is the internet connection).  The following is a quick walkthrough of the process.  I apologize to any non-Map Dorks who may be reading this.

The process begins, of course, with the installation of Ubuntu 10.04 Server Edition.  Download it, burn it to a disk, and install it on the machine you have chosen to be your server.  Read the screens that come up during installation and make the decisions that are appropriate for your life.  The only one of these I feel compelled to comment on is the software selection:

The above image shows my choices (what the hell – install everything, right?).  Definitely install Samba shares.  It allows Linux machines to talk to others.  Also, be sure to install the OpenSSH server.  You’ll need it.  For our purposes, there’s no real reason to install a print server, and installing a mail server will cause the computer to ask you a slew of configuration questions you’re probably not prepared to answer.  Give it a pass.

During the installation process, you will be asked to give your server a name.  I named mine ‘maps’.  So whenever I write ‘maps’, substitute the name you give your own machine.

Once your installation is complete, you will be asked to login to your new server (using the username and password you provided during installation), after which you will be presented with a blinking white underscore (_) on a black screen.  This is a command prompt, and you need not fear it.  I’ll walk you through the process of using it to give yourself a better interface with which to communicate with your server.  Hang tight.

Let’s begin the process by taking control of the machine.  Type in “sudo su” (sans quotation marks) and hit ‘enter’.  The server will ask for your password, and after you supply it, you will be able to do pretty much anything you want.  You are now what is sometimes called a superuser, or root.  What it means is that you are now speaking to your computer in terms it cannot ignore.  This circumstance should be treated with respect.  At this stage, your server will erase itself if you tell it to (and it won’t ask you whether or not you’re sure about it – it’ll just go ahead and obey your orders).  So double-check your typing before you hit ‘enter’.

Now, let’s get ourselves a GUI (Graphical User Interface).  The server edition we’re using doesn’t have its own GUI, and for good reasons (both resource conservation and security).  Instead, we can install Webmin, a software package that allows us to connect to our server using a web browser on another computer on the same network.  We’ll do this using the command line.  Type in (ignore the bullets before each command.  They are only there to let you know where each new line begins):

• wget http://www.webmin.com/download/deb/webmin-current.deb

And hit ‘enter’ (I’m not going to keep repeating this.  Just assume that hitting ‘enter’ is something you should do after entering commands{the dark words}).  Follow this with:

• sudo dpkg -i webmin-current.deb

And finish it up with:

• sudo apt-get -f install

Now we have a GUI in place.  If you open a browser on another computer on your network and type: https://maps:10000 into the location bar (remember to replace ‘maps’ with the name you gave your own server), you’ll be asked to supply your username and password, then you’ll see this (you may also be asked to verify Webmin’s certificate, depending on your browser):

Cool, huh?  Don’t get your hopes up, though.  We’re not done with the command line yet (don’t sweat it – I’ll hold your hand along the way.  Besides – you should learn to be comfortable with the command line).  For the moment, though, let’s take a look around the Webmin interface.  There is a lot this program can do, and if you can find the time and determination it would be a good idea to learn your way through it.  For now, you just really need to know a few options.  The first is that the initial page will notify you if any of your packages (Linux for ‘software’) have available updates.  It’s a good idea to take care of them.  If you want, Webmin can be told to do this automatically (on the update page you get to when you click through).  The other important features are both located under the ‘Other’ menu (on the left).  The first is the file manager (which bears a striking resemblance to the Windows File Manager of old), which gives you the ability to explore and modify the file system on your server (this feature runs on Java, so be sure the browser you’re using can handle it).  The other feature is ‘Upload and Download’ which does what it says it does.  Together, these two features give you the ability to put maps on your map server, something I assume you’ll want to do.

Please note the specs on my server (as pictured above).  It’s not terribly different than Don’s Inspiron.  I’m not suggesting you do the same, but it is worth noting that an old machine can handle this job.

Back to the command line.  Let’s get OpenGeo:

• wget -qO- http://apt.opengeo.org/gpg.key | apt-key add –
• echo “deb http://apt.opengeo.org/ubuntu lucid main” >> /etc/apt/sources.list
• apt-get update
• apt-cache search opengeo
• apt-get install opengeo-suite

Rock and roll.  When your server is done doing what it needs to do, go back to the browser you used for Webmin and type http://maps:8080/dashboard/ into the location bar.  Check out the OpenGeo goodness.

Finally, to make your new server truly headless, you’re going to need some way to login remotely (when you turn the machine on, it won’t do a damn thing until you give it a username and password).  Since you listened to me earlier and installed the OpenSSH server, you’ll be able to do this.  All you need is an SSH client.  If you’re remotely connecting through a Linux machine, chances are you already have one.  In the terminal, just type:

• ssh <username>@<computer name or IP address>

In my case, this would be:

• ssh terry@maps

You’ll be asked for a password, and then you’re in (I hear this works the same in OS X, but I cannot confirm it).

If you’re using a Windows machine – or if you just prefer a GUI – you can use PuTTY.  PuTTY is very simple to use (and it comes as an executable.  I love programs that don’t mess with the registry).  Tell it the name of the computer you want to connect to and it opens a console window asking for your username and password.  Tell it what it wants to know.

It’s not a bad idea to install a new, dedicated browser for use with your new server.  I used Safari, but only because I already use Firefox and Chrome for other purposes.  Also, your network will probably give your server a dynamic IP address.  This is not an issue for you, since your network can identify the machine by name.  If you want to (and there are several valid reasons to do so), you can assign a static IP address to your server.  To find out how to do so, just search around a bit at the extraordinary Ubuntu Forums.  Update:  It seem that Webmin provides an easy method to assign a static IP address to your server.  Go to Networking → Network Configuration → Network Interfaces → Activated at Boot.  Click on the name of your active connection, and you will then be able to assign a static IP address just by filling in boxes.

Enjoy your map server.  If I can find the time, I’ll write up a post on how I added Flex Viewer, TileMill and TileStream to the server:

And 50 bonus points to anyone who understands the image at the top of this post.