In this post we’re going to get a functional OpenStreetMap tile server up and running. We’ll deploy a small map initially just to make sure everything is working correctly.
These instructions are have been written and tested against a fresh install of Ubuntu 20.04 server and follow on from the steps outlined in Secure Ubuntu 20.04. If you have upgraded from an earlier version of Ubuntu this guide probably won’t work.
An OSM tile server has 5 main components:
First up we will install all the tools and utilities we are going to need.
sudo apt install libboost-all-dev git tar unzip wget bzip2 build-essential autoconf libtool libxml2-dev libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev munin-node munin protobuf-c-compiler libfreetype6-dev libtiff5-dev libicu-dev libgdal-dev libcairo2-dev libcairomm-1.0-dev apache2 apache2-dev libagg-dev liblua5.2-dev ttf-unifont lua5.1 liblua5.1-0-dev acl npm tmux
y when prompted to install. It might take a while.
Ubuntu 20.04 has pre-packaged versions of
postgis (some graphical extensions) so we can simply install from the package manager
sudo apt-get install postgresql postgresql-contrib postgis postgresql-12-postgis-3 postgresql-12-postgis-3-scripts
y to install.
See Optimise Postgres for some tips on configuring postgres for a large (full planet) import. You don’t need to do this now, as we’re only going to import a small extract to prove the server is working.
Now we need to create a postgis database, the convention is to call it
gis so we’ll follow along. We also need a user that Mapnik will use to render maps later. We’ll call it
osm but you can call it what ever you like.
sudo -u postgres -i createuser osm createdb -E UTF8 -O osm gis
Next we create the
psql -c "CREATE EXTENSION postgis;" -d gis psql -c "CREATE EXTENSION hstore;" -d gis
osm as the table owner:
psql -c "ALTER TABLE spatial_ref_sys OWNER TO osm;" -d gis psql -c "ALTER TABLE geometry_columns OWNER TO osm;" -d gis
And exit from the postgres user
osm system user so the tile server can run as
sudo adduser --system osm
While we are here, let’s give our user id permission to write to the
osm user’s home directory.
sudo setfacl -R -m u:chris:rwx /home/osm/
remember to replace
chris with your real user name
sudo apt install osm2pgsql
Again we will install the default version in Ubuntu 20.04
sudo apt-get install autoconf apache2-dev libtool libxml2-dev libbz2-dev libgeos-dev libgeos++-dev libproj-dev gdal-bin libmapnik-dev mapnik-utils python3-mapnik
note: We’re using the “switch2osm” branch of https://github.com/SomeoneElseOSM/mod_tile, which is itself forked from https://github.com/openstreetmap/mod_tile, but modified so that it supports Ubuntu 20.04, and with a couple of other changes to work on a standard Ubuntu server rather than one of OSM’s rendering servers.
mkdir -p /home/osm/src && cd /home/osm/src git clone -b switch2osm git://github.com/SomeoneElseOSM/mod_tile.git cd mod_tile ./autogen.sh
That should complete with
autoreconf: Leaving directory `.'
That should complete with
config.status: executing libtool commands
You might see some warnings and it might look like it has hung, but stick with it and it should complete with
make: Leaving directory '/home/osm/src/mod_tile'
sudo make install
That should finish with
make: Leaving directory '/home/osm/src/mod_tile'
sudo make install-mod_tile
That should finish with
chmod 644 /usr/lib/apache2/modules/mod_tile.so
which wont reply at all.
Now we need to download and configure a stylesheet. We’re going to use the ‘standard’ map style from OSM.
cd /home/osm/src git clone git://github.com/gravitystorm/openstreetmap-carto.git cd openstreetmap-carto
Next we install a “carto” compiler
sudo npm install -g carto carto -v
It should respond with a version number of at least
Now we convert
carto into something
mapnik can use.
carto project.mml > mapnik.xml
There will be warnings, but you now have a
mapnik XML stylesheet at
Importing the full planet will take a long time, exactly how long will depend on your machine. So we will start with a small extract:
mkdir /home/osm/data && cd /home/osm/data wget https://download.geofabrik.de/asia/azerbaijan-latest.osm.pbf
Next comes the fun part; importing the data, we switch to the
postgres user first:
sudo -u postgres -i osm2pgsql --database gis --create --slim --multi-geometry --hstore --tag-transform-script /home/osm/src/openstreetmap-carto/openstreetmap-carto.lua --cache 35000 --number-processes 10 --style /home/osm/src/openstreetmap-carto/openstreetmap-carto.style /home/osm/data/azerbaijan-latest.osm.pbf
--database gis: The database to use
--create: Load into an empty db rather than
--appendto existing data
--slim: Store temporary data in the database. This greatly reduces the RAM usage but is much slower. Required if you want to update with
--multi-geometry: Generate multi-geometry features in postgresql tables.
--hstore: Allows tags for which there are no explicit database columns to be used for rendering.
--tag-transform-script <path>: Defines the lua script used for tag processing. This an easy is a way to process OSM tags before the style itself processes them, making the style logic potentially much simpler.
--cache <size>: The size of the in memory cache in Mb.
--number-processes <n>: Specifies the number of parallel processes used for certain operations. I have 10 cores on my server so I use 10.
--style <path>: Location of the style file.
The command will finish with something like:
osm2pgsql took 32s overall node cache: stored: 3008359(100.00%), storage efficiency: 50.08% (dense blocks: 9, sparse nodes: 2966494), hit rate: 100.00%
Once the import completes you need to grant all privileges to
osm user on the
psql -c "GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO osm;" -d gis
Custom indexes are not required, but will speed-up rendering, particularly on full-planet databases or heavy load environments, probably won’t be as helpful for out initial small extract.
cd /home/osm/src/openstreetmap-carto/ psql -d gis -f indexes.sql
The indexes may take a little while to create, especially with a large extract.
Basically the bigger your cache the quicker the import, however it’s a balancing act, too big a value may cause the process to terminate because it runs out of memory. Try with 80% of your free ram and see how you get on. If
osm2pgsql runs out of memory you’ll need to start over with a smaller cache.
total used free shared buff/cache available Mem: 61840 17305 44284 7 250 43969 Swap: 2097 0 2097
So I’d start with
--cache 35000 (44284 * 0.8 = 35427)
Most of the map data needed comes from the OSM data file we down loaded earlier, but we need to get some shape files for things like low-zoom country boundaries.
cd /home/osm/src/openstreetmap-carto/ scripts/get-shapefiles.py
When complete it will display
…script completed in xx.x seconds..
Install the fonts to display place names around the world:
sudo apt-get install ttf-dejavu fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted ttf-unifont
sudo nano /usr/local/etc/renderd.conf
If you are short on ram (2Gb or so) then
XML line to point to the correct location
URI=/hot/ was chosen so that the tiles generated here can more easily be used in place of the HOT tile layer at OpenStreetMap.org. I’ll change it to
osm but you can use what ever you like.
For my server I don’t need 20x zoom so I’m going to set
MAXZOOM to 18, but you can leave it at 20 if you want
[renderd] num_threads=2 tile_dir=/var/lib/mod_tile stats_file=/var/run/renderd/renderd.stats [mapnik] plugins_dir=/usr/lib/mapnik/3.0/input font_dir=/usr/share/fonts/truetype font_dir_recurse=1 [ajt] URI=/osm/ TILEDIR=/var/lib/mod_tile XML=/home/osm/src/openstreetmap-carto/mapnik.xml HOST=localhost TILESIZE=256 MAXZOOM=20
First we create the paths from
[renderd] config above and set the
osm user as owner:
sudo mkdir /var/lib/mod_tile sudo chown osm /var/lib/mod_tile sudo mkdir /var/run/renderd sudo chown osm /var/run/renderd
Now we need to tell
sudo nano /etc/apache2/conf-available/mod_tile.conf
add the following line:
LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so
save it and run:
sudo a2enconf mod_tile
It will say you need to run
systemctl reload apache2 but we won’t do that just yet. We need to tell apache about
sudo nano /etc/apache2/sites-available/000-default.conf
add the following between the
LoadTileConfigFile /usr/local/etc/renderd.conf ModTileRenderdSocketName /var/run/renderd/renderd.sock ## Timeout before giving up for a tile to be rendered ModTileRequestTimeout 0 ## Timeout before giving up for a tile to be rendered that is otherwise missing ModTileMissingRequestTimeout 30
Now we can reload
sudo service apache2 reload
note: you might need to do it twice, I’m not sure why.
If you have enabled the firewall as described in [Part 1 - setup] you’ll need to open the
http port now:
sudo ufw allow http
If you point a web browser at:
http://yourserveripaddress/index.html you should get Ubuntu / apache’s “It works!” page.
Initially we’ll run
renderd in the foreground so we can see any errors if they occur:
sudo -u osm renderd -f -c /usr/local/etc/renderd.conf
This command uses
sudo to switch to the
osm user and run the command
Point your web browser at
http://yourserveripaddress/osm/0/0/0.png (if you didn’t choose
osm as the uri when you configured
renderd make sure you use the uri you chose instead). If all goes well you should see a map of the world in your browser and output on the command line that includes
DEBUG: START TILE and
DEBUG: DONE TILE
If you see any
DEBUG: Failed to read cmd on fd messages you can safely ignore them, they are not errors.
Assuming everything worked you can now
ctrl-c to stop the foreground process.
If you get errors and the tile is not displayed then copy the output and ask about the problem somewhere like stackoverflow or help.openstreetmap.org
First edit the
/home/osm/src/mod_tile/debian/renderd.init file and set the
RUNASUSER property to
sudo nano /home/osm/src/mod_tile/debian/renderd.init ... SCRIPTNAME=/etc/init.d/$NAME RUNASUSER=osm ## Exit if the package is not installed ...
Next copy it to the system directory, set permissions and add the service:
sudo cp /home/osm/src/mod_tile/debian/renderd.init /etc/init.d/renderd sudo chmod u+x /etc/init.d/renderd sudo cp /home/osm/src/mod_tile/debian/renderd.service /lib/systemd/system/
And start it:
sudo /etc/init.d/renderd start
To make it start automatically when the system reboots:
sudo systemctl enable renderd
There is an example html file in mod_tile’s extras folder which we can use as the index.html on our server. It uses leafletjs to display a ‘slippy map’ centered on Azerbaijan.
127.0.0.1with the ip address of your server.
osm(or the uri you have chosen)
[40.36629, 49.83335]to center on your map.
Now replace the
index.html file in
sudo cp /home/osm/src/mod_tile/extra/sample_leaflet.html /var/www/html/index.html
Tail the logs so we can see when a tile is requested:
sudo tail -f /var/log/syslog | grep " TILE "
Note: the spaces around
TILE are important
Now you can point your browser at
and you will see a line printed to your console every time a tile is requested.
If you made it this far you now have a functional tile server… Nice!