- US Geocoding Database
MySQL Client libraries.
Tiger Census Data for Street-Level
Where to get data from:
Has each of the states in the US. The files are organised into US County format and are zipped in this manner. You will need to download the entire .zip file, and put all zip files in one directory. Copy /hcslmap/data/tiger/myunzip.sh into your download directory. Run:
You should see a number of *.RTx files in your download directory.
cp usgeocode/*.txt [your download directory]
How to build.
Ensure that the -L/usr/lib/mysql, and -I/usr/include/mysql correspond with the real paths of the MySQL libraries and include files.
Create a database in MySQL called 'usgeocode'.
mysql -u root -p
CREATE DATABASE usgeocode;
Uncomment the compiler/platform being used. GCC = Unix, BORLANDC = Windows (it should also compile on VCC, although this is untested).
#define HOST ""
#define USERNAME ""
#define PASSWORD ""
#define DATABASE "usgeocode"
#define TIGERPATH "/lbs/usgeocode/tigerdata/california/"
#define TIGERPATHDOUBLE "/lbs/usgeocode/tigerdata/california/"
Where HOST, USERNAME, PASSWORD, and DATABASE all describe the location and access to the database.
TIGERPATH is the path to your unzipped Tiger files - use normal Unix or Windows paths in C 'escaped' format depending on which platform you are compiling. TIGERPATHDOUBLE is a Windows only requirement and is simply the same path with 4 backslashes '\'s between directories. eg. "c:\\\\maps\\\\tiger\\\\california\\\\"
Run the Makefile.
How to run.
The whole process will take several hours to several days depending on the speed of your system and the quantity of the data. Note that 'readtiger' can be run with new Tiger files at a later date, provided that the old files have been removed from the Tiger data directory.
The database for California was approx 1GB in size.
Unresolved problems with usgeocode database 8 Sept 2002
-longitude/latitude are switched over!
-You will need to build an index on the street.zipcode field
mysql -u root -p
CREATE INDEX street_zp ON street(zipcode);
- US Street-level maps
Where to find data:
National Atlas USA Data
Download Shapefiles for
Rail (approx < ~)
City (approx < ~)
States (approx < ~)
Street-Level USA Tiger Data
These are preprepared shape-files.
You will need to register with your e-mail address, and then download on a per state/county basis. You can download up to 20MB in any one session.
The layers to download are:
The roads are the largest layer and can be 100+MB for a whole state. The other layers are closer to 20MB for a state.
ESRI World Data
ESRI World Basemap.
Requires use of ESRI products, and no translation should be made to other formats. A world-wide map is created on selecting the world map. As you zoom in using the interactive web interface, you are introduced to more detailed data. Suggestion is to download one coarse world level and then several areas in your area of focus. TODO: Determine if the EsriBasemap is different from the EsriWorldmap. Approximate size World = 200Kb, Continents < 1MB.
AND Data (commercial dataset)
At one stage available from the ESRI download site. Contact them directly now however.
World-wide 1:250,000 scale Shape files. Good for road network scale mapping.
MapServer - shp2img, tile4ms; Unzip
How to build MapServer:
See the mapserver installation document.
How to set up directory structure and spatially index the map data:
a) Root Master Data Directory
hcslmap/data # index shapefiles, rectangular regions that cover the area of each of the real data files
b) 'Vertical' Scale Layer Main Subdirectories
hcslmap/data/esriworldmap # ESRI world map data layer shapefiles (World Scale)
hcslmap/data/usnational # US National layer shapefiles (City Scale)
hcslmap/data/tiger # Tiger layer shapefiles (Street Scale)
c) 'Horizontal' Region Subdirectories within the same 'Vertical' Scale Layer
d) Multiple Data Types within the same Region
This is an expandable and flexible structure that can have as many Shape map data files as required. Take care with displaying too much data in any one view however ie. when the concentration of too many Shape files from the same scale layer overlap in one geographic region. This will reduce performance.
The actual Shape files come in the following form:
yourfile.shp #The largest file
Generating the index files
A reference file needs to be created for each type of data, at each different scale. This file tells the map server the boundaries for the Shape map data files. It is itself a Shape file and sits in the Root Master Data Directory (a).
(once all data files are in place) creates the index files.
Example entries in make_index.sh:
./tile4ms rdline_index.txt rdline_index
./tile4ms pppoly_index.txt pppoly_index
The utility tile4ms creates a Shape file called rdline_index from rdline_index.txt. rdline_index.txt contains the list of filenames for the Shape data files themselves.
Example entries in a rdline_index.txt file:
./uk/rdline_line.shp #These are Shape file relative locations
-- Each Shape file must be spatially indexed internally to maintain rendering performance. A spatially indexed Shape file-set has an additional .qix file:
yourfile.qix #.qix = Additional spatial index for Shape file
To generate a .qix file you must run
/your_mapserver_path/shptree yourfile.shp 8
-- Fortunately there are some useful scripts in Mobilemaps for handling multiple files. Once you have downloaded all of the .zip versions of a fileset in one directory, copy myunzip.sh, mytree.sh from /hcslmap/data/tiger into your new directory:
./myunzip.sh #Unzips all the .zip files
./mytree.sh #Creates a .qix file for every Shape file.
However be warned that you will need to edit the mytree.sh to include your own absolute path to the mapserver. You will also need a version of 'unzip' on your machine: download and install unzip-5.41-3.i386.rpm from www.redhat.com if you don't have unzip.
-- Special case Tiger files: To make things easier for the large quantities of data in Tiger files, there is a simple install script.
1. Put all layers of downloaded .zip files in one directory. If the files were too large to download in one session, they should be renamed with a '1', '2', '3' after the name.
2. Copy install.sh, myunzip.sh, mytree.sh into the same directory.
3. You should have a directory looking like this:
4. If the number of road .zip files is greater than 8, you will need to edit install.sh and repeat the road paragraph of the script once more, replacing the old number with the new. The same principle applies to the other types.
6. This will take some time to process because of the quantities of data. Once it is complete you should have multiple directories for each type of data.
In each directory should be a list of .zip files (these can be deleted if you want to save space on your drive). The real data is contained in the Shape files: eg.
7. You will need to generate an index file for this fileset in your Root Master Data directory that points to each of the Shape files. The best approach here:
ls *.shp >my_index.txt
Enters all of the Shape file-names into a new text file. Then in a text editor replace 'tgr' with './my_path_relative_to_Root_Master_Data_dir/tgr'
8. Copy the file my_index.txt into the Root Master Data directory (renamed as the data type eg. 'tiger_county_index.txt') and add this index file name to the end of the make_index.sh script.