This document explains how to generate a new 'sox', or 'spatial box', executable that identifies whether a user has typed in a placename amongst a set of search keywords.
The tool allows a portal to identify words that are placenames, which can then be used to identify where to search from in a nearby engine. It features the capability to limit the recognised words to a specific area of the world, or lets an administrator choose an arbitrary placename list.
The process removes any dictionary words from the input list of placenames, and gives fine control over specific words; that is, whether a particular word should be considered a placename or a query term.
- Obtain a placename list to be used in the portal
See lbs_root/INSTALL file, Section 3. "Import placenames" for details on obtaining a global file in this format from mobilemaps.com, and optionally trimming it to suit your portal. You will need this file to obtain the dictionary. Alternatively, create the placename list from your own data - see ~ below.
After you have trimmed the list, export the file from MySQL with a command such as
mysql -u username -p -D database -N -e "SELECT * FROM placenames" > placenames.txt
(replace username, database, with your own terms)
~ Alternatively create the placename list from your own data. The file should be in the following format:
[ID, or 0] [Full placename, Further Details] [Latitude] [Longitude]
(Tab delimited. The file can include duplicates. Any letters after the comma will be ignored by sox eg. 'London, UK', would be read as 'London'. )
For both options: Copy the placenames.txt and dictionary.txt file into your current directory, generally
lbs_root/sox/soxprepare/
ie. This is also known as your 'data directory'.
- Create a database in MySQL
Enter from the command-line
mysql -u username -p
(replace username with your own username)
Enter
CREATE DATABASE soxprepare;
exit
- Configure options
Copy the file soxprepare.conf.sample to soxprepare.conf
cp soxprepare.conf.sample soxprepare.conf
Open the file soxprepare.conf in a text editor. Follow the instructions.
- Prepare the definite_words and definite_placename files
Open 'definite_words.txt' in a text editor. Append or remove any words from this list that should be interpreted as words as opposed to placenames. This list overrides the dictionary.
Open 'definite_placename.txt' in a text editor. Append or remove any words from this list that should be interpreted as placenames as opposed to words. If you include a name on this list that is not in your original placename list it will not be recognised as a placename in the search query.
The 'definite_placename.txt' file already holds some names that were in the dictionary, but are better known as placenames. It is recommended to leave these in place and add to the list, because they represent all words that should not have been in the dictionary.
Don't worry about accidently duplicating words, putting words in a particular order, or capitalization. The software will automatically handle these scenarios.
Take a moment to study the file 'possible_word_placename.txt'. This list has some words that could be included in either the 'definite_words.txt' file or the 'definite_placename.txt' file depending on the users of the nearby engine, and the context of your portal.
- Run the script
Enter
perl soxprepare.pl
You will need to wait for approximately ten or more minutes for the process to finish depending on the size of your placename list. You may be asked to enter the password of your database part way through. Follow the instructions onscreen after the process is complete.
The instructions include copying the files that have been created back into the lbs_root/sox folder, and rebuilding sox.
eg.
cp soxnames* ..
cp customsox.h ..
cp make ..
cd ..
./make
To test that the software is working, run
./sox hello London
Which should return:
London
hello
>
The first line returned is your placenames, the second line is your keywords, and the third line is any additional words you entered, such as state codes, stop words (eg. 'a', 'the'), or secondary placenames.
This sox executable can then be used directly within your web server directory eg. cgi-bin/lbs/sox/sox.
The cause for many errors is a lack of file permissions. For your data files to be read into MySQL, they may need to have extended read permissions.
Enter eg on linux:
