Get some data and import it

Grab populated places points

wget http://edcftp.cr.usgs.gov/pub/data/nationalatlas/citiesx020.tar.gz

Grab US counties map:

wget http://edcftp.cr.usgs.gov/pub/data/nationalatlas/countyp020.tar.gz

Lots more data available in the National Atlas

Extract shapefiles from the two archives

tar -xzvf citiesx020.tar.gz
tar -xzvf  countyp020.tar.gz

Ogrinfo the files to have a look at them

ogrinfo citiesx020.shp
INFO: Open of `citiesx020.shp'
      using driver `ESRI Shapefile' successful.
1: citiesx020 (Point) 

ogrinfo countyp020.shp
INFO: Open of `countyp020.shp'
      using driver `ESRI Shapefile' successful.
1: countyp020 (Polygon)

Notice that the cities shapefile is a point layer and the county layer is a polygon layer.

Import the two shape files into your PostGIS db

To get started with PostGIS see my first tutorial.

ogr2ogr -f "PostgreSQL" -s_srs EPSG:4269 "PG:dbname=mydbname" countyp020.shp -nln us_counties
ogr2ogr -f "PostgreSQL" -s_srs EPSG:4269 "PG:dbname=mydbname" citiesx020.shp -nln us_cities

Make sure you change mydbname to your database’s name.

Checking the Import

psql mydbname
mydbname# \d us_cities
                                   Table "public.us_cities"
    Column    |       Type       |                          Modifiers
--------------+------------------+-------------------------------------------------------------
 ogc_fid      | integer          | not null default nextval('us_cities_ogc_fid_seq'::regclass)
 wkb_geometry | geometry         |
 citiesx020   | double precision |
 feature      | character(27)    |
 name         | character(48)    |
 pop_range    | character(21)    |
 pop_2000     | numeric(8,0)     |
 fips55       | character(5)     |
 county       | character(55)    |
 fips         | character(5)     |
 state        | character(2)     |
 state_fips   | character(2)     |
 display      | numeric(1,0)     |
Indexes:
    "us_cities_pk" PRIMARY KEY, btree (ogc_fid)
    "us_cities_geom_idx" gist (wkb_geometry)
Check constraints:
    "enforce_dims_wkb_geometry" CHECK (ndims(wkb_geometry) = 2)
    "enforce_geotype_wkb_geometry" CHECK (geometrytype(wkb_geometry) = 'POINT'::text OR wkb_geometry IS NULL)
    "enforce_srid_wkb_geometry" CHECK (srid(wkb_geometry) = -1)

You can see that ogr2ogr has imported the shapefile, created a primary key (ogc_fid) and created a spatial index. What you can’t see is that ogr2ogr was also nice enough to add an entry to the geometry_columns table so that postgis tools know that there is a geometry column (wkb_geometry) in this table.

Looking at the Data

Lets fire off qgis and see what we have

Next time, querying with spatial predicates….I promise it won’t take six months for the next post.

sudo apt-get install gdal-bin

Once you have GDAL/OGR installed you get a slew of command line utilities, I’ll try to cover some others in later tutorials, but for now we’re interested in ogr2ogr. ogr2ogr converts between the vector formats that OGR understands.

Example using a shapefile at the city of chicago website:
Download the data

wget http://egov.cityofchicago.org/webportal/COCWebPortal/COC_ATTACH/TIFS2008.zip

Unzip it

unzip TIFS2008.zip

Now we have to do the hard part (not really that hard, but important), look for the projection information in the metadata. I looked in the tifs.shp.xml file and found that the projections is:

NAD_1983_StatePlane_Illinois_East_FIPS_1201_Feet

Google uses WGS84 spatial reference system.
Now we have to lookup EPSG codes that OGR understands for these projections. A good spot to do this is spatialreference.org. EPSG codes provide a short form of expressing projection and spatial reference information.
Once we have that all sorted out we’re ready to run ogr2ogr:

ogr2ogr -f "KML" -s_srs "EPSG:102671" -t_srs "EPSG:4326" tifs.kml tifs.shp

The -f “KML” specifies that we want the output in KML. -s_srs is the source (tifs.shp) spatial reference system and -t_srs is the target spatial reference system found at spatialreference.org, then we specify the output file tifs.kml and the input file tifs.shp. That is it!

Lets shrink it down into a kmz (compressed kml) so that it takes up less disk space.

zip tifs.kmz tifs.kml

To get started in ubuntu 7.04 (should work in 7.10 and debian too):

$ sudo apt-get install postgis postgresql-8.1-postgis

Next we’ll start setting up the PostGIS environment.

Then we need to set up a new database, I called mine geodb:

$ createdb geodb

We then need to bind it to the PostGIS libs with the following:

$ createlang plpgsql geodb

And finally add the PostGIS Tables:

$ psql -d mytestdb -f /usr/share/postgresql-8.1-postgis/lwpostgis.sql
$ psql -d mytestdb -f /usr/share/postgresql-8.1-postgis/spatial_ref_sys.sql

Phew….that was a bit of work.

Now we’re ready to go. We have a database called geodb and we’ve told PostgreSQL that it is a PostGIS database.

So now we can use the shp2pgsql command to import a shapefile into the db:

$ shp2pgsql -W LATIN1 -D -I myshapefile.shp mytablename | psql geodb

-W is for setting the encoding type, I had problems with many shapefiles as my locale is UTF8 and PostgreSQL seems to default to the locale. For me LATIN1 has been working better.

-D sets the SQL output to be in dump format for insertion into the database.

-I flags it to compute the spatial indexing upon import. For large complex vector features this can take awhile, so use with discretion.

All of this is piped to

This gives us a new table called “tablename”, which we can see by doing the following:

$ psql geodb
Welcome to psql 8.1.8, the PostgreSQL interactive terminal.
 
Type:  \\copyright for distribution terms
       \\h for help with SQL commands
       \\? for help with psql commands
       \\g or terminate with semicolon to execute query
       \\q to quit
 
geodb=#

At the PostgreSQL prompt you can enter various SQL commands including queries.

geodb=# \\d mytablename

shows the columns and data types of the data in the table.

Notice the column called: the_geom

This is where the GIS Objects are stored.

The sample query there would show all the data in the shapefile’s associated .dbf as well as a column called

Next time we’ll launch into some querying fun.

Onward to Part II…

I have been hacking on a bunch of convenience code in python for the LTM (Land Transformation Model) which serves as the land use projection portion of my research. Basically just wrapping a bunch of ArcGIS functions that do what I need done and then some file manipulation and config file writing before I launch code written by our postdoc. Going to try and do some time projections for all of the state of Indiana and then go back to my Muskegon study area with the new tools to re-run the model up there. It should make things run much more smoothly.

Geopy lets me code a geocoder in 6 lines of python using the google maps API.

Example:

#!/usr/bin/python
from geopy import geocoders
import sys
g = geocoders.Google('ENTER YOUR API KEY HERE')
address = sys.argv[1]
place, (lat,lng) = g.geocode(address)
print "%s: %.5f, %.5f" % (place,lat,lng)

Then run from the command line like so:

# geoloc "500 Main St. 42367"
500 Main St, Powderly, KY 42367, USA: 37.23329, -87.16244

Nice!