Shapefiles and PRJ - Tying them together

ESRI has developed a de-facto standard for Shapefiles, but sometimes we as users' of Shapefiles forget something, a Shapefile is actually a minimum of three files.

The three mandatory/required files are:

• .shp - the file that stores the feature geometry.
• .shx - the file that stores the index of the feature geometry.
• .dbf - the dBASE file that stores the attribute information of features.

A recent thread on the GDAL Mailing List led to the first link about Shapefiles. The key to the thread was a discussion about where Projections are stored in the Shapefile definition. As it can be seen, there is no location in the required files for projections.

Therefore ESRI adopted a new file type (PRJ) with the extension .prj.

So how are projections defined in this file?

As we know coordinate systems in terms of mapping can either geographic (longitude, latitude) or projected (X, Y). In the PRJ definition, the coordinate system is composed of several objects, with every object having a keyword in uppercase. Objects can be composed of other objects. ESRI calls the string in this file a Projection Engine (PE). The Sole purpose of the Projection Engine is to store the metadata for a coordinate system in a string, or in a .prj file. This string, which ESRI also calls a PE (not for Physical Education!) string, must be continuous and not broken.

Now the scary part: You can define your own units, datums, and spheroids!

An example, taken from the ESRI website, shows how we can define in .prj file a projected or geographic definition.

As we know projected coordinate systems (of which maps are) are based upon a geographic coordinate system (latitude, longitude), so the in their sample file, a projected coordinate system first is defined.

For example, UTM zone 10N on the NAD83 datum is defined as

PROJCS["NAD_1983_UTM_Zone_10N",
,
PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",500000.0],
PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-123.0],
PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_of_Origin",0.0],
UNIT["Meter",1.0]]

The geographic coordinate system name is followed by the datum, the prime meridian, and the angular unit of measure.

The geographic coordinate system string for UTM zone 10N on NAD 1983 is:

GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137,298.257222101]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.0174532925199433]]

The full string representation of NAD 1983 UTM zone 10N is:

PROJCS["NAD_1983_UTM_Zone_10N",
GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137,298.257222101]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",500000.0],
PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-123.0],
PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_of_Origin",0.0],
UNIT["Meter",1.0]]

As I stated earlier, you can define your own to use the predefined names for map projection and parameter object's.

I hope the above information aids in your understanding of Shapefiles and how projections and coordinate systems are defined.

My best advice for dealing with PRJ files; copy one that you already have and use it as a base, then modify the parameters as you need.

Good luck and have fun!

GDAL2Tiles - Summer of Code - EPSG:4326

Being involved in the Open Source world as it applies to Geospatial is interesting.


Google does almost every year, a Summer of Code, and OSGeo was involved in this as well. More information can be found on the Wiki about this years projects.


Tiling and speed have always been issues with Internet Mapping - especially with Raster Images.

Just recently Klokan Petr Pridal described his Summer of Code project as being able "to allow easy publishing of raster maps on the Internet. Your raster file (like TIFF/GeoTIFF, MrSID, ECW, JPEG2000, JPEG, PNG) is converted into a directory structure of small tiles (TMS compatible), which you can just copy to the webserver. Simple webpages with viewers based on Google Maps and OpenLayers are generated as well - so anybody can comfortably explore your maps on-line and you do not need to install or configure any special software (like mapserver) and the map displays very fast in the webbrowser. "


Through Open Source we are seeing innovation in the way software is being developed and the speed of take-up and understanding by developers worldwide. Tools, such as the one's developed by Klokan will soon be on internet maps worldwide, using libraries that have being developed by Frank Warmerdam (FWTools, GDAL) to make mapping easier.


Of course, there are always co-ordinate system problems involved and some restrictions. Currently the software (if using Google) is restricted to EPSG:4326 (Good ol'WGS84!). There is one line that scares us Surveyors - "World files and embeded georeference is used during tile generation, but you can publish a picture without proper georeference too".


No Georeference means - where are we? Be careful out there. Co-ordinates are always important in any mapping application. It is always important to know your:


1) Datum


2) Map Projection


3) Ellipsoid


as a very bare minimum. Depending on the Map Projection used, then you always have to consider the False Easting, False Northing, Central Meridians, Latitudes, etc..


As the technology increases and becomes faster and easier, we can tend to make mistakes easily and make too many assumptions about the data. Not all data is EPSG:4326. Remember, maps are only as good as the people who produce and understand them. When you have people reading and relying on your maps, especially when pulling data (for eg. via WFS), there may be different co-ordinate systems - hence roads may move, lakes may be on cities, etc., etc..


As the blog posting several days back pointed out, boundaries are important and the idea of Redrawing the Map was discussed. We'll see what happens in resolving this political dispute, but again the accuracy of the data and the maps produced will be important.


Now, for an excellent example of GDAL2Tiles - Summer of Code, take a look at the beautiful country of the Czech Republic and see some tiling and speed that can put some commercial software to shame.


An example of tiling using Open Layers is the city of Moscow.


See if you can find the Kremlin and Red Square.


Keep on watching this space and I'll bring more news about Open Source for GeoSpatial and someday soon will be producing some samples using Oil and Gas, Mining and Forestry Data.


It is a whole new world out there and geomatics is changing rapidly.

FOSS4G2007 - Victoria, BC

FOSS4G2007 is happening soon in the current location I've planted some roots - Victoria, B.C., Canada.

Who knows whether these roots will be permanent or not. It seems I move every two years, whether in North America or overseas. The world seems so small with Air Travel nowadays.

The opening and closing plenary speakers have all be selected and they have accepted. The keynote speakers :

Geoff Zeiss, Director of Technology, Autodesk
Tyler Mitchell, Executive Director, OSGeo
Peter Rushforth, Technology Advisor, GeoConnections

There is also a series of Lightening Talks, consisting of eight speakers, with five minutes each, that will try to being as concise an overview of open source geospatial software, community building, open geodata projects, and many demonstrations. Demonstrations always seem to grab the attention of the listeners. A picture is worth a 1000 words!

Damian Conway, will be opening the conference, with his observations of open source culture in Geek Eye for the Straight Guy.

The closing plenary will include a panel discussion featuring:

Peter Batty, Independent Consultant
Tim Bowden, Director, Mapforge Geospatial
Mark Sondheim, BC Integrated Land Management Bureau
Frank Warmerdam, President, OSGeo

And of course, I'll be presenting there, about map projections, datum's, and co-ordinate systems and how to make your way around them using Open Source Geospatial Software.

I'll be instructing this lab with Frank Warmerdam (the creator of GDAL, PROJ.4, and many other Open Source libraries).

L-03: Datums, Coordinate Systems, Map Projections & Datum Transformations

Location. Location. Location. With so many maps and datums out there, how does a person know what datum is correct? How come my GPS coordinates don't match up on my map? Why is there a shift of 100 metres? How do I transform between different datums? What is a datum? What is the EPSG? Why have GIS Vendors and Oracle adopted them? Does offshore or onshore make a difference? How come there are so many datums? This presentation looks to provide some answers to some of these questions and to point out that latitude and longitude are not absolute.

Over the decades that surveyors have been trying to map the Earth, history and politics have shaped the way we see the world. Are the borders actually there? What if one nation adopts a standard, but the other does not? Does really matter what the co-ordinate system is? Why when I draw the a UTM Projection, the lines are curved, not in a grid? Is the OGC adopting these standards? So many questions and this presentation aims to answer some of them and provide some light on a complicated and sometimes unclear topic.

Hope to see everyone there. It looks to be a great conference!