Tuesday, July 7, 2009

Newbie Notes for GIS Web Development

I have spent some time over the past couple of weeks getting to know the Open Source GIS arena, from spatial databases (PostGIS) to server software (GeoServer) to web display techs (OpenLayers). When I started, I decided to map my trail as I explored this space since I will have co-workers following along behind me as new projects in our group ramp up. Hopefully, these breadcrumbs will help other newbies in this area as well.

begin tangent
Have you ever felt that sometimes the worst people to write documentation for something are actually those that know the most about it? Once you are an expert in something, it's really hard to consciously remember all the questions that came up and problems you had to solve and research you had to do to reach nirvana. That's why with this project I tried to jot down questions that I had as I went, and kept the question list even after I figured out the answers. It helped me not forget what I didn't know. I've used this technique before, and strongly encourage new hires who come onto my projects to do the same so we can fill in the holes in our developer documentation. Hmmm... sounds like another blog entry in there...
end tangent

This first entry is about the "what's what" GIS and "who's who" in the Open Source area. It is not intended to provide all the information you need to work on a GIS application. It is intended to tell you where to get that information. It also explains a few questions and misconceptions I had as I started down this path.

What is GIS?

First, to get your head around the GIS terms and concepts, read this short overview on GIS concepts from developerworks. One thing the article does is define what a layer is and relate that to the term feature. What it leaves out is the term FeatureType. A FeatureType defines a type of data, listing the attributes that go along with it, such as name, shape, and other meta-data (e.g. population for cities, road type for a set roads (highway, secondary, dirt, etc)). The layer concept is a way of visualizing a bunch of features of the same FeatureType. For example, in GoogleEarth, you don't think about turning on and off the road FeatureType or the city FeatureType. You think about showing/hiding those layers. Some software (like GeoServer) uses the term FeatureType in places where we more naturally think in layers. Just get used to moving back and forth between the two terms.

What's a Map?

Something that is missing from all the literature is a strict definition of what is a map. Tutorials describe layers, FeatureTypes,and features. Standards define ways of storing geographic information, retrieving that information, applying styles to it, and rendering it. But I couldn't find anything that strictly defines what a map is. What this means is that each application or library has it's own concept for what comprises a map, or it may not have a unified concept of a map at all. OpenLayers (the most popular way of rendering maps in web pages) is based around the concept of a map. It's map has layers of data that can be shown/hidden independently. The map also has tools for zooming, paging, measuring, highlighting, and possibly even editing the data. GeoServer (an Open Source map server), on the other hand, is more focused on individual data sets (i.e. independent layers or FeatureTypes). How those sets of data are combined into a single, visual display that we would call a map is up to the consumer of the data (such as a web app using OpenLayers or a desktop app like uDig).

If you are looking for the least common denominator for the concept of a map, think of a display of layered geospatial information with one or more "base layers" comprised of static (or nearly so) data (e.g. geographic features, political boundaries, rivers, roads, cities, etc) and zero or more "live layers" comprised of data that can change with relatively high frequency (e.g. weather images, traffic patterns, earthquake epicenters, recent Elvis sightings, etc).

Who's Who

To learn about the Open Source standards and tools in the GIS space, flip through Scott Davis' GIS for Web Developers presentation while you listen to his GIS podcast. He has a lot of other great content on his mapmap site. If you like his presentation style, pick up his GIS for web developers book. It comes in a PDF format for instant gratification.

Now that you've been exposed to some of the concepts and heard mention of the major players and most popular apps, you can read what Wikipedia has to say about them. As usual, the Wikipedia pages have links to the organizations' sites as well as important related concepts.
  • Open Geospatial Consortium (OGC) - the first thing to know about the OGC is that they publish the WMS and WFS standards. They have many other standards as well, but those two are the primary protocols by which you will get data from a map server (like GeoServer) to a UI (like OpenLayers).

  • ESRI ArcGIS - ESRI is the 800 lbs. gorilla in the GIS space. It is sort of like the Oracle of GIS. It has it's own commercial, proprietary software suite called ArcGIS. If you have heard of Shape files, this is the company that invented that format. It's not open source, but it's good to know who they are. In my situation, I have existing systems that feed into ArcGIS layers, so I have to work with it as well as with other data via the OGC standards.

  • GeoServer - Highly extensible, open source WMS/WFS server. A good application to keep in mind if you want to run an application that is a single source for both "base layer" data and your app-specific data. Something that was critical for my project is that it can pull data from an ESRI ArcGIS server as well as other sources like PostGIS or raw images. It's online user manual contains some good sections on basic concepts for serving and formating geo data over HTTP, including Styled Layer Descriptor (SLD), WMS, and WFS.

  • PostGIS - Geo-spatial extensions to PostgreSQL. This is the most popular (and powerful) OpenSource geo-enabled DB. MySql also has geo-extensions, as does MS SqlServer and Oracle (called Oracle Spatial). BostonGis has a great tutorial on installing PostGIS and the basics for using it.

  • OpenLayers - JavaScript library for displaying maps and mapping tools. If you see a map on a web site and it isn't an embedded GoogleMap or MapQuest map, it's probably being rendered by OpenLayers.

  • Open Source Geospatial Foundation (OSGeo) - not to be confused with the OGC, above. The OGC is a standards organization. OSGeo is a non-profit that supports open-source geo software projects and related initiatives. They support web mapping, desktop apps, geospatial libraries, and other types of projects, including GeoTools and OpenLayers.
If you thought that keeping OGC and OSGeo straight was confusing, just wait! There's one more. OpenGeo is a company (sorry, a "social enterprise") that integrates the most popular Open Source GIS technologies (like those listed above) into a single, supported stack or application framework. I have no experience with them, but if you need to get a GIS app up and running quickly, they sound like good people to call. I am sure there are other such organizations out there that can help write your software or train your dev team. I just now found that Scott Davis' ThirstyHead company is offering 3-day GIS training course.

GIS Blogs

There are probably a 100 GIS-oriented blogs. Start with planetgs. That is an aggregator for many others. If you find that articles coming from a particular source are good, you can follow it directly. I happen to like Fuzzy Tolerance for it's good content on GIS and Open Source web development in general and concise monthly roundups.

Where to Get Data

If you want to get "base layer" data (geographic, political, structural, etc) to display underneath your app-specific data, browse through these sites:


If you have address data or other geographic text and want to find out how to plot it on a map, there are a few free geocoding services. geocoder.us is a good starting place for testing your app if you just have data in the US. There are sister services for other countries. Google also has a geocoding service, but the license requirement says you have to use the data to display on a google map. (At least, it does in one place. In another place, it just says display on a map, without specifically stating google map.)

Getting Data Via WMS versus WFS

Before going into more detail about some of the above apps and libraries, I want to clear up something that confused me at first. How data is stored (vectors or rasters) is a separate concept from how it is distributed and displayed. When you request data from a WMS service, you will get back an image. It doesn't matter if the data is stored as a jpeg or tiff or as a Shape file or a set of XML files or it is in a database table. Whatever the source, the map server converts that data into an image using some standard styling rules and sends that image over the wire. On the other hand, if you request the same data via WFS, you will get some form of data list, usually in an XML format known as GML. How that data is then transformed into some visual display is up to the client.

The difference between WMS and WFS has an impact on how you can combine data from different sources in the same web-based map when using OpenLayers. WFS layers can be subject to the cross domain scripting limitation. But that is the basis for another post.

Reference Apps

Finally, here are some apps that let you drool over the possibilities of what GIS tools can do for you.
Have your own cool web-based GIS app? Post a comment and I'll add it in.

Next up... A few notes on setting up datasources in GeoServer.

1 comment:

  1. Another app to check out is one over on the USGS site http://maps.ngmdb.us/dataviewer/


Note: Only a member of this blog may post a comment.