Transformations in PostGIS

Lately I've been doing a lot of development centered around PostGIS (http://postgis.refractions.net/) and ran across a problem that has a relatively easy solution (for those well versed in GIS) but finding an answer proved to be somewhat difficult.

Here are the assumptions and the requirements before we look at the problem:

• A point is defined by a latitude and longitude.
• A line is a defined by a series of points.
• A buffer is defined as a shape that envelopes a line. The width of the buffer is specified by the user in meters. The distance value the user supplies is a +/- distance from the supplied line.
• We are assuming WGS 84 as the target map projection.

PostGIS provides a handy buffer function called ST_Buffer and in order to use this function the geometry supplied to the buffer function along with the desired buffer distance need to be in the same coordinate system. See the problem yet? My coordinates are in decimal degrees (i.e., latitude and longitude) and my buffer distance is in meters. So I need to transform the line into a different coordinate system that will allow me represent my geography based shape in metric coordinates. This transformation is non-trivial since converting meters to decimal degrees relies on the use of a map projection because the world isn't flat (sorry!). Before we go any further let's look at the query I've developed so far:

SELECT ST_Buffer(GeomFromText('LINESTRING(-76.543 42.567, -76.012 42.345, -75.890 42.445, -75.543 42.330)'), 300.0) AS buffer_shape

The query is valid but the buffer distance is thought to be in decimal degrees (i.e., WGS84) with respect to PostGIS and therefore produces a shape that pretty much covers the world. I tried guesstimating the conversion from decimal degrees to meters but quickly realized, at best, a guesstimate would be terribly wrong.

To solve this problem - I employed the Transform function from PostGIS. In order to get this function to work properly I had to determine what coordinate system to transform my geometry into...this is where my Google-fu fell relatively short. Luckily I went to http://gis.stackexchange.com and found a reference to the proj4js.org project. Using proj4js I was able to determine the SRID of the desired metric coordinate system (that SRID is 900913). This coordinate system bases coordinates off of meters instead of decimal degrees. So I changed my original query to be the following:

SELECT Transform(ST_Buffer(Transform(GeomFromText('LINESTRING(-76.543 42.567, -76.012 42.345, -75.890 42.445, -75.543 42.330)'), 900913), 300.0), 4326) AS buffer_shape

The above statement transforms the line from WGS 84 to a Mercator projection, performs the buffer operation, and then does another transformation to project the resulting buffer shape back into WGS 84. Voila! Again, nothing earth shattering here but if you don't speak GIS all day then performing the transformations may not be entirely obvious. In my opinion the PostGIS documentation falls short when mentioning the transformation details.

Command Scripts

Recently I had to create a test that simulated a catastrophic failure to ensure the integrity of an embedded database. Development time for the test was limited so I set out for the quickest approach and saved myself the hassle of mocking test cases for another day. The test would initialize the database and then insert a random number of rows. During the test the OS would kill the process while rows were being inserted and then restart the process to ensure the integrity of the database.

I ended up coming up with a command script in Windows XP that I felt was worth sharing due to its relative obscurity. I will save the majority of the details and only highlight the important points.

First, let's look at the script I made to launch the java process responsible for bootstrapping the database:

START java -cp . Sleeper
ping 1.0.0.0 -n 1 -w 5000 >NUL
FOR /F "tokens=1-2" in ('jps') DO (
IF "%%j" == "Sleeper" (
SET PID=%%i
)
)
TASKKILL /PID %PID%

For me, the coolest part of this script was using the jps command to get the PID for the process that I launched in the first step. My second favorite feature (*cough* hack *cough*) was the use of the ping command (not my idea see the link here). It's the only way I could get the script to wait for a few seconds before killing my process - allowing the test running a separate process to reach the portion of the test where the insertion was occurring. It's not the most reliable or sane approach but it worked perfectly for creating a quick simulation.

The second snippet isn't as exciting as the first but think it's a huge timesaver when trying to execute a Java program that has a very large number of dependencies all located in the same directory. This script scans the directory (e.g., the lib/ folder) and adds every filename ending with .jar to the classpath. Again, keep in mind that this isn't optimal - but it's something quick-and-dirty that you can use to get going:

::EnableDelayedExpansion must be turned on in order to
::programmatically append to the classpath
SETLOCAL EnableDelayedExpansion
SET CLASSPATH=.

FOR %%i IN (lib\*.jar) DO SET CLASSPATH=!CLASSPATH!;%%i

%JAVA_HOME%\bin\java -cp %CLASSPATH YourClassGoesHere

Be sure to include the SETLOCAL (http://ss64.com/nt/setlocal.html) directive and to use the ! operator instead of the % operator when doing the assignment.

Quality of Documentation

I spend a lot of team reading API documentation to try and figure out how a particular function works, or most likely, I use the documentation to determine how a set of API calls work together to produce a desirable outcome. On most days I don't consider myself to be one of those developers "who just gets it" so I rely heavily on the completeness and quality of the documentation to complete my task in a timely fashion. Recently I have been developing with Java during the day and PHP (more specifically Drupal) by night. I thought juxtaposing (don't you just love that word?) the documentation for the Java platform with Drupal would provide some meaningful insight for updating Java's javadoc utility.

Since Java's inception, the language has grown to an unimaginable size encompassing everything from Desktop to Enterprise to Mobile development. The javadoc utility has remained consistent throughout each release and produces the same clean, intuitive documentation today as it did several years back. In my opinion Java makes it easy to create quality documentation and the platform documentation that Sun's staff puts out is clear, concise, and objective. Enter the API documentation for Drupal.

The documentation for Drupal is not auto-generated. Someone has to sit down and document a function and then publish the documentation for the rest of the world to see. Pulling yourself away from the code to the documentation will result in a lower quality of documentation because the developer will need to make a mental note to go back through code and extract the documentation - this may not happen as scheduled resulting in incomplete platform documentation. As a result of this, Drupal's documentation is concise but does not provide a big picture view of how a particular function interacts with related functions. Despite this shortcoming, the source code for each function is included with the documentation allowing a developer to dive right into the details to figure out how to best use the function. Additionally, the documentation includes a search utility that makes it easy to find the specific function you're looking for without knowing its exact signature.

PHP and Java differ greatly in functionality so I wouldn't want to see the javadoc utility get cute with an autocomplete, search utility but I would think Sun would update the javadoc utility and provide a "web 2.0" approach to the generation of their documentation. Scrolling through ALL the classes is getting a little old. Has anyone seen a third party javadoc utility that provides a "richer" experience?

Too Much Code

With the numerous software frameworks in existence today, it is easy to question yourself as soon as you associate copying and pasting with software reuse. I use to think that developing user-interfaces for desktop applications involved a lot of redundant (redundant is a technical term for boring) code but had to correct myself the minute I started working with web applications. Instinctively the copying and pasting felt wrong and started searching for more "clever" development practices to apply to a somewhat daunting code base.

With Struts, a developer creates JSP pages, maps the functionality found in the JSP page to an Action via the struts-config file, then implements the Action to execute the desired functionality. You wash, rinse, and repeat for every single JSP page and watch the amount of code pile up even when implementing the simplest of web applications. A developer could spend more time on consolidating the code but runs the risk of decreasing the site's maintainability if the code becomes too generic. Sometimes people forget how clever they really are until they have to revisit a project where everything is handled with general terms. For me, it has been easier to focus on developing maintainable code through refactoring and documentation then worry about the number of files a web application contains.

Until I investigate more, I found a way to optimize the annoying "copying and pasting" design pattern by using code templates in your favorite IDE (my favorite is Eclipse). With Eclipse a user can make use of a number of built-in code templates and can add their own pretty easily. This is a pretty simple utility that can save a developer time but is often overlooked. I found two articles (part 1 and part 2) that go through the basics of using and creating code templates within Eclipse.

Mock Objects

Obviously testing software before it is released to the masses is important. Testing is also a great way to verify project requirements and in some extreme cases, drive the entire development of an application. When you start thinking about large web applications that operate in a distributed environment and service a large number of clients - testing should be the first thing that comes to mind. However, it seems that in certain cases, the size and complexity of enterprise applications make testing a resource intensive, daunting task that gets ignored. As a naive enterprise developer, I set out this past week to investigate the various testing strategies and break the testing stigma that plagues smaller enterprise development boutiques.

From the research I conducted, the most popular approach seems to be making use of mock objects during development. Since most web applications require interaction between multiple components or services it is often hard to write a test case for an individual component without first implementing all collaborating components. The basic principal behind mock objects is to provide basic, functional collaborating objects without disrupting the task at hand. Aside from being useful during development of an individual component, using mock objects also allows you to perform additional testing on the component without impacting live data or configurations (this is handy for verifying requirements). The mock objects approach promotes the use of a factory design pattern to provide a single point of entry for "mock" components. As someone who is new to the enterprise scene, this concept took a while to understand. I hope that all the work I spent digesting this information pays off when I take my very first web app from soup to nuts.

Enterprise impressions from a desktop perspective

Up until this point in my career I have only dealt with client (or desktop) programming. My initial, albeit overly optimistic, impression was that web programming was a simple enough technology that I could pick it up in a matter of hours if need be. Before I go any further let me give you a little background, I graduated college 3 years ago and started my professional career working for a research and development company that helped clients transform theoretical knowledge into software prototypes. Most of the development was done in Java with the occasional performance-centric work being completed in C. Towards the end of my time at my previous company I began to get curious about object relational mapping frameworks (a la Hibernate). After working with Hibernate for a few months I quickly began to appreciate the amount of work that goes into building enterprise systems. Coincidentally I had also started hunting for a new job so I thought it was an excellent opportunity to broaden my horizons and dive into enterprise development.

The choices for a web programmer (web programming does not include design related tasks) are limitless. Speaking from a Java perspective alone, there are a wide variety of frameworks and configurations to choose from when developing a new web application. In my new job I will initially be working with the Struts framework and the common controls presentation framework all running inside of the Tomcat web container. I spent the first few days sifting through the different point releases of the servlet specification and the struts framework trying to figure out how everything fit together. Needless to say it was overwhelming but the exploration quickly changed to challenging and enjoyable. So far the only real frustration I have encountered is there seems to be a serious gap in unit-testing solutions for struts based applications. Sure, a few exist but none seem to be as complete as I'd like.

I would encourage you to take the time to at least survey the enterprise landscape if you haven't done so already. Over the next few months I intend to update this blog with my impressions based on my journey and hope it will be useful information for developers following a similar path.