Using GDAL/OGR in Ruby to generate a Shapefile

Lately I have been playing with the GDAL/OGR library to generate a GIS Shapefile. The good thing is that there are Ruby bindings for this library. The bad thing is that there is absolutely no documentation for those bindings. So when something fails you get stuck pretty often. This is not a call for documentation but more of a summary where I got stuck and what to watch out for.

First, get the latest version. If you do not like playing with make like me, you will definitely appreciate this ubuntu repository with quite new versions of binary packages.

My first task was to build a ESRI Shapefile and for that I needed the following code to work:

driver = Ogr.get_driver_by_name('ESRI Shapefile')
data_source = driver.create_datasource('output.shp')

Two days I was stuck at this, trying a lot of stuff. But then irb came to the rescue. I canot tell you why but the magic is in how you require the library into your script. This is what I did when I started to write the script:

import 'gdal/gdal'
import 'gdal/ogr'
import 'gdal/osr'

And I was getting NoMethodError for create_data_source. Try reducing prevoius lines to:

import 'gdal/ogr'
import 'gdal/osr'

And bang! It all works. So after I got this solved everything else was a piece of cake. Just open up the Python bindings documentation and the translation is rather easy. SomeMethodName is some_method_name in Ruby. Ogr::SOMEConstant is Ogr::SOMECONSTANT in Ruby etc.

Doing gem update --system might lose all your gems on Debian/Ubuntu

Well, if you installed RubyGems on your Ubuntu/Debian system via the package manager and are tired of waiting for the .deb package to get updated you might try doing this:

$ gem update --system

This caused a funny thing for me, all the installed gems disappeared! Well it is not that bad if you have your gem list backed up. If you do not. Than it is bad and you have some finding out to do. So remember back up you installed gems with

$ gem list > gems-list.txt

UPDATE: One more thingi I forgot is this.

Rails 2.1: Gem dependecies causing apps not running

Well i case you run into something like this while using new gem dependecy features in Rails 2.1:

ruby/site_ruby/1.8/rubygems/version.rb:237:
in `initialize': undefined method `collect' for # (NoMethodError)

Try updating your RubyGems installation:

$ sudo gem update --system

or on Windows just

gem update --system

And you should be just fine.

Care about your database!

Interesting post at Ruby Fleebie today. I did not realize that it could take so long for a person to realize that an index is missing.

Now here is my solution to the problem. First if a query runs slow use MySQL's feature DESCRIBE QUERY and it will tell you why it is slow (well it will not tell you, but it will provide you with some pretty helpful information).

But this should not happen (but it happens). The first thing you should really do is something else. It is so simple. When I am writing a migration I apply following rule:

For each line

t.column "column_name", :some_numeric_type

that has column_name ending with _id I immediately add one more line

add_index "table_name", "column_name"

See? This way all my foreign keys are indexed from the beginning.

It is simple to care about your databases. Do it. It is essential.

Rails deployment problem? No problem!

Disclaimer: This is just a stream o thoughts I have been wanting to get out of my chest for some time. I have published some stuff in Czech (my native language) and I feel like putting something down in english. I am not sure there will be a conclusion but I will do everything to stay with facts and leave personal stuff and feelings out.

In the past I had some experience with Ruby On Rails on shared hosting and recently I have tried virtual server hosting and ultimately moved to my own dedicated mashine placed in a housing center. This said I think it is safe to say I have at least some overview of what issues people on shared hosts face every day and due to close cooperation with my hosting provider and also my own admin experience I also have some experience of what issues hosting providers/administrators face when setting up environment for their clients.

Right now there are multiple Rails deployment models. I will not go though each and every one separately, but I think they can be easily categorized into two groups. Sice Ruby is an interpreted language you need an instance of the interpreter running for your app to work. Since the classic Ruby interpreter is not in blazing speed it is good to load up the interpreter cache up the code in memory and then execute it. The question is "Who starts those instances?" and this is how I categorize Rails deployment models:

1. Instances of interpreter are started by the webserver (apache)
   


2. Instances are started externaly and independently on webserver

If you have any experience with Rails deployment you will agree with me that nowadays the only viable option is number two. You want your code to be cached and serve several requests before being reloaded. And when updating your application you canot restart the whole webserver which is propaby serving dozens of other sites. So what now?

This is mainly the issue of shared hostings. It is simply unacceptable to allow your users interfeere with the front end webserver (option number one is out). But allowing users to start some back-end processes is also risky because controlling who uses which port is really not easy. Still no conclusion.

When I started setting up my own production server I first reached out to FastCGI (more accurately mod_fcgid for apache) and I was really supprised by the performance. The application just felt blazinly fast. The headaches came when setting up multiple application deploying with Capistrano. Each deploy caused restart of Apache. Which is not pretty. I started looking for alternatives.

Then I gazed on SwitchPipe. A project I read about some time ago but completely forgot it (since it was no good for me at shared host). I actually think SwtichPipe-like kind of thing might be the ultimate answear to Rails deployment.

Rails is single threaded so a situation similar to Java/Tomcat stack when several Java threads are serving requests from Apache is not applicable. What Threads to in Tomcat, Mongrel (or any other Rails container) has to do with multiple instances. Again there are several options how to archieve this. For example this tutorial describes rather througly how to setup apache to proxy requests to your local Mongrel instances. Its a cool tutorial but when I saw all the configuration needed my head started to spin. This not Rails-like at all. No way I am gona do this.

The great thing about SwitchPipe is that all the configuration (even if you want to change almost everything) is about 10 lines max for a signle Rails/Mongrel application.

For Apache to use your SwitchPipe cluster you need to do the follwing in your .htaccess:

  RewriteEngine On
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteRule ^(.*)$ http://127.0.0.1:10000/myapp/$1 [P]

All the lengthy stuff Rails generates for you can go away. It is pretty lovable. Now the actual SwitchPipe config for your site:

 
path: /path/to/your/app
type: mongrel_rails
max_instances: 2 # whatever you want
timeout: 30
user: yourUserName 
# this is good for shared hosts, apps run with permissions of actual users
# files get created right and users can work with them

I might be short sightet. I sure do not know all the issues faced by the hosting providers/clients but this kind of setup is pretty easy to automatize. Restarts of the cluster are curently handled by touching the SwitchPipe configuration file, but this is promised to get improved (I myself am hoping to try and create the patch).

Still, Rails deployment is far different from what the PHP folks are used to. Rails is just different. So what? Get used to it or dont use it! Yes, for me it is that simple. After my recent experience with setting up multiple virtual-host webserver I have came to conclusion that the Rails Deployment problem is just a FUD, nothing else. You know Java, right? You know .NET right? There are not plenty of hostings for applications like this. I say they have a deployment problem, not Rails. Whether you are setting up your own (virtual)server or shared hosting environment for your users, learn something about Rails and Ruby and Mongrel, check out SwitchPipe and enjoy. Rails world is really fun.

RailsEnvy is over?

This is definitely shocking. Well I was in shock for like thirty seconds. Than I realized what day it was when guys published it and well I got a big laugh. Great joke guys, great one!

Using Rails and Capistrano on Ubuntu from gems

If you (as I did) install Capistrano and Rails as gems on your Ubuntu mashine, you might get into troube that bash simply does not recognise those fancy shmancy rails and cap commands. As Neoarch pointed out, the solution is adding the following...

#The following sets the path for Capistrano
export PATH=$PATH:/var/lib/gems/1.8/bin

...to your ~/.bashrc file.

Enjoy.