Mollusca Code - Home

Importer v0.2

2009-08-22T12:44:00Z

Danlucraft pointed something out the other day: Why would Importer require that you use a hash? And of course he was right. So after a little review: Importer, second iteration:

module Importer

  module ClassMethods
    def importer(package, classes)
        classes.each do |item|
          path = package + "." + item
          import path
        end
    end
  end

  def self.included(receiver)
    receiver.extend ClassMethods
  end

end

This will remove the pesky hash notation:

require 'importer'

class Banzaiii
  include Importer
  importer 'org.eclipse.swt.widgets', \
      %w{Display, Shell, Button, Canvas}
  importer 'org.eclipse.swt', ['SWT']

  def build
    puts SWT::BOLD | SWT::ITALIC 
  end
end

Maybe this will evolve into something that enables you to do the follwing:

class Widgets
  include Importer
  importer 'org.eclipse.swt.widgets.*'
end

# and thus
canvas = Widgets::Canvas.new

Which is exactly what is currently missing from jruby…

Cleaning up imports in jRuby

2009-08-13T14:01:00Z

Import Puts Noise In My Class

Today I stumbled accross something like this:

class SwtBuilder

  import org.eclipse.swt.widgets.Display
  import org.eclipse.swt.widgets.Shell
  import org.eclipse.swt.widgets.Button
  import org.eclipse.swt.widgets.Canvas
  import org.eclipse.swt.SWT

  import ch.mollusca.foo.Bar
  import ch.mollusca.foo.Jiffy

  def build
    puts SWT::BOLD | SWT::ITALIC
  end

end

SwtBuilder.new.build

Alas, jRuby doesn’t support

import org.eclipse.swt.widgets.*

there is plenty of noise throughout this class definition. It’s a lot of vertical clutter and I don’t like it. This will even get worse for any class that ties together a few java classes.

Enter Importer

I tried to figure out a way to clean this up. Maybe I’ve hit the spot, but I think it’s not quite there yet. Anyway:

module Importer

  module ClassMethods
    def importer(hash)
      unless hash[:path] && hash[:items]
        puts "need to specify path and items!" 
      else
        hash[:items].each do |item|
          path = hash[:path] + "." + item
          import path
        end
      end
    end
  end

  def self.included(receiver)
    receiver.extend ClassMethods
  end

end

This allows me to write:

class SwtBuilder

  include Importer
  importer :path => 'org.eclipse.swt.widgets', \
    :items => ['Display', 'Shell', 'Button', 'Canvas']
  importer :path => 'org.eclipse.swt', :items => ['SWT']

  def build
    puts SWT::BOLD | SWT::ITALIC
  end

end

SwtBuilder.new.build

It’s formatted in two lines for layout reasons, but in principle can nicely be written on one line. Any comments on the idea?

OSX Java Update breaks PostgreSQL JDBC4

2009-06-17T10:09:00Z

It seems like the latest Java update from Apple breaks the JDBC4 version of the postgresql database driver.

I currently have two Grails apps doing similar things. One (by accident or luck, your call) uses postgresql-604.jdbc3 as it’s database driver. The other one uses a different driver, namely postgres-604-jdbc4.

Today I updated my installed Java on the Mac using software update.

Suddenly one of the Grails apps was broken.

.... tries to work out what has gone wrong

I got a ClassNotFound Exception while loading the driver. So I tried to figure out where the two apps differ. It turns out that the only relevant difference between the two apps (database-driver-wise) is that one uses jdbc3 and the other jdbc4.

I’m not blaming anyone, but I suspect the update broke it.

So if anyone faces the same problem while running a Grails app, there you go: Replace jdbc4 with jdbc3!

And now part of the stacktrace for googleability:

Starting integration tests ...
Cannot load JDBC driver class 'org.postgresql.Driver'
java.lang.ClassNotFoundException: org.postgresql.Driver
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at org.codehaus.groovy.grails.cli.support.GrailsRootLoader.loadClass(GrailsRootLoader.java:45)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:164)

Addendum

And yes, I’m aware of the fact that the JVM on OSX is 1.5. But previously it used to work, so I figured this might still be of some use to some people. Maybe JDBC4 was a bad idea all along the way, however.

Grails + Spatial Data 2.0

2009-05-11T07:59:00Z

This is intended as a follow up to this artile: Grails + Spatial Data

We recently discovered (actually we were nudged in the right direction…) that one can completely do away with the manual hibernate mapping, by adding the following code to the domain class that contains spatial data:

class Event {
  String title
  String description
  Url url
  Point spatial

  static mapping = {
    columns {
      spatial type: org.hibernatespatial.GeometryUserType
    }
  }
}

I think it doesn’t get any sweeter than that.

The rules of Code Club

2009-04-27T14:22:00Z

Something I just came up with, while typing for my students:

These are the rules of Code Club:
---------------------------------
1)  The first rule of Code Club is, you do talk at Code Club.
2)  The second rule of Code Club is, you DO TALK at Code Club.
3)  If someone forgets to write a test first, coding is over.
4)  Two guys to a screen.
5)  One thing at a time. Especially in classes and methods.
5)  Coding will go on as long, it takes to pass the tests.
6)  If this is your first day at Code Club, you have to code.

How much info is there in a RAW-File?

2009-03-09T20:16:00Z

I’m simply amazed once again, how much information there is in a single RAW-file. I with my humble editing skills am able to pull of stunts like the following:

Image the way that Canon DPP and Lightroom both agreed more or less agreed should look like:

Here’s what there is to do to get the following result:

The white balance is off. I used a flash, but bounced it off a wall, so it’s not the cameras, DPPs or LRs fault, that it’s set to “flash”
Tweak in LR ‘till cooked right
1. Pull up exposure a bit.
2. Adjust white balance.
3. Pull up the exposure some more, so we have a black spot and a white spot (at least something close to both)
Off to editing in LAB mode
1. Tweak curve of L-channel to get the highlights punchy.
2. Some more monkeying around with the L-Curve
3. Apply very moderate unsharp mask
4. steepen the b-curve a tad, remove slight color cast at the same time.
done.

Programming Groovy

2008-12-11T16:18:00Z

I will probably not this a lot, but this one is necessary. I just received my copy of the book “Programming Groovy” by Venkat Subramaniam, read it through in a few days and I gotta say:

It’s awesome!

Very clear and concise explanations. Even MOPping is explained in very short and clear examples. I like that!

My new notebook has arrived!

2008-10-17T20:15:00Z

Yay! I’ll have another beer on that.

The new notebook has arrived. Behold! Here goes the Lenovo x301.

One of the nicest pieces of technology I’ve seen. This eats Airbooks for breakfast.

Here’s a few point to consider:

1440×900 resolution
DVD-Burner
Solid State Disk (hell yeah…)
more than one USB port (pokes Airbook)
VGA Port
DisplayPort port (that one will rock)
Awesome speakers. Best I’ve heard so far in any notebook.

Since I ordered it without an operating system, I installed my favourite i686-distro: Archlinux. Install went really smoothly. Everything that matters to me seems to be working. Had a few hickups of Xorg along the way, but I think it’s fine now. Did I mention the keyboard rocks? And that it’s only 1.4kg?? :)

I will definitely post some more on this. KDE 4.1 is running smothly and boot times are ridiculously short. The SSD is definitely the way to go, if you want fast load times. That’s true for software loading time as well. This thingie loads firefox faster than my desktop…

Some more geeking on this mervelous piece now…

Calling stored procedures from Grails

2008-10-16T08:18:00Z

Calling stored procedures from Grails is not that hard to achieve. One needs to wade through the HQL documentation though, which I find slightly counter-intuitive. I’d rather see stuff like that on the grails page.

So, how do we call a stored procedure then?

What stored procedure?

Here’s the situation:

The events from the last post are in the database now.
The database has been spatially enabled according to the PostGIS plan.

We now want to find the closest events, to a certain point on the earth. Let’s assume we have successfully geocoded the point, so we have its coordinates in lat;long form.

The basic function that we want to call in plain SQL is:

select * from event e
  order by distance_sphere(
  Geometry('POINT(lat, long)'),e.location_id)

The distance_sphere() function calculates the distance of two points on the earth given in lat;long form. However the function takes two argumenty of type geometry, so we have to create one first.

Calling it from grails

To call this function from grails and get a nice an clean Collection of objects back (we don’t want to mess with ResultSet, do we?) we have to grab a hold of the current hibernate session. This is pretty easy: We just have it injected by grails:

Add the following line to the EventController class:

def sessionFactory

The sessionFactory object is now defined inside of this controller and will have its value injected by grails. Assuming that the function is called by some form using POST, the code for our controller function boils down to this:

def searchByLocation = {
  if (request.method == "POST"){
     def lat = params['lat']
     def long = params['long']

     def session = sessionFactory.currentSession

     def sql = "select * from " +
     "event e order by distance_sphere(setsrid" +
     "(geometry('POINT(${lat} ${long})'), 0)," +
     "e.location_id) limit 20;" 

     def results = session.createSQLQuery(sql).
       addEntity(Event.class).list()
  }
}

The setSRID() function from PostGIS has to be called, because by default grails in combination with hibernatespatial stores the points with the SRID value set to 0. This is no problem, as long as all coordinates you save are in the same reference system and you do not have to do any coordinate transformation. Otherwise you would have to handle that as well.

What’s nice about this method of calling the stored procedure through hibernate is that by calling addEntity() we can set the returned type and calling list() we get a nice and clean java collection.

That’s it so far…

Important note (This is new)

As pointed out by Matt, this code is vulnerable to SQL-Injection type attacks, if used directly like that in a setting where this is of any use to the attacker.

Please, with sugar on top, always sanitize form input data, before parsing it into a database query. In this case, this would have meant more code, but I should have added a note to the code. Mea culpa. To fix this simply make sure that the entered text looks exactly like float data and nothing else.

Switch to Mephisto

2008-10-13T16:21:00Z

I just had to do this. Sorry to all those who already liked the old blog :D

Something had to be done about the PHP bloat that was flying around before this. So I switched to this nice and clean ruby based blogging engine.

Thanks to Passenger (mod_rails) that was relatively easy to achieve. Alas, the server is running Debian, there where some problems during the initial setup of the ruby and rails environment. Why the hell does Debian ship a 1.5 years old package in the stable branch? I mean come on 1.8.5?? WTF!

Anyway. Here we go now. Shiny new blog, leaner and meaner software, where I can actually read the code and edit it and where I can also add new features. We’ll see about that though.

I will soon move the old articles over here. Actually as soon as I have syntax highlighting like pastie.org :D

Ah, and yeah, mephisto is in fact not dead. Go see the mephisto git repo.

Grails + Spatial Data (PostGIS)

2008-10-04T07:41:00Z

For a client/server application I’m currently developing, I chose grails (www.grails.org) as the framework to develop the web application and webservice part. Grails is a framework very similar in concepts to Ruby on Rails, which I personally like a lot for it’s convention over configuration attitude.

The main downside of grails, as far as I’m concerned is the huge amount of *.jar dependencies that it comes along with. This is, as long as applications do not stray from the standard data types of a web framework, very well hidden from the developer.

In this blog post I will outline a few points, on how to save geo-spatial points (no pun intended) using grails.

For said application, I have to save GIS spatial data in the database and I want to access them from the web application. I also want to call the special GIS-related stored procedures on the server. This seemed to be very hard at first, since almost no documentation on the topic can be found, especially in the documentation of the grails framework (not only for GIS data, but also for other custom data types, that is). On the mailing lists of Hibernate and Spring (libraries both used by grails), no working solutions could be found, but only the problem statements.

Digging deeper however revealed the solution to the problem, which I’m going to present now.

Let’s say we want to create a webapp, that can do the following things:

Store upcoming and past events, whereas each event has:

A Date,
a title,
a description,
and a location (This will be the GIS data)

and store members with:

Events they are attending.
A name.
A birthday.

I suppose creating a web-app using grails is already clear to readers at this point.

The domain classes

Let’s have a look at the domain classes, shall we?

# Event.groovy
class Event {
    String name
    String description
    Date date

    static hasMany = [participants : Participant]
    static belongsTo = [creator : Participant]

    public void findWithinLimit(double distance){
    }
}

# Participant.groovy
class Participant {
    String name
    Date birthDate

    static hasMany = [events : Event]
}

I intentionally left out the GIS attribute of the event class. You can easily start off with this as your domain classes and add GIS functionality later.

Adding GIS functions to PostgreSQL

Now, lets add the GIS functionality to the database. Assuming you use PostgreSQL this can be done in the following way:

Add the plpgsql language to the database (createlang command on shell)
Execute the update script on the database (my database is called geoEvent) to add the PostGIS functions and data types. On my computer that script is called lwpostgis.sql and sits in /usr/share. I execute the script using pgadmin3, which is my method of choice to work with postgresql databases…

This will make your database ready to store GIS data.

Now, first things first. Let’s add all the libraries we need to make grails aware of the data types we’re going to be using.

You’ll need:

hibernate-spatial.jar (find this on the hibernatespatial site)
hibernate-spatial-postgis.jar (same site)
jts-1.8.jar (JTS site)
postgis_1.4.0.jar (PostGIS site)
postgresql-8.2-VERSION.jdbc3.jar (it may be possible to run this with jdbc4, but I haven’t tested that one yet. Version works fine, however)

Enough libraries. ;)

Adding GIS to the domain classes

Now let’s get those GIS data types into the domain classes.

# Event.groovy
import com.vividsolutions.jts.geom.Point
class Event {
    String name
    String description
    Date date
    Point location

    static hasMany = [participants : Participant]

    public void findWithinLimit(double distance){

    }
}

Hooooold your horses, we’re not yet ready to fire up the grails app again. :D

Bind hibernate

You will first have to tell hibernate, that there is new data types that are mapped to the database in a certain way.

In grails-app/conf/hibernate create a file called hibernate.cfg.xml

<hibernate-configuration>
  <session-factory>
    <!-- SPATIAL SQL dialect -->
    <property name="dialect">
      org.hibernatespatial.postgis.PostgisDialect
    </property>

    <!-- Echo all executed SQL to stdout -->
    <property name="show_sql">false</property>

    <mapping resource="Event.hbm.xml" />
  </session-factory>
</hibernate-configuration>

This tells hibernate, that there is a special SQL dialect to learn and that there is a mapping for the type Event (This is by convention, I’d say).

Event.hbm.xml looks like this:

<hibernate-mapping>
  <class name="Event" table="event">
    <id name="id" column="id">
      <generator class="native" />
    </id>
    <property name="location" 
      type="org.hibernatespatial.GeometryUserType">
      <column name="location" sql-type="GEOMETRY" />
    </property>
    <property name="name" />
    <property name="description" />
    <property name="date" />
  </class>
</hibernate-mapping>

That’s it! Now you can fire up your application again and shine with GIS data. At least it seems that way. The tables all get created in the database, the attributes are all there and so are the foreign keys and link tables. Read on the next page how to add data to your database using your grails scaffolds.

Ticket to forms and back

To have grails render your Point in a way, that a user can add new data to the database using the scaffolded code you will have to add a PropertyEditor (which is a concept from the Spring framework…).

A property editor tells your application, how to translate a certain data type into String format and back. This doesn’t look so bad after all:

package com.vividsolutions.jts.geom;

import java.beans.PropertyEditorSupport;
import com.vividsolutions.jts.geom.Point;
import com.vividsolutions.jts.io.ParseException;
import com.vividsolutions.jts.io.WKTReader;

public class PointEditor
        extends PropertyEditorSupport{

       private Point point;

    @Override
    public String getAsText() {
        point = (Point) super.getValue();
        return String.format("POINT(%f %f)",
                    point.getX(), point.getY());
    }

    @Override
    public void setAsText(String text)
                throws IllegalArgumentException {
        WKTReader reader = new WKTReader();
        try {
            point = (Point)reader.read(text);

            super.setValue(point);
        } catch (ParseException e) {
            e.printStackTrace();
        }
    }
}

Now, grails will pick up this PropertyEditor automatically, iff you put it in the correct place. You will have to put it in same directory as the class you are trying to render and parse from a String. So this goes into the src/java/com.vividsolutions.jts.geom package. This will enable you to render and enter Points as Strings of the form:

POINT(48.8 8.8)
POINT(20 30 40) // 3d-Point

That’s quite enough for today. I will follow this post up with a tutorial on how to use the stored procedures to find nearby Points efficiently, as is already hinted in the code by the function findWithinLimit(distance).

Here we go...

2008-10-04T07:37:00Z

So this is it.

Grails + Hibernatespatial made me open a blog of my own.

Since grails in combination with spatial data is documented quite badly, I will try to fix this issue by posting my solution to the problem (Which basically boils down to a spring+hibernate problem). I hope this will make the string “grails hibernatespatial” google-able. At least all the solutions that pop up on google right now, are highly outdated or downright wrong XX(

The working solution will be up on monday, when I’m at work and have access to my code-base…