• ### Persistence in Squeak

Ah, deliciously punny post title. You’ll see, assuming you make it to the end of this thing… don’t worry, I won’t blame you if you don’t.

Over the last week or so, I’ve been working on a little toy project, partly to fill a need I have, and partly to fiddle around with developing a web application in Seaside and Smalltalk, and specifically the Squeak implementation of Smalltalk.

Now, there are many parts needed to build a functional web-based application. Obviously you need a web server to actually serve the application. You need some sort of language to implement the application in. You need a framework in which to actually build that application (okay, sure, back in the days of the wild west, people built their own, but you’d be a fool to do that today given the plethora of frameworks available which simplify the web development process). And last but not least, in all probability, you need a data persistence solution.

Of course, the first thing that comes to mind when turning ones thoughts to persistence is a good old fashioned relational database, which has been the cornerstone of data persistence for many a decade now. But when one is working in a deeply object-oriented language like Smalltalk, working with a relational database becomes rather cumbersome due to the rather substantial impedance mismatch between relational and object-oriented data modeling. As a result, we as an industry have turned to tools such as automated object-relational mappers (eg, Hibernate, etc) to try and ease the pain of this mismatch, but in general, the results aren’t what I would call pretty.

Which is why, during my first hack at leveraging a persistence solution for my little Seaside application, I decided to try something entirely different: an object-oriented database called Magma. Unfortunately, it didn’t go too well.

#### On Magma

Magma is a very interesting project. As a persistence solution, it really aims for the same space occupied by Gemstone/S: to act as a completely transparent persistence solution for object-oriented data models. By that I mean the idea is that you hand Magma an object graph, and it persists it to it’s own custom data format on disk. When you pull it back out, Magma reifies parts of the object graph you’re interested in, and when you modify the graph, Magma spots the changes and reflects them back into the persistent store.

Of course, on the face of it, this seems like absolute magic. You simply work with your objects. When you want to persist a change, you just do something like:

session commit: [ model doStuff ].


And voila, everything just, well, works. Of course, persistence is about more than just simple object storage, in that you also need to be able to query the data model, and be able to do so in an efficient manner. To that end, Magma provides a few specialized collection objects, such as the MagmaCollection class, which provide interfaces for applying indexes, querying, sorting, and so forth.

So, on it’s face, Magma looks like a fantastic solution! The transparent persistence model makes it dead easy to manipulate your data model, and you no longer have to jump through all the object-relational modeling hoops that one would normally have to deal with.

But, alas, it’s just not that easy.

Unfortunately, Magma has one serious fault that rules it out for all but the most basic data-driven applications: It’s slow. Additionally, because Magma absolutely requires per-attribute indexes for any collection you want to query, the number of indexes in a data model can grow substantially, particularly in data mining/exploration tools. Worse, Magma steps on a rather nasty performance problem in Squeak whereby large numbers of files in a single directory (as in, thousands) causes the FileDirectory class to bog down… and guess what happens when you create a large number of indexes? That’s right, a lot of files get created in a single directory, and so you get utterly dismal performance when any index is initially opened.

And as if that weren’t enough, in order to really squeeze decent performance out of Magma, you must start tweaking what are called “read strategies”. See, when you start reifying an object graph, you have to make a decision on how deep to go before you stop. After all, if you have a deep tree of objects, unless you plan to traverse that whole tree at some point, it’s a waste of time to load the whole thing all at once. So the “read strategy” dictates at what depth various parts of the object graph are read. But ultimately, what this equates to is deep micromanagement of the database behaviour, and, quite frankly, I have absolutely no interest in that.

Thus, after many days of fighting, I’ve decided to throw out Magma. Which is rather painful, as I already have an object model built up assuming it’s use. Fortunately, the very nature of Magma means you don’t really tailor the object model too tightly to the database, but things do leak through here and there, and the model itself must, to some extent, be designed to facilitate querying, traversals, and so forth. Thus, any movement away from an RDBMS will necessitate rethinking my data model.

#### A Way Forward?

So what now? Well, I’ve decided to take the hit and switch to a solution based on Glorp, an object-relational mapping system for Smalltalk, and PostgreSQL, that venerable RDBMS. Of course, this will likely come with it’s own issues, first and foremost one of installation…

Unfortunately, while Squeak package management has taken a step forward with Monticello, the management of dependencies between packages, and inconsistencies between platforms (eg, Pharo vs Squeak) means that things are a lot harder for the user than they need to be. In this particular case, the original Glorp port is rather old. So the folks developing SqueakDBX have worked to port over the latest version to Squeak, with some success. Unfortunately, their installation script doesn’t appear to work in Pharo. So I had to resort to pulling in their loader classes and then manually executing the installation steps by hand. Tedious, to say the least.

But, on the bright side, I have a Pharo image that seems to have a functional Postgres client and Glorp install, so I can start fiddling with those tools to see if they can meet my needs.

Which brings me back to the double entendre. Because returning to the Squeak world has reminded me of one thing: Occasionally the tools get in your way as much as they clear it out for you, and so sometimes you really do need to be incredibly… yes, I’m gonna say it, get ready… here it comes… persistent.

• ### Running in the Rain - Wetter or Drier?

Well, as anyone living in Edmonton knows, the weather in our area has been, well… rather crappy. Cold, rainy, windy, it feels more like the fall than waning summer. And through it all, I’ve persisted in cycle commuting, mostly because it allows me to justify (excuse) a rather gastronomically decadent lifestyle. Consequently, I’ve found myself caught in more than a few showers over the last few weeks, resulting in much dampness, and, oddly enough, a bit of inspiration.

Now, a favorite show of many folks, myself included, is Mythbusters. They attempt to perform “scientific” experiments to verify or debunk various myths, preconceived notions, and so forth. Now, one of the topics they tackled was: Does moving faster in the rain keep you dry, or get you wetter? Well, in their “experiment”, I seem to recall they found little difference between slow or fast walking, which I found a little surprising, and during a recent bike trip, I found myself pondering how it is they could have found the results they did.

Meanwhile, I’ve also been digging more deeply into the joyous language that is Smalltalk, specifically the Squeak implementation, and a related web application framework called Seaside. However, I’ve been at a loss for a small-scale project to hack up that would allow me to flex my rather atrophed Smalltalk muscles. And so it was that, a couple days ago, while cycling home in the rain, I realized, why not simulate a person walking through a rain storm, and determine whether the Mythbusters results were accurate?

Now, before I get into the details, I should point out this really is pretty non-scientific. I’m sure there are details that I’ve missed which make this simulation completely unrealistic. But, it was fun. :) Now, a bit of explanation about my methodology. First, the simulation is two-dimensional, since I didn’t think the added complexity of doing a full, 3D simulation would generate sufficiently different results. Second, rather than moving my subject through a shower of rain drops at varying speeds, I decided to apply a uniform direction vector to the drops themselves (basically move the drops instead of the subject… the effect is the same, but the implementation is a lot easier). With that said, the experiment is set up as follows (note, these parameters are all configurable, but this is what I chose… they’re entirely arbitrary):

1. The rain drop spawn field is 20m by 20m.
2. The rain drops are created at a rate of 80 every second, distributed randomly across the top of the spawn field.
3. Rain drops fall at the terminal velocity for a typical drop indicated [http://www.grow.arizona.edu/water/raindropvelocity.shtml here] (6.25 m/s).
4. The subject is a rectangle approximately 6 feet tall by 6 inches wide.
5. The subject’s walking speed varies from 1 to 8 m/s, stepping 0.25 m/s per experiment.
6. The subject “walks” a fixed 20m during each experiment.
7. Each experiment was repeated 10 times and the results averaged (since rain drops are spawned in random positions).

The final tallies can be seen in the graph below:

Granted, it looks a bit noisy, but the general trend appears to indicate that moving faster through a rain storm helps keep you drier! Though, the advantage does seem to level off (it looks like a roughly exponential decay, to me, with the limit at some non-zero value). Remember that, folks… the weather doesn’t look like it’s going to improve. :(

Incidentally, working on this in Squeak has been quite enjoyable. The richness of the class library made many tasks far easier than they would be in other languages, and the ability to fix bugs as I go, and then continue running the code is, to say the least, incredibly cool. And, frankly, I think Smalltalk is the most elegant programming language I’ve ever worked with. :)

Update:

Found an oversight in my simulation, but the above graph now reflects the latest version. In short, I had to make sure the playfield was populated with raindrops before beginning each walk. Otherwise, the subject could complete the walk before a drop ever fell low enough to hit him!

Update 2:

Woo! I win a gold star!