The fly to mars. Steps bringing us closer?

Ok, this is going to be a hard post for me.

I have to admit a lot of mistakes I did, and some of them not caused because of “circumstantial” issues but for my poor understanding of some of the issues I found.

First, a small summarize of what we have.

I presented Mars at ESUG’10, with a version “almost working”, with a set of working widgets and a set of tools implemented with old tool builder, just as a way to show the advance so far.

There were an ugly bug on the ObjectiveCBridge logic, as I demonstrated at ESUG by crashing the system several times, who was stopping me to continue developing the framework, because the error was too heavy and too “omnipresent” to continue. Fortunately, this bug was not taken into account so severely, and Mars won the third price of the technology awards! (And I’m really grateful about, is very encouraging)

Stuck by ignorance

But so far... Mars development was stopped until I can solve the bridge bug. And my knowledge of the insides of the callback logic wasn’t really good enough to let me fix the problem.

Fortunately, 2010 edition of Smalltalks conference was very “deep” oriented and there were several virtual machine presentations, and Eliot Miranda was here... and he solved my bug in about 15’. Yes, that fast.

Also he showed me his work on the threaded and callback supporting FFI. That, and some misunderstandings derived from my poor english become relevant next couple of months, as you will see.

Resume and stuck again

I resume working in Mars, trying hard to finish the year with an usable version. But then, two things happened:

First, John contacted me and offered me become the new vm builder for the macs. After thought about a little, I accepted, and then the time I can spare to work on Mars was dramatically reduced, while learning and getting inside all the vm stuff (and I’m also there, I still don’t understand too many things).

Second, a new problem emerged: The ObjectiveCBridge, with the bug fixed, was not being fast enough to allow me to handle Cocoa delegates in a usable way (this problem was evident using it with Deimos -the iPhone derivate).

I need to explain in short what is a cocoa delegate: It is one of the ways cocoa handles the interaction with the program, allowing to add some behavior without extending the hierarchy. It is also the preferred way to handle user interface events. For example, if you want to validate if a window can be closed, or a table cell is editable. And finally, It is also the way you feed a table (or list, etc.) with the data you want to display. In this last case, the methods you implement are invoked a lot of times... and you need to provide an answer really fast, or your application will look really frustrating.

So... the ObjectiveCBridge wasn’t doing me acceptable response times in this datasource-delegates. For the iPhone, I couldn’t get less time than 110ms for circuit (callback execute and process answer), and it needed to be less than 40ms to be responsible as expected (with the cool effect a table has in an iPhone). For the OSX, I couldn’t get less than 50ms, and this last time, using a questionable design, firing different processes in the image to handle the where an immediate answer wasn’t needed... which was producing an ultimate deadlock when filling a browser.

So, what is the correct path?

I had two paths to follow, to try solve this problem, first one was really awful: start creating “support classes”, in Objective-C, to use them as delegates and just call Pharo when absolutely needed. I hated this solution because it would have meant a significant loose of functionality, and also an increment of startup times (if I need to fill a delegate with the list of items I want to show... is a time-consuming task to, and I can’t use any of the optimizations a cocoa widget has to avoid loose time with this).

The decision I took was radical, but I think it was ultimately correct: To drop the use of ObjectiveCBridge and adopt another callback approach: the implemented by Alien... the LibC qsort example was very impressive, giving me a 100 number sort in 7ms (in a non-cog vm), and that meant more than 600 callbacks executed!

I started to work. But for that, I needed another ObjectiveCBridge... an Alien version from the old one, to interact properly with the cocoa classes. Fortunately right there was the Alien-MacOSX package. Unfortunately, the package was very outdated and some of it’s functionality was incomplete and never worked (like the proxy stuff).

Anyway, it was better than none (and I really hate reinvent the wheel every time) so I started to fix it. After some weeks (yes, sadly I measure my Mars dedication in weeks and not days... the problems of not having enough time to spare) I finished a working version... (this can be taken as a “side project” now, and can be used for anyone who needs it, and finally I can restart working in Mars.

The examples I build to test if my answer was correct was very promising... just 115ms rendering a table with the 3500 classes and traits of my system, including it’s browser icons!

What now?

Next steps will be to finish the 1.0 version of Mars :)

In the mean time, while trying different forms of communication with the system, I redesigned many things of Mars, for instance, first version was using traditional event handling approach (with changed:/update: observer mechanism) and now I’m adapting it to use announcements, but that is material for another post.

The real huge improvement is that there are some people joining this exiting adventure. They are Carla (you know her from SmallUML ESUG 2010 project), Germán (he did Namespaces ESUG 2010 project) and Nicolás (you don’t know him, but I hope you will soon)

Together, we are deeply commited to finish a working version of Mars, to show in ESUG 2011, and also with some little “surprises” we are preparing.

The journey to Mars continues... and we are going to know this new world really soon :)

Thanks, Esteban