Smoking toooooo much PHP

The documentation for seed gobject introspection is improving continually, I now have a jhbuild virtual machine, which is picking up the latest versions from git.

Job hunting is going pretty slowly (if anyone at ubuntu is listening hint hint ;) , and projects are pretty much done, so I get to work on really cool stuff.

View Extended Entry

Unfortunately my internet line died over the weekend, and I was left without a connection until Tuesday. I had a bit of offline time to have a look at some interesting new(ish) technology.

If you have paid any attention to Gnome development, there are mentions of gnome-shell, and something about making it easier to develop add-on/applets etc. using Javascript. I have not really had time to look at this much, however given the fact I had downloaded all the components prior to my loss of connectivity, I spent some time over the weekend and monday looking at this in more detail.

It looks like there are obviously things going on in IRC and mailing lists, so most of my impressions are from the websites, and code. There are basically 2 projects currently.

GJS - This is the mozilla Javascript engine bound to the gobject introspection system
Seed - This is the webkit Javascript engine bound to the gobject introspection system

Of the two, Seed apears to be a little more advanced and more time has gone into it, based on the fact there appear to be more core features in Seed (eg. I could not find anything like Seed.print() in GJS), and there are very few examples in GJS

Both of these use the GObject introspection method to bind to Gtk, and a large and growing number of other libraries. This is something I found very interesting having gone through a similar process with rooscript.

The fundimental issue of creating bindings to Gtk (and anything that uses the GObject system) is that historically, almost everyone (PHP, Python etc.) who did it used this method.

* Parse the HTML documentation, or the .h files (extracting the classes, enums, methods, arguments etc.)
* Generate Binding code for each library, which wraps these methods, loads the '.so' libraries and links it all together.
* Compile a Module (or statically link it in some cases)
* Run... Test.. Fix bugs in wrapping code... Run ... Test.

What GObject inspection introduces is an very thin layer that can be used to expose any of these methods without writing any binding code (other than to GObject inspection). The result is that to add more features (linking to another library) - you just have to generate a XML file describing the interface, compile it with g-ir-compile and put it in the right folder, and you have magically added support to a new Library, without generating an C code!

This also means that fixing the binding is considerably simpler (fix the XML file, re-compile etc.) and you have solved bugs in any language that is using it... (as I found with gtk_tree_store_set_column_types()

Playing with the languages.

GJS and Seed present the Gtk API very slightly differently, which at present appears to be a bit of a blocker for deciding which to use (or even to bother for some). The key differences are for authoring

* Javascript 'let' features in GJS - not available in Seed.. - This is rather a big blocker as code designed for GJS will fail in Seed totally.. (and visa-versa probably)
* Different call signatures to signals� � �  SEED:OBJECT.signal.connect(method)�  vs�  GJS:OBJECT.connect('signal', method)�  - I think personally that GJS way is more true to the concept and cleaner, however as you can see below there are ways around this.

Seed has a git repo on gnome 'seed-examples', which contains a large number of examples, although their depth is rather thin unfortunatly.. (code coverage is probably 1% at best)...
Both have pretty much Zero in the documentation stakes.. - A rather trivial task to create - which I'm tempted to have a go at....

Anyway back to my playing around..

I've been doing some serious shit in Javascript recently, GUI builders, Code generators, Hacking GTK bindings etc. so I've seen the good, bad and ugly as far as Javascript goes (Bad sometimes being my code).�  I've been slowly moving to what I regard as an effecient, productive use of Javascript, most of which can be seen in the xtype support in RooJS

After runing the first few examples in seed-examples, I decided to see if using an xtype / Roo structure could be done with Seed. After a bit of tweaking, and bugfixing of Seed, I finally got this code to work.

var win = XN.xnew({
    xtype : Gtk.Window,
    type: Gtk.WindowType.TOPLEVEL,
    listeners : {
        'delete-event' : function (widget, event) {
            return false;
        },
        destroy  : function (widget) {
            Gtk.main_quit();
        }
   },
   
   set : {
        set_border_width : [ 10 ],
        resize : [300, 300],
        show_all : []
   },
   items : [
        {
            xtype : Gtk.VBox,
           
            items : [
                {
                    xtype : Gtk.TreeView,
                    
                    
                    
                    onRender : function()
                    {
                        this.column = XN.xnew({
                            xtype : Gtk.TreeViewColumn,
                            items : [
                                {
                                    xtype : Gtk.CellRendererText
                                }
                            ]
                        });
                        
                        this.column.add_attribute(this.column.items[0], "text", 0);
                        this.append_column(this.column);

                        this.set_model( XN.xnew( {
                            
                            xtype : Gtk.TreeStore,
                            
                            onRender : function() {
                                 
                                this.set_column_types ( 1, [GObject.TYPE_INT] );
                                this.iter = new Gtk.TreeIter();
                                this.append(this.iter);
                                
                                this.set_value(this.iter, 0, [GObject.TYPE_INT, 3]);
                                this.append(this.iter);
                                this.set_value(this.iter, 0, [GObject.TYPE_INT, 2]);
                                this.append(this.iter);
                                this.set_value(this.iter, 0, [GObject.TYPE_INT, 9]);
                            }
                        }));
                    }
                     
                }
            
            ]
         }
    ]
    
 });
 

While you may say, so what, it's a bit prettier than the standard examples, but not exactly much different... While that is true, the use of this nested tree, and xtype method is that it enables you to use code and ui builders to put together an interface, and re-use the components very easily. It also solves one of the GJS/Seed issues of signal binding issues (as we wrap up all listeners).

The backend code to make this happen is here.. - It's pretty trivial again, but obvously could be expanded to handle the different binding methods for how different components are combined (add/pack_start/append etc...)

createDelegate = function(method, obj, args, appendArgs){
    
    return function() {
        var callArgs = args || arguments;
        if(appendArgs === true){
            callArgs = Array.prototype.slice.call(arguments, 0);
            callArgs = callArgs.concat(args);
        }else if(typeof appendArgs == "number"){
            callArgs = Array.prototype.slice.call(arguments, 0); // copy arguments first
                var applyArgs = [appendArgs, 0].concat(args); // create method call params
                Array.prototype.splice.apply(callArgs, applyArgs); // splice them in
            }
            return method.apply(obj || window, callArgs);
        };
};


xnew = function(o)
{
    var ret  = o.self ||  new o.xtype(o);
    
    o.items = o.items || [];
    //Seed.print(o.pack.length);
    // packing 
    var addm = typeof(ret['add']) == 'undefined' ?  'pack_start' : 'add';
    ret.items = [];
    for( var i =0; i < o.items.length;i++) {
        ret.items[i] = xnew(o.items[i]);
        ret[ o.items.length == 1 ? addm : 'pack_start'](ret.items[i]);
        ret.items[i].xparent = ret;
    }
    o.set = o.set || {}; 
    for (var i in o.set) {
         
        ret[i].apply(ret, o.set[i]);
    }
     
     
    o.listeners = o.listeners || {};
    for (var i in o.listeners) {
         var _li = createDelegate(o.listeners[i],ret);
         Seed.print(typeof(_li));
         ret.signal[i].connect(_li);
    }
   
    // apply functions..
    for(var i in o) {
        if (typeof(ret[i]) == 'undefined') {
            ret[i] = o[i];
        }
    }
    if (ret.onRender) {
        ret.onRender.call(ret);
    }
    
    return ret;

}

Missing Features

One area that I was quite interested in, unfortuntatly did not look to easy, that being Javascript tokenizing. In rooscript, I hacked in the ability to retrieve the tokens of a Javascript file and expose it to a Javascript interface. This enabled me to write Javascript compressors, and hack jstoolkit to use this. Leaving�  the painfull task of parsing javascript to the compiled part, and not having to deal with a messy parser in Javascript. Looking at Webkit code, It was not immediatly obvious how this could be done however.

For the time being I will leave that component of my Roo Builder application using rooscript.

Well, a peek into the secret world of Roo development. I've been a bit frustrated with the speed of development of the applications recently, so I decided to speed it up a bit..

Below is� a little video of a module I've added to the Iconstruction framework (which might get released one day..) - which enables me to very rapidly build applications.

The knock on effect of developing the builder has been that there is now far better support for xtype object constructors in Roo, along with far more acurate and complete documenation (as the builder tool needs that to work)

If you have ever used googles translate API, it can quickly become a love hate relationship. You love the features it provides, but you begin to hate that fact that google's server are slow and flaky for loading the libraries that they recommend.

This situation was getting especially annoying yesterday, as the load time of my application (that is getting continually re-loaded while I'm working on it) was getting worse and worse, and google's API's was the culprit.

So after some further reading on that page, I realized that the translation call was really just a simple HTTP request with the correct parameters.. no need for huge google framework API etc.

So here's the ~20 line javascript to replace the slow loading 50k+ library that google recommends....

/**
 * usage 
 * gtranslate('hello', 'en', 'es', function (res) {
 *     if (typeof(res) == 'object') { return; } // failure
 *     console.log(res); // success...
 * });

*/

function gtranslate(str, src, dest, cb) 
{
        var x = new Roo.data.ScriptTagProxy({ 
            url:  'http://ajax.googleapis.com/ajax/services/language/translate', 
            callbackParam : 'callback' 
        });
        x.load(
            {
                v: '1.0',
                q : str,
                langpair : src + '|' +dest,
            }, // end params.
            { // reader
                readRecords : function (o) {
                    if (!o.responseData) {
                        return o;
                    }
                    return o.responseData.translatedText;
                }
            }, 
            function (result) {
                cb(result);
            },
            this,
            []
        );
            
        
    } 
    

Remember the good old days where phpnuke was the worst code since sliced cheese, and thousands of idiot's had installed in on their server waiting to be hacked into..

Well, It's a bit like how I felt this week trying to solve my svg graphing issues..

A bit of background, In the last week I just pushed out the latest version of our Anti-spam / Mail control application, the big new feature was a beta version of the statistic's� and analysis section. Where we showed nice little graphs about how much work your staff where doing and if they had really been sending jokes all day long or just reading facebook spam

Open Flash Charts

While the database side , the design is quite interesting, I did not really spend to much time thinking about the frontend - as I like to keep that flexible (read throw away). So for the first beta I decided to use Open Flash Charts. The current version is extremely well designed, in terms of the chart generation consists of delivering a JSON file to a static single Flash file. The JSON file sets the graph type, and date etc..

It's very quick to set up and work with, and the graphs are quite pretty, and a cutely interactive. And for our first beta it's just about OK.

However for anything other than a beta,� there are rather serious problems with it.
a) It's dog ass slow, to load, render on anything but a brand new overspec'd PC
b) The flash file is quite large, and yet another overhead to a already large application.
c) Having multiple Flash graphs on a single page, just makes the above two problems worse.
d) It depends on closed source tools (Actionscript compilers etc.) - Never a good idea.
e) Printing of Flash in firefox is broken (at least on linux)
or basically all summed up... Flash sucks

PlotKit

So after beta 1 was released to testing, I started the hunt for an alternative. Since the 'correct' method to do graphs on the web is really SVG, and as I like to shift all the pointless processing to the end users PC.� JavaScript to convert the data into a chart is really the ideal solution.

Pretty high up on google's list is PlotKit, which has some really nice demo's on the site, It depends on two other pieces of code, excanvas (as nice self contained javascript file that handles SVG compatibility issues with IE/FF/Safari etc.)� and MochiKit.

MochiKit

Since MochiKit is none too small, and would add to my javascript overhead, I thought I'd check out what it was, and see if it was really needed. What I saw, was more "nightmare on nuke street".

The front page of the web site claims quite large "MochiKit makes JavaScript suck less", That should be paraphrased, "only if you are a Python nut-head" - otherwise it make your Javascript code into complete jibberish. __self__, __new__, __screwed_up_language__ everywhere....

<big rant> The dumb f*** who wrote it decided that Javascript was not Python, so they should try and make it's syntax as close to python as possible. Ignoring the fact that Javascript syntax just solves� problems differently, it does not need fixing or even breaking to be as badly designed as python.. </big rant>

Anyway after digging through the code to plotkit, and mochikit, I came to the conclusion that they had developed a piece of code that was completely unmaintainable or extend-able. So was forced to keep looking for an alternative.

Plotr

Quite a way down google's list (search svg javascript graph) I finally found Plotr, Basically someone else had obviously looked at PlotKit, and decided that Mochikit was just so ugly, that removing it from the application was the only sane thing left to do.

What they chose rather than MochiKit, was Prototype, which in general is considerably better than MochiKit. And basically removed all the Mochikit dependancies and replaced them with Prototype. This made the code quite a bit simple to read and understand, (they also added comments), although I'm not to keen on this type of commenting, as I find it add's noise, and makes the code less readable.

xxxx : function (/** some_type **/ xxxxx, /** another_type **/) {

Anyway, otherwise Plotr looks clean and simple.

** It looks like the author of Plotr as gone on to start Flotr - another graphing library.. - did not notice that until I'd commited to working with Plotr...

Prototype

What I wanted to do however was reduce the size of the Prototype dependancy. So I started having a look at Prototype's internals.

Splattering the namespace

Unlike Ext, and Roo, Prototype's core creates quite a few global properties, $, $R, $$, JSONSomething and Element which by the looks of things are totally unnecessary, and make it difficult to work with other frameworks (or potentially other code), This is downright bad design, and could easily be rectified by putting almost all of those under the Prototype.* object.

Along with this, there is quite excessive use of adding to Global Prototypes, String.prototype gains about 20 extra methods, some quite useful, or wrappers for compatibility, other seem more like features for features sake, and should really be moved to a Prototype.String object or just removed.

What make me wonder the most was the way that prototype adds properties to HTML and DOM objects, $("xxxx") - get's does a getElementById( "xxxx") and returns the DOM/HTML object overlaid with all of the properties from the Element object.� In Ext/Roo's 'Roo.get("xxx")' method� create's something like the 'Element' object , Roo.Element, it puts the original DOM object in the return value ".dom" property. Hence making documentation clearer and code considerably more managable. Not so much of WTF did that method come from? when reading code.

Anyway - I managed to give Prototype a bit of a diet, so it's� not so much of a Namespace/bandwidth hog. You can download the lightwight code here:
http://www.akbkhome.com/svn/javascript/plotr
Note the full bundle (protolite, excanvas + plotr comes in at 54K or ~ 15K if you are serving javascript with the mod_gz library...

Just to prove it still works
http://devel.akbkhome.com/plotr/testplot.html

See here for the original code
http://code.google.com/p/plotr/source/browse/

Just a small update on roojs1 - I've added in-line comments to the RooJs1 documentation - so if you think a method or class is not well explained, add your comment, example code, or bug note right on the documentation. - Something I think makes a huge difference to the usability of code...

Other small changes are also beginning to be added - including the fields property of a Roo.toolbar, and the new Roo.form.Hidden element, to make creating forms with hidden variables simpler.

Anyway, rumors are swirling around that there is a recommendation to avoid ExtJS among certain linux distributors due to uncertainties and general lack of confidence in the license, it's continual changing and general poor behavior by Jack in not understanding the implications of the bait and switch changes.

It also appears that Jack's Legal team are working harder than his coding team again, sending out threatening letters to people involved in various ExtJs2 derivative works. (which is the key reason I'm focusing on v1 - clear, definitive separation and licenses that do not allow for such shenanigans)

While ExtJS2 offered some interesting new features. The significant change to the object rendering model, from my experience has been broken on every release prior to the GPL one, (and since I never tested it after that release, I suspect it's still broken).

Other changes relating to a clearer heirachy of object model look like they should be a simple addition to ExtJS1, along with better support for non DOM element dependent constructors. Hence I think building on ExtJS1 appears to be a better long term move, and sounds like a few more are beginning to see that wisdom.

Another update on progress on RooJS (the ExtJS v1 Fork)

• Form's now have CSS, so the dynamic form examples work quite well
  
• The CSS was created by documenting how the existing examples where rendered in ExtJS - A document describing this was created (see form_design.svg in the css folder). Then that document was used to create a brand new CSS file. I believe this can be regarded as not breaking the copyright and having a reasonable reverse engineering process.
  
• the x-box borders are missing, although not critical
• the HTML editor needs to be fixed.
  
• 
  
• I have done the design diagram for CSS of the menu's and will be working on that soon.
• The doc's are improving as well, The doc's appear to be listing all the correct elements now, future work will focus on tidying up the templates.
• Most of the improvements came from replacing the Walker code in the jsdoctoolkit so that it understands scope in a similar way to the compressor. I also had to add a few extra comments in the source to give the documentation tool a few hints.
  
• /** @scope Roo.somescope */� -- changes the scope in the documentation engine (as sometimes it just can not guess what the scope is.
• /** @scopeAlias avar=Roo.xyz.prototype */ -- adds an alias to the parser engine alias map, so when it sees 'avar', it will replace it with Roo.xyz.prototype
• The compresser is working well, someone asked on #roojs on freenode about using it for other code, including ExtJs2.0 - There should be no problem doing that - have a look at the bundle build file to see how to set up a builder for any project.
• One thing to note is that the compresser uses hints when it sees 'eval'. The YUI compresser basically turns off compression for a large chunk of the code if it sees 'eval', I've added code to pick up a comment before the eval statement:
• /** eval:var:abcd */� will turn variable compresion back on, and exclude 'abcd' from the list of variables to be compressed (use multiple eval:var:... statement on multiple lines to exclude multiple variables)

Anyway back to work tommorow, after celebrating (eg. doing nothing) the glorious reunification with the motherland day (HKSAR day)

CSS hackers needed.

Yes, the first test version of roojs1 is built and working you can have a look here to see the old ExtJS1 examples (which where sourced from a 0.40 release licenced under a BSD licence)
http://www.akbkhome.com/roojs1/examples

Warning this may break occasionally as I do daily builds of the library

Most of the main layout, windows, grid and tree work pretty well, but there are some problems with toolbars, menus and forms. Basically I could not find complete licence free sources for any of them. So if you know anyone who can create CSS for the missing parts (and can illustrate reasonably well that it was not based on the original ExtJS CSS) send them over this way. That will help RooJS2 as well, as the CSS is probably pretty similar.

Have a look at http://www.akbkhome.com/roojs1/css/roojs-debug.css for details on the status of the css build..

A big leap forward in building a truely free high quality JS UI toolkit.

Anyway onto my geeking techo fun.

Rooscript compressor

As part of the whole build toolkit, I've ported the yui compressor to rooscript (the dmdscript based Javascript interpreter), The results are pretty good.

The current output from the compressor is 8K less that the ~497K that ExtJS 1.1.1 files are (built I presume using yui compressor).� Along with this, it's pretty easy to tweak, and remove Files and add your own application (eg. create an all in one� javascript file for your application)

Speedwise I've been trying to get the compressor to go faster (It normally caches the minification result to a file and can do a 'reparse' in arround 2 seconds, but for a fresh build it takes about 6 minutes to parse all the ~100 files in Roojs1

In the process of speeding it up, I've added a few switches to rooscript

• -p for a very simple profiler (gives you the total time taken for all the opcodes exculuding call/new etc. in each function) along with how many times they where called.
• -U to turn on Unicode for strings, as I've disabled it by default, although I'm not that sure now, that it produced the speed hit that I thought it might. ** mostly affect things like xx.indexOf("\n")
  
• -d (to show all the opcodes as they run) - old but handy sometimes...

The code is reasonably simple, so If you have any great ideas on how to improve compression, it's extremely easy to test how well they work. (rather than the usual compile/run cycle that all the Java versions have)

In looking at the resulting code, one simple opmization that I've started adding to RooJs is changing the constructor code in the object from Roo.data.Connection.superclass.constructor.call(this,....) to Roo.superC(this,....) and Roo.data.Connection.superclass.somemethod.call(this,....) to Roo.superM('somemethod', this,.....)

I'm guessing that's could be ~ 2K of rather wastefull code removed..


Basically since the compressor is working now, I'm going to re-look at the Docs, and sort out the bug's in those - probably replacing the scoping code in jsdoctoolkit, with something closer to the ported yui compresser version.