In addition many of the documentation details have been expanded, including
Interfaces, Enums
callback methods are now documented
More libraries have been added.
More doc comments can be picked up
With better documentation it's finally possible to figure out how to use the API. A classic example of this was a small problem I tried to solve before the API documentation was available. Listing a directory asynchronously, It took me over an hour to get close to figuring out how to do this, I eventually had to give up. as digging through the source, C reference and GIR files took so long.
However within 5 minutes with the documentation, I was able to write a small script to do this.
Gio = imports.gi.Gio;
Gtk = imports.gi.Gtk;
var f = Gio.file_new_for_path('/home/');
f.enumerate_children_async (
"*",
Gio.FileQueryInfoFlags.NONE,
GLib.PRIORITY_DEFAULT,
null,
function(o,ar) {
// listing completed..
var fe = f.enumerate_children_finish(ar); var ch = false; while (ch = fe.next_file(null)) {
Seed.print(ch.get_name());
}
Seed.quit();
},
null);
Gtk.main();
Just about finished a gumblar cleanup, for a small Hong Kong company. This is not the first crack I've seen in the last few months, I fixed another server last month that got ssh brute force attacked. It looks like cracking is on the up, so if you need help fixing a site, by someone who knows what they are doing, and at the same time you will help out a number of open source projects - give me a bell (alan@akbkhome.com)
The gumblar (or derivative) attack I was looking at was quite interesting, the first indication the owner got was that browsers kept showing the "Reported Attack Site!" or "Warning: Visiting this site may harm your computer" message. So I get the call to find out what's going on.
When you ignore the message and go through to the site, look at the HTML the first thing you see is that there is a <script> tag added just before the body pointing to a gifimg.php file. After that you have a long hunt around google to find out what's going on.
At the time of writing, the exact attack vector does not look like it's been confirmed, but is either a brute force ftp attack (I think is quite unlikely considering the username/pass combo on this sample site). Or more likely a PDF desktop attack to a machine that has access to the site.
My first assumption was that it was a Wordpress exploit, but the more I examined the situation, it seemed less likely. However I highly suspect that the PDF attack vector having got the ftp credentials goes looking for standard locations of wordpress installations (eg. '/wordpress) - so hint one is not to install your software in such obvious places.
Cleaning it out
The first step in sorting out the mess was to mirror the original site, with virus and all onto a offline location. (both as a precaution that if we broke things we had a backup, and so we can use this as a source to replace the hacked files with new ones).
After that it was a matter of googling for details of the attack and writing a gumblar cleaner script. It basically checks for infected file types, then preg_replaces out the hacked additions. These include
php files with an eval/base64_encode line
javascript files with document.write lines
html, shtml and htm files with <script tags.
I used ftpput, and check return values, to ensure that each file was successfully replaced before overwriting the local copy and making a nice copy for my reference into the virus folder.
Inside out of the attack.
The infection is quite interesting, and in this case was quite painful, due to the nature of how Wordpress publishes files.
Initially I suspect the core code in the PDF actually has some ftp code which will try and modify standard set of PHP files to add a small base64_encode script.. (phplist, and wordpress appear to be core targets, and I'm sure there are more.)
This is a snippet of some of the code that get's added (it's all eval, base64_encoded - read up on my blog post about idiot ways to protect your PHP code using this idea.)
After that wordpress does it's wonders and infects the rest of the site for you. As all the generated pages suddenly get the extra <script tags> when publishing and your wordpress outputs the infection into the admin system.
Note: I only dissected one of the php scripts, which changed output buffering adding the <script tag, but did not see the document.write changer. I suspect there may be another variant of the script above that i did not look at that modifies the javascript files, or that it's done remotely.
Anyway all cleaned up after a few days (due to the long time the original backup took) . After this the recommendations for the owner where, stop using adobe PDF viewer (there are alternatives out there) - stop using IE, ask all staff to use Firefox with noscript. and keep a backup!
Job hunting is going pretty slowly (if anyone at ubuntu is listening hint hint ;) , and projects are pretty much done, so I get to work on really cool stuff.
While developing in seed is pretty productive, hunting down the documentation, or discovering the right method to use has been one of the roadblocks I have been bumping up against. So wondered yesterday how well the introspection code in seed works.
Turns out, that the built in Seed.introspection() method is pretty pointless, even after committing a few extra features, I discovered it was never going to be much help. However girepostory has it's on gir/typelib file, so you can call introspection directly.
After a few experiments, I worked out to inspect basically every method, and class in Gnome. Hence there is now full documentation for every Gnome Class that seed can talk to (this does not include libxml however, as it does not use gir to talk to that.
Support for doc comments (by parsing the gir files) is there, but the machines I build on unfortunately have rather old gir files, which do not include all the docs comments.
This code could be very easily refactored to generate docs for pygi, gjs or vala.. or any other gir based binding..
If it's not working, it's probably as I'm working on it and have broken it.
The code is in my seed subversion repo the key files that do all the work are JSDOC/Introspection.js, and docs.js The rest of the code just deals with Template parsing and handling (/docs/* are the templates)
To run it you need cutting edge seed (which can take quite a while to set up as it depends on a recent webkit build)
As part of my, "What to do while looking for a Job" project, I'm exploring the Gtk/Gobject introspection bindings in seed.
It's one of those fun, not documented anywhere, and you have to dig around all the source to understand how to do things projects. And part of a grander plan to develop a generic application builder loosely based around the RooJs builder application I blogged about previously.
The idea is that using Gtk, GtkSourceView, and the concepts I developed in the RooJS Builder, I would be able to create Dynamic Web applications (and eventually Gtk ones) very rapidly due to the ability to closely tie the action code to the interface design.
The core reasons why it speeds up development are:
- Syntax checking is feasible when creating code.
- Adding, editing and selecting properties and event handlers is faster, less error prone, and access to documentation is instant.
- Visually changing and seeing the effect of a change (the change - reload - test cycle) is reduced to milliseconds rather than tens of seconds.
So far most of the application is mocked up, (and relivant patches have been submitted to seeds' bugzilla for review). However one of the key components I was looking at over the Chinese New Year Break was drag and drop of Palete Items onto the Rendered View of the application (Webkit embedded), (and eventually the tree that makes it up). To do this involved seriously testing the Gobject introspection bindings and how the interact with the drag and drop methods in Gtk.
So Read on if you are interested in how it all comes together.
Unfortunately my internet line died over the weekend, and I was left without a connection until Tuesday. I had a bit of offline time to have a look at some interesting new(ish) technology.
If you have paid any attention to Gnome development, there are mentions of gnome-shell, and something about making it easier to develop add-on/applets etc. using Javascript. I have not really had time to look at this much, however given the fact I had downloaded all the components prior to my loss of connectivity, I spent some time over the weekend and monday looking at this in more detail.
It looks like there are obviously things going on in IRC and mailing lists, so most of my impressions are from the websites, and code. There are basically 2 projects currently.
GJS - This is the mozilla Javascript engine bound to the gobject introspection system Seed - This is the webkit Javascript engine bound to the gobject introspection system
Of the two, Seed apears to be a little more advanced and more time has gone into it, based on the fact there appear to be more core features in Seed (eg. I could not find anything like Seed.print() in GJS), and there are very few examples in GJS
Both of these use the GObject introspection method to bind to Gtk, and a large and growing number of other libraries. This is something I found very interesting having gone through a similar process with rooscript.
The fundimental issue of creating bindings to Gtk (and anything that uses the GObject system) is that historically, almost everyone (PHP, Python etc.) who did it used this method.
* Parse the HTML documentation, or the .h files (extracting the classes, enums, methods, arguments etc.) * Generate Binding code for each library, which wraps these methods, loads the '.so' libraries and links it all together. * Compile a Module (or statically link it in some cases) * Run... Test.. Fix bugs in wrapping code... Run ... Test.
What GObject inspection introduces is an very thin layer that can be used to expose any of these methods without writing any binding code (other than to GObject inspection). The result is that to add more features (linking to another library) - you just have to generate a XML file describing the interface, compile it with g-ir-compile and put it in the right folder, and you have magically added support to a new Library, without generating an C code!
This also means that fixing the binding is considerably simpler (fix the XML file, re-compile etc.) and you have solved bugs in any language that is using it... (as I found with gtk_tree_store_set_column_types()
Playing with the languages.
GJS and Seed present the Gtk API very slightly differently, which at present appears to be a bit of a blocker for deciding which to use (or even to bother for some). The key differences are for authoring
* Javascript 'let' features in GJS - not available in Seed.. - This is rather a big blocker as code designed for GJS will fail in Seed totally.. (and visa-versa probably) * Different call signatures to signals� � � SEED:OBJECT.signal.connect(method)� vs� GJS:OBJECT.connect('signal', method)� - I think personally that GJS way is more true to the concept and cleaner, however as you can see below there are ways around this.
Seed has a git repo on gnome 'seed-examples', which contains a large number of examples, although their depth is rather thin unfortunatly.. (code coverage is probably 1% at best)... Both have pretty much Zero in the documentation stakes.. - A rather trivial task to create - which I'm tempted to have a go at....
Anyway back to my playing around..
I've been doing some serious shit in Javascript recently, GUI builders, Code generators, Hacking GTK bindings etc. so I've seen the good, bad and ugly as far as Javascript goes (Bad sometimes being my code).� I've been slowly moving to what I regard as an effecient, productive use of Javascript, most of which can be seen in the xtype support in RooJS
After runing the first few examples in seed-examples, I decided to see if using an xtype / Roo structure could be done with Seed. After a bit of tweaking, and bugfixing of Seed, I finally got this code to work.
var win = XN.xnew({ xtype : Gtk.Window, type: Gtk.WindowType.TOPLEVEL, listeners : { 'delete-event' : function (widget, event) { return false; }, destroy : function (widget) { Gtk.main_quit(); } },
While you may say, so what, it's a bit prettier than the standard examples, but not exactly much different... While that is true, the use of this nested tree, and xtype method is that it enables you to use code and ui builders to put together an interface, and re-use the components very easily. It also solves one of the GJS/Seed issues of signal binding issues (as we wrap up all listeners).
The backend code to make this happen is here.. - It's pretty trivial again, but obvously could be expanded to handle the different binding methods for how different components are combined (add/pack_start/append etc...)
xnew = function(o) { var ret = o.self || new o.xtype(o);
o.items = o.items || []; //Seed.print(o.pack.length); // packing var addm = typeof(ret['add']) == 'undefined' ? 'pack_start' : 'add'; ret.items = []; for( var i =0; i < o.items.length;i++) { ret.items[i] = xnew(o.items[i]); ret[ o.items.length == 1 ? addm : 'pack_start'](ret.items[i]); ret.items[i].xparent = ret; } o.set = o.set || {}; for (var i in o.set) {
ret[i].apply(ret, o.set[i]); }
o.listeners = o.listeners || {}; for (var i in o.listeners) { var _li = createDelegate(o.listeners[i],ret); Seed.print(typeof(_li)); ret.signal[i].connect(_li); }
// apply functions.. for(var i in o) { if (typeof(ret[i]) == 'undefined') { ret[i] = o[i]; } } if (ret.onRender) { ret.onRender.call(ret); }
return ret;
}
Missing Features
One area that I was quite interested in, unfortuntatly did not look to easy, that being Javascript tokenizing. In rooscript, I hacked in the ability to retrieve the tokens of a Javascript file and expose it to a Javascript interface. This enabled me to write Javascript compressors, and hack jstoolkit to use this. Leaving� the painfull task of parsing javascript to the compiled part, and not having to deal with a messy parser in Javascript. Looking at Webkit code, It was not immediatly obvious how this could be done however.
For the time being I will leave that component of my Roo Builder application using rooscript.
Well, a peek into the secret world of Roo development. I've been a bit frustrated with the speed of development of the applications recently, so I decided to speed it up a bit..
Below is a little video of a module I've added to the Iconstruction framework (which might get released one day..) - which enables me to very rapidly build applications.
The knock on effect of developing the builder has been that there is now far better support for xtype object constructors in Roo, along with far more acurate and complete documenation (as the builder tool needs that to work)
Looks like ubuntu released karmic today (as I guess they could not rename it 9.11).
I've been testing karmic for months, (which at times have been very painful - but if you join the beta ride, what else would it be), but in the last few weeks it looked like it was really coming together.
The only big regression was the synaptics touchpad resetting the 'tap' setting after suspend resume cycle.
However, there was a kernel update last week whichcompletelybroke suspend (it never really sleeps the machine), and attempts to report it via the apport tool result in a invalid message about the kernel not being a ubuntu one.. (which someone has reported)
I'd really suggest anyone requiring resume/suspend to work, to hold of for a little while until they sort out the remaining issues.
If you have ever used googles translate API, it can quickly become a love hate relationship. You love the features it provides, but you begin to hate that fact that google's server are slow and flaky for loading the libraries that they recommend.
This situation was getting especially annoying yesterday, as the load time of my application (that is getting continually re-loaded while I'm working on it) was getting worse and worse, and google's API's was the culprit.
So after some further reading on that page, I realized that the translation call was really just a simple HTTP request with the correct parameters.. no need for huge google framework API etc.
So here's the ~20 line javascript to replace the slow loading 50k+ library that google recommends....
Just before I saw Brendan's post about PHP4 compatibility in PEAR, I had been getting a few queries about making a couple of my PEAR packages more 'PHP5' compatible or PEAR2 ready.
From
my perspective, pretty much all of the packages I maintain (As far as I
know) are PHP5 'compatible'. however they may emit E_STRICT errors.
This
brings up the interesting question, which I guess all the current
maintainers, users and contributors have come across, how much value is
added to the existing packages by adding that level of support.
From
an 'ideal world' / 'perfect coding' perspective, they would benefit
from this changes. but as somebody who earns an income by delivering
projects as quickly and efficiently as possible, the return on
investment for making those changes is very tiny, if not negative.
Two
of the biggest changes I'm aware of for this 'PHP5 compatibility' issue
are the 'static' method prefix and getting rid of 'var' which
completely break PHP4 compatibility (and yes we still maintain PHP4
applications, and clients rarely have budget to make changes like
this). Doing these changes would mean that I would have to either
freeze or depreciate PHP4 support, or start maintaining dual versions.
(Personally I would prefer a hook in the package builder that would do
the replacement for me, so I could upload 2 package on each release).
Going
forward, PEAR2 is still in a gestation period, (as PHP5.3 and
namespaces support has just come out.) Resulting in any code that had
targeted PHP5.3/PEAR2 aging very quickly (eg. requiring changes to
handle the final changes to the namespace syntax.). This may start
changing soon, however I suspect it would really take some significant
effort in time to start creating PEAR2 packages for existing code
(which has a rather poor return on investment) . And without a
reasonable base number of packages, the attraction of submitting code
to PEAR2 is lessened. A classic chicken and egg situation.
At
the same time, there is no real alternative to PEAR2, pretty much all
other 'framework' solutions have been built around the assumption that
you have to accept a majority of the 'framework' to utilize the single
packages. Which is even worse that the pains that PEAR(1) imposes on
you.
All that said, if you want to send me patches to fix any
big PHP5 issues in my packages please don't hesitate, I will try and
make the changes.
Went along to Barcamp in Hong Kong this weekend, Have to thank all the guys & girls involved in organizing it. As it was a chance to meet some interesting people, who I did not realize where even in Hong Kong.
Barcamp Hong Kong, is perhaps a little different from what you might get elsewhere. First off it's not so much a techy thing. (Evidenced by the sight of 1 Linux desktop = mine...) I gave one talk on 'D', and went to a couple of others with lesser technical emphasis - drupal / OO JS etc. but the majority of talks where either business development style (raising funds, startups). or iPhone related (with out the coding stuff).
I guess this comes from the general attitude here that most programmers tend to regard it as a stepping stone to management or something, and have very little passion about the whole industry.. (as someone described it to me).
The technical talks gathered at most 20 people (out of I guess 200-300 people at the camp). And unfortunately, my talk/chat/discussion got rather hijacked by someone who wandered in, was completely non-technical, and started asking irrelivant questions. (self reminder, politely say, this is for programmers only next time)
Overall it was not much of a learning experience, rather the chance to actually meet developers with similar interests, (and see that they actually exist) - Shame I had to leave early for family stuff, as the drinks would have been good...