Smoking toooooo much PHP
Thursday 28 June 2007
To generate the Gtk bindings (and adding other features) for gtkDjs I use a code generator. This is pretty common to all the Gtk bindings. I worked on the PHP-GTK ones a long time ago, which in turn where based loosely off the Python bindings.
For GtkDjs, the starting point was the GtkD binding code. While the current code is still similar to it's parent, It has grown considerably since it's original birth. So In the vain of "if someone ever feels like helping out with the bindings", or wants to write and send me bindings for SQLite, Mysql, curl or libsoup etc. Here's the inside story of them.
The basic concept.
In essence, the bindings work using "generator scripts" called APILookup*.txt, which contain a series of either information, or instructions to do tasks.
Other than the overall configuration settings for the generator, or libraries, the core part of generating bindings is to create classes. In Gtk this is normally done by reading the HTML documentation for a specific Gtk Struct, and using that information to generate data about the Enums, Signals and Methods that are defined. This is all done by GtkClassParser.d
Finally after the File has been parsed, the Output classes "ClassOut, FuncOut, EnumOut, LibraryOut, PhobosOut, SignalOut" generate the code files in the src directory.
Unlike the original GtkD bindings GtkDjs does a double pass at generating each of the packages, as we use the first pass to calculate dependencies and inheritance, rather than relying on the list of imports in the APILookup files. This means that the APILookup*.txt files do not really have to list all the required imports.
More details on the APILookup files.
While I'm not going to explain all the possible commands available in the files, I will explain the concept and a few key ones. The file "GtkWrap.d" contains the parser, with a complete list of keywords and actions, it's a reasonably simple and obvious piece of code. I'm still adding new commands when I see them as necessary, and may even remove a few as some are pretty pointless or unused.. (like code: *** which is ignored but is usefull for reference on how GtkD did things).
The basic concept of the file is a list of data in the formats:
#for simple data
keyword : Value
# for building a list of key value pairs
keyword : key valuepair
#for long data - like code etc.
keyword : subject
......
keyword : end
#for long data - (older style)
keyword : start
......
keyword : end
Order is quite important, as the parser for the APIWrap is not forgiving and can easily go into infinite loops if you get the wrong or unknown keyword in the wrong place. Fortunately using the existing files as a basis for a new one should help out a bit.
The core APILookup.txt files contains a few key pieces of information
- aliases of Gtk base types to D types (and a few kludges to work around other issues)
- mappings of the D-Types to Javascript Types (really related to the method calls on dmdscript.value)
- where the files are, like the gtk documentation and where the generated files should be written to.
- List of packages, and how they related to directories (Note that the GtkDjs bindings also use a jsPackage command in the individual wrapping files to determine how they are exposed to Javascript)
- A small bit of code to copy the dl() loader into the generated directory.
- And finally a "include" like statement "lookup:" which tells the parser to start using the new file as it's input.
The individual Binding for the specific libraries (or packages as the tend to be called) are structured a little differently.
- At present there are alot of add*: commands, which are generally ignored by GtkDjs, and will probably be removed. - As we auto generate most things
- In the Gobject bindings, you will see an example of structCode: which is how structs are described if they can not be done by just pretending they are pointers to void.
- Again in the Gobject you can see a manual enum description. necessary as the real Enum is not actually described in the documentation.
- next are the two key information, wrap and jsPackage - wrap indicates which package it is wrapping (from those listed in APILookup) - so it knows where to write the files. and jsPackage tells it how to expose it to Javascript
- structWrap: is used simply ensure that the struct is available usually as an empty struct.
- nostruct: is used to prevent a struct from being bound.. usually an indicator that something needs fixing..
- finally we have a small batch of code for each class to be written. The class: command indicates the filename of the code to be written, the real classname is based on the struct that is being wrapped usually.
- some of the files have jscode, and jssig elements describing how the automatic code and documentation generation should be overridden.
The generator code.
In order of how they are used.
- GtkWrap.d (includes main()) and is the command interpreter, data from the commands is stored in a ConvParms class (convparms.d), and passed to the Outputters. Any packages that are defined are stored in a class defined in Packages.d.
- DefReader.d is the helper class that does the low level conversion of the APILookup.txt files into commands.
- GtkClassParser.d does the legwork of collection all the described definitions in the HTML file into Classes from GtkStruct.d GtkFunc.d, GtkEnum.d (note signals re-use the GtkFunc class). The HtmlStrip.d library just removes the HTML tags making the documentation files a bit easier to parse.
- Finally after a file is parsed, usually the outFile: command is found and the class code is generated by ClassOut.d. Which in turn loops through the functions, signals and generates the code using FuncOut.d and SignalOut.d
- At the end of each package file, the loader, struct listing and enum lists are generated from LibraryOut.d which uses EnumsOut.d and GtkStruct.d (although that wasn't the best of design hacks)
- PhobosOut.d is used by the non-gtk code to generate code that does not need library links.
The design differs a little from the original GtkD generator in that I separated the parsing of the documentation and the code generation into separate classes. I felt the original code was pretty huge and unwieldy done that way.
Anyway comment away if you have any further thoughts...
Wednesday 27 June 2007
Two minor updates this week, First the GtkDjs manual now works in IE, along with featuring user comments, - you can just add normal comments, code examples, rewrite the introduction, or note a bug. Eventually I plan to add an approval flag so it moves the comment into the documentation, along with the ability to create a comment based on another comments or the existing documentation. And maybe HTML comments... As for the bindings I've been looking at the GtkTreeView and Gtk.TreeStore. To push it a bit, I built a little file navigator (in the tests folder). Key to this was the ability to store Javascript objects in the Treestore. My initial idea of just using a gpointer to the class turned into a bit of a disaster, as the garbage collector assumed that the object was not being used, (even though a pointer to it remained in the store), and free'ed it. This leed to segfaults later when trying to read it. To solve this I ended up using boxed types, so hopefully I've implemented so the objects now get free'd correctly.. I've also simplified the setting method for the store (next on the list is to look at the fetching methods). So storing data in a node/row is a mater of: store.append(iter,parentIter);
store.set(iter, 0, "node title");
store.set(iter, 1, {directory: pdir + name + "/" }); Getting a directory listing uses the new Phobos (or generic D binding backend) binding code. var filelist = File.listdir("/home/alan");I did notice one little gotcha for the dmdscript backend, that it does not support the "in" comparison operator So if ("fred" in myobject) {...}
becomes
if (myobject.hasOwnProperty("fred")) {....}
Talking of gotcha's I spent quite a long time getting the manual to work in IE. My little extjs tricks noted that comma's are a bit of a nightmare in IE, in that It doesnt like trailing comma's in lists of object properties. As typical with any Microsoft product, this is totally inconsistant with the behaviour for arrays, in which case it quite happily accepts trailing commans. Not only does it accept them, it adds an undefined element on the end of the array... - which broke my method list horribly in IE. Otherwise getting it to work in IE basically consisted of using Ext.DomQuery rather than trying to use standard DOM calls which never seem to work as expected. Anyway comment away...
Monday 18 June 2007
Other than Enums which are now created and available in Javascript, I've tidied up the generation code quite a bit, so static methods are generated correctly. Less brute force generation is used, and made a start on the documentation generation. So now most of the donkey work of the bindings are done, it's down to the last 20% (the long tail from what I've been reading recently). Something like the main developer[s] get's 80% of the project done, but since there's always those little niggly things that dont work it takes 1000's of developers to fix the last 20%... - interesting statistic, which explains why it's such a pain fixing all those bugs that crop up.. Anyway The GtkDjavascript Manual has been born.. While again not quite ready for production, (It desperately needs user comments, Enums, Signals ,hrefs linking all together and many other things.). I think it's a step in the right direction. The navbar, and the XML method descriptions are generated from the code generation script. (The main body is a nice example of extjs - and should be portable to gtkdjs eventually) - The biggest bug on the docs is that stuff that is manually overriden (eg. Gtk.init()) is flagged as broken in the manual. as I've not added any code to add the fact I've overridden it to the manual docs.. Anyway.. ticking along nicely.. now I wonder how much use it is....
Wednesday 13 June 2007
I'm still digging through the gtk api, checking it works as I go. (and still deciding on a name for the bindings). I've been looking at perhaps one of the two most complex widgets in Gtk, GtkTreeView. The tutorial for which looks like it weighs in at half the Full Gtk manual. At present I've got through the first section, and got a roughed out GtkListStore to work with it. (TreeStore should come next). For those who dont know GtkTreeView, it uses the concept of a Model, and a View. The Model contains all the data (in rows or a tree) - GtkListStore is a model. The View (eg. GtkTreeView and it's columns) describe how to render the view, and what columns to use from the model. This API is pretty raw at present, Some of it can be tidied up in Javascript, others may need more work in the bindings First part of using the tree is creating the Store. var store = new Gtk.ListStore(
"gchararray",
"gfloat",
"gpointer",
"gboolean",
"GdkPixbuf"); At present I've used the Gtype names to define what is being stored where. - This may need simplifying, and integrating somehow with the writing mechanism as validating data types written to the tree is a bit haphazard at present. Next, to add data, you must use what I think is the worst named Class in Gtk, Gtk.TreeIter. a better name would have probably been something like GtkRowReaderAndWriter (bit of a mouthfull, but alot clearer.) To use this Row reader/writer, we first have to create a blank version. Then call the append Method on the store (a bit ass backwards, but again this could be fixed with some Javascript prototype modifications. var iter = new Gtk.TreeIter();
store.append(iter);
The append method loads the Row reader writer, so that it can access the new row. Next we add data to the row. This is done via G.Values at present, although this may change in the future... store.set(iter, 0, new G.Value("test"));
store.set(iter, 1, new G.Value(1.0));
store.set(iter, 2, new G.Value({ a: 1, c: 2}));
store.set(iter, 3, new G.Value(true) );
var pixbuf = Gdk.Pixbuf.newFromFile("gnome-logo-icon.png");
store.set(iter, 4, new G.Value(pixbuf) );
all this could easily be abstracted by defining a simple prototype Gtk.ListStore.prototype.appendRow = function(rdata) {
var iter = new Gtk.TreeIter();
this.append(iter);
for(var i =0;i<rdata.length;i++) {
this.set(iter,i, new G.Value(rdata[i]));
}
}
.....
store.appendRow(["test", 1.0, {a: 1}, true, Gtk.Pixbuf.newFromFile(....)]);
Next step is to build the Widget. And the renderers (the classes that turn the data into visable data). Gtk comes with a few renderers, It will be interesting to see if creating them in Javascript is feasible. var view = new Gtk.TreeView();
var renderer = new Gtk.CellRendererText ();
var pixrenderer = new Gtk.CellRendererPixbuf ();
Adding columns is again a little raw, as we cant use the varargs methods the C does, so we have to combine a few calls.
var col = new Gtk.TreeViewColumn();
col.setTitle("Name");
col.packStart(renderer, true);
col.addAttribute(renderer, "text", 0);
view.insertColumn( col, -1);
Again a nice wrapper could be written. Gtk.TreeView.prototype.insertColumnAuto = function(config) {
var col = new Gtk.TreeViewColumn();
col.setTitle(config.title);
col.packStart(config.renderer, true);
col.addAttribute(config.renderer,
config.renderer.isPrototypeOf(GtkCellRenderText) ?
"text" : "pixbuf", 0);
this.insertColumn( col, -1);
}
...
view.insertColumnAuto({
title: "text here",
renderer: new GtkCellRendererText(),
}
Merging the data and the view is done simply by setting the model. view.setModel ( store );
window.add(view); // add it to a GtkWindow..
window.showAll();
Gtk.main();
I also had a few experiments with callbacks. view.connect("row-activated", function(view, path, col) {
var model = view.getModel();
var iter = new Gtk.TreeIter();
if (model.getIter(iter, path)) {
var name = new G.Value();
var name2 = new G.Value();
model.getValue(iter, 0, name);
model.getValue(iter, 1, name2);
println (
"The row containing the name '"+
name.getString() +
"' has been double-clicked. holding" +
name2.getFloat()
);
model = new Gtk.ListStore(model);
model.remove(iter);
println ("removed?");
}
});
At present this is a little klunky, but again with some prototype modifications it should be pretty easy to solve. Fetching values has to be done by creating an empty G.Value, them fetching and converting them. The last line illustrates removing a line. - since we cant cast in Javascript, I've modified the ListStore constructor to accept either a list of strings (eg. creating it) or a store object which does effectively the same as casting.. enabling you to call the remove method on GtkListStore.
Anyway back to fighting fires with real work today...
Wednesday 6 June 2007
As I dig away at the Gtk wrappings, I'm testing the ideas that should make the bindings interesting. One of the ideas is that extending core Gtk Objects should enable easier usage. I started with the basics of interface design, Making a Menubar as simple as possible to build, and bind.. var menuBar = new Gtk.MenuBar.quick([
new Gtk.MenuItem.quick("File" , function () {
println("File");
}),
new Gtk.MenuItem.quick("Edit" , function () {
println("Edit");
}),
new Gtk.MenuItem.quick("Search" , function () {
println("Search");
})
]);
In the above example, I've created an extended class Gtk.MenuItem.quick, and Gtk.MenuBar.quick (it could just be Ext.MenuBar / Ext.MenuItem...) - Just adding it to the Gtk.Menubar Class namespace seemed clever at the time. As you can see, creating the menu, along with adding the handlers is very simple. - I could even use a standard Javascript object as the constructor (even more like ExtJs): eg. new Gtk.MenuItem.quick({
title: "Search" ,
icon: Gtk.Stock.SEARCH, // this doesnt work yet!
handler: function () {
println("Search");
}
})
This morphing of the Gtk bindings is all done in the Javascript code, Gtk.MenuItem.quick = function(label, activate) {
Gtk.MenuItem.quick.superclass.constructor.call(this,label);
this.connect("activate", activate);
}
Ext.extend(Gtk.MenuItem.quick, Gtk.MenuItem, {});
Gtk.MenuBar.quick = function(ar)
{
Gtk.MenuBar.quick.superclass.constructor.call(this,label);
for (var i in ar) {
this.append(ar[i]);
}
}
Ext.extend(Gtk.MenuBar.quick , Gtk.MenuBar, {});
(using snippets of code from Extjs) to implement the extending. On the backend however, I had to make a few changes to dmdscript, both the DObject definition (which is the class that handles Javascript objects in the script engine). Noteably adding generic support for bound void*'s to all Javascript objects. (so binding things like curl, mysql, sqlite etc. should be considerably easier..) Along with this, dmdscript as available from digitalmars did not support the {function}.call(this, args) syntax, a stub was there, and it was pretty simple to add. Anyway the next big challange is understanding GtkTreeView, GtkTreeModel and GtkTreeIter so that building Trees can be made more Javascript (and Extjs) like..
Sunday 3 June 2007
Signals for the most part are working. I went with the GTK'ish style, rather than the more JS style, eg. var a = new Gtk.Window(0);
a.connect("delete", function() { Gtk.exit(1); });
That's not to say that it's not trivial to use a more javascript style = in the client side!!!! Gtk.Widget.prototype.on = function (sig, call) { return this.connect(sig,call); }There is still quite a bit todo, especially around the more complex widget's like textview, tree and lists. but I suspect the majority of the main widgets should just work. I'm kind of wondering If I'm announcing this to the wrong audience though, as I dont really get any sense of excitement. I have this wierd vision that GTK and Javascript could be a leap forward in application development. Having written a couple of projects both in scripted and compiled languages, The whole save/[compile]/run/test loop can get very slow, especially as the application grows. The idea, that to edit the toolbar, and it's behaviour, I would pull up the debugger/editor in the running application (That it'self would be written in gtkjs). Navigate in the tree of objects (AKA Firebug DOM inspector) to the class definition or prototype for the menu. - click edit - bring up a scintilla or GtkSourceView editor, edit the code.. save it.. (editor both saves to disk overwriting the specific code that it needs to), along with compiling and altering the in-memory method - Then type something in the debug command line to for the refresh of the menu... - Or even have the editor know that saving that function would need some js run to test it.. - and it affecting the running application - changing the behavior and look!. Not only that, the program would not crash, only output to the debug console any unhandled exceptions... - (with line/file/method etc.) I'm just in awe of the concept of adding a small feature or fixing a bug would by so mind blowingly simple, that you would lower the barriers even further to make more of your users your development team.. Then again other days I wonder if I've been doing to many of those drugs....
Friday 1 June 2007
My last entry described how to create objects for dmdscript, I made a brief comment, that gtk javascript might be quite cool.. Well after a bit of hacking I got hello world to work. (well and a bit more..)
Gtk.init();
var w = new Gtk.Window(0);
w.setTitle("hello world");
w.showAll();
Gtk.main();
run with /tmp/gtkjs hello.ds I started with the GtkD bindings code generator, and extracted out the HTML parser code (it parses the HTML API docs). The based on the code from the last post, and some of the ideas from GtkD generated all the classes for dmdscript. At present: - 317 Classes, Including Atk, Glib, Gobject, gthread, gdk, gdkpixbuf, glade gtk and pango
- 2554 Methods should be working.
What's left to do.. - Signals - I've got a prototype working for this, so It's mostly a matter of writing the generation code.
- Enums - They are in the D code, but no exported to the Javascript side.
- Argument checking.. - Should be pretty trivial to add..
- Return checking.. again pretty trivial..
- Custom code for non-generated methods, Structs etc.
- Tiding up the APILookup files. - - they are pretty similar to the GtkD code at present, and my generator just ignores alot of them.
- Lots of code tidy ups.. (the class registration is a bit of a brute force hack)
If you want to try it out... (developers only.. dont expect to write applications with it yet.) svn co http://www.akbkhome.com/svn/gtkDS
cd gtkDS/wrap
#download API Files (needs curl / tar etc.)
sh downloadFiles.sh
#compile it - needs dmd installed..
sh buildit.sh
cd ..
#build the bindings - requires compd from dsource.org/projects/dool
compd gdkds.compd
/tmp/gtkjs test/hello.ds
Now I just wonder what use it is ;)
|
