Smoking toooooo much PHP

I recieved a very nice email, enquiring about some of my code hosted on my dev server. For those not in the know, I try and keep my libraries in
http://devel.akbkhome.com/svn/index.php/akpear/

This includes alot of my non-pear submitted code, Why not submit it to PEAR, you may ask.

• Maintainance - until the bugs and feature requests calm down on my existing packages, I'm already stretched quite far.
  
• Standards - PEAR standards are very strict (for good reason), but For some of the code, which I use once, adding full documentation is often not within the budget.
• Proof of Concept - eg. DWG_Parser, which could be improved to solve a bigger issue. But I've not got the time/budget to do that, so it lives as a proof of concept, and a starting point if someone wants to use it.
  

Some of the highlights include

• HTML_FlexyFramework - url to object loader and mapper
  
• Config_Singleton - 1/2 finished generic Config Store.
• DWG_Parser - read Autocad Dwg files (extracts the icon only)
• Gtk_ValidateManager - really a HTML form validation framework (interface to build validation rules, and runtime code to apply them to HTML pages.)
• HTML_DataObject - browse and edit code for automating browsing of databases and updating them. (This is about the 3rd effort at this, and it's still got serious issues. - but it does work!)
• SpreadSheet_Generic - a tool that at present, can load a Gnumeric file, and write an XLS file, with templating and block replacement abilities.
• XML_SvgToPdf - takes SVG (from things like sodipodi, writes PDF's - with facilities for templating)
  
• Backup_DVD - a nice routine to automate backing up of data

Along with this, a few projects I've worked on , the clients have allow to code to be open sourced, These tend to be highly specialized projects, so that making a big fanfare of their existance is not really that usefull. However they do provide a good illustration of some of the PEAR classes.

Interestingly if you look at the code size for each of the applications, the size of the library folder (eg. pear) is at least 3 times larger than the source code! - pretty good code reuse.

The two key Projects that are available are:

Hebe
http://devel.akbkhome.com/svn/index.php/hebehaven2/
A Yacht club Membership management, billing application, It illustrates alot of the code above in action. I wrote this in 2003, and still make minor updates, but the project is paid for and pretty much done.

Since this project was done, I've stopped using certain features of the design.

• The default entry page class in Hebe is Hebe/index.php (Hebe_index)
• In later projects the base class matches the project name = eg. Hebe in /Hebe.php
• The template folder was in the top level folder eg. /templates
  
• In late projects that was move to the Project folder eg. /Hebe/templates

FlexyShipping
http://devel.akbkhome.com/svn/team.php/flexyshipping5/
This is a Shipping management application in XUL, I did the specification and outsourced it to a team to build. After the majority was completed, I took it back onboard, and have been working with the end user to complete all the features they want.
It is interesting for a number of reasons other than just using XUL (which simply blows away HTML for intranet applications)

On the positive side:

• It meant alot of additions to HTML_Template_Flexy to support XUL.
• There is a very nice library in there for form submiting of XUL interfaces (and some nice tree stuff) common.js
  
• The code design while not perfect enabled the outsource team to work on a simple concept, and ensure that it was maintainable when It went into the feature finalization stage.
• Due to the Intranet nature of the project, almost all the data validation is done client side.. (so in theory we only check if the return values from update/insert where ok on the server side).

On the not so great side

• There where initial problems that the contractor didnt know how to use DataObject correctly (and missed some of the key features, eg. setFrom(), factory()
  
• The file layout (alot of layered directories is an absolute pig to maintain.. ) eg.
• FlexyShipping_Add_Order_Item was in FlexyShipping/Add/Order/item.php
  
• This should have been FlexyShipping_OrderItemAdd,
  most of the generic subclasses should really have ended up in the top level directory, with perhaps only some of the more specialized page classes in a 3rd level folder (like reports perhaps)
• There is a considerable amount of repitition (eg. lots of small classes which do little more that list/get/set )which something like HTML_DataObject solved in Hebe. However I suspect that this is actually clearer than the HTML_DataObject solutions, which eventually become horrifically complex.

Akbkhome
Of course the latest in the line of projects is the akbkhome website, which is trying to make highly modular the whole Framework concept, with some mixed success. which can be seen now via it'self

I've been having a little attempt revamping my subversion viewer, seeing if I can

1. Integrate it into the new site
2. Use VersionControl_SVN, rather than the quick hack I wrote, Subversion_Stream.

I have to admit, though, the API for VersionControl is not exacly clean and user friendly. This is a general overview of the gripes.

• Uses PEAR_ErrorStack, which hides errors, even worse than PEAR_Error used to.. DataObjects uses a ErrorStack style system, you can print_r the dataobject, and it shows you the last error as a propery of the object... (simple).
  
  Error stack seems to insist on you loading it, then only using it's methods to check stuff. - the actually error is hidden away in the stack object..
  
  Trouble is, that a number of the functions in DataObjects needed to return true/false (fetch, for example), so returning an PEAR_Error, was problematic then (in hindsight, alot of the other methods should have returned PEAR_Error, but that broke BC - and in the future it should throw exceptions).
  So, DataObjects had some logical justification for messing with the standard a little, VersionControl_SVN hasnt really, and it just adds a dependancy on a eventually redundant package.
  
  
  
• Trying to make things easy ending up with a messier API,
  
  $x = VersionControl_SVN::factory(array('list','cat'), $options);
  $x->list->run()....
  
  while this is cute, it adds an horrific amount of complexity to the API, and only saves you a little bit of typing, at the expense of clarity and readibility.
  
  
• Counterintuative program flow.
  

class VC_SVN {
.... function run() {
....... $this->prepare();
....... exec in here..
....... $this->parseOutput();
}

VC_SVN_List extends VC_SVN {
..... function prepare() {

So you call run on the base class, and the body of what happens is in the extended class. - so if you look at the 'action' classes, it is impossible to follow the flow of the application, without having a perfect memory of the base class..

it would have been far better to have a run method in each action class, and call some generic methods from the base class.


• direct API mapping makes it a little unclear how it should work:
  example -
  
  require: svn list -v {mysite}
  
  command:
  $svn->list->run(array({mysite}), array('v'=>true));
  
  (and if you get it wrong, not much happens)
  ok - how is this simpler than:
  $svn->run('list -v ', $mysite);
  
  

Defer unknowns has proved very successful, with success rates of more than 1000 spams per day removed from my email box.

My original code parsed the exim log, and built black,white and greylists by selecting checkboxes ona web page. It was effective, but involved a little maintenaince. (daily review, and mostly blacklisting IP addresses.)

By looking at the general pattern of this, it became clear, that most spammers fire off a large number of hijacked PC's and just run through a big email list. If it fails (eg. defer), they just give up on that machine, and pass it along to the next (often changing the signature). - It's a known trick to do defer greylisting on this. Basically first time that ip contacts you, you respond, defer, try later.. next time, you let it through. (In my new scheme, I only black/grey/white list the ones that tried more than once - which should significantly reduce the amount of maintenance, and makes spotting good IP addresses alot easier.

The other beauty of the new solution is that it doesnt involve parsing logs anymore, it's almost a pure exim/mysql solutions, with my manual categorizing a considerably simpler web page.

The exim config I'm using is available in the extended entry.
or have a look at the simple spam manager interface

I guess if you want to run this on a bigger site, you might want to go to the mysql conference where you can find out reall answers form mysql developers, and experts. (and if you go to the php conference at the same place/time, you can see me talking about php5 and pear.)

View Extended Entry

That last entry noted the huge torpedo that Windows had fired at my Racing yacht (DBDO). I spent a good day or two exploring options, apart from the ones mentioned previously (using pdo as a backend). I looked at having a way using callbacks to implement the libgda stuff in clientside php on windows.

All in all the options where looking pretty dire, I was going to have to spend alot of time just to create a crippleware version for windows (well I guess if you use windows you shouldnt expect better). But It was not really what I regarded as a interesting or usefull use of my spare time.

On the sidelines, I had also looked at building libgda on windows, (or Win4Lin in my case), I had tried mingw, dev-cpp, but having to deal with a operating system that was never designed to make building things easy, it just became fustrating and annoying.

I dont know how I came across it, perhaps researching mingw, but I accidentally ended up on a page discussing cross-compiling mingw. It seemed to infer that you could build the windows .exe and .dll's on linux, and not have to go near the windows hell..

After some experimentation, (trying the debian package for it, and downloading a whole enviroment - google for cross compiling mingw), I got something resembling an enviroment that ran configure, and even actually made a few files.

After tying to build libgda for a while,I concluded that using configure/make on it was pretty pointless, (It adds a huge overhead, got various thing wrong most of the time). So I ended up hacking up a php build script for it which successfully compiles libsql, and libgda (although I havent linked or tested it yet)

Ah well, back to real work for a while, panic at least partially over...

I really hate it when I overlook something. This week it was libgda, which is the current backend for DBDO. It was after a few commits by the pecl team to the config.w32 file in cvs. The question came to me.... Could DBDO actually be built on windows?

Unfortunatly after some reseach, the answer was a definate maybe. libgda was designed to be buildable on windows, but no-one has really taken up the gauntlet and run with it. Browsing through the archives, it looks like a few people have tried, but no-one had actually decided to make it their baby. So no dll's or lib's exist for libgda.

This does leave DBDO in a bit of a state of flux.

• Should I continue on, as on Unix, libgda provides an excellent backend, and builds really easily and quickly. (on the offchance that someone gets passionate about win32 support and spend a few nights figuring out how to build libgda)
• Should I look at the alternative backends:
• libdbi - which lacks
• has no global connection/config cache
  
• a schema retreival API
• a column type to abstract column type API
• pdo - which lacks
• has no global connection/config cache
• a schema retreival API
• a column type to abstract column type API
• copy & pasting chunks of libgda (impossible due to the LGPL...), let alone a pain to implement..

To implement the same thing using PDO, would require

• copying the driver header file (and keeping up with any changes)
• copying the driver loader code.
• implementing a config cache.
• implementing a connection cache.
  
• adding schema query and fetchers for all the backends.
• adding schema type to enum type mappers.
  
• enriching the type enum? maybe

It may be worth attempting this by emulating the gda calls, and relaying them through the pdo backends??

On a side note, It became clear earlier that the joinAdd design in DataObjects is probably far better done on the client side, - most of it relates to building the join part based on the links configuration. As far as I can see, this is hardly touches on the special advantages of coding it up in C.

Sometimes, it's those little problems that have annoyed you for years, and you never bothered looking at how to solve them. Today it was alternating highlights on a table.

A horrifically simple problem, rendering a table of data, and to make it easier to read, you alternate the rows with a light grey background. I did a whole application, that was heavily focused on displaying data like this, and came back this week to add an extra page.

The old code did a standard DataObject/Flexy Template trick:

$hightlight = 0;

while ($do->fetch()) {
	$r = clone($do);
	$r->highlight = "row" . ((int) $highlight);
	$hightlight = !$highlight;
	$this->rows[] = $r;
}

and in the template:

<tr flexy:foreach="rows,row" class="{row.highlight}">
	<td>{row.somedata}</td>
	...
</tr>

Ok, that wasnt too bad, but somehow the messing around in PHP each time, just to do the highlighting felt kind of messy.., I had looked at this again recently for another project, and discovered there is a way to do it using CSS2 (but IE doesnt support it.. - what a supprise). I had also seen another suggestion, adding a behaviour to the style for the table, this seemed reasonable, however, when I'm prototyping, I like to keep things together, so it was a bit annoying to have this loose bit of javascript hanging around in a file on it's own.

So today I got a bit closer to an ideal solution.

while ($do->fetch()) {
	$this->rows[] = clone($do);
}

And in the template:

<script type="text/javascript">


function highlightTables()
{
	var tables=document.getElementsByType('table');
	for (var t = 0; t < tables.length; t++) {
		if (tables[t].getAttribute('class') != 'stripes') {
			continue;
		}
		oRows = tables[t].rows;
		var   len = oRows.length;
		for (i=1; i<oRows.length; i++) {
	        	oRows[i].setAttribute("class", 
				(oRows[i].rowIndex % 2) ? 'row1' : 'row0');
    		}
	}
}
// run it on load, or just call the function when you startup.
window.onload = hightlightTables

</script>

<tr flexy:foreach="rows,row" class="stripes">
	<td>{row.somedata}</td>
	...
</tr>