Smoking toooooo much PHP

Just before I saw Brendan's post about PHP4 compatibility in PEAR, I had been getting a few queries about making a couple of my PEAR packages more 'PHP5' compatible or PEAR2 ready.

From my perspective, pretty much all of the packages I maintain (As far as I know) are PHP5 'compatible'. however they may emit E_STRICT errors.

This brings up the interesting question, which I guess all the current maintainers, users and contributors have come across, how much value is added to the existing packages by adding that level of support.

From an 'ideal world' / 'perfect coding' perspective, they would benefit from this changes. but as somebody who earns an income by delivering projects as quickly and efficiently as possible, the return on investment for making those changes is very tiny, if not negative.

Since the packages generally just work, making the changes required, would not really change that 'just work' situation, and as� Jamie Zawinski famously said "How will this software get him laid?"

Two of the biggest changes I'm aware of for this 'PHP5 compatibility' issue are the 'static' method prefix and getting rid of 'var' which completely break PHP4 compatibility (and yes we still maintain PHP4 applications, and clients rarely have budget to make changes like this). Doing these changes would mean that I would have to either freeze or depreciate PHP4 support, or start maintaining dual versions. (Personally I would prefer a hook in the package builder that would do the replacement for me, so I could upload 2 package on each release).

Going forward, PEAR2 is still in a gestation period, (as PHP5.3 and namespaces support has just come out.) Resulting in any code that had� targeted PHP5.3/PEAR2 aging very quickly (eg. requiring changes to handle the final changes to the namespace syntax.). This may start changing soon, however I suspect it would really take some significant effort in time to start creating PEAR2 packages for existing code (which has a rather poor return on investment) . And without a reasonable base number of packages, the attraction of submitting code to PEAR2 is lessened. A classic chicken and egg situation.

At the same time, there is no real alternative to PEAR2, pretty much all other 'framework' solutions have been built around the assumption that you have to accept a majority of the 'framework' to utilize the single packages. Which is even worse that the pains that PEAR(1) imposes on you.

All that said, if you want to send me patches to fix any big PHP5 issues in my packages please don't hesitate, I will try and make the changes.

Well, after a busy few months, things have gone quiet again. Hopefully it's a short term thing, but it has given me a bit of time to do get back to the less profitable things in life (like blogging).

PEAR releases

Yes, after a quite a few emails bugging me to release updates to DataObjects and Template Flexy, I finally got round to getting them out the door� (and even fixing a few bugs after the got out). I've also made an effort to get Services_JSON onto the pear release system, as I use it quite a bit, and it's been sitting in as the proposal for Services_JSON for well over a year.

Facebook

Yes, that other great waster of time, I've finally set up an account for me - "AK BK Consulting" - After my first effort of using facebook fell apart, due to the mess it made of mixing family stuff and professional stuff, (I use my wife's account for my family stuff). I set up that one so I can join up with PHP developers anywhere and IT people in Hong Kong. So feel free to add me as a friend (as I dont have many ;) - this blog should be� syndicated into my page...

ExtJS / Roo builder for HTML_FlexyFramework

I was messing around this week writing a builder for HTML_FlexyFramework, my little lightweight page loader, that integrates quite nicely with Dataobjects, Template Flexy and pear in general. Part of the incentive for this was seeing a little project that a potential client had developed, in some windows application that generated an how site starting with the database schema.

The idea� was quite nice, and the interface of the builder was quite friendly. But the downside was that the resulting code was just unusable jibberish. So rather than work out how to add features to it, I wondered if using DataObject's Generator as a core, I could generate a whole ExtJS/RooJS interface from the database, and then edit that to quickly get a frontend up and running.

The code's sitting in my akpear repo RooJS_DB_DataObject (it actually writes ExtJS1.1 code) and does the basic tasks of setting up a grid for each table, a simple form to edit each table, along with some limited joins based on the links.ini

If you want to try it out, it runs like this:

php  RooJS_DB_DataObject/cli.php 'mysql://dbuser:dbpass@dbhost/dbname'

Nice little proof of concept.. It's got some idea how to 'update' it's generated code, but I've disabled that at present. It should however give you a quick way to jumpstart an ExtJS application.

There's been a long thread the last couple of weeks covering require_once and some rather crazy ideas to change the PEAR standards to do all sorts of strange things to load code up.

What is most ridiculus is the benchmarking that is being done to 'prove' that we should all move our code loaders to some 'allfiles.php' and we are magicially going to running be 15% faster.

What this whole concepts fails to put into place is loading all the PHP files up, either tiered one to another or all from one place is such a small part of a PHP page responding from a request.

Think of what is happening when you look at this page.

• You start by looking at a bootstrapper (that sets config option) - probably quite quick as it just set's some variables.
• Then you run into the Framework class - that works out which file to load based on the URL you are typing. - This does quite a bit of text maniplation, making best guesses about where things might be. (this is quite generic and could be made specific to the application quite easily)
  
• We then do the page action stuff, like pulling data from databases which requires quite a few PEAR libraries. And does some really slow crap like connecting to databases and pulling data. This stage probably does far to many queries and pulls down to much data that it doesnt use.
• Then we do output stuff, Normally we are using a compiled template, (so we dont load all the template parsing code - well at least we save a bit of effort here). so we need to pull down at least the Flexy class, then the compiled template.

Now looking at that whole process and thinking it's a bit slow, You would probably go through the following steps to optimize it.

• replace it with a staticly generated page!!!

Well, yeah, that's it!....

Only kidding... - but if you do this to a normal page, you would probably want to do the following

• reduce / remove database calls
• cache as much as possible.
  
• reduce / remove libraries needed and code used.
• move any data intensive code into C
• use an opcode cacher.
  

At no point would you get so silly that you would re-jig the 'require_once' calls just to get 15% saving on the code loading component, because wait for it..... the code loading component of your application (when using APC) would take such an insignificant percentage of the total running time.

Working out how to remove a whole file or pre-process some bit of data would be a far better use of your time. And if you where running a team focusing on this you would probably have fired the guy who wasted all that time on such a pointless task as moving the require_once lines around.

Sounds like moving the deck chairs on the titanic?

Having just released the increadibly minor update to the Flexy Template Engine in pear, I can now release the documentation that goes with it...

FlexyFramework, with HTML_Template_Flexy include the ability quickly create multilingual websites, by translating the templates.

I've set this up a few times now, and kept meaning to document it. - So here it goes.

Having set up FlexyFramework, (see the previous post about this),

... The full instructions are in the Extended entry....


View Extended Entry

After seeing quite a few "fixed XSS problem" cvs commits on the mailing lists, I finally remembered that a release of Flexy was overdue. The thought, "that would not have happened if they had used flexy" came to mind a bit too often.. Flexy is designed to be safe by default (like a firewall where you have to try hard to open ports)

HTML_Template_Flexy has not had much added to it in the last 6 months, probably as it just 'works', tries not to get in your way, or do to much.. But there have been a few bugs left over from the last release in January, along with a few tiny feature requests. So I finally got round to tidying up a release.

There is nothing mind shattering in this release, But users and potential users may be interested in the extra pages in the manual. They should come online by monday:

• flexy:nameuses
  
• flexy:tojavascript jsvar="phpvar"

Along with a pretty complete list of configuration options that I updated a few weeks ago.

As far as I know there is not to much more missing.. but some of these are a bit critical

• {_( translation markers )_} , and how to use the translation toolkit.
• flexy:raw="somevar" for putting text within tags (as otherwise the syntax would break HTML editors.)
• Using some of the modifiers from the Savant Plugin (hint {xxx():numberformat}

I'm sure there is more, but that's all I could think of at present..

Whenever someone starts saying template engines, there's an equally vocal community that gently suggests that PHP is a great template engine. Well, I think this week that sounded alot like bollocks...

The pear website, while not a masterpiece for PHP code, has however been written by some pretty smart people, and uses (in parts) the concept of PHP as a template engine. Last week however we got a very polite email to the group mentioning that it was possible to do Cross site scripting attacks on some pages.

The root of the issue was that it was outputing variables (either directly from input or indirectly) which had not been escaped correctly for HTML or javascript, so it was possible to make your favourite javascript hacks work through the url..

While the issues with pearweb where not that serious, it did illustrate the problem of simple PHP templating against more complex engines like Flexy.

When I wrote Flexy, I'd been doing webdev for quite a while, and realized that like everyone else, I make mistakes (some may say like my opinions on this blog). So to some degree, I tend to prefer my applications to protect me from myself, while at the same time allow me to deliberatly break things.

One of the more unusual features of Flexy, is that all tags eg. {stuffThatOutputsVariables} or the method calls are by default html escaped. (unless you explicitly add the :h modifier). Not only this, these tags within javascript blocks, just dont work. You are forced to use the <flexy:tojavascript tags to send variables to the javascript code, again, reducing the chances of accidentally letting your friendly hacker have fun with your site..

So while PHP templates have some advantages, in that it lacks the requirement for compiling. That penalty seems a small price to pay for the extra protection.. so Flexy's new catchphrase may be, "Put your condom on, and use a Flexy Template Engine..."

For a change, I've taken break from bashing internals, and got back to real work. (More on DBDO later this week hopefully)

One of my on-going projects, that has been dragging on longer than I would of liked is a shipping management application. I think it's mentioned in the archives, but for anyone who missed it, it is a mid sized XUL application which deals primarily with the management of a trading companies shipping requirements. I originally outsourced the main development, and have been tidying up and refining the code as we near final deployment (which as usual has taken longer than expected.)

This week I sat down and focused on the last major part of the project, reporting. Almost all the requirements for reporting include the ability to download an excel file of the data. So previously I had been making heavy use of PEAR's Spreadsheet_Excel_Writer. In using it, I had gone through various stages of evolution

• Writing raw Excel_Writer code in PHP, This however becomes very tedious, is not amazingly readable, kind of breaks the seperation of display/computation. And tends to be less flexible over a long period of time.
• Using a gnumeric as a template and using XML_Tree to merge data with it and output via Spreadsheet_Excel_Writer, again this helped in terms of enabling a simpler API for spreadsheet writing, and moving some of the layout/look and feel into the Gnumeric template. But the code for doing this was not quite as elegant as I would have liked.
• Using Javascript to read HTML tables and create a CSV file, that is sent to the server, and back again as text/csv mimetype (forcing the browser to open it in excel/openoffice etc.). Which was nice from an architectural point of view, by lacked any formating.
• And finally this week. Using javascript to generate a Spreadsheet_Excel_Writer specific XML file (by mixing a XML template file and the HTML content of the page), sending it to the server, and then letting PHP use the DOM extension and simple iteration with Spreadsheet_Excel_Writer to generate the page.

This weeks solution while not quite complete has a number of key advantages, some of which appeared after I started using it.

• No display level code goes into the Action->Data manipulation stage (we just store the data ready for the template engine/ template to render)
• It is possible to visualize the data prior to it ending up in the excel file.
• hence debugging the data output and finding issues is a lot quicker
• More code reuse,
• the library for XML to Excel is simple to reuse,
  
• the code for extracting the data from the html and generating XML is simple enough for copy & paste. and maybe possible to create a js library eventually.
• It offers infinate possibilities for formating, and changing layout.
• Less memory intensive, the data retrieval/storage and excel file create are broken up into two seperate processes.
  

The extended entry includes a few more details....

View Extended Entry

Seeing Davey's missery, and Sebastian's / Kristian's fixes to Serendipity to stop trackback spamming. I struck me that perhaps trackbacks combined with blog spammers are always going to be a disaster waiting to happen.

The concept is quite nice, being able to see related articles, when reading a blog, but the trackback concept is a little too trusting, unless you want to go in and moderate trackbacks.

It inspired me however to hack up a smart referer log for my blog entries. (visit the site to find out more..)

View Extended Entry

Alot of people started using blogs as a slightly better media for technical information, but it's becomming evident that with subjects ruby on rails, and some of MS astroturfing with marketing material, that blog aggregators like Artima have been abused heavily with rather second rate blogging, about 3rd rate tools.

While ruby on rails is probably a good tool, it fails in a huge part from the flawed thinking that one provider can deliver a complete solution. It took me a long while to realize that attempting the complete toolkit that ror promises is often fruitless. It rarely delivers much beyhond the intial demostratable examples. What normally happens is that in designing for a single solution (A super fast web interface to databases), you often end up with libraries that are rather poor for generic usage.

The fact that PHP already has 3 or 4 projects that are based on PEAR, that deliver pretty much the same solution as ror. Indicates that the concept of small flexible libraries, maintained by seperate individuals, rather than one super mega project is always more valuable, although I guess you miss out on the hype more.

After quite a few email conversations about HTML_Template_Flexy, I finally got round to documenting one of the javascript libraries I've modified. This is still a bit buggy around the edges, but is pretty much working.

Javascript Calendar started off with a few hacks around the dynCalendar on pear.php.net, and evolved into nice clean simple Calender renderer.

The main design criteria for this was that it should be independant of the HTML page, not requiring any specific javascript, or changing the backend Template code. Just include the javascript file, set the style, and add a few extra attributes to the input elements.