Smoking toooooo much PHP

There's been a long thread the last couple of weeks covering require_once and some rather crazy ideas to change the PEAR standards to do all sorts of strange things to load code up.

What is most ridiculus is the benchmarking that is being done to 'prove' that we should all move our code loaders to some 'allfiles.php' and we are magicially going to running be 15% faster.

What this whole concepts fails to put into place is loading all the PHP files up, either tiered one to another or all from one place is such a small part of a PHP page responding from a request.

Think of what is happening when you look at this page.

• You start by looking at a bootstrapper (that sets config option) - probably quite quick as it just set's some variables.
• Then you run into the Framework class - that works out which file to load based on the URL you are typing. - This does quite a bit of text maniplation, making best guesses about where things might be. (this is quite generic and could be made specific to the application quite easily)
  
• We then do the page action stuff, like pulling data from databases which requires quite a few PEAR libraries. And does some really slow crap like connecting to databases and pulling data. This stage probably does far to many queries and pulls down to much data that it doesnt use.
• Then we do output stuff, Normally we are using a compiled template, (so we dont load all the template parsing code - well at least we save a bit of effort here). so we need to pull down at least the Flexy class, then the compiled template.

Now looking at that whole process and thinking it's a bit slow, You would probably go through the following steps to optimize it.

• replace it with a staticly generated page!!!

Well, yeah, that's it!....

Only kidding... - but if you do this to a normal page, you would probably want to do the following

• reduce / remove database calls
• cache as much as possible.
  
• reduce / remove libraries needed and code used.
• move any data intensive code into C
• use an opcode cacher.
  

At no point would you get so silly that you would re-jig the 'require_once' calls just to get 15% saving on the code loading component, because wait for it..... the code loading component of your application (when using APC) would take such an insignificant percentage of the total running time.

Working out how to remove a whole file or pre-process some bit of data would be a far better use of your time. And if you where running a team focusing on this you would probably have fired the guy who wasted all that time on such a pointless task as moving the require_once lines around.

Sounds like moving the deck chairs on the titanic?

Having just released the increadibly minor update to the Flexy Template Engine in pear, I can now release the documentation that goes with it...

FlexyFramework, with HTML_Template_Flexy include the ability quickly create multilingual websites, by translating the templates.

I've set this up a few times now, and kept meaning to document it. - So here it goes.

Having set up FlexyFramework, (see the previous post about this),

... The full instructions are in the Extended entry....


View Extended Entry

After seeing quite a few "fixed XSS problem" cvs commits on the mailing lists, I finally remembered that a release of Flexy was overdue. The thought, "that would not have happened if they had used flexy" came to mind a bit too often.. Flexy is designed to be safe by default (like a firewall where you have to try hard to open ports)

HTML_Template_Flexy has not had much added to it in the last 6 months, probably as it just 'works', tries not to get in your way, or do to much.. But there have been a few bugs left over from the last release in January, along with a few tiny feature requests. So I finally got round to tidying up a release.

There is nothing mind shattering in this release, But users and potential users may be interested in the extra pages in the manual. They should come online by monday:

• flexy:nameuses
  
• flexy:tojavascript jsvar="phpvar"

Along with a pretty complete list of configuration options that I updated a few weeks ago.

As far as I know there is not to much more missing.. but some of these are a bit critical

• {_( translation markers )_} , and how to use the translation toolkit.
• flexy:raw="somevar" for putting text within tags (as otherwise the syntax would break HTML editors.)
• Using some of the modifiers from the Savant Plugin (hint {xxx():numberformat}

I'm sure there is more, but that's all I could think of at present..

Whenever someone starts saying template engines, there's an equally vocal community that gently suggests that PHP is a great template engine. Well, I think this week that sounded alot like bollocks...

The pear website, while not a masterpiece for PHP code, has however been written by some pretty smart people, and uses (in parts) the concept of PHP as a template engine. Last week however we got a very polite email to the group mentioning that it was possible to do Cross site scripting attacks on some pages.

The root of the issue was that it was outputing variables (either directly from input or indirectly) which had not been escaped correctly for HTML or javascript, so it was possible to make your favourite javascript hacks work through the url..

While the issues with pearweb where not that serious, it did illustrate the problem of simple PHP templating against more complex engines like Flexy.

When I wrote Flexy, I'd been doing webdev for quite a while, and realized that like everyone else, I make mistakes (some may say like my opinions on this blog). So to some degree, I tend to prefer my applications to protect me from myself, while at the same time allow me to deliberatly break things.

One of the more unusual features of Flexy, is that all tags eg. {stuffThatOutputsVariables} or the method calls are by default html escaped. (unless you explicitly add the :h modifier). Not only this, these tags within javascript blocks, just dont work. You are forced to use the <flexy:tojavascript tags to send variables to the javascript code, again, reducing the chances of accidentally letting your friendly hacker have fun with your site..

So while PHP templates have some advantages, in that it lacks the requirement for compiling. That penalty seems a small price to pay for the extra protection.. so Flexy's new catchphrase may be, "Put your condom on, and use a Flexy Template Engine..."

For a change, I've taken break from bashing internals, and got back to real work. (More on DBDO later this week hopefully)

One of my on-going projects, that has been dragging on longer than I would of liked is a shipping management application. I think it's mentioned in the archives, but for anyone who missed it, it is a mid sized XUL application which deals primarily with the management of a trading companies shipping requirements. I originally outsourced the main development, and have been tidying up and refining the code as we near final deployment (which as usual has taken longer than expected.)

This week I sat down and focused on the last major part of the project, reporting. Almost all the requirements for reporting include the ability to download an excel file of the data. So previously I had been making heavy use of PEAR's Spreadsheet_Excel_Writer. In using it, I had gone through various stages of evolution

• Writing raw Excel_Writer code in PHP, This however becomes very tedious, is not amazingly readable, kind of breaks the seperation of display/computation. And tends to be less flexible over a long period of time.
• Using a gnumeric as a template and using XML_Tree to merge data with it and output via Spreadsheet_Excel_Writer, again this helped in terms of enabling a simpler API for spreadsheet writing, and moving some of the layout/look and feel into the Gnumeric template. But the code for doing this was not quite as elegant as I would have liked.
• Using Javascript to read HTML tables and create a CSV file, that is sent to the server, and back again as text/csv mimetype (forcing the browser to open it in excel/openoffice etc.). Which was nice from an architectural point of view, by lacked any formating.
• And finally this week. Using javascript to generate a Spreadsheet_Excel_Writer specific XML file (by mixing a XML template file and the HTML content of the page), sending it to the server, and then letting PHP use the DOM extension and simple iteration with Spreadsheet_Excel_Writer to generate the page.

This weeks solution while not quite complete has a number of key advantages, some of which appeared after I started using it.

• No display level code goes into the Action->Data manipulation stage (we just store the data ready for the template engine/ template to render)
• It is possible to visualize the data prior to it ending up in the excel file.
• hence debugging the data output and finding issues is a lot quicker
• More code reuse,
• the library for XML to Excel is simple to reuse,
  
• the code for extracting the data from the html and generating XML is simple enough for copy & paste. and maybe possible to create a js library eventually.
• It offers infinate possibilities for formating, and changing layout.
• Less memory intensive, the data retrieval/storage and excel file create are broken up into two seperate processes.
  

The extended entry includes a few more details....

View Extended Entry

Seeing Davey's missery, and Sebastian's / Kristian's fixes to Serendipity to stop trackback spamming. I struck me that perhaps trackbacks combined with blog spammers are always going to be a disaster waiting to happen.

The concept is quite nice, being able to see related articles, when reading a blog, but the trackback concept is a little too trusting, unless you want to go in and moderate trackbacks.

It inspired me however to hack up a smart referer log for my blog entries. (visit the site to find out more..)

View Extended Entry

Alot of people started using blogs as a slightly better media for technical information, but it's becomming evident that with subjects ruby on rails, and some of MS astroturfing with marketing material, that blog aggregators like Artima have been abused heavily with rather second rate blogging, about 3rd rate tools.

While ruby on rails is probably a good tool, it fails in a huge part from the flawed thinking that one provider can deliver a complete solution. It took me a long while to realize that attempting the complete toolkit that ror promises is often fruitless. It rarely delivers much beyhond the intial demostratable examples. What normally happens is that in designing for a single solution (A super fast web interface to databases), you often end up with libraries that are rather poor for generic usage.

The fact that PHP already has 3 or 4 projects that are based on PEAR, that deliver pretty much the same solution as ror. Indicates that the concept of small flexible libraries, maintained by seperate individuals, rather than one super mega project is always more valuable, although I guess you miss out on the hype more.

After quite a few email conversations about HTML_Template_Flexy, I finally got round to documenting one of the javascript libraries I've modified. This is still a bit buggy around the edges, but is pretty much working.

Javascript Calendar started off with a few hacks around the dynCalendar on pear.php.net, and evolved into nice clean simple Calender renderer.

The main design criteria for this was that it should be independant of the HTML page, not requiring any specific javascript, or changing the backend Template code. Just include the javascript file, set the style, and add a few extra attributes to the input elements.

I've been working on a typical web application, and given the fact that deadlines are not a big issue for a change, a little focus has gone into thinking 'is there a better way to do this?'

The challenge is really, how to do client side javascript validation and at the same time add the ability for flexy to pick up the calls and perform the matching serverside validation..

Like alot of my client side javascript, i'm going with a model that basically involves adding these lines to the end of the file:

<script type="text/javascript" src="/js/validate.js" / >
<script type="text/javascript">
//<!--
Validate_onload();
//-->
</script>

The onload method adds a onsubmit hander to all the forms, and which in turn looks for all the input elements and does the required validation.

I've tended to avoid onchange handlers on the form, partly as I feel it is a bit confusing for an end user to be continually be reminded that they have got something wrong, even while they are filling in the form (and perhaps havent finished entering the data)

the onload'er method is also quite nice, as it doesnt involve to many changes to the form.

Now comes the crunch.. where to add the checks for validation, A long while ago I tried something like this:

<input name="age" validate:rule="isempty();number({ min: 8, max: 100});">

Using javascript, it should be easy to extract the rule, split it on the ');' bit, and eval each of the expressions, using the input's value.

On the Flexy side, it does involve a little messing around to convert the tests to pear's Validate class, but the principle is quite nice in the fact you can test the validation just by viewing the template, without actually installing it in the application and running it through the template engine.

Javascript allows me nicely to change the background colour if there is an error (either directy or setting it's style), but again another issue raises its head:
Where do I put the message about what is wrong.

• On the simple version you really want it to stick add a div layer, and put a message under the box.
• On a more complex version, you want it to pull the phrase from a hidden element on the page, and display a translated version if applicable.

The other thing to consider is how to re-integrate the error messaging with stuff coming back after a post (on the extremely unlikely situation these days that someone has turned of javascript )

But at least it did make me think that I should javascript enable the comment boxes on my blog, only making the comment box appear if you are using javascript.. - at least it should stop a few blog spammers.

I have a whistle stop trip over to Frankfurt for my one talk PEAR and PHP5, at the PHP Conference. Sad thing is, that I arrive on monday evening, and fly out wednesday afternoon - so I guess other than jetlag and my talk, I'm not going to get too much done..

Anyway - come and see the talk, if you want to see all the gotcha's for PEAR & PHP5..