PHP's great claim to fame has always been a solution to a domain specific problem, Delivering Web pages. It's relatively easy mixing of HTML and code (while at times very nasty), have proved quite a good solution for many people.

But as technolgy moves forward, I wonder if PHP will be required much longer. Why you may ask, since I've got craploads of PHP code lying around in my subversion folders. Well In addition to most of my new projects using ExtJS to deliver the interfaces, which has almost totally removed PHP from rendering tasks. The last few weeks I have been using Mysql's newish Stored procedures, which move the alot of application logic and rules into the database, and seriously reduce rather kludgy code in PHP. Not only that it also reduces the data going back and forth between PHP and Mysql.

So PHP is doing less and less of the work, to the point where if there was an apache module that did mysql stored procedure calls based on the request URL, and returned JSON, I suspect PHP would be practically obsolite.....

Anyway started playing with apache modules today... - not enough time to do anything substantial, but there are a few fixes needed to this page.
http://www.ibm.com/developerworks/linux/library/l-apache/
The build lines should be something like this.

gcc -fPIC  `apxs2 -q CFLAGS ` -I`apxs2 -q INCLUDEDIR` 
   -c time_cookie.c -o time_cookie.o

gcc -shared  -lapr-1 -laprutil-1 time_cookie.o -o time_cookie.so

And the FOREACH loop in it should use

curr_bucket = APR_BRIGADE_FIRST(bb);
while (curr_bucket != APR_BRIGADE_SENTINEL(bb)) {
    ...
   curr_bucket = APR_BUCKET_NEXT(curr_bucket);
}

Anyway I wonder if anyone has done a Mysql/Json apache module.

View Extended Entry

Posted by in PHP | Add / View Comments()

Most of the fun development projects had to be put on hold over the last few months, due to good ole paying projects.

ExtJS security holes

The majority of my paid projects are now using ExtJS, (and one has now used ExtJS 2.0, which is a definite improvement in both features and design). But I did run into what I considered to be a rather serious problem. ExtJS by design is a XSS nightmare... so be careful.

Basically the grid component and a few of the other components are designed to use JSON data comming from the server and render the data it recieves onto a styled HTML table. The problem I discovered, was that rather than using DOM.createTextElement() or similar to render the data it recieves. It uses innerHTML. So data is not escaped in any way prior to rendering onto the page.

It was discovered while testing one of my first major products, A complete Spam/Mail control solution, I discovered that it was not rendering email addresses <someaddress@example.com> , due to the <> characters. I dug into the code and realized the XSS hole, and posted it as a bug on the extjs forum.

Unfortunately it appears that rather than regarding this as a huge hole, they would rather trust developers to read the fine print in the documentation about rendering data to the screen. Personally I would be embarrassed by this attitude on my project.. 'security is someone elses problem'.. but I guess that's their problem... - I have to keep security patches for extjs handy....

GtkDjs updates

I finally got a short time yesterday to update the code in Gtkjs to use the latest dmdscript source, overlayed with my JS2 stuff and language binding add-ons. It builds and runs all the test code in the test folder, and Walter fixed the closure issues alot cleaner than I had...

I've also updated the 'downloadFiles.sh' script so that It fetches the correct language HTML files for the generator. It appears that gnome has moved all the documentation around. (It also means the generated code is more in sync with newer releases of Gtk and the many extensions.)

Mailfort teaser..

As I mentioned above, I'm working heavily on a new product, Mailfort, anyone sending email to me, who I dont know, is likely to get a Identify confirmation email. We are currently testing with friends and family, along with a couple of companies, and hopefully more details will appear soon. But the teaser is that I've been working on Spam and mail control solutions for myself and a few clients, which has finally evolved in something generic enough that we can begin to sell. A number of my friends have been complaining about their Spam problems, so they are now slowly being introduced to the beta test product (with very positive feedback) so hopefully we will get the full website and product rolled out soon.

View Extended Entry

Posted by in Javascript | Add / View Comments()

If you know anyone in Hong Kong (or who wants to move here and can get a visa) with PHP experience, and some linux experience (although not essential) , who is looking for full time job, feel free to contact me alan at qsolutions dot hk dot com.

This is a permanent job, working with me in a small company based in Central. The projects are varied and if you can program in D, C or any other language there are even more opportunities.

View Extended Entry

Posted by in Hong Kong | Add / View Comments()

Daily builds / downloads available

I've set up a cron job now, so all the bindings and binaries are built daily, you can download them from here.
http://devel.akbkhome.com/gtkjs/gtkjs_snapshot.tar.bz2
This includes gtkjs, mjs and jsfastcgi, In theory it would not take too much to build this on Windows... - Send me the fixes if you  attempt it...

Just download, unzip and try out

./gtkjs test/filetree.ds

etc.

the gtkdjs Documentation is updated daily to match

And I still need a better name for the project;)

Closure and scoping fixes.

After seeing a post on the DMDScript newsgroup concerning closures and scoping issues, I spent quite a while looking at the issue.

The basic problem, was that the original DMDscript code was designed to create a cache of function definitions, and re-use them whenever they where refered to. Along with this, they had no concept of creationScope. so variables in the creating scope where not available. Fixing this involved quite a few changes to the compiler and runtime. Basically creating new Function instances when they are found, rather than only once at compile time. Along with storing the scope inside the Function instance. so when they are called they understand the correct scope. Anyway standard Javascript scoping, and closures all work as expected.

Undefined Warnings improved

I mentioned in my last post that I added Warnings when undefined variables where accessed, I've modified this slightly as undefined is a valid type in Javascript, so unassigned arguments to a function call are flagged as undefined, but accessing them should not issue a warning. So only when the scope.Get(variablename) calls return null, rather than a Value.vtype[V_UNDEFINED] flag this warning now. 

LibSoup extension

LibSoup provides HTTP support, at present, basic sync requests work, the code is there for async calls, but I've not tested it yet. test code is in tests/Soup/.

This little bit of code get's my web site, and shows headers, then extracts all the href from all the <a> tags.

var sess = new Soup.SessionSync();

var msg = new Soup.Message("GET", "http://www.akbkhome.com/");
status = sess.sendMessage(msg);

println("Status is " + status);
println("ResponseHeaders is " + msg.responseHeaders.toSource());


var imp = new Dom.Implementation;
var doc = imp.createHtmlDocFromMemory(msg.response);
var a_s = doc.getElementsByTagName("a");
for(var i =0; i < a_s.length; i++) {
    println("got href: " + a_s.item(i).getAttribute("href"));
}

XML2 limited support

To enable parsing of html documents for Gdome, I've added limited support for XML2, it's mostly used by the Gdome method:  Dom.Implementation.prototype.createHtmlDocFromMemory( String htmltext )
The Generator code is currently tied into the libxml header files on my system.. this needs fixing so it uses the gtkwrap download script. (but the code is a  nice example of how to automatically create bindings based on .h files..)

Gtk.SourceView support

In preperation for testing text editing (and eventually self-editing / morphing of applications), I've bound the GtkSourceView widget. Unlike the standard Leds editor which uses scintilla, I thought I'd try and see if using GtkSourceView would prove more robust. - The bindings had already been done for GtkD, so there was not much to change to the APILookupSourceView.txt code. I only used the full name "sourceview" rather than "gsv"

Handling of missing functions and Libraries improved

When I was testing GtkToolbar on one of my machines which had an old Gtk library, I discovered a segfault when it called a newer function that did not exist in the old library. To fix this, the Loader.d file now binds a simple function that throws and exception to any method it can not bind. So you can catch in javascript any call to unsupported methods in a library.

Phobos Path module added

all the methods from std.path are now exported to javascript, and in the manual. This enables a few usefull functions like

var file = "/etc/passwd"
println("Directory = " + Path.getDirName(file));
println("Filename  = " + Path.getBaseName(file));
println("combined filename  = " + Path.join("etc","passwd"));

Regex Fixes

In testing extjs parsing, I came across two bugs in D's regex implementation, that prevented it's loading.

• forgetful parsing (?:abc|cde|xyz) - the fix for this was trivial and is in D's bugtracker, and there is a modified regex file in the dmdscript distribution.
• the use of special chars in character range matching after the '-'. eg.  [\w-\*] and [\w-\.] which should be interpreted as [\w\*-] and [\w\.-].  These currently parse correctly however I've not got round to fixing the implementation properly so they are pretty broken at present. (basically matching everything from \w onwards...) - the bugs in D's bugtraker, but the fix is not... I may get back to this...

Simplified exporting of Structures

One of the missing pieces for the Gtk Bindings, that I came accross with jLeds, was the event structures had not been bound. Since they are very simple structs, and you only need to get access to their properties, I developed a new action for the wrap file "structDump" which autogenerates the class file, forces the full Struct definition to be written and writes getter methods for all the exportable properties.

Bindings builder flexibility

Since we now have about 20 libraries bound, the Wrapping builder script can take a little while on slow systems to build all the bindings code. So I've added the ability to specify which APILookup files to action.
eg. to build LibSoup you can do.

sh buildit.sh APILookupGLib.txt APILookupGObject.txt APILookupSoup.txt 

It cant do the dependancy resolution yet, so you have to list all the pre-requisite packages so it has access to all the required types when building the library you need.

The struct parser is also a  little more robust now, handling comments and lists of elements eg.
ushort x, y;

Minor Gtk Fixes

• G.Object.setProperty(String key, Any value) now works, and converts value into G.Values automatically.
• GtkTreePath[] Gtk.TreeSelection.prototype.getSelectedRows(Gtk.TreeModel model)  now works - although I need to look at generic ways to generate array of Objects from GList..
• new Gtk.TreeStore(Array coltypes) now works (where coltypes is either an array of strings, or G.Types.) - this works along with the other constructor methods - eg. varargs......
• Gtk.TreeStore.prototype.set(Gtk.TreeIter row, Array values) now works to quickly set the values of a tree row, rather than calling set(GtkTreeIter row, Number col, Mixed value)
• Gtk.Container.prototype.remove(Number item) and Gtk.Container.prototype.remove(Gtk.Widget widget) are now working - I need to change this slightly, so you can remove all by sending '0' as the item, and waiting until it returns false...
• Most of the  GtkEvent Structures are now exported, and all the properties are available as getters.
• new GdkColor(Number color)  is based on 0xRRGGBB hexidecimal values, rather than longintegers.. (you can use the long version to do more precise colours: new GdkColor(Number red, Number green , Number blue);
• new Gdk.Pixbuf(Array xpmdata) is now supported so converting xpm data for use as icons is very simple (just replace "const char**" with "var")

ExtJs loading

One of the goals is to make the engine compatible with existing code out there, a nice test of this was extjs, which it can now load correctly (with a few lines of javascript prefixing the loading). I've not actually tested any of the functionality yet, as XmlHttpRequest needs building from the libsoup bindings.

jLeds port underway..

What's a language without an editor (or an email client - if you know the old joke about every application eventually evolves until it can read email.).. So partly to test the bindings I've started looking at porting Leds to gtkjs - the code is in tests/jLeds, and gives a good example of some of the new Javascript2 features, along with the gtk bindings.


Try the newsgroup if you need support.
news://news.digitalmars.com/DMDScript

View Extended Entry

Posted by in Gtk | Add / View Comments()

Add one feature, and create a few bugs.. seems like it's always the way..

The two features added over the weekend where class syntax support and include support. My initial effort proved to be a little off the mark.

Read on if you want to know the nitty gritty details about writing an interperted language runtime... Along with a list of the Gtk binding improvements..

View Extended Entry

Posted by in Gtk | Add / View Comments()

A good friend of mine asked what's the difference between the dmdscript/fastcgijs/gtkjs stuff I'm doing and Adobe AIR.

I've not really looked at Adobe AIR much, noticed that it was mentioned on the extjs site, so I thought I'd do a quick "what's the difference"..

So what's the key differences

For Adobe AIR

• Backed by a huge company with unknown reasons for hooking you in.
• Lot's of support for Adobe technology, but any other libraries, you can forget about.
• Probably written in C++ or C, so there are probably a few nice security holes difficult to find in the memory allocation. Even if you did have access to the source - it would be a pretty complex task to write bindings for libraries.
  
• Great support for Windows, pretty good for Mac, and bugger all for other un*x's

For GtkJs / Fastcgijs / dmdscript

• Absolutly no support ;) - obviously only done, because it's interesting and fun!
  
• Lots of support for any open source library, as long as you are willing to waste a few days working out how mindblowingly simple it is to write bindings.
• Written in D, which makes it pretty secure (well if you fix some of the bugs in my generator to do a small amount of sanity checks). And due to the fact that it's so similar in syntax to Javascript that makes the whole process of moving slow parts into compiled code considerable simpler than any other scripted language..
• Great support for linux, and it might just work on any other platform, as long as you can find someone who knows those platforms.
  

Personally I think the Adobe AIR idea is pretty good, Javascript is a good basis for a language. But to be honest for the reasons above, I would not touch it with a barge pole..

View Extended Entry

Posted by in Gtk | Add / View Comments()

Last week I had a go using my little bindings for something useful, rather than just being a little toy. One of the projects I was working on was a hack to exim4, that logged all incomming and outgoing attachments into mysql and stored them on the filesystem.

The basic idea was that the company needed the ability to review if any sensitive data was being sent out by email. The exim hacks are pretty simple for this (hook into the mime handler). The web viewer uses extjs and PHP at the backend, it's a small addition to their current email management interface.

But one area that was needed was a little cron job that cleaned up the directories and deleted data and files older than 2 weeks (so they dont get too clogged up.)

I thought this would be an interesting test of the mysql and file system bindings. So I threw together a horrifically simplistic version of DataObjects, and a short script that did an SQL query and based on the results deleted records and files.

One issue was that I could not really do the DataObjects thing of overlaying the results, or query options into the dataobject as object variables in javascript are overlaid into the same namespaces as methods, so you could accidentally delete methods if your database had matching names. - So the data just gets put in (dataobject).data.* 

The other issue was that the syntax for javascript while fun and flexible could really do with a class construct. If you look through the specifications for ECMAScript4, There is a huge amount of changes making Javascript as we know it far more Java / PHP5'y.. with interfaces, public private etc. So I had a go at adding some of the new features into the dmdscript core.

I've managed to get class's and include working. although it probably needs quite a bit more testing. This basic syntax now works

include "mytest.ds"; // this is run at compile time (not runtime!)

class A {
        function A() {
                println("This is the constructor A");

        }
        var c = 12;
        function B() {
                println("This is B and c is " + this.c);
        }


}
class C extends A 
{
        function C() {
                println("This is the constructor C");
                A.prototype.constructor.call(this);
        }
        function D() {
                println("This is D and c is " + this.c);
        }
}
 
 
var a = new A();
a.B();

var c = new C();
c.B();
c.D();

Will successfully output

Hello from mytest
This is the constructor A
This is B and c is 12
This is the constructor C
This is the constructor A
This is B and c is 12
This is D and c is 12

Now I can start writing tidy little javascript libraries...

View Extended Entry

Posted by in Gtk | Add / View Comments()

There's been a long thread the last couple of weeks covering require_once and some rather crazy ideas to change the PEAR standards to do all sorts of strange things to load code up.

What is most ridiculus is the benchmarking that is being done to 'prove' that we should all move our code loaders to some 'allfiles.php' and we are magicially going to running be 15% faster.

What this whole concepts fails to put into place is loading all the PHP files up, either tiered one to another or all from one place is such a small part of a PHP page responding from a request.

Think of what is happening when you look at this page.

• You start by looking at a bootstrapper (that sets config option) - probably quite quick as it just set's some variables.
• Then you run into the Framework class - that works out which file to load based on the URL you are typing. - This does quite a bit of text maniplation, making best guesses about where things might be. (this is quite generic and could be made specific to the application quite easily)
  
• We then do the page action stuff, like pulling data from databases which requires quite a few PEAR libraries. And does some really slow crap like connecting to databases and pulling data. This stage probably does far to many queries and pulls down to much data that it doesnt use.
• Then we do output stuff, Normally we are using a compiled template, (so we dont load all the template parsing code - well at least we save a bit of effort here). so we need to pull down at least the Flexy class, then the compiled template.

Now looking at that whole process and thinking it's a bit slow, You would probably go through the following steps to optimize it.

• replace it with a staticly generated page!!!

Well, yeah, that's it!....

Only kidding... - but if you do this to a normal page, you would probably want to do the following

• reduce / remove database calls
• cache as much as possible.
  
• reduce / remove libraries needed and code used.
• move any data intensive code into C
• use an opcode cacher.
  

At no point would you get so silly that you would re-jig the 'require_once' calls just to get 15% saving on the code loading component, because wait for it..... the code loading component of your application (when using APC) would take such an insignificant percentage of the total running time.

Working out how to remove a whole file or pre-process some bit of data would be a far better use of your time. And if you where running a team focusing on this you would probably have fired the guy who wasted all that time on such a pointless task as moving the require_once lines around.

Sounds like moving the deck chairs on the titanic?

View Extended Entry

Posted by in PEAR | Add / View Comments()

I've been slowly ticking away adding features to the gtk javascript bindings, in doing so, I'm adding extra libraries - Mysql works quite well, and Seeing the recent post on planet.dprogramming.com about fastcgi4d, I wondered if I could run dmdscript from fastcgi.

To enable a web version of dmdscript with my bindings, I had to do quite a bit of re-organization, enabling it to be built with random libraries added in and exclude the Gtk stuff. Core to this was to move all the registration code into the directories that hold the binding code. Each binding directory now contains 3 files, binding***.d, binding***type.d and register***.d. Which manage the dynamic loading of .so (or .dll if windows actually works). and the registering of all the javascript objects and methods.

For fastcgi, I had a look at the code described on the blog post, but was very reluctant to use it, as it

• required tango, which is not a current requirement, and I'm a little concerned about using.
• required knowlege of templates, which are still a bit of a black art. and I've only used very sparingly when absolutly every other alternative has been ruled out.

So after considerable digging through the code and trying to make head or tail of the code on fastcgi.com, I finally found the python fastcgi code, which ported easily to D.

The core files libraries that are helpers in dealing with fastcgi are here.
http://www.akbkhome.com/svn/gtkDS/src/fastcgi/

They need the loader.d and paths.d file from here
http://www.akbkhome.com/svn/gtkDS/wrap/
to be usable.

and the current, non-threaded simple responder is here
http://www.akbkhome.com/svn/gtkDS/src/fcgi.d

So for hello world example, this simple piece of code dumps all the server variables and says hello world.

println(Request.toSource());

println("hellow world from javascript");

Next job is to look at how GET/POST data is passed around and how to escape data on output. among a list of 100 ideas for how it could all work.

View Extended Entry

Posted by in Gtk | Add / View Comments()

View Extended Entry

Posted by in Gtk | Add / View Comments()