Unfortunately my internet line died over the weekend, and I was left without a connection until Tuesday. I had a bit of offline time to have a look at some interesting new(ish) technology.

If you have paid any attention to Gnome development, there are mentions of gnome-shell, and something about making it easier to develop add-on/applets etc. using Javascript. I have not really had time to look at this much, however given the fact I had downloaded all the components prior to my loss of connectivity, I spent some time over the weekend and monday looking at this in more detail.

It looks like there are obviously things going on in IRC and mailing lists, so most of my impressions are from the websites, and code. There are basically 2 projects currently.

GJS - This is the mozilla Javascript engine bound to the gobject introspection system
Seed - This is the webkit Javascript engine bound to the gobject introspection system

Of the two, Seed apears to be a little more advanced and more time has gone into it, based on the fact there appear to be more core features in Seed (eg. I could not find anything like Seed.print() in GJS), and there are very few examples in GJS

Both of these use the GObject introspection method to bind to Gtk, and a large and growing number of other libraries. This is something I found very interesting having gone through a similar process with rooscript.

The fundimental issue of creating bindings to Gtk (and anything that uses the GObject system) is that historically, almost everyone (PHP, Python etc.) who did it used this method.

* Parse the HTML documentation, or the .h files (extracting the classes, enums, methods, arguments etc.)
* Generate Binding code for each library, which wraps these methods, loads the '.so' libraries and links it all together.
* Compile a Module (or statically link it in some cases)
* Run... Test.. Fix bugs in wrapping code... Run ... Test.

What GObject inspection introduces is an very thin layer that can be used to expose any of these methods without writing any binding code (other than to GObject inspection). The result is that to add more features (linking to another library) - you just have to generate a XML file describing the interface, compile it with g-ir-compile and put it in the right folder, and you have magically added support to a new Library, without generating an C code!

This also means that fixing the binding is considerably simpler (fix the XML file, re-compile etc.) and you have solved bugs in any language that is using it... (as I found with gtk_tree_store_set_column_types()

Playing with the languages.

GJS and Seed present the Gtk API very slightly differently, which at present appears to be a bit of a blocker for deciding which to use (or even to bother for some). The key differences are for authoring

* Javascript 'let' features in GJS - not available in Seed.. - This is rather a big blocker as code designed for GJS will fail in Seed totally.. (and visa-versa probably)
* Different call signatures to signals� � �  SEED:OBJECT.signal.connect(method)�  vs�  GJS:OBJECT.connect('signal', method)�  - I think personally that GJS way is more true to the concept and cleaner, however as you can see below there are ways around this.

Seed has a git repo on gnome 'seed-examples', which contains a large number of examples, although their depth is rather thin unfortunatly.. (code coverage is probably 1% at best)...
Both have pretty much Zero in the documentation stakes.. - A rather trivial task to create - which I'm tempted to have a go at....

Anyway back to my playing around..

I've been doing some serious shit in Javascript recently, GUI builders, Code generators, Hacking GTK bindings etc. so I've seen the good, bad and ugly as far as Javascript goes (Bad sometimes being my code).�  I've been slowly moving to what I regard as an effecient, productive use of Javascript, most of which can be seen in the xtype support in RooJS

After runing the first few examples in seed-examples, I decided to see if using an xtype / Roo structure could be done with Seed. After a bit of tweaking, and bugfixing of Seed, I finally got this code to work.

var win = XN.xnew({
    xtype : Gtk.Window,
    type: Gtk.WindowType.TOPLEVEL,
    listeners : {
        'delete-event' : function (widget, event) {
            return false;
        },
        destroy  : function (widget) {
            Gtk.main_quit();
        }
   },
   
   set : {
        set_border_width : [ 10 ],
        resize : [300, 300],
        show_all : []
   },
   items : [
        {
            xtype : Gtk.VBox,
           
            items : [
                {
                    xtype : Gtk.TreeView,
                    
                    
                    
                    onRender : function()
                    {
                        this.column = XN.xnew({
                            xtype : Gtk.TreeViewColumn,
                            items : [
                                {
                                    xtype : Gtk.CellRendererText
                                }
                            ]
                        });
                        
                        this.column.add_attribute(this.column.items[0], "text", 0);
                        this.append_column(this.column);

                        this.set_model( XN.xnew( {
                            
                            xtype : Gtk.TreeStore,
                            
                            onRender : function() {
                                 
                                this.set_column_types ( 1, [GObject.TYPE_INT] );
                                this.iter = new Gtk.TreeIter();
                                this.append(this.iter);
                                
                                this.set_value(this.iter, 0, [GObject.TYPE_INT, 3]);
                                this.append(this.iter);
                                this.set_value(this.iter, 0, [GObject.TYPE_INT, 2]);
                                this.append(this.iter);
                                this.set_value(this.iter, 0, [GObject.TYPE_INT, 9]);
                            }
                        }));
                    }
                     
                }
            
            ]
         }
    ]
    
 });
 

While you may say, so what, it's a bit prettier than the standard examples, but not exactly much different... While that is true, the use of this nested tree, and xtype method is that it enables you to use code and ui builders to put together an interface, and re-use the components very easily. It also solves one of the GJS/Seed issues of signal binding issues (as we wrap up all listeners).

The backend code to make this happen is here.. - It's pretty trivial again, but obvously could be expanded to handle the different binding methods for how different components are combined (add/pack_start/append etc...)

createDelegate = function(method, obj, args, appendArgs){
    
    return function() {
        var callArgs = args || arguments;
        if(appendArgs === true){
            callArgs = Array.prototype.slice.call(arguments, 0);
            callArgs = callArgs.concat(args);
        }else if(typeof appendArgs == "number"){
            callArgs = Array.prototype.slice.call(arguments, 0); // copy arguments first
                var applyArgs = [appendArgs, 0].concat(args); // create method call params
                Array.prototype.splice.apply(callArgs, applyArgs); // splice them in
            }
            return method.apply(obj || window, callArgs);
        };
};


xnew = function(o)
{
    var ret  = o.self ||  new o.xtype(o);
    
    o.items = o.items || [];
    //Seed.print(o.pack.length);
    // packing 
    var addm = typeof(ret['add']) == 'undefined' ?  'pack_start' : 'add';
    ret.items = [];
    for( var i =0; i < o.items.length;i++) {
        ret.items[i] = xnew(o.items[i]);
        ret[ o.items.length == 1 ? addm : 'pack_start'](ret.items[i]);
        ret.items[i].xparent = ret;
    }
    o.set = o.set || {}; 
    for (var i in o.set) {
         
        ret[i].apply(ret, o.set[i]);
    }
     
     
    o.listeners = o.listeners || {};
    for (var i in o.listeners) {
         var _li = createDelegate(o.listeners[i],ret);
         Seed.print(typeof(_li));
         ret.signal[i].connect(_li);
    }
   
    // apply functions..
    for(var i in o) {
        if (typeof(ret[i]) == 'undefined') {
            ret[i] = o[i];
        }
    }
    if (ret.onRender) {
        ret.onRender.call(ret);
    }
    
    return ret;

}

Missing Features

One area that I was quite interested in, unfortuntatly did not look to easy, that being Javascript tokenizing. In rooscript, I hacked in the ability to retrieve the tokens of a Javascript file and expose it to a Javascript interface. This enabled me to write Javascript compressors, and hack jstoolkit to use this. Leaving�  the painfull task of parsing javascript to the compiled part, and not having to deal with a messy parser in Javascript. Looking at Webkit code, It was not immediatly obvious how this could be done however.

For the time being I will leave that component of my Roo Builder application using rooscript.

View Extended Entry

Posted by in Javascript | Add / View Comments()

Well, a peek into the secret world of Roo development. I've been a bit frustrated with the speed of development of the applications recently, so I decided to speed it up a bit..

Below is  a little video of a module I've added to the Iconstruction framework (which might get released one day..) - which enables me to very rapidly build applications.

The knock on effect of developing the builder has been that there is now far better support for xtype object constructors in Roo, along with far more acurate and complete documenation (as the builder tool needs that to work)

View Extended Entry

Posted by in Javascript | Add / View Comments()

Looks like ubuntu released karmic today (as I guess they could not rename it 9.11). 

View Extended Entry

Posted by in Linux | Add / View Comments()

If you have ever used googles translate API, it can quickly become a love hate relationship. You love the features it provides, but you begin to hate that fact that google's server are slow and flaky for loading the libraries that they recommend.

This situation was getting especially annoying yesterday, as the load time of my application (that is getting continually re-loaded while I'm working on it) was getting worse and worse, and google's API's was the culprit.

So after some further reading on that page, I realized that the translation call was really just a simple HTTP request with the correct parameters.. no need for huge google framework API etc.

So here's the ~20 line javascript to replace the slow loading 50k+ library that google recommends....

/**
 * usage 
 * gtranslate('hello', 'en', 'es', function (res) {
 *     if (typeof(res) == 'object') { return; } // failure
 *     console.log(res); // success...
 * });

*/

function gtranslate(str, src, dest, cb) 
{
        var x = new Roo.data.ScriptTagProxy({ 
            url:  'http://ajax.googleapis.com/ajax/services/language/translate', 
            callbackParam : 'callback' 
        });
        x.load(
            {
                v: '1.0',
                q : str,
                langpair : src + '|' +dest,
            }, // end params.
            { // reader
                readRecords : function (o) {
                    if (!o.responseData) {
                        return o;
                    }
                    return o.responseData.translatedText;
                }
            }, 
            function (result) {
                cb(result);
            },
            this,
            []
        );
            
        
    } 
    

View Extended Entry

Posted by in Javascript | Add / View Comments()

Just before I saw Brendan's post about PHP4 compatibility in PEAR, I had been getting a few queries about making a couple of my PEAR packages more 'PHP5' compatible or PEAR2 ready.

From my perspective, pretty much all of the packages I maintain (As far as I know) are PHP5 'compatible'. however they may emit E_STRICT errors.

This brings up the interesting question, which I guess all the current maintainers, users and contributors have come across, how much value is added to the existing packages by adding that level of support.

From an 'ideal world' / 'perfect coding' perspective, they would benefit from this changes. but as somebody who earns an income by delivering projects as quickly and efficiently as possible, the return on investment for making those changes is very tiny, if not negative.

Since the packages generally just work, making the changes required, would not really change that 'just work' situation, and as  Jamie Zawinski famously said "How will this software get him laid?"

Two of the biggest changes I'm aware of for this 'PHP5 compatibility' issue are the 'static' method prefix and getting rid of 'var' which completely break PHP4 compatibility (and yes we still maintain PHP4 applications, and clients rarely have budget to make changes like this). Doing these changes would mean that I would have to either freeze or depreciate PHP4 support, or start maintaining dual versions. (Personally I would prefer a hook in the package builder that would do the replacement for me, so I could upload 2 package on each release).

Going forward, PEAR2 is still in a gestation period, (as PHP5.3 and namespaces support has just come out.) Resulting in any code that had  targeted PHP5.3/PEAR2 aging very quickly (eg. requiring changes to handle the final changes to the namespace syntax.). This may start changing soon, however I suspect it would really take some significant effort in time to start creating PEAR2 packages for existing code (which has a rather poor return on investment) . And without a reasonable base number of packages, the attraction of submitting code to PEAR2 is lessened. A classic chicken and egg situation.

At the same time, there is no real alternative to PEAR2, pretty much all other 'framework' solutions have been built around the assumption that you have to accept a majority of the 'framework' to utilize the single packages. Which is even worse that the pains that PEAR(1) imposes on you.

All that said, if you want to send me patches to fix any big PHP5 issues in my packages please don't hesitate, I will try and make the changes.

View Extended Entry

Posted by in PEAR | Add / View Comments()

Went along to Barcamp in Hong Kong this weekend, Have to thank all the guys & girls involved in organizing it. As it was a chance to meet some interesting people, who I did not realize where even in Hong Kong.

Barcamp Hong Kong, is perhaps a little different from what you might get elsewhere. First off it's not so much a techy thing. (Evidenced by the sight of 1 Linux desktop = mine...) I gave one talk on 'D', and went to a couple of others with lesser technical emphasis - drupal / OO JS etc. but the majority of talks where either business development style (raising funds, startups). or iPhone related (with out the coding stuff).

I guess this comes from the general attitude here that most programmers tend to regard it as a stepping stone to management or something, and have very little passion about the whole industry.. (as someone described it to me).

The technical talks gathered at most 20 people (out of I guess 200-300 people at the camp). And unfortunately, my talk/chat/discussion got rather hijacked by someone who wandered in, was completely non-technical, and started asking irrelivant questions. (self reminder, politely say, this is for programmers only next time)

Overall it was not much of a learning experience, rather the chance to actually meet developers with similar interests, (and see that they actually exist) - Shame I had to leave early for family stuff, as the drinks would have been good...

View Extended Entry

Posted by in Hong Kong | Add / View Comments()

Storing tree data in Mysql databases, is relatively common, however, all the existing documentation about doing this, makes the whole process rather complex.

If you google it, you will probably find the quite definitive answer at mysql.com, describing the classic  parent_id method, and the left/right numbering process. Both of these methods involve rather complex SQL to fetch and update the tree.

In seeking a better solutions for a tree that was infrequently updated, but frequently queried, I thought I'd try seeing if I could write a few stored procedures to simplify the process.

Our basic database structure looks like this:

CREATE TABLE _TREE_ (
    id int(11) NOT NULL auto_increment,
    parent_id   int(11)  NOT NULL DEFAULT 0,
    seqid int(11)  NOT NULL DEFAULT 0,
    depth int(11)  NOT NULL DEFAULT 0,
    leaf int(1)  NOT NULL DEFAULT 0,
    name varchar(128) default '',
    fullpath TEXT default '',
    PRIMARY KEY  (`id`),
    INDEX qlookup( parent_id , seqid , depth)
);

our key components are

• id (the nodes id)
  
• parent_id (the nodes parent - pretty clasic)
• name (the textual name of the node)
• seqid - the generated order item for the whole tree
• depth - how deep the node is (usefull for indenting)
• leaf - is it a leaf node (eg. has no children) - usefull for icons

The trick of the stored procedures is to correctly generate the seqid, this consists of two parts, the top level wrapper and the recursive prodedure to update the table.

DROP PROCEDURE IF EXISTS _TREE__resequence;

DELIMITER $$
CREATE PROCEDURE _TREE__resequence(i_sep VARCHAR(4))   DETERMINISTIC
  BEGIN
        DECLARE v_p, v_d, v_s INT(11);
        DECLARE v_fp TEXT;
        SET v_fp = '';
        SET v_p =0;
        SET v_d =0;
        SET v_s =0;
        SET max_sp_recursion_depth=255;
        CALL _TREE__resequence_sub(v_p, v_d, v_fp, i_sep, v_s);
        
        
    END $$
DELIMITER ;

   



DROP PROCEDURE IF EXISTS _TREE__resequence_sub;

DELIMITER $$
CREATE PROCEDURE _TREE__resequence_sub(
    i_parent INT(11),
    i_depth INT(11),
    i_fullpath TEXT,
    i_sep VARCHAR(4),
    INOUT i_seqid INT(11)
    )   DETERMINISTIC
    BEGIN
        DECLARE v_nid, v_ex_seqid  INT(11);
        DECLARE v_name VARCHAR(128);
        DECLARE v_leaf INT(1);
        DECLARE v_fullpath TEXT;
        DECLARE done INT DEFAULT 0;

        DECLARE qry CURSOR FOR SELECT id, seqid, name  FROM _TREE_ 
              WHERE parent_id = i_parent ORDER BY seqid;
        DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
        OPEN qry;
        
        REPEAT
            FETCH qry INTO v_nid, v_ex_seqid, v_name;
            
            IF NOT done THEN
                IF v_ex_seqid != i_seqid THEN 
                    UPDATE _TREE_ SET seqid = i_seqid, depth=i_depth WHERE id=v_nid;
                END IF;
                IF i_depth > 0 THEN 
                    SET v_fullpath = CONCAT(i_fullpath, i_sep, v_name);
                ELSE 
                    SET v_fullpath = v_name;
                END IF;
                SET v_leaf =0;
                SELECT COUNT(id) INTO v_leaf FROM _TREE_ where parent_id = v_nid;
                
                UPDATE _TREE_ SET 
                    fullpath = v_fullpath,
                    leaf = IF (v_leaf > 0, 0 , 1)
                    WHERE id=v_nid;
                
                SET i_seqid = i_seqid +1;
                #// do the children..
                CALL _TREE__resequence_sub(v_nid, i_depth+1, v_fullpath, i_sep, i_seqid);
            END IF;
        UNTIL done END REPEAT;
        
        CLOSE qry;
    
    END $$
DELIMITER ;

This does the hard work of iterating through the tree, and updating the sequence number, depth, leaf field and filling in the fullpath field

Now our simple add node code just adds the node in the correct place, bumps the seqid along, so that you can then regenerate the tree.

DROP FUNCTION IF EXISTS _TREE__add_node;

DELIMITER $$
CREATE FUNCTION _TREE__add_node(
    i_parent INT(11),
    i_after INT(11),
    i_name VARCHAR(128)
    ) RETURNS INT(11) DETERMINISTIC
    BEGIN
        
        DECLARE v_depth INT(11);
        DECLARE v_seqid INT(11);
        DECLARE v_ret INT(11);
        DECLARE v_tmp INT(11);
         
        SET v_depth = 0;
        SET v_seqid = 0;
        SET v_ret = 0;
        SET v_tmp = 0;
         
        
        #// grap parent depth.
        SELECT depth +1, seqid + 1 INTO v_depth, v_seqid FROM _TREE_ WHERE  id= i_parent LIMIT 1;
        #// grab previous id..
        IF i_after > 0 THEN 
            SELECT seqid INTO v_tmp FROM _TREE_ WHERE id = i_after AND parent_id = i_parent LIMIT 1;
            IF v_tmp > -1 THEN 
                SET v_seqid = v_tmp+1;
            END IF;
        END IF;
            
        
        
        INSERT INTO _TREE_ SET 
            depth = v_depth , 
            seqid = v_seqid , 
            parent_id =  i_parent,
            name = i_name;
        
        
        SELECT LAST_INSERT_ID() INTO v_ret;
        
        #// fix the seqend id 
        
        
        UPDATE   _TREE_ SET seqid = seqid +1   WHERE seqid >= v_seqid  AND id != v_ret;
        
        RETURN v_ret;
        
    END $$
DELIMITER ;

You can now create a generic tree code by replacing my _TREE_ word with your table name:
eg.

#sed -e "s/_TREE_/yourtable/g" tree.js | mysql yourdb -f 

Once all that is done the SQL to create tree nodes is very simple:

 delete from nametree; alter table nametree AUTO_INCREMENT =1 ;
 
 select nametree_add_node(0,0, 'one at top'); 
 call nametree_resequence(':');

 select nametree_add_node(1,0, 'two on one'); 
 call nametree_resequence(':');

 select nametree_add_node(1,0, 'three on one (first)'); 
 call nametree_resequence(':');
 
 select nametree_add_node(1,2, 'four on one (last)'); 
 call nametree_resequence(':');
 
 select nametree_add_node(0,1, 'five on top (last)');
 call nametree_resequence(':');
 
 select nametree_add_node(0,0, 'six is first'); 
 call nametree_resequence(':');

 SELECT * FROM nametree ORDER BY seqid;

Gives you a nice little table

+----+-----------+-------+-------+------+----------------------+---------------------------------+
| id | parent_id | seqid | depth | leaf | name                 | fullpath                        |
+----+-----------+-------+-------+------+----------------------+---------------------------------+
|  6 |         0 |     0 |     0 |    1 | six is first         | six is first                    | 
|  1 |         0 |     1 |     0 |    0 | one at top           | one at top                      | 
|  3 |         1 |     2 |     1 |    1 | three on one (first) | one at top:three on one (first) | 
|  2 |         1 |     3 |     1 |    1 | two on one           | one at top:two on one           | 
|  4 |         1 |     4 |     1 |    1 | four on one (last)   | one at top:four on one (last)   | 
|  5 |         0 |     5 |     0 |    1 | five on top (last)   | five on top (last)              | 
+----+-----------+-------+-------+------+----------------------+---------------------------------+

Enjoy tree.js (I name my mysql stored procs as js, just so my editor works better with them)

View Extended Entry

Posted by in General Crap | Add / View Comments()

After months of being left on the side burner, I just got round to fixing Digital TV with the ASUS My Cinema and MythTV.

Having finally got the correct source code from ASUS a few months ago, I've gone through it and made it thread safe (get rid of the global locks). So you can use multiple sticks on a single PC. and got rid of most of the shutdown kernel panics.

The final fixes where related to the really bad locking code, that basically was failing too quickly and constantly changing the GI settings. The kludge I've done is to remove the flipping (as all of the HK channels are on GI_945). and this card only really works in HK & China.

This resolve the last issue of waiting for a lock to a signal taking over a minute to milliseconds. - Finally making the card usable in MythTv (as it doesnt really like really slow locking cards..)

Anyway, code is in my repo - I need to start thinking about syncing with v4l-dvb guys and finding out how to rename the files so they are not just overwriting the existing dib3000mc driver.

For your pleasure

ASUS My Cinema U3100 Mini DMB-TH Linux Driver for v4l-dvb

If you need frequency details have a look at my last post about the ASUS stick.

View Extended Entry

Posted by in Linux | Add / View Comments()

Well, after a busy few months, things have gone quiet again. Hopefully it's a short term thing, but it has given me a bit of time to do get back to the less profitable things in life (like blogging).

PEAR releases

Yes, after a quite a few emails bugging me to release updates to DataObjects and Template Flexy, I finally got round to getting them out the door  (and even fixing a few bugs after the got out). I've also made an effort to get Services_JSON onto the pear release system, as I use it quite a bit, and it's been sitting in as the proposal for Services_JSON for well over a year.

Facebook

Yes, that other great waster of time, I've finally set up an account for me - "AK BK Consulting" - After my first effort of using facebook fell apart, due to the mess it made of mixing family stuff and professional stuff, (I use my wife's account for my family stuff). I set up that one so I can join up with PHP developers anywhere and IT people in Hong Kong. So feel free to add me as a friend (as I dont have many ;) - this blog should be  syndicated into my page...

ExtJS / Roo builder for HTML_FlexyFramework

I was messing around this week writing a builder for HTML_FlexyFramework, my little lightweight page loader, that integrates quite nicely with Dataobjects, Template Flexy and pear in general. Part of the incentive for this was seeing a little project that a potential client had developed, in some windows application that generated an how site starting with the database schema.

The idea  was quite nice, and the interface of the builder was quite friendly. But the downside was that the resulting code was just unusable jibberish. So rather than work out how to add features to it, I wondered if using DataObject's Generator as a core, I could generate a whole ExtJS/RooJS interface from the database, and then edit that to quickly get a frontend up and running.

The code's sitting in my akpear repo RooJS_DB_DataObject (it actually writes ExtJS1.1 code) and does the basic tasks of setting up a grid for each table, a simple form to edit each table, along with some limited joins based on the links.ini

If you want to try it out, it runs like this:

php  RooJS_DB_DataObject/cli.php 'mysql://dbuser:dbpass@dbhost/dbname'

Nice little proof of concept.. It's got some idea how to 'update' it's generated code, but I've disabled that at present. It should however give you a quick way to jumpstart an ExtJS application.

View Extended Entry

Posted by in PEAR | Add / View Comments()

As my thumbs are twiddling waiting to see if projects start coming in next year, I thought I'd do a roundup of this years projects (open and closed). I was pondering if everyone else was seeing the effects of the recession, so I'd be interested to hear if you've also seen work slacking off.

Let's hope next year we see some more life from the Sales and Marketing, as my current reliance on word of mouth, may not keep feeding my mouth at this rate.

Anyway Merry Christmas and Happy new year to all.

Read on for my Year of Projects

View Extended Entry

Posted by in General Crap | Add / View Comments()