27 May 2008
Quick ExtJs Fork update
On the global plan, this is what I've been looking at
- I have found a copy of yui-ext 0.33, which was released under a BSD style licence, and includes all the CSS and Images. I've not got round to merging and testing it with the 1.1.1 code, but I can't see any major issues, as from my recollection 1.0 evolved from this.
- I got about halfway through adding all the javascript files (v1.1.1) to my subversion repo, along with modifying the headers to remove the licence link reference which is now irrelivant. The code can sit there until a shared repo becomes available.
- Made a dependancy list for the files (so that building specialized bundles should be easier)
- started testing a few javascript bundling tools (that work better than my simple tokenizer tool)
- Started looking at docs
So I started looking at jsdoc toolkit (on google code) - It's dependancies are a bit annoying (java + mozilla js engine AFAIR) - but the thing is written in javascript - (very well designed) and looks like a better bet.
My idea was to take this code, and use my dmdscript fork to run it and generate documentation without that java dependancy. My little dmdscript fork has a few benefits that make it very suited to this task - Built in File access, a built in tokenizer to start. And it's piss easy to extend.
However it turned out to be a bit more complex than that. One of the major problems turned out to be that dmdscript uses digital mars's D's regex engine, which unfortunatly is seriously feature flawed. To the point that most regex's that you give it fall over badly.
I did manage to hack in forgetfull matching (?:.....) but when I started finding quite a few more oddities, ([\s\S]+\s)? the whole task of trying to fix the code became to much, and I came to the conclusion that just ripping it out would be simpler. In it's place I've used the excellent pcre library by Phil Hazel (who also wrote one of the best MTA's out there.)
The binding code for the pcre library was already available in dsource, hidden away in the dwin project, so after understanding the API for that, I've almost got to the point where a semi modified version of jsdoctookit can parse the Code. I still need to do some more fixes to the regex engein so it can render the templates. -But it's getting there.
11 May 2008
ExtJs - talking forking
I've started looking at forking ExtJS, after some considerable thought, I'm pretty close to the concluding that forking ExtJS is really option left, to retain the investment I've already made in it..
So, as I have a Zip file of 1.1.1 and it explicitly says that the Javascript code is Licensed under LGPL, (not ifs/ no but's) - I'm slowly putting the code into my subversion repo under www.akkbkhome.com/svn/extjs1 (*I'm looking at extjs1.1.1 as I have been using it and tend to prefer it, but there is nothing in the discusion below that precludes anyone helping/leadingwith the last extjs2 version that was released)
Now if this actually becomes a full fork (most forks fail BTW), It will need a bit of work, so If anyone is interested in helping out. I've no idea where this could go. But this fork ain't changing the license no-more...
The plan
I think there's quite a few things to do here... so Ideas or contributions.. -- feel free to email me, or just comment on this post. - At worst, it could form the brainstorming for anyone else actually doing this. (If you want to comment on if/should this be done - do it on my previous post, otherwise I will delete the comment)
The Code
- Tidy up Ext Js's source - see if it can be stored in one file per class, and have a simple classname->filename mapping... (eg. Ext.Dom in Ext/Dom.js) ** started..
- Sort out the CSS
- If Jack ever released any old CSS/images under a open licence (ext-yui source?) see if that can be used.?
- or Go through the classes and see what their requirements are for CSS
- Probably in batches - write a short document listing the required classes needed by the batch, enabling someone to contribute a CSS file that works.
- Posibly create enough CSS to make enough of it useable (probably with no images)
- Document image requirements (see above), then see if other open projects already have images that can be used? - otherwise see if someone want's to contribute themes..
- Break up Source into managable packages - Distribute ownership! - GIT or multiple SVN repos??
The Project
- Build infrastructure
- Create a command line tool that can compress then merge all the required components (and allows end users to do this)
- Allows source to be taken from multiple sources (so you can create builds with extensions / or without stuff you don't need)
- Forums
- Set up a mailing list! (and archived, with a search feature!) - Anyone know a good hosted one available? - or should I just go off and set the infrastructure here..
- No more 'premium' ... users! - equal, like open source is supposed to be!
- Manual
- Work out how to build a manual from the source!-- While Ext's manual is not bad, take the opportunitythis time do it right, and have a bug reporting / user comments on it....
- A name??/ - May need some thought...
The reasoning.
I have to admitafter some reflection to being pretty pissed at this change, ExtJs was useful in a number of ways, other than being a reasonably well written, the forums where search-able, so you occasionally found fixes to issues that you where having. The doc's where not to bad. etc.
But basically I've committed 1000's of hours of time to learning, and writing huge codebases that depend on ExtJS, under the basic premise that it was availably for Free, with the only Caveat that If I modified ExtJS, then I would have to give back those changes. "Quid Quo-pro" as they say.
The Change to GPL has altered that equation in such a radical way that If this was not a 'software' product, and was something physical. you would be down at the consumer council, and filing a class action against Jack for things like Breach of trust, financial gain by deception etc.And filing claims for the loss of your time, and the cost of replacing his library..
I have seen postings that appear to claim Jack plan's to 'send notice' to people using a fork, but as far as I can see, he released the Javascript code as LGPL, and from every reading I've seen of that, I have the absolute right to distribute the Javascript code, along with any modifications. - This is the purpose of the license!!! - so by claiming otherwise he is not honoring his own license, not a good omen for the future of ExtJs even under GPL!
I'm floating this, as a plan... - shout if you are interested/ have some ideas..?? - (or you can find real technical flaws - not FUD flaws please).
07 May 2008
Ouch, what a mess, - Extjs goes GPL3
I think that basically sums up my reaction to Jack's GPLv3'ing a library. For those who are not aware, As I was not, when I visited the extjs web site a few days ago to grab a copy of the old 1.1 version and found all references to download it had disappeared. Digging a bit deeper on the site, I started spotting a few comments about the new licensing.
While I can say that Jack as the author (of I presume most of extjs?) has the right to change the license to whatever he likes, I think he has probably just destroyed the project. I could not commit to writing new code with a "Library" that is GPL, unless I was working on a GPL project (which is unlikely at present - got bills to pay). And for commercial or spec projects, that are not turning revenues yet, I can't really justify my time in committing to develop stuff that may, or may-not be able to fund today's and whatever Jack feels like charging in the future for the non-commercial license.
I've seen too many people burned by this closed source dependencies that they build their businesses around, only to have the effective rental for their office yanked through the roof, and no other option than a huge effort moving to another library or software causing chaos.
All that said, I'm not sure if v2.0 and v2.1 are really worth bothering with anyway, I've tried them on 2 projects so far, and the general sense I get, is that compared to v1.1 they are a little finicky, and tend to produce slightly unpredictable results. Which is tempting me to stick with 1.1..
The problem I see though is that the community that has built up around extjs has been supported by quite a few handy tools, the wiki, the doc's and the rather nasty forum (which is a good way to waste time finding answers to issues). So I guess someone setting up a openext? (I saw something on the net about it) should probably sort out those issues first, then start solving the technical issues about how to replace all the images and css in extjs which where not previouslylicensed under LGPL.
I would hate to have to go looking at the alternatives again, So sticking with v1.1+ hacks may be the best long term plan for me anyway.
30 Mar 2008
Little mice squeek to much - feedback on some of the coding issues..
Have to admit the reaction online to some of these posts have been quite funny, Cake guys, don't take this so seriously.... - Most of what I said about Cake I pretty much stand by, the responses I've seen don't seem to show that they knew they where making compromises in designing it the way it is, rather they knew best, and live with it (or without it..)
.. I also wonder what I got myself into...
More muses on the responses.....
** Apologies to Planet D for the Abiword CSS in the RSS / might be worth using the D RSS feed for Planet D? **
26 Mar 2008
CakePHP taking it apart, and the better written world of sinners..
Yeah, at one end we have our happy little hackers trying to make a quick CMS by throwing together a few includes and hiding functions everywhere in their wonderful directory structure, at the other we have those glorious frameworks.
So If you enjoy my rather questionable grammar, and High Horse views, here's the dig through CakePHP.
Dissent always welcome so comment away...
22 Mar 2008
Another 7 deadly sins for PHP
Ok, Since I had some really bad seafood on Good Friday (hence not turning out to be that good). resulting in me vegitating in bed today, I thought considering how much crap I gave some of those CMS projects, they deserve a bit of an explanation about why they got nailed.
Otherwise known as the deadly sins re-run post...
18 Mar 2008
Licence to release PHP code?
Basically if I need to use any of this for a commerical project, the first question will be, can we maintain and add to it, knowing that it's going to be maintainable. Well. the basic answer from all of the code in the 'Lite' section is. Forget about it. Most of the code in that section, you would be lucky if you server was not hacked. Installing a virus might be more efficient..
Anyway this is the summary from looking at the code.. - it's not pretty and I've not got much in the way of complements for them.. - but it's quite entertaining...
Translucid
BAD - Uses defines for configuration
GOOD - Bootstraper (index.php seems reasonably sensible)
BAD - Stupid directory layout
BAD - Namespacing on classes not sensible
BAD - re-written database abs. layer
BAD - stupid file naming *.class.php
BAD - loads of smarty assignvar's
Snews
BAD - functional based single file crap
Siteman
BAD - site was hacked!
BAD - Code unreadable
BAD - include style running
pivot
BAD - uses functions everywhere
BAD - directories splattered with unorganized files.
BAD - Code unreadable
BAD - php Serialized data in Files pretending to be a database
phpcms
BAD - Messy mix of HTML + PHP
GOOD - a bit of structure to it
BAD - horrific class naming sturcture.
BAD - uses capital letters for GLOBAL variables everywhere.
BAD - Functional crap
lokicms
BAD - functional Crap
BAD - farcical mix of HTML and PHP
lightneasy
BAD - functional Crap
BAD - unreadable code
BAD - if .. if .. if ... interpreting of page actions..
jaf-cms
BAD - Mix HTML and PHP
BAD - trys to get arround register globals!
BAD - functional crap
guppy
BAD - stupid licencing
BAD - stupid file naming *.inc
BAD - functional crap
BAD - mix HTML + PHP
BAD - unreadable code
eyeos
BAD - losts of functional crap
GOOD - uses pear
BAD - mindblowingly bad filenaming
BAD - not lite or simple (WTF was it in this category for..)
doop
BAD - not available
cutephp
BAD - functional crap
BAD - stupid file naming standard - .mdu!!
BAD - mix HTML and PHP
09 Mar 2008
Chinese Xapian search and indexing
its funny how you can often end up solving pretty much the same problem twice, dejavu for coding. Last weeks challenge was a free text search engine for email archives including support for chinese.
About 7 years ago, I remember hacking on mnogosearch to solve a pretty similar problem. After some research this time, I settled on Xapian, some of the reasons included,
- utf8 internal support
- nice bindings for PHP & c (via gcc/C++)
- a working set of command line tools (omindex & quest)
- database independant ~ no mysql dependancies
And the test that always makes the deal is that after apt-get'tting the package, it just worked! Creating a working store and runing queries is quite simple
The only trouble was that although it says it supports utf8, actual support for chinese is a bit more complex.
Unlike western langages, each character needs to be treated like a word. Ideally Xapian would realize this, however without wanting to hack the C++ code, I decided it would be quicker to create files based on the original email and pad the chinese characters with spaces. This means, in the short term, I can use omindex, rather than, binding the D code directly to xapian API.
the way this is done in the D index builder is
- parse the email with callbacks for each mime part. I have ported the mime code from binc imap for this, and called it dinc (silly name for the week)
- if the part is text, or html, convert it to utf8 using iconv
- stream read each line of the body (using memory streams)
- convert each line to utf32/dchars
- loop through the dchars and see if the are chinese, japanese or korean (see this for the simple check for cjk characters). pad the ouput line with spaces when found
- convert the resulting utf32 array to utf8/char array, and write it to the output stream (file stream),
all this code should be quite simple to extend when i get round to the d direct access to xapian api. It should also be pretty memmory efficient, and fast when i rewrite the iconv code to work like a stream filter...
On the other end of this was making the extjs/php5 front end do the searching. Again, the end user would be expected to search in chinese as a series of characters, eg. type 'XXX' rather than type 'X X X' so in php. I needed to convert the search string into utf32, and compare each block of 4 characters against the previous list of cjk charcters, padding with spaces. then converting back to utf8 prior to sending the query. This is all pretty simple using iconv or mb_string.
All in all, not that difficult to do, however actually finding/working how to do this was quite a challenge.
24 Jan 2008
Bluetooth GPRS via an A1200 and peoples.net
for starters: /etc/bluetooth/rfcomm.conf
(you need to find out your bluetooth id - plenty of results if you google "bt grps howto's" )
rfcomm0 {
bind yes;
device ***device id here **;
channel 5;
comment "Alans phone";
}you need to configure pairing - I think this is the one that worked
/etc/bluetooth/hcid.conf
options {
autoinit yes;
security user;
pairing multi;
passkey "**MAKE UP A PASSKEY ***";
}And now for the magical stuff.. (the 0.0.0.0:10.0.0.1 was the magic bit that solved all the connection problems)
/etc/ppp/peers/peoples
debug
connect "/usr/sbin/chat -v -f /etc/chatscripts/people-connect"
user "ptc"
password "ptc"
/dev/rfcomm0 115200
show-password
noipdefault
usepeerdns
defaultroute
noauth
0.0.0.0:10.0.0.1
ipcp-accept-local
ipcp-accept-remote
debug
kdebug 7
#nodeflate
novj
noccp
#remove if you want pon to run in the background
modem -detach
the chat script
/etc/chatscripts/peoples-connect
TIMEOUT 35
ECHO ON
ABORT '\nBUSY\r'
ABORT '\nERROR\r'
ABORT '\nNO ANSWER\r'
ABORT '\nNO CARRIER\r'
ABORT '\nNO DIALTONE\r'
ABORT '\nRINGING\r\n\r\nRINGING\r'
'' ATZ
OK 'ATM1L1'
OK 'AT&f+cgdcont=1,"IP","peoples.net","",0,0'
OK 'ATDT*99***1#'
CONNECT ""
This is disconnect (not really used it, but might help...)
/etc/chatscripts/people-disconnect
SAY 'Starting GPRS disconnect script\n'
"" \K
"" +++ATH0
And hopefully when you do
#pon peoplesyou get an internet connection from anywhere in Hong Kong..
13 Dec 2007
Script Crusher
- Its slow to load
- Can you translale it into their language
- Loading of many javascript files (With our currently quite small applications, there are still more than 30 single javascript files - one class per file, which can take over 10 seconds to load.)
- The javascript contains comments and whitespace (As I like to make my code readable and maintainable)
To address these issues, there are a few javascript compression tools out there, however, most have rather painful setup requirements, like getting java working, and are not very hacker friendly.
Now that all my interface is embedded into javascript code, including all the button labels etc. l have been wondering for quite a while how I was going to translate the application and keep it up-to-date (in the Flexy template engine we extract the strings from the HTML, and have a tool that manages PO files based on them). Since this did not seem feasible for Javascript, I was begining to dread the thought of finding all the strings and wrapping them with something like a gettext type call.
So it turned out my little dmdscript hacking may have some use...
It took a small amount of hacking to add tokenizing support to dmdscript, by adding a new class for the javascript interface and few minor changes to dmdscript's internal tokenizer I got this little script to work.
var data = File.read(fname);
var x = Script.tokenize(data);
for(var i =0; i < x.length; i++) {
if (x[i].tok == "whitespace") {
if (x[i].val.indexOf("\n") > -1) {
print("\n");
continue;
}
print(" ");
continue;
}
print(x[i].val);
}
print(";\n");
Basically it reads the file, then creates an array of objects, each one representing a token. (I should perhaps change the token to an integer and use Script.Token.whitespace as a constant... ).
The script will basically strip all the whitespace and comments, and convert whitespace into either a line break or a space.. (as linebreaks are actually relivant in Javascript).
The next issue was how to deal with language conversion. The code above can be used to test for strings by doing:
for(var i =0; i < x.length; i++) {
if (x[i].tok == "string") {
println("got string: " + x[i].val);
}
}
So I came up with the idea of using the quote style to determine if a string needs translating.
single quotes = does not need translating
double quotes = a translable string.
It was amazingly easy to change my existing code to do this, I could then generate a list of strings for each file, and output a JSON structured file which can be used to store the translations mapping.
#djs ScriptCrusher.js -- --strings /tmp/Hello.js > /tmp/zh_HK.jsWhich can be edited and translated.
Would generate a file:
"Hello.js" : {
"Hello": "Hello",
"World": "World",
}
Then when Crunching the script up, it uses the map to convert strings from one language into another.
eg.
#djs ScriptCrusher.js -- --langfile /tmp/zh_HK.js /tmp/Hello.js \
> /tmp/Hello.zh_HK.js
#djs ScriptCrusher.js --/tmp/Hello.js > /tmp/Hello.en.js
(I've not done the merging of old/new yet, but it should be pretty trivial).
djs should be pretty simple to build: (requires gdc - available in Debian and Ubuntu + probably others)
#svn co http://www.akbkhome.com/svn/gtkDS
#cd gtkDS
#sh djs.gdc.sh
The full ScriptCrusher is here..
http://www.akbkhome.com/svn/gtkDS/test/ScriptCrusher.js
Happy Hacking..
Follow us
-
- Some thoughts on the language server and its usefulness in the roobuilder
- Roo Builder for Gtk4 moving forward
- Clustered Web Applications - Mysql and File replication
- GitLive - Branching - Merging
- PDO_DataObject Released
- PDO_DataObject is under way
- Mass email Marketing and anti-spam - some of the how-to..
- Hydra - Recruitment done right
Blog Latest
-
Twitter - @Roojs