These are my notes/ comments on getting it to work (partly), check the blog again every month or so to see if I ever get the thing working fully..

After leaving the Digital TV sticks lying around for a few months, I finally got back to looking at the issues with drivers. ASUS have been absolutely useless, and have ceased responding to all support requests. The Source code supplied on the site, and emailed to me directly appears to be the completely the wrong source code (based on comparisons using strings on the binary driver,  I'll have more confirmation of this when I review the dmesg outputs as described below..)

Having given up on the source code, I decided just to see if I could get them to work in any way. We had a go with the Windows drivers on a PC at work, but did not really have much success (as I found out later, probably due to signal issues with the little antenna). But it did give me the idea of using virtual machines and running the EEEPC development kit. So here's the process so far just to test the cards. - details in the extended bit..

View Extended Entry

Posted by in Linux | Add / View Comments()

For this weeks hack, I've been looking at Lightning, the calendaring extension to thunderbird.

What brought this on was we where discussing a bug tracker for the Anti-spam / Mail management software, and none of the software that i've seen for this either is elegantly written, easy to set up or simple to use.

To me the 'ideal' way to handle this, having played around with Lightning, is to have a calender provider (that's shared) and have all the key people able to drag a 'requirement email' into the TODO task list, or similar. Where people can submit bug's by signing up to a calendar/todo list and just add it as a task.

Obviously not suited to all types of projects..., but I think you get the jist that using lightning to edit these bugs would be pretty compelling due to it's integration with your primary mailer.

State of the current providers.

So to see if this could work, I had a look at the current list of network providers available. ICAL, CalDav, Sun's Protocol, and there's a google provider.

On the face of things it looks like that's quite a choice, but if you look closely at all of these, one thing stands out like a sore thumb. The backend server to deal with all these protocols is an absolute nightmare to create, or modify to suit other types of applications (think of a shipping managment application that we could integrate order delivery information with the calendar, so you could add notes about the delivery etc. via the calendar item.).

Part of the core problem with the existing protocols can be summed up with one dirty word "XML"! - CalDav and google being the worst of this. If you look at the problem today and tried to solve it, the first thing that would come to mind would be JSON/REST. It's the perfect set of protocols to ensure that backend server would be trivial to write, and the front end would not be such a mess. (especially as Lightning's written mostly in Javascript anyway).

Hacking a provider

There are really only two examples I could find of calendar providers for lightning. Google is one, and there is also an extension called hcalendar - that pulls calendar's in from web pages (as far as I could see). The second of these two proved to be the goldmine for creating a JSON calendar provider extension.

hcalendar provides most of the code needed to implement any kind of provider, it only implements the getItems() method. However it has stub's for all the others.

You can browse through the code here, calJSONCalendar.js, I did run into a few issues with Async requests though, as It looks like you need to implement a request object (as per caldav and google calendar), otherwise you run into problems with transactions on updating. So as a simple workaround, the extension uses sync requests for updates and adding.

How it works

getItems() - get a list of calender or todo items.

is a GET request, with params
- aCount = max number of resulsts
- aRangeStart = date start ISO format
- aRangeEnd = date end ISO format
- caltype = calendar data type (C = calendar, T= todo)
- filter (not yet)
eg.
http://your_calendar?caltype=C&aCount=0&aRangeStart=2008-01-01....

this returns a json array something like this.

[
   {
	id: 1,                      // your data base id (ignored)
	uid: "XXXXXXXXXXXXXXXXX",
        dtstart: "2008-10-10 12:12:12",
        dtend: "2008-10-10 12:12:12",
	caltype : 'C',
	ical : "  <<< Calender data in ical format >>" 
	
	// all the other data is optional (if you do not have a ical record, it can build 
        // most of the event data from properties

	privacy: "default";
	created: "2008.....",
	last_modified: "2008....."
   }, 
   ......
]

modifyItem() / addItem()

is a POST request, with the params pretty much the same as the json data above (with the ical record in the ical POST entry) - the resulting entry is returned by the call in the same format as above. At present ical is the main thing that is posted, however I think it needs to send quite a bit of the generic information.

deleteItem()

is a POST request, with the property _delete_={uid},
(currently we ignore the response) - it should probably be
{ success: true|false, msgid : 1, msg: "A string error" }

the body of this message contains a example server implementation for the extension. - you can download the extension here: jsoncalendar-0.2.xpi, (REQUIRES LIGHTNING 0.9!!) or have a look at my svn repo JsonCalendarProvider for how to develop test it.

Read on for the SERVER BACKEND CODE...

View Extended Entry

Posted by in XUL | Add / View Comments()

Remember the good old days where phpnuke was the worst code since sliced cheese, and thousands of idiot's had installed in on their server waiting to be hacked into..

Well, It's a bit like how I felt this week trying to solve my svg graphing issues..

A bit of background, In the last week I just pushed out the latest version of our Anti-spam / Mail control application, the big new feature was a beta version of the statistic's  and analysis section. Where we showed nice little graphs about how much work your staff where doing and if they had really been sending jokes all day long or just reading facebook spam

Open Flash Charts

While the database side , the design is quite interesting, I did not really spend to much time thinking about the frontend - as I like to keep that flexible (read throw away). So for the first beta I decided to use Open Flash Charts. The current version is extremely well designed, in terms of the chart generation consists of delivering a JSON file to a static single Flash file. The JSON file sets the graph type, and date etc..

It's very quick to set up and work with, and the graphs are quite pretty, and a cutely interactive. And for our first beta it's just about OK.

However for anything other than a beta,  there are rather serious problems with it.
a) It's dog ass slow, to load, render on anything but a brand new overspec'd PC
b) The flash file is quite large, and yet another overhead to a already large application.
c) Having multiple Flash graphs on a single page, just makes the above two problems worse.
d) It depends on closed source tools (Actionscript compilers etc.) - Never a good idea.
e) Printing of Flash in firefox is broken (at least on linux)
or basically all summed up... Flash sucks

PlotKit

So after beta 1 was released to testing, I started the hunt for an alternative. Since the 'correct' method to do graphs on the web is really SVG, and as I like to shift all the pointless processing to the end users PC.  JavaScript to convert the data into a chart is really the ideal solution.

Pretty high up on google's list is PlotKit, which has some really nice demo's on the site, It depends on two other pieces of code, excanvas (as nice self contained javascript file that handles SVG compatibility issues with IE/FF/Safari etc.)  and MochiKit.

MochiKit

Since MochiKit is none too small, and would add to my javascript overhead, I thought I'd check out what it was, and see if it was really needed. What I saw, was more "nightmare on nuke street".

The front page of the web site claims quite large "MochiKit makes JavaScript suck less", That should be paraphrased, "only if you are a Python nut-head" - otherwise it make your Javascript code into complete jibberish. __self__, __new__, __screwed_up_language__ everywhere....

<big rant> The dumb f*** who wrote it decided that Javascript was not Python, so they should try and make it's syntax as close to python as possible. Ignoring the fact that Javascript syntax just solves  problems differently, it does not need fixing or even breaking to be as badly designed as python.. </big rant>

Anyway after digging through the code to plotkit, and mochikit, I came to the conclusion that they had developed a piece of code that was completely unmaintainable or extend-able. So was forced to keep looking for an alternative.

Plotr

Quite a way down google's list (search svg javascript graph) I finally found Plotr, Basically someone else had obviously looked at PlotKit, and decided that Mochikit was just so ugly, that removing it from the application was the only sane thing left to do.

What they chose rather than MochiKit, was Prototype, which in general is considerably better than MochiKit. And basically removed all the Mochikit dependancies and replaced them with Prototype. This made the code quite a bit simple to read and understand, (they also added comments), although I'm not to keen on this type of commenting, as I find it add's noise, and makes the code less readable.

xxxx : function (/** some_type **/ xxxxx, /** another_type **/) {

Anyway, otherwise Plotr looks clean and simple.

** It looks like the author of Plotr as gone on to start Flotr - another graphing library.. - did not notice that until I'd commited to working with Plotr...

Prototype

What I wanted to do however was reduce the size of the Prototype dependancy. So I started having a look at Prototype's internals.

Splattering the namespace

Unlike Ext, and Roo, Prototype's core creates quite a few global properties, $, $R, $$, JSONSomething and Element which by the looks of things are totally unnecessary, and make it difficult to work with other frameworks (or potentially other code), This is downright bad design, and could easily be rectified by putting almost all of those under the Prototype.* object.

Along with this, there is quite excessive use of adding to Global Prototypes, String.prototype gains about 20 extra methods, some quite useful, or wrappers for compatibility, other seem more like features for features sake, and should really be moved to a Prototype.String object or just removed.

What make me wonder the most was the way that prototype adds properties to HTML and DOM objects, $("xxxx") - get's does a getElementById( "xxxx") and returns the DOM/HTML object overlaid with all of the properties from the Element object.  In Ext/Roo's 'Roo.get("xxx")' method  create's something like the 'Element' object , Roo.Element, it puts the original DOM object in the return value ".dom" property. Hence making documentation clearer and code considerably more managable. Not so much of WTF did that method come from? when reading code.

Anyway - I managed to give Prototype a bit of a diet, so it's  not so much of a Namespace/bandwidth hog. You can download the lightwight code here:
http://www.akbkhome.com/svn/javascript/plotr
Note the full bundle (protolite, excanvas + plotr comes in at 54K or ~ 15K if you are serving javascript with the mod_gz library...

Just to prove it still works
http://devel.akbkhome.com/plotr/testplot.html

See here for the original code
http://code.google.com/p/plotr/source/browse/

View Extended Entry

Posted by in Javascript | Add / View Comments()

Just a small update on roojs1 - I've added in-line comments to the RooJs1 documentation - so if you think a method or class is not well explained, add your comment, example code, or bug note right on the documentation. - Something I think makes a huge difference to the usability of code...

Other small changes are also beginning to be added - including the fields property of a Roo.toolbar, and the new Roo.form.Hidden element, to make creating forms with hidden variables simpler.

Anyway, rumors are swirling around that there is a recommendation to avoid ExtJS among certain linux distributors due to uncertainties and general lack of confidence in the license, it's continual changing and general poor behavior by Jack in not understanding the implications of the bait and switch changes.

It also appears that Jack's Legal team are working harder than his coding team again, sending out threatening letters to people involved in various ExtJs2 derivative works. (which is the key reason I'm focusing on v1 - clear, definitive separation and licenses that do not allow for such shenanigans)

While ExtJS2 offered some interesting new features. The significant change to the object rendering model, from my experience has been broken on every release prior to the GPL one, (and since I never tested it after that release, I suspect it's still broken).

Other changes relating to a clearer heirachy of object model look like they should be a simple addition to ExtJS1, along with better support for non DOM element dependent constructors. Hence I think building on ExtJS1 appears to be a better long term move, and sounds like a few more are beginning to see that wisdom.

View Extended Entry

Posted by in Javascript | Add / View Comments()

View Extended Entry

Posted by in Digitalmars D | Add / View Comments()

After a little wait (~ 3 weeks), having asked ASUS to provide me with the source code for the drivers to the DMB-TH Digital TV receiver, I finally got two tarballs of the source code. One step closer to an open source friendly solution to watch Digital TV in China and Hong Kong.

It took a little persuasion to convince the technical support team at ASUS that I was entitled to the source as it was based on GPL code, but once they understood, and had asked their supervisor/ R&D department about it. It was just a matter of waiting for the R&D dept to get time to do it.

I was very impressed with ASUS in general - their tech response times for product support where pretty amazing (usually answering within a few hours). If you have the option of buying ASUS, I can highly recommend their email tech support. Friendly, positive, and responsive.

The driver modifications are all to drivers/media/dvb/*, and consist mostly of 2 new files dealing with the demodulator, and a rather unfortunate hack at db3000mc.c - Basically replacing the core of that file with the routines for the Legend chip. Rather than creating a new file (which looks like it would have been a far better idea).

Anyway, I'm going to try building these changes against a recent kernel, then see if I can sort out the db3000mc mess. If anyone want's to help out, drop me an email and I can send you the files.

View Extended Entry

Posted by in Linux | Add / View Comments()

Another update on progress on RooJS (the ExtJS v1 Fork)

• Form's now have CSS, so the dynamic form examples work quite well
  
• The CSS was created by documenting how the existing examples where rendered in ExtJS - A document describing this was created (see form_design.svg in the css folder). Then that document was used to create a brand new CSS file. I believe this can be regarded as not breaking the copyright and having a reasonable reverse engineering process.
  
• the x-box borders are missing, although not critical
• the HTML editor needs to be fixed.
  
• 
  
• I have done the design diagram for CSS of the menu's and will be working on that soon.
• The doc's are improving as well, The doc's appear to be listing all the correct elements now, future work will focus on tidying up the templates.
• Most of the improvements came from replacing the Walker code in the jsdoctoolkit so that it understands scope in a similar way to the compressor. I also had to add a few extra comments in the source to give the documentation tool a few hints.
  
• /** @scope Roo.somescope */ -- changes the scope in the documentation engine (as sometimes it just can not guess what the scope is.
• /** @scopeAlias avar=Roo.xyz.prototype */ -- adds an alias to the parser engine alias map, so when it sees 'avar', it will replace it with Roo.xyz.prototype
• The compresser is working well, someone asked on #roojs on freenode about using it for other code, including ExtJs2.0 - There should be no problem doing that - have a look at the bundle build file to see how to set up a builder for any project.
• One thing to note is that the compresser uses hints when it sees 'eval'. The YUI compresser basically turns off compression for a large chunk of the code if it sees 'eval', I've added code to pick up a comment before the eval statement:
• /** eval:var:abcd */ will turn variable compresion back on, and exclude 'abcd' from the list of variables to be compressed (use multiple eval:var:... statement on multiple lines to exclude multiple variables)

Anyway back to work tommorow, after celebrating (eg. doing nothing) the glorious reunification with the motherland day (HKSAR day)

View Extended Entry

Posted by in Javascript | Add / View Comments()

CSS hackers needed.

Yes, the first test version of roojs1 is built and working you can have a look here to see the old ExtJS1 examples (which where sourced from a 0.40 release licenced under a BSD licence)
http://www.akbkhome.com/roojs1/examples

Warning this may break occasionally as I do daily builds of the library

Most of the main layout, windows, grid and tree work pretty well, but there are some problems with toolbars, menus and forms. Basically I could not find complete licence free sources for any of them. So if you know anyone who can create CSS for the missing parts (and can illustrate reasonably well that it was not based on the original ExtJS CSS) send them over this way. That will help RooJS2 as well, as the CSS is probably pretty similar.

Have a look at http://www.akbkhome.com/roojs1/css/roojs-debug.css for details on the status of the css build..

A big leap forward in building a truely free high quality JS UI toolkit.

Anyway onto my geeking techo fun.

Rooscript compressor

As part of the whole build toolkit, I've ported the yui compressor to rooscript (the dmdscript based Javascript interpreter), The results are pretty good.

The current output from the compressor is 8K less that the ~497K that ExtJS 1.1.1 files are (built I presume using yui compressor).  Along with this, it's pretty easy to tweak, and remove Files and add your own application (eg. create an all in one  javascript file for your application)

Speedwise I've been trying to get the compressor to go faster (It normally caches the minification result to a file and can do a 'reparse' in arround 2 seconds, but for a fresh build it takes about 6 minutes to parse all the ~100 files in Roojs1

In the process of speeding it up, I've added a few switches to rooscript

• -p for a very simple profiler (gives you the total time taken for all the opcodes exculuding call/new etc. in each function) along with how many times they where called.
• -U to turn on Unicode for strings, as I've disabled it by default, although I'm not that sure now, that it produced the speed hit that I thought it might. ** mostly affect things like xx.indexOf("\n")
  
• -d (to show all the opcodes as they run) - old but handy sometimes...

The code is reasonably simple, so If you have any great ideas on how to improve compression, it's extremely easy to test how well they work. (rather than the usual compile/run cycle that all the Java versions have)

In looking at the resulting code, one simple opmization that I've started adding to RooJs is changing the constructor code in the object from Roo.data.Connection.superclass.constructor.call(this,....) to Roo.superC(this,....) and Roo.data.Connection.superclass.somemethod.call(this,....) to Roo.superM('somemethod', this,.....)

I'm guessing that's could be ~ 2K of rather wastefull code removed..


Basically since the compressor is working now, I'm going to re-look at the Docs, and sort out the bug's in those - probably replacing the scoping code in jsdoctoolkit, with something closer to the ported yui compresser version.

View Extended Entry

Posted by in Javascript | Add / View Comments()

View Extended Entry

Posted by in Hong Kong | Add / View Comments()

View Extended Entry

Posted by in Javascript | Add / View Comments()