D sockets, and rooscript, roojs updates

Non-blocking socketstreams in D

I'm working on a threaded server app in D, and wanted to use non-blocking as it gives far more control over timeout rules and DOS attacks. - Unfortunately D's standard socketstream is not really designed to work with non-blocking sockets.

While defiantly not perfect, I think this will give you an idea of what's needed to get it to work.

Extend the standard socket stream, and turn off blocking in the constructor.

class SocketStreamB: SocketStream
{
    this(Socket sock, FileMode mode) {
         super(sock, mode);
	 this.socket.blocking(false);
    }

Now for the not so clean bit.. - I've created a _recv wrapper, that allows errors, and restricts to a max length . - not really relivant to the details of how to do this, but gives you some idea of what's going on later.

char[] _recv(int len, SocketFlags flags)
{
	 assert(len < 1029);
	 char[1028] buf;
	 int read = std.c.linux.socket.recv(
                this.socket.handle(), cast(void*) buf.ptr, len, cast(int)flags);
	 if (read < -1) { // allow -1?
   	     throw new Exception(
                   "Socked returned " ~ std.string.toString( read) ~ " bytes");
	 }
	 return read  > 0 ? buf[0..read].dup : "";
	}

I added a new method "isAvailable", this peeks to see if data is available. - quite useful if you need to test if data has been sent too early.

bool isAvailable(int timeout) {
	 d_time end_time = cast(d_time) (std.date.getUTCtime()  + 
	  (timeout > 0 ? (timeout * std.date.TicksPerSecond) : 100)
	  );
	 char[] buf;
	 while (std.date.getUTCtime()  < end_time) {
            // see if data is available?
	  // peek a fetch..
	  try {
	   buf = _recv(1, SocketFlags.PEEK); 
	  } catch (Exception e) {
	   buf = "";
	  }
	  
	  if (buf.length < 1) {
	   std.c.time.usleep(500);
	   continue; // no data yet..
	  }
	  //std.stdio.writefln("Available Char %x '%s'", buf[0],buf);
	  return true;
	 }
	 return false;
	}

Now for a key example - by peeking for data being available, finding the \n in it, we can then fetch all the data we need only.

char[] readLine(int timeout = 60) // return "" on timeout..
	{
		d_time end_time = cast(d_time) (std.date.getUTCtime()  + (timeout * std.date.TicksPerSecond));

        char[] ret;
		char[] buf;
		
         
        while (std.date.getUTCtime()  < end_time) {
            // see if data is available?
			// peek a fetch..
			
			buf = _recv(1028, SocketFlags.PEEK); 
			 
			if (buf.length < 1) {
				std.c.time.usleep(500);
				continue; // no data yet..
			}
			// peak max - 
			int rpos = std.string.find(buf, "\n");
			
			if (rpos < 0) {
				ret ~= _recv(buf.length, SocketFlags.NONE);
				// std.c.time.usleep(500);??? sleep needed
				continue;
			}
			
			
			// got a "\n"; - eat up until it..
			ret ~= _recv(rpos+1, SocketFlags.NONE);
			if ((ret.length > 1) && (ret[ret.length-2] == '\r')) {
				return ret[0..ret.length-2];
			}
			return ret[0..ret.length-1];
			
			// should we sleep a bit?
			
		}
		// otherwise wait until data is ready.
		throw new Exception("timeout waiting for data");
		return "";
	}

One thing I did notice is that writeLine/readLine in D are system specific - eg. based on standard line endings on each platform, when you are dealing with socket based application this default becomes irrelevant, and rather annoying - One of those "probably not a good idea in hindsight"

Unix Sockets in D

This was another missing part to phobos - using Unix sockets - this appears to work OK on gdc. (note the use of std.c.unix.unix, rather than std.c.linux.socket) - I think that is gdc specific.

//private import std.c.linux.socket;
private import std.stdio;
private import std.socket;	
private import std.c.unix.unix;
private import std.stdint;

extern(C) {
		
	struct sockaddr_un {
		int16_t sun_family = AF_UNIX;
		ubyte[108] sun_path;
	}
}
  

class UnixAddress : std.socket.Address 
{
	
	
	protected sockaddr_un sun;
	
	
	protected sockaddr* name() { 
		return cast(sockaddr*)&sun; 
	}
	protected int nameLen() { 
		return sun.sizeof; 
	}
	

	
	public:
	this(char[] path) {
		
		strcpy(cast(char*)sun.sun_path, std.string.toStringz(path));
	}
	
	AddressFamily addressFamily()
	{
		return cast(AddressFamily)AddressFamily.UNIX;
	}
	
	
	string toString()
	{
		return std.string.toString(cast(char*)sun.sun_path);
	}

}
 

Rooscript updates - System, and GDC cached building.

I did a little hacking on Rooscript over the weekend. - key to this was the new "System" object, which currently only supports a run method, to run an external script and return the exit value. - not all that usefull, but good enough to enable me to write a GDC speed rebuilder.

Basically the GDC shell scripts you can use to build rooscript are extremely simple and easy to understand, however building the roogtk version can be very slow, as it has to build alot of files. Unlike a classic make which only builds files that have changed, using that script a second time is just as slow, so making small fixes and see if the work took a bit to long. So if you have a look at examples/gtkbuild.js this little script will do effectively the same as gdc, except that it stores the temporary .o files and compares the modification date/time to the original .d and only builds it again if something has change - making rebuilds practically instant..

RooJS updates - examples in the manual

I've sorted out most of the CSS now, and added links to the examples to the doc system. (using Roo's new wiki).  I have a improved HTML editor component ready to replace the one currently in Roo, that allows multiple usage, and better configuration of toolbars, just got to find time to do merge the codebase. I've been bugging everyone on #roojs to get the hg server up, so that anyone can check it out and use it. see the latest docs here: http://www.akbkhome.com/roojs1/docs/