Pretty cool!

Google published Google Gears a plugin that exposes an APIs to Javascript applications for off-line storage.

It also comes with an embedded Sqlite3 database, very nice.

Occupation 101 has finally been released, after years in production the movie can now be purchased on DVD.

This is an incredibly important movie to understand the living conditions of the palestinian population in the West Bank.

The Production

The movie was directed, shot and produced by two guys which explains why it took so long for the movie to be released.

Resources

B'Tselem: The Israeli Information Center for Human Rights in the Occupied Territories (West Bank and Gaza) has extensive information.

John Pilger's Palestine is Still the Issue documentary is now available on video.google.com.

Nat was today in San Francisco to announce Novell's partnership with the EFF to work on Patent Reform.

The details can be found here.

"EFF is partnering with Novell to try to get rid of software patents that are hurting innovation all over the world," stated Shari Steele, Executive Director of the EFF in an interview prior to the panel session.

In essence, Novell is committed to working with the EFF to improve patent quality, while at the same time work to lobby with government agencies to reform existing patent policies and litigation, according to Nat Friedman. Novell's Chief Technology and Strategy Officer for Open Source. Specifically, Novell will assist the EFF in two ways.

First, they will work with and support the EFF's existing Patent Busting Project, which targets existing patents that cover technology concepts that are perhaps to fundamental or already have prior art.

"They're awarded for fundamental concepts in computer science on a too regular basis," Friedman stated. "Things like XOR, the ISNOT operator."

"If you go on our Web site," Steele added, "we asked Internet users to identify the ten most egregious patents out there. We have our 'Ten Most Wanted.' And we've been one by one hacking away away at them."

The second part of the partnership will have Novell working with the EFF and legislators to lobby for patent reform, initially in the US, but also branching out to Europe, where patent problems continue to arise. The EFF and Novell will also work with standards groups to assist in patent reform.

The news is notable because it's the first time, Steele confirmed, that a corporate entity has publicly thrown in this level of support for the EFF on the patent issue. Normally, Steele said, companies have been much more circumspect and allowed the EFF to solely take the lead on this issue.

The goal is to continue defending Linux from IP threats (as we have done in the SCO lawsuit that has costed Novell millions of dollars) and as we did when we were one of the founding members that contributed significantly to the creation of the Open Invention Network.

For those of you that could not attend the Compiler Lab but would like to see the slides, John Lam has published two slide decks:

• DLR Experiences.
• Zen of the DLR.

Today was a bonding day for those of us that attended the Compiler Lab, after the morning session and the lightning talks were over most of us stayed around and talk until the end of the day.

In the afternoon, while some of us were busy schmoozing and debating important topics like what was better: Jon Stewart or the Colbert report and lamenting the results of the French elections some other people were actually busy hacking in the afternoon.

Apparently some folks that are working on a ColdFussion compiler got their Cold Fusion compiler using the DLR. Rodrigo and JB were also quite busy hacking in the afternoon.

Good news for Boo users: after I posted my message yesterday about readline-like functionality Cédric Vivier pointed out that he had just implemented the readline capabilities in Booish. You can see the details here.

Hosting and the Case for Multiple Hosts

The morning sessions focused on how to host DLR-bound languages into an application and the various services offered by the DLR infrastructure.

A question that came up is whether it is necessary to support more than one hosting "context" in a single application. Where global variables and types would be shared across multiple scripting instances.

Although today the DLR does not support more than one context, I believe that it should support independent contexts. A case was made for Processes and AppDomains already existing and that adding a scripting contexts was not necessary but if there were interesting scenarios for this setup they could look into it.

In my opinion, AppDomains are too heavy of a separation barrier in an application. They are hard to setup properly and depending on them requires an entire application to be restructured around this new boundary. In the particular case of Boo, multiple hosting environments can exist in a single application and I think this is a good thing.

The best use can I can think of for the need to have multiple hosting contexts would be if the DLR is hosted in two independent libraries A and B developed by two independent groups and then having both of those libraries consumed by a third party. The developers of A and B should not have to talk to each other.

Another example: I could imagine a setup where I want a scripting context for extending the internals of my application and on the same process, I would like to have a second scripting context that does not have access to the internals of my application, but instead is used by the user to write/prototype whatever it is that my application hosts.

An accounting application that is heavily scripted internally would use one hosting context for its internal scripting facilities: forms, rendering, accounting and another scripting context would be used for doing report scripting: a much more limited environment that would not get access to all the application internals and types and globals, but only those that make sense for the reporting infrastructure.

Anyways my vote: yes, we need multiple hosts per AppDomain.

There was a demo of some scripting editor during the talk that hosted the DLR itself but Google does not turn up anything useful about it (Nessie), it looked cute.

DLR Code Generation Scenarios

In the second part of the sessions they showed some examples of what kind of code IronPython, IronRuby and the Javascript compilers generate.

The DLR in particular provides support for capturing the environment (to create proper lambda functions). A key point during the presentations was that providing as much information the DLR as possible allowed it to optimize the code.

Rodrigo -the author of Boo- pointed out during one of the breaks that he did like the optimizations that scripting developers were getting for free.

Milo: Compiler Infrastructure

In a similar spirit to what the DLR team did by refactoring IronPython into the DLR and then growing languages out of it, Rodrigo and JB talked about a similar project.

Based on the Boo codebase and their joint work on Cecil while they both worked at db4objects they created Milo (which is available here: http://code.google.com/p/milo/.

Milo's goal is to simplify the development of a compiler that targets the .NET framework

Cute Anecdotes

Their new DLR-based Javascript compiler (that implements the current ECMA spec) was written by two developers in four months. Nandan Prabhu was at the conference and he explained that if they were to rewrite it today it would probably take three months for a programmer to implement as such a person would not have to keep up with the daily changes in the DLR.

The current sizes for the runtime and the compiler are:

-rw-r--r-- 1 root root 114688 2007-05-02 04:07 Microsoft.JScript.Compiler.dll
-rw-r--r-- 1 root root 299008 2007-05-02 04:07 Microsoft.JScript.Runtime.dll
	

Extrapolating that from our compiler source code it seems that the code is roughly 35,000 lines of code for both the compiler and the runtime. And the compiler is only one fourth of that, so something in the 9,000 line range.

Another interesting detail: the new Javascript compiler is written in Visual Basic.NET.

Dinner

The day ended up with a fascinating discussion in the kitchen with the Jims (Miller and Hugunin) about languages, big integer support, scheme, python, academia and practice and tons of anecdotes of the early days of the CLR and Jim's days at MIT.

As part of my Silverlight for Linux research I have prototyped a native code library that uses Cairo and ffmpeg to prototype some of the ideas from the Silverlight canvas.

The above picture shows 2 videos playing, one of them rotated and a triangle object rendered in the middle as well. This was just a test to get an idea of what is involved.

The code does no attempt to keep up with the clock or do anything reasonable with time: it just renders stuff as fast as it can and does have no audio support. The next step is tune this a little bit (I do too many copies right now, too much stuff is done on the critical path) and then contrast this with an Agg-based implementation.

The canvas implementation that am using right now is inspired by the Gnome Canvas in many regards.

Compiler Lab: First Day

Today was the first day of the Compiler Lab and got to see again many good friends and some folks I had not seen for ages. This will feel like a name-dropping blog post, but here it goes anyways.

Scott Gunthrie introduced the day with a general overview of where they are and where they are going with .NET, Silverlight and WPF.

JB Evain and Rodrigo Oliveira were both late for the opening session, but they looked like they had been hacking all Sunday.

Jim Hugunin introduced the Dynamic Language Runtime in the second half of the morning and went through the design decisions behind the DLR. Although he has covered some of this information in blog this presentation was colored with the history of how decisions were made and the engineering choices that they had made that took them from IronPython to work on a general purpose framework for hosting dynamic languages.

The historical bits are always the most interesting to me. Jim talked about how they decided to go for a general purpose framework as people started asking for support for more dynamic languages. They were not really interested in writing all the compiler themselves, so instead they decided to turn their experience in building IronPython in some reusable work that could be used by others.

To accomplish this, they chose to implement four different dynamic languages extracting the common infrastructure into a reusable framework. Jim did not say how much they had learned from each language (which I guess is a good question for tomorrow), but he mentioned that the DLRConsole sample for Silverlight originally was pure Python. That it took him about three hours to add Javascript, three days to get Ruby in shape to work on the DLR and their new VisualBasic required almost no work.

I have extensive notes, but they are too extensive and am not sure I want to edit those, but you can get a good idea of what the presentations were all about from his blog posts:

• A Dynamic Language Runtime
• The One True Object.
• The One True Object, part 2: where Jim introduces the idea of message passing as the universal mechanism for hosting dynamic languages instead of object wrapping.
• DLR Trees: the default AST representation that dynamic language authors can use to take advantage of the DLR.

The DLR is pretty serious, there were at least six people in the audience from the DLR team. I finally got to meet Dino Viehland, Bill Chiles was handing out copies of ToyScript (a reference small interpreter), Martin Maly was there, John Lam sat on the back (we all presume he was working furiously on his presentation for tomorrow) and Mahesh Prakriya was there as well.

Jim Miller of Scheme fame was there and made things all the more amusing by pointing out the similarities of certain features to Scheme. He also brought up a few issues that would need to be included in the DLR to support languages like Scheme as the day went by.

Jim Hugunin has always found interesting ways of demoing IronPython. He used to build a calculator with Avalon and rotate the buttons. Today he showed us some XNA space ship controlled with an Xbox control and later used IronPython to control a Lego Robot that his nine year old son had built.

John Gough was also here and he has retired from teaching at QUT. Nigel Perry has left the University too and is now working at Microsoft.

John Lam is organizing the lightning talks, and he has divided things neatly into three categories: Monday was the day for 5 minute talks. Tuesday was the day for 10 minute talks and Wednesday is the day for 15 minute talks.

When I left the building there were zero talks scheduled for 5 or 15 minutes. Everyone is listed on the 10-minute column for tomorrow.

I had a fantastic dinner with Scott Gunthrie and Jason Zander. I got a chance to pepper them with questions about the early history of the .NET framework for my pet project on the archaeology of .NET to be published in May 2nd, 2076. They were also kind enough to explain to me some of the technicalities about the Silverlight implementation. I should be updating the wiki page with the details later.

Help Rodrigo

Apparently "Console.ReadLine" on Cygwin in some way gets line editing even from C# applications. Am not sure how this works, but Rodrigo can not live without this feature in Booish.

I remember that a few years ago there was a Unix program that will provide readline capabilities to any program that lacked them by running a process under a pseudo terminal and providing line editing functionality to tools like ftp, nslookup or any other Unix tool that did not provide its own command-line editing functionality.

In this interview Rob Connery explains how he came up with SubSonic. A tool that helps developers build applications:

That someday came when I was refactoring the Commerce Starter Kit, trying to leverage as much Data access code into the base classes as I could. I was using generics in a way that they probably weren’t designed to be used, but in the process I stumbled on a really cool pattern for generating very little code for a full data access layer. When I first got it to work I literally jumped out of my chair, swearing to myself.

I gave it about 2 weeks and tested and tested and when it didn’t break or slow down, I remember thinking "I’ve got something here!". I had been doing some Ruby/Rails stuff at the time, and I had the idea that much of that ΄love‘ could be brought over to the ASP world. Although the real joy of Rails is the language Ruby - but maybe someday RubyCLR will be finished (or some other dynamic language package for .NET) and that’s when I think SubSonic will come into it’s own.

Through Phil Haack

Once Subsonic runs on Mono, we should integrate it into MonoDevelop. Rob did a great demo of it at Mix.

I will be arriving at noon to Redmond on Sunday and spend Monday and Tuesday (skipping Wednesday) at the Compiler Lab meeting at Microsoft.

Drop me a note if you want to get together on Sunday afternoon or swing by and say hi during the meeting.

Rodrigo de Oliveira from Boo fame and JB Evain will be both there as well.

Mono and the Dynamic Language Runtime (DLR)

Thanks to Zoltan Varga, Marek Safar and Sanghyeon Seo it is now possible to run the Dynamic Language Runtime in Mono and run the new DLR-based IronPython 2.0 in Mono.

Update: Mono already supports the pre-Alpha releases of IronPython and has for a long time. What is different is that this is the DLR-based IronPython.

The changes did not make it into Mono 1.2.4 (Mono 1.2.4 was branched before MIX 07 even started, or the DLR was announced). So you will need to build Mono from the source on SVN or wait for Mono 1.2.5 to include the fixes.

We have used Paint.NET for a long time as a test for our Windows.Forms implementation. In the past I have blogged about it (here and here).

Until now, releasing the port was either painful (because I had botched the 1.x port) or not very useful (because newer versions of Paint.NET depended on 2.x features that we were missing). In Mono 1.2.4 we finally have an implementation of the *Strip classes that are heavily used in Paint.NET for the menus, toolbars and floating editing gadgets so now Paint.NET looks great.

The port is not complete, but it works. Am releasing it to the public since so many people asked about it and volunteered to contribute to complete the port.

Links:

• The project home page.
• The Porting Strategy document.
• The discussion group for the port.

Feel free to join the effort.

Our just released Mono 1.2.4 contains a class that replaces System.BitConverter: Mono.DataConvert.

We created this interface a few months ago when we ran into problems with the limitations in System.BitConverter in Mono and when we noticed that it was not possible to fix this API. Mono is switching internally to use this instead of BitConverter as it leads to subtle errors when making your code work in big-endian systems.

Mono.DataConvert offers conversions from and to .NET types to raw byte sequences:

• Host types to native byte arrays, little endian byte arrays and big endian byte arrays.
• native, little endian, big endian to Host types.
• Helper routines for Stream I/O.
• Perl-like convenience routines for packing and unpacking data structures.

I like the Perl-like interface, for example:

	// The following means:
	// Little endian encoding of a Int16, followed by an aligned
	// int32 value.
	byte [] r = DataConverter.Pack ("_s!i", 0x7b, 0x12345678);

	DataConverter.Pack ("^ii", 0x1234abcd, 0x7fadb007)
	// Result: 12 34 ab cd 7f ad b0 07
	

We also have a more traditional APIs, for example:

	// The instance-based API
	byte [] x;
	x = DataConverter.LittleEndian.GetBytes (my_double_value);
	x = DataConverter.Host.GetBytes (my_double_value);
	x = DataConverter.BigEndian.GetBytes (my_double_value);

	// Or a static version of it:
	x = DataConverter.GetBytesLE (my_double_val);
	x = DataConverter.GetBytesNative (my_double_val);
	x = DataConverter.GetBytesBE (my_double_val);
	

The code is licensed under the MIT X11 license, and we encourage developers on both Mono and .NET to use this library as opposed to the fundamentally limited and broken BitConverter.

This class is bundled as part of corlib, and is by default "internal". Since this is a small class, developers are encouraged to copy the source code into their own applications. See this for details