The Jesus General makes an appeal to the Gates Foundation to rebuild Noah's Arc given that the Foundation is funding the Discovery Institute to promote ``intelligent design'' (through CrooksAndLiars).

Update: A reader comments that the funding to the Discovery Institute is for project Cascadia for transportation alternatives in the Northwest. Seems like that ruins the fun of the Jesus General post.

I enjoy reading the Jesus General: `An 11 on the Manly Scale of Absolute Gender'.

In the meantime, one of my favorite BillMon's is his post: "Bring Me the Head of Hugo Chavez".

Airlines

I just got an email from one of the Airlines I fly on notifying me of a terrific opportunity: they will now have more seats in coach. In the email they seemed to be pretty psyched about it. I mean, really excited about it they even mention that they will continue to provide snacks.

Meanwhile I was thinking `There goes the legroom'.

The new HTTP application pipeline in Mono is in my opinion a beautiful work of art. Let me explain.

The HTTP application pipeline processes an incoming HTTP request. When the request comes in it has to go through a number of steps before the request is actually handed over to the developer code: authentication, authorization, cache lookup, session state acquisition. There are a similar set of steps processed after your code is complete.

The runtime by default provides a few modules. These are listed in the machine.config file in the <httpModules> section. Mono by default includes: FormsAuthentication, UrlAuthorization, Session and the OutputCache modules. These hook up to one or more of the stages in the application pipeline.

These stages are part of the HttpApplication class. You typically have one of these per "directory" where you have deployed an ASP.NET application. As a developer, you can hook up to various points in the processing pipeline. For example, you could add your own authentication system by listing a dynamic module in the web.config file, and then hooking up to the processing pipeline:

	app.AuthenticateRequest += my_authenticator;

Now, typically when processing a request the various hooks are invoked synchronously: one after another. But in some cases your hook code might want to perform a lengthy operation, for example contacting a remote authentication server. Instead of blocking the executing thread, you want to queue the work using the CIL asynchronous framework and release the current thread from its duties so it can be used to process another request. To do this, you must register your hook using a different API:

	app.AddOnAuthenticateRequestAsync (begin_authentication, end_authentication);

Where begin_authentication is the method that will initiate the authentication asynchronously and end_authentication is the method that will be invoked when the asynchronous operation has completed. As I said before, while the asynchronous operation is pending the thread should be returned to the threadpool so it can handle the next request. When the asynchronous operation is completed it will be queued for execution again and when a thread becomes available it will execute the completion routine and resume execution.

The challenge is that you might have both synchronous and asynchronous hooks in a given application and also that at any stage the pipeline can be stopped (for example if authorization failed).

Now the challenge was to come up with a maintainable and clean design for the application pipeline. Here is where iterators came into play. The first step to make this simple was to treat all asynchronous registrations as synchronous at the event layer:

	//
	// AsyncInvoker is merely a wrapper class to hold the `b' and
	// `e' events, it does not actually invoke anything.
	// 
	public void AddAsync (BeginEventHandler b, EndEventHandler e)
	{
		AsyncInvoker invoker = new AsyncInvoker (b, e);
		Hook += new EventHandler (invoker.Invoke);
	}

The Pipeline is written like this:

	IEnumerator Pipeline ()
	{
		if (Authentication != null)
			foreach (bool stop in RunHooks (Authentication))
				yield return stop;

		if (Authorization != null)
			foreach (bool stop in RunHooks (Authorization))
				yield return stop;

		[...]
		
		done.Set ();
	}

Now the trick is how to implement RunHooks which takes a list of events, here it is:

	IEnumerable RunHooks (string stage, Delegate list)
	{
		Delegate [] delegates = list.GetInvocationList ();

		foreach (EventHandler d in delegates){
			if (d.Target != null && d.Target.GetType () is AsyncInvoker){
				AsyncInvoker ai = (AsyncInvoker) d.Target;

				ai.begin (this, EventArgs.Empty, resume, ai);
				yield return false;
			} else 
				d (this, EventArgs.Empty);

			if (stop_processing)
				yield return true;
		}
	}

Notice that we basically are using nested yield-based enumerators. The return value from "RunHooks" indicates whether the pipeline must be stopped (true) or not (false). RunHooks will execute as many synchronous operations as it can in order until it finds an asynchronous operation. At that point it initiates the operation calling the "begin" method and then it yields the control. The control is transfered to the Pipeline method which also yields and returns control to the caller.

The pipeline is kicked into action by:

	void Start (object callback)
	{
		done.Reset ();
		pipeline = Pipeline ();

		Execute ();
	}

Now the actual processing engine lives in the "Execute" method:

	void Execute ()
	{
		if (pipeline.MoveNext ())
			if ((bool)pipeline.Current){
				Console.WriteLine (prefix + "Stop requested");
				done.Set ();
			}
	}

	// This restarts the pipeline after an async call completes.
	void resume (IAsyncResult ar)
	{
		AsyncInvoker ai = (AsyncInvoker) ar.AsyncState;
		if (ai.end != null)
			ai.end (ar);

		Console.WriteLine (prefix + "Completed async operation: {0}", ar.GetType ());
		Execute ();
	}

Execute is basically using the IEnumerator interface directly while the Pipeline method uses the conveniece foreach method that iterates over every step.

The complete sample can be obtained here. This is the prototype I used as a proof of concept. The actual implementation (with different routine names) that we landed on System.Web is more complete is available here (you must scroll down).

At 1000 lines of code for the full file its one of the clean and small hacks that am most proud of.

George which kicked ass on his US Senate declaration and did not let any of the smears stick is touring the US.

Am sadly going to miss him in Boston as am out of town on September 13th, but maybe I will catch up with him later on his US Peace Tour.

Ports: August has been a good month for Mono ports. Zoltan has completed the JIT port to the IA64 platform, another 64 bit port of the Mono VM. Wade has made some RPMs based on a CVS snapshot that you can install on SLES9/IA64 (Update: Link to an external site with the snapshots).

In the meantime Paolo has made significant progress on the ARM JIT port. It is now running on both little-endian and big-endian machines. Geoff even managed to build Mono and run Mono on his 133 Mhz Linksys machine

Plenty of progress on the students from Google's Summer of Code:

XBuild: Marek reports that his implementation of msbuild (xbuild) is now able to compile itself. Congratulations!

ASP.NET Editor: Michael has merged Blago's code into the ASP.NET editor and it is now possible to create ASP.NET controls and also edit the HTML with the Mozilla editor. You can see his screenshot here. The code is available from SVN from the module `aspeditor'.

XAML Compiler: Iain has reached feature completion on his XAML compiler.

Bug Finder: Aaron report talks about some real bugs being found by his bug finding tool. His bug finding tool uses the Cecil library to read the metadata and analizes the program for known mistakes that developers incur on.

DIVA: This is probably the flashiest and most visual of all the summer of code projects. This is a video editor that uses GStreamer. I recommend you read Michael's blog which is gives a detailed description of his progress and has various videos of the various UI widgets he has created to create an open source video editor with Mono and Gtk#. Michael just integrated Mono.Cairo into his application to polish various of his widgets.

Compilers: Florian assisted Cesar in various JScript tests and the results are very promising. Mono's JScript compiler is passing more and more of the Rhino tests. Jaen checked in his Ruby.NET compiler into the repository and Florian has been assisting a bit. Discussion is happening on the #ruby.net channel on irc.gnome.org.

Work on GCC CIL has made great progress but suffered a set back as Jey broke his wrist this week.

And finally PHP.NET is on schedule to complete the goals that Raffa signed up for in the summer of code. He will continue working on the other milestones as part of his university work after the summer is over.

Cecil: Cecil continues to make progress. The best way of tracking Cecil progress is on JB's blog. It is now capable of writing assemblies and hopefully soon we can start on the Mono diet linker.

Monodoc: has received plenty of improvements: now it uses Mozilla, has font changing features, can summarize contributions, supports CSS rendering and integrated Lucene indexing to search the contents of documentation.

DataGrid: Pedro continues to make progress on the Winforms DataGrid widget. His code hopefully can soon go into SVN.

ASP.NET: We just landed a large patch to our implementation of ASP.NET.

Libgdiplus: Now that Cairo 1.0 and the API is frozen we ported Libgdiplus to use the new version of Cairo. The code is almost ready to be landed on the main branch but there are a few loose ends still required before we commit it. The good news is that Windows.Forms applications are now significantly faster.

MSDN Browser: Ben wrote a beautiful piece of code: a Gtk# client application for browsing MSDN documentation. The beauty? 220 lines of C# take a look at it and marvel at all the features Ben is using.

IronPython: Zoltan has fixed pretty much all bugs in Mono that prevented Mono from running IronPython 0.9 and its included regression test suite. We have identified a couple of problems that still must be fixed, but hopefully they will be in place by the time we release Mono 1.1.9.

MCS Just when we thought that we were done with the C# compiler Microsoft rectified a problem in the Nullable Types specification. We are all glad that they did and Hari is working on fixing this. In the meantime Marek and Atsushi continue on a quest to eliminate MCS bugs and to make the compiler stricter and more useful.

There is one regression in the usefulness of mcs that I have noticed recently. In the past the compiler would continue reporting errors for other problem areas in your source, but now it stops too early. Way too early. So when you have been hacking for a few hours straight you might need a few builds to get all the errors ironed out. You always do, but we used to need less.

The problem originates in that in a few places in the compiler we decided that if we can not compile a chunk it was best to stop processing and not continue as the compiler might depend on the types being defined and that might cause the compiler to crash. At the time, I felt that it was best to stop the compiler, but now am thinking that maybe we would serve our users best if we coped with incomplete trees in the compiler and produce more errors. It wont be easy to change this though.

Stephen Walli an ex-Microsoftie weights in on the Mono BOF at the PDC:

They published the CLR and C# language specifications as international standards through ECMA and ISO, but it still looks like a proprietary controlled technology to the customer base. Standards are a message in the marketplace that encourage multiple implementations. As long as there is only one commercial implementation, there is no standard, and the customers know it regardless of the number of trees killed to produce specifications.

He goes on saying that we should take our impromptu gathering beyond the hallway meeting into something larger:

Instead of gathering in a hallway like stubborn refugees, get a meeting room in a neighbouring hotel, or take over a bar like a Dick's Last Resort.
[..]
Hand everyone a copy of the latest SuSE distribution, and whatever Mono tools can be put together. Hand out t-shirts to the first 100 people through the door and make them GREAT t-shirts.

Last night's Jon Stewart show kept me on the brink of suspense since Jon announced that Christopher Hitchens would be the guest on the program. Hitchens being probably the best informed person and probably the only articulate person backing up the war on Iraq is a difficult bone to chew, and I knew this was going to be an interesting interview.

Jon sometimes chooses to not confront head-on his guests on the show when they representing dissenting ideas. With Hitchens it was different, he asked directly "explain to me why I am wrong" and had the best comments on the debate.

The episode was fantastic, if you missed it Crooks and Liars has the interview and in my opinion one of the best openings on talking points.

My other show

Nat recently introduced me to Discovery Channel's Mythbusters a TV show that looks at urban legends and looks at how plausible those myths really are. Am hooked on it.

English

Not being a native speaker means that sometimes my spelling is not great. I depend on Emacs ispell and Evolutions speller for many things, but recently I have started using Google's "define:word" to look up word definitions. Its just two keystrokes away: C-j define:word ENTER.

There are many new things in the new Mono that are worth showing and talking about to folks at the PDC this year.

Am afraid this year the Mono BOF will not be accepted again at the Microsoft PDC. My submission was reviewed on Tuesday and it has still not shown up in the voting list for BOFs, chances are voting for it will only happen (if it happens at all) on Saturday/Sunday, clearly the worst days to get any votes on.

At the last PDC the Mono BOF had the largest number of votes when half the spots were still available and it got dropped out of the list. When I asked the various people in charge what happened they kept pointing fingers at someone else until it reached full circle. Nobody could tell me why the most voted BOF proposal did not get selected. I would be happy with an honest answer even if it is "We do not want to promote open source/Mono/Novell" instead I heard a number of variations on "The problem is that `New frontiers for 6502 assembly language in the copy-editing industry had more votes'" (it didnt).

Anyways, this PDC ts looking just like the last one. So it is time to get ready for a Mono meeting like we hadthe last time: in the middle of the hallway. Last time we picked a spot in the middle of two concourses which had enough space and chairs to hold our meeting (about 80-100 people). I will do some scouting on Monday and find a good spot and a good time to hold the meeting and announce a meeting place here.

If you get to the PDC early, and you know the spot where we had the meeting the last time and you can provide directions, please email me.

Am on Google IM now, [email protected]

Please only request to be added if we have IMed, talked in person or emailed each other in the past.

Instructions for Gnome/Gaim users: here. Google's IM is based on the open Jabber protocol which not only allows for many implementations and creative uses of IM, but in addition Jabber is an interesting protocol that can be used to route arbitrary XML messages across the internet (not necessarily IM only).

The protocol has now a big service provider with a big user base that will help realize its potential.

Robert Fisk has gone back to Baghdad and reports in a series of articles on life outside the green zone. Fisk refuses to be an "hotel journalist" a problem that starts as he reported on his first day back in Baghdad as:

Once you let Iraqis buy your food on the streets, tell you what people are saying, come back to you with their observations, you have entered the pointless hothouse of hotel journalism, the reporter with the mobile phone trapped in his room who might as well be broadcasting or writing from Co Mayo.

His Pity the Nation book has many colorful stories of hotel journalism as practiced by many in the days of the Lebanon civil war: from the bar at the Commodore Hotel when things got risky to reporting on the crisis from the nearby Island of Cyprus.

There is a new twist to "hotel journalism" as things go from bad to worse in Iraq. The Lebanon years of Hotel-journalism are gone, replaced now with Prison-journalism:

I head off to the Palestine Hotel where one of the largest Western news agencies has its headquarters. I take the lift to an upper floor only to be met by a guard and a vast steel wall which blocks off the hotel corridor. He searches me, sends in my card and after a few minutes an Iraqi guard stares at me through a grille and opens an iron door.

I enter to find another vast steel wall in front of me. Once he has clanged the outer door shut, the inner door is opened and I am in the grotty old hotel corridor.

The reporters are sitting in a fuggy room with a small window from which they can see the Tigris river. One of the American staff admits he has not been outside "for months". An Arab reporter does their street reporting; an American travels around Iraq - but only as an "embed" with US troops. No American journalists from this bureau travel the streets of Baghdad. This is not hotel journalism, as I once described it. This is prison journalism.

He has been reporting back from the streets of Baghdad since August 12. Where accidents and death are treated in a way similar to Terry Gillian's Brazil's futuristic state: fill form completely.

Billmon predicts that the new Iraqi constitution has enough sectarian and non-secular elements that will lead to a Lebanon-like civil war. The signs are there already:

In all - and this was only an initial count - 43 civilians were killed and more than 80 wounded in the deadliest bombing in Baghdad this month.
[..]
For once, it seemed, there were no suicide bombers involved, just old-fashioned car bombs, packed with explosives to kill the largest number of innocents in the least possible time.

Most troublesome is the new culture of revenge and the new culture of kidnapping which are emerging in one of the bloodiest months of the occupation:

"I consider this a quiet day," one of the mortuary officials said to me as we stood close to the dead. So in just 36 hours - from dawn on Sunday to midday on Monday, 62 Baghdad civilians had been killed. No Western official, no Iraqi government minister, no civil servant, no press release from the authorities, no newspaper, mentioned this terrible statistic. The dead of Iraq - as they have from the beginning of our illegal invasion - were simply written out of the script. Officially they do not exist.

Thus there has been no disclosure of the fact that in July 2003 - three months after the invasion - 700 corpses were brought to the mortuary in Baghdad. In July of 2004, this rose to around 800. The mortuary records the violent death toll for June of this year as 879 - 764 of them male, 115 female. Of the men, 480 had been killed by firearms, along with 25 of the women. By comparison, equivalent figures for July 1997, 1998 and 1999 were all below 200.

As for the constitution, I asked an old Iraqi friend what he thought yesterday. "Sure, it'ss important," he said. "But my family lives in fear of kidnapping, I'm too afraid to tell my father I work for journalists, and we only have one hour in six of electricity and we can't even keep our food from going bad in the fridge. Federalism? You can't eat federalism and you can't use it to fuel your car and it doesn't make my fridge work."

And then someone told me the other day, and am not kidding you "at least they have democracy now". Fisk listens to a military commander weight on the bus station bombing:

And that night, I flip on the television again and find the local US military commander in the Sadr City district of Baghdad - close to the bus station - remarking blithely that while local people had been very angry, they supported the local "security" forces (ie the Americans) and were giving them more help than ever and that we were - wait for it - "on the path to democracy".

You cant make stuff up like this.

I started blogging today about Utah's events. But they now seem minor issues.

Last week I broke my elbow when I fell off my bike. The humerous "tip" got broken and required surgery and three large screws to hold it in place. I wish at least I had broken it as part of a facinating story, involving a remote country, spies and a distress call. But all I got is "I was coming back from work and fell".

I have tried to reconstruct the tragic events of 8/12 to find the failure on the system. But I have come up empty handed so far.

What I do know is that in the morning I used Nat's air pump on my bike and got a kick out of it. I remember thinking "I think I over inflated these bad boys".

That afternoon after a frantic hacking expedition removing "throw NotImplementedExceptions" from our new code and filing some compiler bugs, I realized I only had 20 minutes left to meet my wife on bthe opposite side of the river.

The lock holder had loosened up, I took the lock out and put it on my back pack.

Within 10 meters of getting on the bike I landed emarasingly in front of a bunch of guests of the Residence Inn, the hotel next door to the office. Being the proud man that I am, I picked the bike and got moving right away, took me a while to realize that I could not move my left arm.

My doctors have been awesome. My only complain is that I had to wait six days to get my surgery done and on the ER it took them the best part of four hours to give me a pain killer.

I started right away my rehabilitation. Took me almost two days to get my hand to lift again, and my fingers are moving now, but they cant really exert any pressure. My nose has more strength. I also painfully learned that you are supposed to keep your affected limbs above your hearth, so even if the fingers can move (and I managed to type a paragraph the other day) I cant dfind a position that wont make my hand swell.

This comes at a very bad time: this is the third year on a row that I have missed FooCamp; The Google sponsored Mono students are progressing by leaps and bounds on their projects, Mono is getting ported to two new platforms, Minosse was completed. and my hacking has stopped for the time being. In particular, am very proud of a new htp pipeline design that we are about to land in Mono which uses iterators extensively to maximize the use of the threadpool. Luckly it got done before the accident.

Reading books has also slown down, its very uncomfortable so am doing most of my reading from the computer. I almost finish reading all the web.

The new version of Gtk+, a library used to build GUI applications on Unix and the heart of the Gnome desktop ñwas announced this weekend.

This is a major upgrade for developers creating GUI applications for Linux and Gnome as it makes it simpler to create visually reach applications. This new version of Gtk+ is built on top of the 2D graphics engine Cairo: every widget is now written using Cairo operations and most importantly developers can now draw their own widgets using the PDF-like rendering model offered by Cairo.

Historically, the most painful part of building Gnome applications was the drawing layer: any interesting visualization or custom widget that needed to draw had to deal with some fairly low level APIs designed in the 80's: visuals, colormaps, GCs, collor allocation, resource acquisition and release (brushes, color resources, pens, patterns). With Cairo all of these problems will become history.

In addition to the productivity boost and the innovative potential to create visually rich applications, Cairo also brings to the end user nice touches like anti-aliased rendering for a more pleasant experience. Gtk builds on this new functionality to bring vector-based themes to the desktop as well.

I just noticed the new notification UI for iFolder. I love the balloon:

iFolder now ships with a sample server for those of you not lucky enough to have your sysadmins run one. The HOWTO is here.

Atsushi Enomoto has a great list of hints on developers using XML and Mono's XML if they care about performance, his list is available here.

String Collation

Mono originally was using IBM's Classes for Unicode (ICU) library. A C library that provides many tools to handle internationalized strings (like comparing strings, finding substrings, handling case sensitivity in a culture-aware fashion) and so on.

Basically all the managed code in Mono would call into the C runtime and the C runtime would use ICU's functionality to carry out the job. Unluckly Microsoft's behavior of the unicode operations differed from ICU implementation and the fixes that we applied in our wrapper code that use ICU were insufficient to provide the same semantics. Developers were running into various unexpected problems and erratic behavior that came out of our mapping which prompted us first discourage the use of ICU, and later to completely disable the ICU support code in Mono.

A few months ago, Atsushi was wrapping up his work on System.XML 2.x and asked me what should he look into as his next task. I asked Atsushi to look into implementing a replacement for ICU that we could use for Mono. He took this challenge very seriously and this past week he finally landed the new string collation code in the repository.

His latest blog post has some performance information and he links to the various posts that detail his quest into implementing string collation for Mono.

On Friday I will be presenting some of the recent work being done at Novell on improving Linux including Beagle, Xgl and other Mono-based applications.

There are a few Mono sessions and a Mono BOF that am planning on attending to discuss recent developments.