I will be doing the keynote at the Usenix Virtual Machine Research and Technology Symposium on May 6 and 7 in California next month.

Am also attending the Usenix Conference in Boston this summer.

Amazon Search

Nice new search engine from Amazon.

Greg was nice enough to point me to the PathScale compiler suite: a high performance compiler suite for AMD64. Pathscale's compiler is based on Open64, but has reportedly updated its C/C++ compiler frontends to a recent GCC (as opposed to the current Open64 derivative compilers).

OpenScale is lead by Fred Chow, an ex-SGI developer that created WHIRL, and later Pro64.

So as a starting point for a Managed C++ compiler, PathScale sources might be a better option.

This product shows one of the splits that people were afraid of to circumvent the GPL: the front-ends which are fairly complex (C++) has been split out from the backend and a proprietary highly optimizing compiler has been developed for the AMD64. It was just a matter of time before someone did this though.

A fresh patch and a full tarball for MCS.

Managed C++

I read Anthony Green's feedback on using Open64 as the foundation for implementing a Managed C++ compiler. I respectfully disagree with his opinion.

Let me explain why Open64 is the best starting point for implementing Managed C++.

First, the requirements:

• A C++ compiler that can generate images in the Common Intermediate Language (CIL). The intermediate language for the ECMA 335 virtual machine.
  Essentially, any existing C or C++ code needs to be compiled into CIL code.
• Expand the language to include the Managed C++ extensions, in some cases the extensions might restrict what you can do.
• High level information must propagate to the low-level CIL layer to provide introspection capabilities, attributes, and become a full CLS consumer and producer.

Since C++ is a language of titanic dimensions, it is not one that you want to reimplement. Your best choice is to pick an existing C++ compiler. In the case of the free software community, that means gcc or any of its forks.

The question is whether GCC's internal IR can be retargetted to produce code for the stack-based CIL and whether you can propagate the extra available metadata. The later seems like a problem that we would have in both Open64 and gcc.

Now what makes Open64 interesting is the fact that we can achieve the first goal without touching GCC: C and C++ compilation would be performed with the Open64 compiler down to WHIRL and then a new generic WHIRL-to-CIL compiler is produced that generates the files we want. We do not have to mess or touch any of the existing GCC internals (it is a difficult code base to work with).

The above is very similar to IKVM's JVM to CIL compiler: the input "language" is an IR, the output language is a different IR.

The fact that Open64 does not target the x86 in a way is irrelevant, because we are not interested in targeting the x86. We are interested in targeting the CIL.

If we were to use the current GCC, we would have to intercept a good stage in the compiler, and most likely deal with RTL and produce bytecodes for CIL. The RTL is hard to penetrate and deeply linked to gcc internals. WHIRL is independent, well documented, and has various tools to process, consume and analyze the various WHIRL stages.

Finally there is the point of the FSF and the GCC maintainers refusing to make structural changes to GCC on philosophical grounds. A split that would encourage the creation of proprietary front-end and back-end systems.

Not only does this mean that its better to work with the Open64 fork of GCC which has already made this pragmatic decision and is a better foundation for targeting the CIL, but our goals are more aligned than those of the GCC maintainers.

Update: Open64

The latest version of Open64 folded in the changes from Intel and ICT.

Some people think that I wrote the Mono JIT. I did not. Dietmar, Paolo, Zoltan and Martin are the people who have hacked on the JIT engine in Mono.

Recently, as a side project, I have been playing with the JIT a bit to improve some code generation here and there, and together with Ben we are learning how it works internally, it is fascinating to watch it in action.

We have been using a simple example: String.GetHashCode hoping to get things to the point where the JIT output matches GCC -O3. The code is slightly smaller than GCC, but it is wasting two registers, so in a tight loop it ends up placing a local variable on the stack which has a high impact on a tight loop.

Managed C++ and Mono

Someone on IRC pointed me a few days ago to Open64. A compiler suite that includes C, C++ and Fortrans that generate a new intermediate representation called `WHIRL'.

Some of the compiler front-ends are the GNU compiler front ends that have been re-targeted to generate WHIRL instead of RTL. This is fascinating because it means we can now seriously consider implementing the ECMA managed extensions to C++ (the job of moving to a non-RTL IR was done by SGI).

It also means that any compiler that is moved to generate WHIRL could run on any ECMA VM.

You can get the WHIRL IR documentation from here

CIL image manipulation

This topic keeps coming up: are there tools to manipulate CIL images (instrument code: pre- and post- processing, add classes, remove classes, trim methods, replace methods, append instructions) and sometime ago the fine folks at the University of Coimbra have implemented such a tool: Runtime Assembly Instrumentation Library

In Chicago

The other day Nat and I met David Vaskevitch, Microsoft's CTO. We went dancing.

Blogs are the TV of the Internet. Reading them is fun, but they are not particularly challenging.

Plenty of channels to choose from though.

My e-mail now consists of 40% of spam after it has gone through SpamAssassin, in the mornings or weekends it is 50% of it.

These days I use Mutt to manually remove the spam before I incorporate my e-mail into Evolution where the mail gets split in plenty of folders (and would make spam removal harder).

Is everyone else getting this same volume of trash?

Big News of the Day

Of course, the big news is that Microsoft has released an internal project called "WiX" as open source and is being hosted in SourceForge. A good move, since it gives a lot of people confidence that the license used is an open source license, without having to spend some quality time reading every sentence of a new license.

From the blog entry from one of the authors you can see that Stephen Walli was behind making this happen. I first saw Stephen doing a talk on startups and VCs at Usenix, on the early days of Ximian, and I remember taking notes frantically. Later I had a chance to meet him at OSCON in Portland.

As I make slow progress on Anonymous Methods on the C# compiler -due to some very embarassing assumption on my part- but most issues have been solved now. Am now implementing parameter capturing and must finish the field access reference from an anonymous method, fix a couple of regressions with iterators and commit quickly to CVS. Martin Baulig has been going through the C# generics spec item by item double-checking that his compiler has everything that we are supposed to have (Generic tests today amount to 20% of our positive tests, and 10% of our negative tests). He has finished with the generics spec as far as the ECMA working draft is concerned, but its lacking some of the latest additions that were freshly submitted in January.

Martin also worked on the new alias features in C# 2.0 and some of that code is already on the CVS tree.

Reflection.Emit is sadly not complete enough to build our generics C# compiler (even with the recently released .NET Framework 2.0 preview) so we had to resort to more hidden methods in our Reflection.Emit API which are only available to our compiler. We hope to post the list of our API needs in case Microsoft is interested in exposing these kind of changes in their implementation.

At the PDC in one of the compiler BOFs, Brad Abrams asked if people had written a compiler that followed the Common Language Specification (CLS) rules and very few people actually said they had. Although we had a C# compiler, we had not implemented the CLS rules for it.

Marek Safar recently contributed the CLS support for our C# compiler. The initial patch is about two months old, we went through various code reviews before the code could land on CVS: the traidional review for maintanability, style and performance was done before we could take his patch. Marek did a fantastic job addressing every concern that Martin and myself had. It turned out that the CLS tests have a minimal impact on compilation time.

Partial Classes is another feature that more and more people are asking about. On the surface you would think its trivial to implement it. And in fact most of it is trivial to implement, except that there is a tiny little clause in the spec that states that type lookup must be performed using the current list of using declarations. The problem is that the list of using declarations might be different from file to file, so in two different pieces of a partial class "MyType" might have a different meaning.

Partial classes are debatable an ugly hack, but at least this rules make sense in terms of doing what the programmer expects.

Implementation wise, it means that we need to do a lot of code shuffling to implement it, and it is not something that we are likely going to have by Mono 1.0.

Erik pointed me to this nice writeup on the usability of Gnome given recent debates on the topic.

I am counting the minutes for Sun to ship our Mono implementation for Solaris. Maybe we can still make it to the Solaris 10 release.

Just picture the benefits, out of the box free C# compiler on Solaris SPARC and Solaris Intel. Out of the box ASP.NET and ADO.NET on SPARC, and the Gtk# bindings for writing applications for the Java Desktop System.

Not to mention that they get the industry's most sexy JIT compiler for free.

I am walking with an extra cell phone battery in case McNealy or Schwartz decide to call me up over the weekend to discuss potential agreements (if I dont pick up, please leave a message, the wonders of ATT wireless).

Thanks to Pedro Santos I got a full list of the Mono Brainshare PPT files:

• Introducing Mono for Developers [ppt];
• Advanced Mono Development: Best Practices [ppt];
• Mono: Developing and Deploying .NET Applications for Linux [ppt];
• Mono: Rapid GUI Application Development on Linux [ppt];
• Developing ASP .NET on Linux with Mono - ainda n�o h� o ppt dispon�vel
• Using Mono to Develop the Next Version of Novell iFolder [ppt];
• Developing with the System.DirectoryServices Namespace API for Mono [ppt].

Notice that the slides that Erik and myself used were slightly different than those that we published for the event, as we continue the long tradition of `Redo your presentation the night before'.