Mono's C# Compiler as a Service on Windows.

by Miguel de Icaza

The Mono team is proud to bring you a preview of C# 5.0 a few years before our friends in Building 41 do.

A snapshot of the code is available in the demo-repl.zip file. This contains the csharp.exe C# REPL shell and the Mono.CSharp.dll compiler-as-a-service assembly.

With Today's code drop all those scenarios now work:

  • Run Mono's C# compiler on .NET
  • Embed Mono's Evaluator in your C# app.
  • Use Mono's REPL with the .NET framework runtime.

Update April 28th: I have now uploaded a new version that fixes the bug that people were getting when importing other libraries. Thanks to Marek for the rapid fix.

Background

Although we have had a compiler-as-a-service framework since September of 2008 it has been so far limited to Mono users, which effectively means only Linux and OSX users could use our C# REPL and the compiler as a service.

The reason is that the Mono's C# compiler uses System.Reflection.Emit as its code generation backend. This API was fine for implementing C# 1.0 but required a few changes to support compiling its own core library (mscorlib, the equivalent of libc in C or rt in Java).

When we started to work on our C# 2.0 support, we were working on our compiler as the language was being standardized at ECMA. Our compiler evolved faster than the underlying support for Reflection and Reflection.Emit did and this lead to more Mono-specific extensions being added to our Reflection and Reflection.Emit code. The more extensions that we depended on, the fewer chances we had of running with Microsoft's runtime.

As time went by, Microsoft did improve their System.Reflection.Emit, but at that point, it was too late for us to go back to it.

This had the unfortunate side effects that our compiler could not run on the Microsoft runtime. Now, this per se is not a major problem since Windows users likely use Microsoft's CSC compiler.

But this also meant that our awesome compiler as a service could not be used by .NET developers and it also meant that our REPL shell could not be used with .NET and most importantly, neither one would run on Silverlight.

Our compiler also relied heavily on our extended System.Reflection and System.Type as its own type system. And we could not just change our APIs for Microsoft's APIs to run on the Microsoft runtime.

Today's commit at the coref replaces our dependency on System.Type and introduced our own Mono.CSharp.TypeSpec which lifts many of the restrictions that we had on System.Type. All of the hard computations that we had to do for inflating generics types are done here now and we can query the types without the backward Reflection limitations.

With today's changes, not only we became more portable, but the move from System.Type to TypeSpec and MemberCache improved the compiler performance by 30% and fixes dozens of bugs that were very hard to fix with the previous compiler architecture.

REPL Love

We have a small guide on using the C# REPL on Linux.

Caveat: both the Microsoft and the Mono C# compilers loads libraries from their "base directory", never from the Global Assembly Cache.

In Mono, the csharp command happens to be in the same directory as the rest of Mono, so loading implicit libraries just works (Loading System.Core for example). But when running our csharp command on Windows, unless you put the binary in the same location as csc.exe the REPL wont be able to load libraries using their short names.

This is particularly obnoxious because to use LINQ you need to use the full assembly name, or copy csharp.exe to the directory that contains System.Core so you can just do:

	csharp> using System.Linq;
	csharp> from x in "Foo" select x;
	

Future work

Public API: There is still some work left to do: we need to turn 99% of the public API in that assembly into a private API, the only class you should care about is the Mono.CSharp.Evaluator class, the rest you should consider internal.

The Evaluator API needs to evolve, right now it is a big singleton that exposes all of the variables defined as global variables. They really should be tied to some kind of context so we can support multiple and independent execution contexts on the same address space.

Chances are that we want to expose some of the internals of the compiler to the world, but we first need to figure out what makes sense.

Now that the code runs on .NET, hopefully those of you that wanted to embed it can provide us some feedback on how you would like to see this API change or even better, provide patches for it.

C# as a DLR language: Someone had suggested to also provide a hook into the DLR, so you can instantiate C# eval engines with the same API that you instantiate IronPython and IronRuby eval engines.

Silverlight support: in theory this works out of the box, but we have not tested it yet.

Contributions: Yes, we are accepting contributions to this awesome compiler.

Keybindings: Currently the csharp command line repl uses Emacs keybindings as part of my fabulous getline.cs command line editor. We are aware that developers of different faiths might find other keybindings more appropriate. We are taking patches.

Add support for IKVM Reflection: Jeroen has written a drop-in replacement for System.Reflection[.Emit] that will allow us to decouple the compile from the profile that it compiles code for (Monoistas are familiar with dmcs, smcs, gmcs and mcs; We will be able to have a single compiler).

Source Code and Hacking

The source code for Mono's C# compiler, the Compiler as a Service and the interactive REPL are all available from SVN, do a checkout from our subversion repository for the module "mcs".

Download from GitHub the "mono" module

Then open the VS2008 solution in mcs/tools/csharp. This will build the Mono.CSharp compiler as a service library and the csharp tool.

We do not typically provide VS Solution files for most of Mono, but we are making an exception for the compiler as a service to encourage .NET developers to play with it.

Update: Our compiler is MIT X11 licensed, so even developers at Microsoft can download this code and look at it. It is all kosher guys!

Posted on 27 Apr 2010