During the last HackWeek, I had a chance to work on a project that we had been discussing in the Mono team: an interactive C# shell.

The idea was simple: create an interactive C# shell by altering the compiler to generate and execute code dynamically as opposed to merely generating static code.

The result is the csharp command. The command provides an read-eval-print loop for entering C# statements and expressions:

	csharp> 1;
	1
	csharp> "Hello, World".IndexOf (",");
	5
	csharp> Console.WriteLine ("Hello");
	Hello
	

Statements are executed; Expressions are evaluated, and the result displayed. There is support for rendering arrays, IEnumerables and dictionaries specially, consider the following C# 3 declaration:

	csharp> new Hashtable () { { "/bin", "executable files" }, { "/etc", "configuration files" } };
	

You will get this back when rendered:

	{{ "/tmp", "Temporary files" }, { "/bin", "executable files" }}
	

Statements can span multiple lines; In those cases the interactive shell will use a different prompt to indicate that more input is expected, for example:

	csharp> 1 +
	      > 2;
	3
	csharp>
	

One of the main advantages of this shell is that you can try out your LINQ expressions directly on the shell, for example, the following query works on the result from Directory.GetFiles:

	csharp> using System.IO;
	csharp> var last_week = DateTime.Now - TimeSpan.FromDays (7);
	csharp> from f in Directory.GetFiles ("/etc") 
	      >    let fi = new FileInfo (f)
	      >    where fi.LastWriteTime < last_week
              >      select f;
        { "/etc/adjtime", "/etc/asound.state", "/etc/ld.so.cache",
	"/etc/mtab", "/etc/printcap", "/etc/resolv.conf" }
	

The LINQ expressions are not limited to working on IEnumerables, you can also use LINQ to XML or LINQ to any database provider supported by Mono by loading the assembly:

	csharp> LoadLibrary ("System.Xml.Linq");
	csharp> using System.Xml.Linq;
	csharp> var xml = new XElement("CompilerSources",
	      >   from f in Directory.GetFiles ("/cvs/mcs/mcs")
	      >   let fi = new FileInfo (f)
	      >   orderby fi.Length
	      >  select new XElement ("file", new XAttribute ("name", f), new XAttribute ("size", fi.Length)));
	csharp> xml;
	<CompilerSources>
	  <file name="/cvs/mcs/mcs/mcs.exe.config" size="395" />
	  <file name="/cvs/mcs/mcs/gmcs.exe.config" size="464" />
	  <file name="/cvs/mcs/mcs/OPTIMIZE" size="498" />
	  <file name="/cvs/mcs/mcs/lambda.todo" size="658" />
	  <file name="/cvs/mcs/mcs/smcs.exe.sources" size="726" />
	  [...]
	</CompilerSources>
	

A differences between csharp and the C# language is that I felt that for interactive use, it would be important to change the type of a variable, so the following is allowed:

	csharp> var a = 1;
	csharp> a;
	1
	csharp> a.GetType ();
	System.Int32
	csharp> var a = "Foo";
	csharp> a;
	"Foo"
	csharp> a.GetType ();
	System.String
	csharp>  
	

To load code interactive I added two methods: LoadAssembly and LoadPackage.

LoadAssembly is the equivalent of passing the -r command line argument to csharp or mcs, this example shows System.Xml.Linq in use:

csharp> LoadAssembly ("System.Xml.Linq");
csharp> using System.Xml.Linq;
csharp> XDocument doc = new XDocument(
      >     new XDeclaration("1.0", "utf-8", "yes"),
      >     new XComment("Sample RSS Feed"),
      >     new XElement("rss", 
      >         new XAttribute("version", "2.0"),
      >         new XElement ("channel",
      >             new XElement("title", "RSS Channel Title"),
      >             new XElement("description", "RSS Channel Description."),
      >             new XElement("link", "https://tirania.org"),
      >             new XElement("item",
      >                 new XElement("title", "First article title"),
      >                 new XElement("description", "First Article Description"),
      >                 new XElement("pubDate", DateTime.Now.ToUniversalTime()),
      >                 new XElement("guid", Guid.NewGuid())),
      >             new XElement("item",
      >                 new XElement("title", "Second article title"),
      >                 new XElement("description", "Second Article Description"),
      >                 new XElement("pubDate", DateTime.Now.ToUniversalTime()),
      >                 new XElement("guid", Guid.NewGuid()))
      >             )
      >         )
      >      );
	

The variable doc is then rendered:

csharp> doc;
<?xml version="1.0" encoding="utf-16" standalone="yes"?>
<!--Sample RSS Feed-->
<rss version="2.0">
  <channel>
    <title>RSS Channel Title</title>
    <description>RSS Channel Description.</description>
    <link>https://tirania.org</link>
    <item>
      <title>First article title</title>
      <description>First Article Description</description>
      <pubDate>9/8/2008 5:13:34 PM</pubDate>
      <guid>bc6825ab-f1ab-4347-ad6e-3cf076011379</guid>
    </item>
    <item>
      <title>Second article title</title>
      <description>Second Article Description</description>
      <pubDate>9/8/2008 5:13:34 PM</pubDate>
      <guid>a474b2bb-deba-4973-9581-762857b24b53</guid>
    </item>
  </channel>
</rss>

	

LoadPackage is the equivalent of invoking the compiler with the -pkg: flag. This is a Mono-ism that integrates Mono libraries with Unix's pkg-config. Packages allow definitions and multiple assemblies to be loaded in a single call, for example, this loads the Gtk# 2.0 package:

	csharp> LoadPackage ("gtk-sharp-2.0");
	csharp> using Gtk;
	csharp> Application.Init ();
	csharp> var b = new Button ("Hello Interactive Shell");
	csharp> var w = new Window ("So cute!");
	csharp> b.Clicked += delegate { Application.Quit (); };
	csharp> w.Add (b);
	csharp> w.ShowAll ();
	csharp> Application.Run ();
	

Pleasure

This shell has been incredibly useful to debug things in the last few weeks, as it avoids the tedious typing to try out APIs and to see what some function might return. Launch csharp and test things away.

To improve the experience of the command line editing, I wrote a managed readline replacement, it provides history, keyboard navigation and searching.

Also the System, System.Linq, System.Collections, and System.Collections.Generic namespaces are imported by default.

Additionally, the csharp command will load any assemblies and C# scripts that you have in the ~/.config/csharp directory. If you are working constantly say on LINQ, you can put there all of your using and LoadLibrary statements.

Availability

The csharp command will be available in Mono 2.2, which is still a few months away, or it can be obtained by compiling Mono from SVN today.

Interactive Commands

Currently there are a few static methods and properties that you can invoke from the command line, like the "help" property that will display some help, and the "quit" property that terminates the shell as well as things like LoadLibrary and LoadPackage that I have described before. A complete list is available on the CsharpRepl page.

I am interested in hearing which other features should be added to the shell. I think we need some special commands to describe types and objects, like monop.

Additionally, currently values are displayed by using ToString(), but perhaps we need a method Inspect(object) that would show the values of all public fields and properties, like a debugger would.

Which other commands should be added?

Update: Microsoft has now changed the license to the MEF to the open source MS-PL license. Please see the follow up post for details.

As a .NET developer, you should avoid using the newly released Managed Extensibility Framework as its license prevents its use beyond the Windows platform. This will prevent your .NET software from running on Linux or MacOS in the future.

Luckily, there is a cross platform solution available today that has no platform limitations: Mono.Addins, a technology inspired by Eclipse's own plugin system. We have tutorials, reference manuals, API docs, our FAQ our public groups and multiple large applications with source code available that you can learn from (MonoDevelop, Banshee, F-Spot and Gnome-Do).

The rule obviously applies to any new APIs that are built for .NET as they are not immediately available for Mono. But unlike the binary-only APIs, these half-open source code releases pose additional problems for the open source CLI:

• More people are exposed to the source code, preventing them from working on a fully open source implementation of it.
• There is a smaller desire from the community to reimplement the code, as it is relatively easy to just use the existing implementation or ignore the issue for the time being.
• Some folks might not care about the license restriction, and ignore it altogether. A practice I do not endorse.

There are two additional issues that are worth pointing out. Should non-open source libraries be hosted on CodePlex to begin with? CodePlex is branded as the "Open Source Project Hosting" from Microsoft, yet, this license is clearly not open source and does not qualify as open source according to the definition and is a clear violation of Codeplex.Com's requirements:

The above link points to http://en.wikipedia.org/wiki/Open_source_licenses.

MEF should be pulled out of the CodePlex site along with all other platform-limiting software, like ASP.NET MVC (h/t to gblock for pointing out that MVC is also violating the terms). Unless CodePlex cooks up a special exception that some software and restrictions are more equal than others.

The second point that is worth making is that picking licenses like the MS-LPL for .NET software is shooting the .NET community in the foot. The LPL license is obviously an effort to tie things into the Windows platform, putting company first, community and developers second.

The MS-LPL is a poisonous license, do not use it (do not confuse with the MS-PL which is a decent license, the extra "L" makes a big difference).

Update: gblock points out that CodePlex technically only requires a "license" to be picked, but does not require it to be OSI compliant. But even if the wording on the FAQ does allow for that interpretation, it is misleading as there is a link to OSI licenses next to it, and the main page clearly states that Codeplex is an "Open Source Project Hosting" site. Capital "O" and capital "S".

The "CodePlex Basics: Getting Started" page also points to the Open Source License Definition, there are no words about custom licenses or about hosting proprietary code with source code available.

Update2: Glenn has reached out to the community and clearly wants to improve the status quo when it comes to MEF, open source, CodePlex and the official policies. Good luck to Glenn.

It is also worth pointing out that Glenn was one of the early adopters and advocates of the MS-PL at Microsoft (to clarify: the MS-PL is an open source license).

The following screenshots shows Visual Studio debugging a remote Mono process running on a Linux box.

Breakpoints, current line, and stack traces.

The setup works like this: you run a debugging server on Linux. On Windows you install a Visual Studio extension that provides a Debugging Engine and the configuration tools to start your application.

Then you start your application from Visual Studio (Tools/Launch with Mono) which will send the binaries and debug information over to Linux and execute it.

Showing the disassembled code generated by Mono's JIT

The core of the above work is to get the Windows to Linux debugging functionality enabled. We will have more news when the debugging experience is more complete and we are approaching the time to test drive it.

If you are interested in trying it out, sign up for our preview on our web site.

Debugging Support

Mono 2.0 will for the first time include a debugger, the command line mdb command.

Beyond this, we want to offer GUI debugging. The native solution is being built on top of MonoDevelop and will be available when we release MonoDevelop 2.0. It will work on both Linux and MacOS hosts.

The second GUI engine will be the above Visual Studio plugin for developers that use Windows as their main OS and Visual Studio as their IDE.

More Visual Studio Integration

In addition to this work to integrate Visual Studio with Mono for remote debugging, thanks to the Summer of Code (Ed Ropple) and Jonathan Pobst's work in this hack week we got some support to run applications on Mono/Windows and run our Mono Migration Analysis tool.

Jonathan's work is available today as an MSI, check out his post for more details.

You can also check Ed's summer of code report on CloverLeaf.

In an age where the Unix shell is more relevant every passing minute, we need to have proper command line editing tools everywhere.

For a project of mine, this weekend I put together a command-line editing class for .NET shell applications. The Mono.Terminal.LineEdit class can be used by shell applications to get readline-like capabilities without depending on any external C libraries.

To use it, just do:

	using Mono.Terminal;

	LineEditor le = new LineEditor ("MyApp");
	while ((s = le.Edit ("prompt> ", "")) != null)
		Console.WriteLine ("You typed: " + s);
	
	

It supports the regular cursor editing, Emacs-like editing commands, history, incremental search in the history as well as history loading and saving.

The code is self-contained, and can be easily reused outside of my project. To use it, you just need to include the getline.cs file in your project. This is built on top of System.Console, so it does not have external library dependencies and will work on both Mono and .NET (finally bringing joy to people using command-line applications that use Console.ReadLine).

It is licensed under the MIT X11 license and the Apache 2.0 license so there are no annoying licensing issues and can be mixed with anything out there.

Update, Jan 2016: New popup dialog for completion has been added.

Cool performance demos comparing SecondLife's LSL engine vs LSL running on Mono's VM.

Big news for the Mono team today.

Linden has started the roll out of their Mono-powered simulation servers.

Users can now opt into having their LSL scripts executed using the Mono VM (it remains an opt-in feature, because some scripts are timing sensitive and the performance increase could break code).

Some choice quotes from Jim Purbrick's blog post:

As well as providing immediate benefits, the integration of the Mono virtual machine makes many future improvements possible: the use of mainstream languages to script Second Life alongside the existing LSL language; the removal of arbitrary per-script limits on script resource usage and the use of mainstream libraries and tools for scripting to name a few.

And:

The integration of Mono is the first step in the evolution of Second Life into a true software development platform. Thank you to all the residents who have helped us take this first step.

Congrats to Linden on their launch!

The Technology

From a SecondLife developer perspective, some of the technical details about how Mono is integrated into the Second Life simulators can be found on their Wiki.

When a user opts into using Mono, a special LSL compiler that generates ECMA CLI byte codes is used. The resulting CLI byte codes are then instrumented with some magic (more below) and then the code is exectuted using the Mono VM which translated the bytecodes into native x86 code.

I find the SecondLife technology fascinating. Embedding Mono into SecondLife was not an ordinary task, it was not just a matter of linking with Mono and writing an LSL to CIL compiler.

SecondLife distributes the virtual world on hundreds of servers, and as visitors move through the virtual world, their inventory, character and scripts migrates from server to server.

This migration requires that running scripts be suspended, serialized to a database and their execution resumed on a different server. This means that all the state of a script, including the current location must be preserved while the user navigates across servers.

The technology to do this is absolutely brilliant. I strongly recommend the Lang.NET presentation that Cory and Jim did in 2006.

The first half of the video is an introduction to Second Life, the second delves into the hackery necessary to make it happen.

This are clearly hugenormous news for us, and for everyone that worked with Linden, for everyone that fixed bugs and implemented new features in Mono to run under the conditions that Linden has.

The RemObject folks have been doing some tutorials on how to build applications with Visual Studio and Interface Builder to target both Windows and MacOS with .NET and Mono respectively.

Check out their tutorial and their notes on cross platform development. The lessons in the tutorial also apply to C#-based development.

It is also worth noting that recent versions of their Oxygene compiler now support generics.

I just got my Google Android present! An awesome Pleo Dinosaur!.

So cute. SO CUTE!

Jon Skeet has an in-depth explanation of how to improve the performance of code that needs to dynamically invoke methods through reflections.

The bottom line is that if you have performance sensitive code that needs to invoke methods that you fetch from reflection, you should avoid using MethodInfo.Invoke() and instead create a delegate from the MethodInfo, and perform the invocations that way:

[...]Using a delegate invocation is only about 10% slower than direct invocation, whereas using reflection takes over 600 times as long. Of course these figures will depend on the method being called - if the direct invocation can be inlined, I'd expect that to make a significant difference in some cases.

This is a well-known trick, but Jon provides a great exploration of the subject.

Protocol Buffers for .NET

Additionally, you can see that Jon's effort to port Google's Protocol Buffers to C# are almost complete.

There are currently three separate approaches to support Protocol Buffers in .NET. Jon's effort essentially mimics the existing support for C# and integrated with the Google implementation and compilers. The other efforts have taken slightly different approaches, one of them is designed with the WCF approach in mind: use C# classes/interfaces as the actual public contract, as opposed to the .proto files.

Our first preview for Mono 2.0 is out; It has been almost six months since we branched version 1.9 so this is a gigantic update to Mono.

Many of the features are listed on the release notes, but the release notes do not even begin to capture the enormous number of fixes, performance improvements, tuning and work that went into this release.

As usual, this is our "best release ever", but it is also the longest we have gone without doing interim releases, so it is possible that we might have regressed where our test suite lacks tests. We would love to get folks to test this, with their code, and to bug reports on any issues they find before our final 2.0 release.

See our Roadmap for more details on the release schedule and the upcoming post-2.0 releases.