Credit where credit is due

by Miguel de Icaza

The csharp interactive shell and the embeddable compiler hacks could only have been possible thanks to the work that Marek Safar has put into implementing C# 3.0, cleaning up the compiler and laying down the foundation for a reusable compiler.

Some time ago, our regression test suite for the compiler was taking 20-30 minutes to run. Marek refactored the compiler so that we could reuse the same instance without launching a new process for each of our 1,500 individual tests.

Without Marek's dedication and love for the compiler, we would never have gotten here.

Posted on 11 Sep 2008


C# Eval: An Embeddable Compiler

by Miguel de Icaza

Hello Internets and C# Lovers!

After the C# Interactive Shell started working, it became obvious that we now had C#'s eval and that it would be incredibly interesting to enable people to embed a C# Eval in their application.

I did a little bit of refactoring on the codebase last night, and now we expose a few methods that allow developers to evaluate C# expressions and statements dynamically.

Usage is very simple, you must reference the `gmcs.exe' assembly, this contains the C# Eval, then you invoke the CSharpEvaluator.Evaluate method, and get the results back.

The following is a small command line calculator that takes C# expressions:

using Mono.CSharp;
using System;

class MyCalculator {
	static void Main (string [] args)
	{
		Console.WriteLine (CSharpEvaluator.Evaluate (args [0] + ";"));
	}
}
	

Compile the above, and execute:

$ mono calculator.exe '1+2'
3
	

There are a number of overload methods for Evaluate that give more control over how things are evaluated. The csharp command is now written entirely using the CSharpEvaluator API. The code is now pretty simple: pretty printing of values, dealing with partial input, loading startup files (if any) and import the default namespaces.

A few uses that come to mind:

  • Use C# as a scripting language for your application.
  • Use it for startup scripts of your own application (similar to .emacs, but with C# instead).
  • Use it to explore the state of your application live.
  • Use it to debug your application as it is running.
  • Embed the compiler on Silverlight and run C# on your web browser, similar to how Jimmy did it for Ruby.
  • Embed it into your app and expose it as a service that you can connect to, and do it live!

The four or five public methods of the API are documented in the source using C# doc tags. When Mono 2.2 goes out, we will publish docs for them.

As we announced back in April, Mono's C# compiler is now licensed under the MIT X11 license, so you can embed at will.

Manhole

Joe Shaw mentioned a few days ago that it would be nice to support something like Twisted's Manhole where your application is listening on a port and then you can connect to it and debug it. It should take a dozen lines to write it, but I am heading out for dinner now.

But if I were to do it, I would probably make the csharp command (all 240 lines of it) support a --server command line option and have it connect to the server and send strings over the wire instead of executing locally. That way you get to use the awesome getline.cs command line editor and history locally and let the server run the code.

Getting Your Hands on This

To enjoy this hack, you have to either wait for Mono 2.2 or install Mono from SVN, all of the code is checked in.

Update: Clarification

Since it was not obvious, we do support C# 3.0, which includes the entire LINQ stack in the above expression evaluator:

using Mono.CSharp;
using System.Collections;
using System;

class X {
	static void Main (string [] args)
	{
		Evaluator.Run ("using System; using System.Linq;");
		bool ress;
		object res;
		string s = Evaluator.Evaluate (
			"from x in System.IO.Directory.GetFiles (\"/etc\") where x.Length < 10 select x;",
			out res, out ress);

		foreach (var v in (IEnumerable) res){
			Console.Write (v);
			Console.Write (' ');
		}
	}
}

The above uses a LINQ expression, and this is the result in my system (files whose full pathname is less than 10 characters):

/etc/motd /etc/mtab /etc/raw /etc/rpc 

Posted on 10 Sep 2008


Enterprise Library 4.0 now Open Source

by Miguel de Icaza

Hello Internets!

Some very good news: As part of the ongoing discussion on the MS Limited Permissive License, I was doing some Google searches on it and came across the Enterprise Library 4.0 release.

Enterprise Library was a project that came out of the Patterns and Practices group at Microsoft, and they had historically used licenses that were not open source friendly. This new release is now licensed under the terms of the open-source friendly MS-PL license.

In the past, various people have asked us to implement an open source version of it, so they could use those building blocks on Unix with Mono. We never had any spare cycles to look into this, so the blocks provided by Enterprise Library were always beyond Mono users.

As it turns out, since May 2008 it has been possible to use the library with Mono on non-Windows platforms.

Now all there is left to do is to package it in a convenient form.

Posted on 09 Sep 2008


Interactive C# Shell

by Miguel de Icaza

During the last HackWeek, I had a chance to work on a project that we had been discussing in the Mono team: an interactive C# shell.

The idea was simple: create an interactive C# shell by altering the compiler to generate and execute code dynamically as opposed to merely generating static code.

The result is the csharp command. The command provides an read-eval-print loop for entering C# statements and expressions:

	csharp> 1;
	1
	csharp> "Hello, World".IndexOf (",");
	5
	csharp> Console.WriteLine ("Hello");
	Hello
	

Statements are executed; Expressions are evaluated, and the result displayed. There is support for rendering arrays, IEnumerables and dictionaries specially, consider the following C# 3 declaration:

	csharp> new Hashtable () { { "/bin", "executable files" }, { "/etc", "configuration files" } };
	

You will get this back when rendered:

	{{ "/tmp", "Temporary files" }, { "/bin", "executable files" }}
	

Statements can span multiple lines; In those cases the interactive shell will use a different prompt to indicate that more input is expected, for example:

	csharp> 1 +
	      > 2;
	3
	csharp>
	

One of the main advantages of this shell is that you can try out your LINQ expressions directly on the shell, for example, the following query works on the result from Directory.GetFiles:

	csharp> using System.IO;
	csharp> var last_week = DateTime.Now - TimeSpan.FromDays (7);
	csharp> from f in Directory.GetFiles ("/etc") 
	      >    let fi = new FileInfo (f)
	      >    where fi.LastWriteTime < last_week
              >      select f;
        { "/etc/adjtime", "/etc/asound.state", "/etc/ld.so.cache",
	"/etc/mtab", "/etc/printcap", "/etc/resolv.conf" }
	

The LINQ expressions are not limited to working on IEnumerables, you can also use LINQ to XML or LINQ to any database provider supported by Mono by loading the assembly:

	csharp> LoadLibrary ("System.Xml.Linq");
	csharp> using System.Xml.Linq;
	csharp> var xml = new XElement("CompilerSources",
	      >   from f in Directory.GetFiles ("/cvs/mcs/mcs")
	      >   let fi = new FileInfo (f)
	      >   orderby fi.Length
	      >  select new XElement ("file", new XAttribute ("name", f), new XAttribute ("size", fi.Length)));
	csharp> xml;
	<CompilerSources>
	  <file name="/cvs/mcs/mcs/mcs.exe.config" size="395" />
	  <file name="/cvs/mcs/mcs/gmcs.exe.config" size="464" />
	  <file name="/cvs/mcs/mcs/OPTIMIZE" size="498" />
	  <file name="/cvs/mcs/mcs/lambda.todo" size="658" />
	  <file name="/cvs/mcs/mcs/smcs.exe.sources" size="726" />
	  [...]
	</CompilerSources>
	

A differences between csharp and the C# language is that I felt that for interactive use, it would be important to change the type of a variable, so the following is allowed:

	csharp> var a = 1;
	csharp> a;
	1
	csharp> a.GetType ();
	System.Int32
	csharp> var a = "Foo";
	csharp> a;
	"Foo"
	csharp> a.GetType ();
	System.String
	csharp>  
	

To load code interactive I added two methods: LoadAssembly and LoadPackage.

LoadAssembly is the equivalent of passing the -r command line argument to csharp or mcs, this example shows System.Xml.Linq in use:

csharp> LoadAssembly ("System.Xml.Linq");
csharp> using System.Xml.Linq;
csharp> XDocument doc = new XDocument(
      >     new XDeclaration("1.0", "utf-8", "yes"),
      >     new XComment("Sample RSS Feed"),
      >     new XElement("rss", 
      >         new XAttribute("version", "2.0"),
      >         new XElement ("channel",
      >             new XElement("title", "RSS Channel Title"),
      >             new XElement("description", "RSS Channel Description."),
      >             new XElement("link", "http://tirania.org"),
      >             new XElement("item",
      >                 new XElement("title", "First article title"),
      >                 new XElement("description", "First Article Description"),
      >                 new XElement("pubDate", DateTime.Now.ToUniversalTime()),
      >                 new XElement("guid", Guid.NewGuid())),
      >             new XElement("item",
      >                 new XElement("title", "Second article title"),
      >                 new XElement("description", "Second Article Description"),
      >                 new XElement("pubDate", DateTime.Now.ToUniversalTime()),
      >                 new XElement("guid", Guid.NewGuid()))
      >             )
      >         )
      >      );
	

The variable doc is then rendered:

csharp> doc;
<?xml version="1.0" encoding="utf-16" standalone="yes"?>
<!--Sample RSS Feed-->
<rss version="2.0">
  <channel>
    <title>RSS Channel Title</title>
    <description>RSS Channel Description.</description>
    <link>http://tirania.org</link>
    <item>
      <title>First article title</title>
      <description>First Article Description</description>
      <pubDate>9/8/2008 5:13:34 PM</pubDate>
      <guid>bc6825ab-f1ab-4347-ad6e-3cf076011379</guid>
    </item>
    <item>
      <title>Second article title</title>
      <description>Second Article Description</description>
      <pubDate>9/8/2008 5:13:34 PM</pubDate>
      <guid>a474b2bb-deba-4973-9581-762857b24b53</guid>
    </item>
  </channel>
</rss>

	

LoadPackage is the equivalent of invoking the compiler with the -pkg: flag. This is a Mono-ism that integrates Mono libraries with Unix's pkg-config. Packages allow definitions and multiple assemblies to be loaded in a single call, for example, this loads the Gtk# 2.0 package:

	csharp> LoadPackage ("gtk-sharp-2.0");
	csharp> using Gtk;
	csharp> Application.Init ();
	csharp> var b = new Button ("Hello Interactive Shell");
	csharp> var w = new Window ("So cute!");
	csharp> b.Clicked += delegate { Application.Quit (); };
	csharp> w.Add (b);
	csharp> w.ShowAll ();
	csharp> Application.Run ();
	

Pleasure

This shell has been incredibly useful to debug things in the last few weeks, as it avoids the tedious typing to try out APIs and to see what some function might return. Launch csharp and test things away.

To improve the experience of the command line editing, I wrote a managed readline replacement, it provides history, keyboard navigation and searching.

Also the System, System.Linq, System.Collections, and System.Collections.Generic namespaces are imported by default.

Additionally, the csharp command will load any assemblies and C# scripts that you have in the ~/.config/csharp directory. If you are working constantly say on LINQ, you can put there all of your using and LoadLibrary statements.

Availability

The csharp command will be available in Mono 2.2, which is still a few months away, or it can be obtained by compiling Mono from SVN today.

Interactive Commands

Currently there are a few static methods and properties that you can invoke from the command line, like the "help" property that will display some help, and the "quit" property that terminates the shell as well as things like LoadLibrary and LoadPackage that I have described before. A complete list is available on the CsharpRepl page.

I am interested in hearing which other features should be added to the shell. I think we need some special commands to describe types and objects, like monop.

Additionally, currently values are displayed by using ToString(), but perhaps we need a method Inspect(object) that would show the values of all public fields and properties, like a debugger would.

Which other commands should be added?

Posted on 08 Sep 2008


Avoid the Managed Extensibility Framework.

by Miguel de Icaza

Update: Microsoft has now changed the license to the MEF to the open source MS-PL license. Please see the follow up post for details.

As a .NET developer, you should avoid using the newly released Managed Extensibility Framework as its license prevents its use beyond the Windows platform. This will prevent your .NET software from running on Linux or MacOS in the future.

Luckily, there is a cross platform solution available today that has no platform limitations: Mono.Addins, a technology inspired by Eclipse's own plugin system. We have tutorials, reference manuals, API docs, our FAQ our public groups and multiple large applications with source code available that you can learn from (MonoDevelop, Banshee, F-Spot and Gnome-Do).

The rule obviously applies to any new APIs that are built for .NET as they are not immediately available for Mono. But unlike the binary-only APIs, these half-open source code releases pose additional problems for the open source CLI:

  • More people are exposed to the source code, preventing them from working on a fully open source implementation of it.
  • There is a smaller desire from the community to reimplement the code, as it is relatively easy to just use the existing implementation or ignore the issue for the time being.
  • Some folks might not care about the license restriction, and ignore it altogether. A practice I do not endorse.

There are two additional issues that are worth pointing out. Should non-open source libraries be hosted on CodePlex to begin with? CodePlex is branded as the "Open Source Project Hosting" from Microsoft, yet, this license is clearly not open source and does not qualify as open source according to the definition and is a clear violation of Codeplex.Com's requirements:

The above link points to http://en.wikipedia.org/wiki/Open_source_licenses.

MEF should be pulled out of the CodePlex site along with all other platform-limiting software, like ASP.NET MVC (h/t to gblock for pointing out that MVC is also violating the terms). Unless CodePlex cooks up a special exception that some software and restrictions are more equal than others.

The second point that is worth making is that picking licenses like the MS-LPL for .NET software is shooting the .NET community in the foot. The LPL license is obviously an effort to tie things into the Windows platform, putting company first, community and developers second.

The MS-LPL is a poisonous license, do not use it (do not confuse with the MS-PL which is a decent license, the extra "L" makes a big difference).

Update: gblock points out that CodePlex technically only requires a "license" to be picked, but does not require it to be OSI compliant. But even if the wording on the FAQ does allow for that interpretation, it is misleading as there is a link to OSI licenses next to it, and the main page clearly states that Codeplex is an "Open Source Project Hosting" site. Capital "O" and capital "S".

The "CodePlex Basics: Getting Started" page also points to the Open Source License Definition, there are no words about custom licenses or about hosting proprietary code with source code available.

Update2: Glenn has reached out to the community and clearly wants to improve the status quo when it comes to MEF, open source, CodePlex and the official policies. Good luck to Glenn.

It is also worth pointing out that Glenn was one of the early adopters and advocates of the MS-PL at Microsoft (to clarify: the MS-PL is an open source license).

Posted on 07 Sep 2008


Using Visual Studio to Debug Mono

by Miguel de Icaza

The following screenshots shows Visual Studio debugging a remote Mono process running on a Linux box.

Breakpoints, current line, and stack traces.

The setup works like this: you run a debugging server on Linux. On Windows you install a Visual Studio extension that provides a Debugging Engine and the configuration tools to start your application.

Then you start your application from Visual Studio (Tools/Launch with Mono) which will send the binaries and debug information over to Linux and execute it.

Showing the disassembled code generated by Mono's JIT

The core of the above work is to get the Windows to Linux debugging functionality enabled. We will have more news when the debugging experience is more complete and we are approaching the time to test drive it.

If you are interested in trying it out, sign up for our preview on our web site.

Debugging Support

Mono 2.0 will for the first time include a debugger, the command line mdb command.

Beyond this, we want to offer GUI debugging. The native solution is being built on top of MonoDevelop and will be available when we release MonoDevelop 2.0. It will work on both Linux and MacOS hosts.

The second GUI engine will be the above Visual Studio plugin for developers that use Windows as their main OS and Visual Studio as their IDE.

More Visual Studio Integration

In addition to this work to integrate Visual Studio with Mono for remote debugging, thanks to the Summer of Code (Ed Ropple) and Jonathan Pobst's work in this hack week we got some support to run applications on Mono/Windows and run our Mono Migration Analysis tool.

Jonathan's work is available today as an MSI, check out his post for more details.

You can also check Ed's summer of code report on CloverLeaf.

Posted on 04 Sep 2008


getline.cs: Partying like its 1988

by Miguel de Icaza

In an age where the Unix shell is more relevant every passing minute, we need to have proper command line editing tools everywhere.

For a project of mine, this weekend I put together a command-line editing class for .NET shell applications. The Mono.Terminal.LineEdit class can be used by shell applications to get readline-like capabilities without depending on any external C libraries.

To use it, just do:

	using Mono.Terminal;

	LineEditor le = new LineEditor ("MyApp");
	while ((s = le.Edit ("prompt> ", "")) != null)
		Console.WriteLine ("You typed: " + s);
	
	

It supports the regular cursor editing, Emacs-like editing commands, history, incremental search in the history as well as history loading and saving.

The code is self-contained, and can be easily reused outside of my project. To use it, you just need to include the getline.cs file in your project. This is built on top of System.Console, so it does not have external library dependencies and will work on both Mono and .NET (finally bringing joy to people using command-line applications that use Console.ReadLine).

It is licensed under the MIT X11 license and the Apache 2.0 license so there are no annoying licensing issues and can be mixed with anything out there.

Update, Jan 2016: New popup dialog for completion has been added.

Posted on 26 Aug 2008


Second Life Demos

by Miguel de Icaza

Cool performance demos comparing SecondLife's LSL engine vs LSL running on Mono's VM.

Posted on 21 Aug 2008


SecondLife rolls out Mono-powered servers

by Miguel de Icaza

Big news for the Mono team today.

Linden has started the roll out of their Mono-powered simulation servers.

Users can now opt into having their LSL scripts executed using the Mono VM (it remains an opt-in feature, because some scripts are timing sensitive and the performance increase could break code).

Some choice quotes from Jim Purbrick's blog post:

As well as providing immediate benefits, the integration of the Mono virtual machine makes many future improvements possible: the use of mainstream languages to script Second Life alongside the existing LSL language; the removal of arbitrary per-script limits on script resource usage and the use of mainstream libraries and tools for scripting to name a few.

And:

The integration of Mono is the first step in the evolution of Second Life into a true software development platform. Thank you to all the residents who have helped us take this first step.

Congrats to Linden on their launch!

The Technology

From a SecondLife developer perspective, some of the technical details about how Mono is integrated into the Second Life simulators can be found on their Wiki.

When a user opts into using Mono, a special LSL compiler that generates ECMA CLI byte codes is used. The resulting CLI byte codes are then instrumented with some magic (more below) and then the code is exectuted using the Mono VM which translated the bytecodes into native x86 code.

I find the SecondLife technology fascinating. Embedding Mono into SecondLife was not an ordinary task, it was not just a matter of linking with Mono and writing an LSL to CIL compiler.

SecondLife distributes the virtual world on hundreds of servers, and as visitors move through the virtual world, their inventory, character and scripts migrates from server to server.

This migration requires that running scripts be suspended, serialized to a database and their execution resumed on a different server. This means that all the state of a script, including the current location must be preserved while the user navigates across servers.

The technology to do this is absolutely brilliant. I strongly recommend the Lang.NET presentation that Cory and Jim did in 2006.

The first half of the video is an introduction to Second Life, the second delves into the hackery necessary to make it happen.

This are clearly hugenormous news for us, and for everyone that worked with Linden, for everyone that fixed bugs and implemented new features in Mono to run under the conditions that Linden has.

Posted on 20 Aug 2008


OSX Development with Oxygene, Cocoa and Mono

by Miguel de Icaza

The RemObject folks have been doing some tutorials on how to build applications with Visual Studio and Interface Builder to target both Windows and MacOS with .NET and Mono respectively.

Check out their tutorial and their notes on cross platform development. The lessons in the tutorial also apply to C#-based development.

It is also worth noting that recent versions of their Oxygene compiler now support generics.

Posted on 20 Aug 2008


« Newer entries | Older entries »