People are loving the C# Interactive Shell.

In the past people have embedded languages like Boo into their applications (MonoDevelop and Banshee for example both have options to embed Boo and a shell).

Zoltan designed a new system for Mono that allows developers to inject and execute code into running Mono applications. Paolo provided significant feedback and design guidelines for the code to be incorporated into Mono.

Thanks to both Zoltan and Paolo this functionality is now available through the Mono.Management assembly: you provide an executable and a PID, and the executable is injected and its Main method executed on the target process:

	using Mono.Attach;

	[..]

	// Get a handle to a running Mono process.
	VirtualMachine vm = new VirtualMachine (pid);

	// Load hello.exe into the target process, and run it
	vm.Attach ("/tmp/hello.exe", "");
	

You can use this for billions of things of course. You could patch running programs, you could attach logging software to a running program, or you could inject a C# evaluator into a live application and examine its state, or tweak it live.

Zoltan came up with a really cool extension to the the csharp command (this is the command-line C# Interactive Shell). The csharp command now takes an --attach command line argument and a PID.

The csharp shell can now use the Mono.Attach.VirtualMachine to injects itself into the remote process and then the client and server communicate through a couple of sockets.

For example, this is the sample that Zoltan used to pitch his idea for supporting attaching to the virtual machine. With the following you can pause a live Banshee process:

$ ps aux | grep banshee
miguel   12359 17.0  2.7 141372 55708 pts/5    Sl+  14:30   0:02 banshee-1 /usr/lib64/banshee-1/Nereid.exe
$ csharp --attach 12359
csharp> using Banshee.ServiceStack;
csharp> using Banshee.Gui;
csharp> var s = ServiceManager.Get ();
csharp> s.PlaybackActions ["PlayPauseAction"].Activate ();
	

All of the code is now on SVN, you need both the mono and mcs modules from trunk.

A GUI Shell

The above commands are a tiny bit risky and are also limited to the shell.

The above commands will execute on a separate thread from the application, and any commands that you execute would be executed on a separate thread which could corrupt the state of your application.

So this weekend, I wrote a tool that integrates with Gtk# applications, its called gsharp and you can find it in the mono-tools module (it borrows some code from Banshee).

gsharp is a Gtk# version of the csharp command. What is important is that it also supports injection of the code on other programs, but makes sure that all the code executes in the Gtk# thread, by issuing all of its commands from the Gtk# idle handler. This means that it is safe to use gsharp with your Gtk# applications.

GUI version of the tool.

Here you can select which project you want to inject the GUI into:

Needs some work, show process names.

This version shows the gsharp tool attached to F-Spot, as I am not very familiar with the codebase, I can not do very much with it:

We will need to implement one of these as well for Windows.Forms applications. This should luckily be easy to do as most of the smarts live in the Mono.CSharp assembly (our embeddable compiler).

Security of Agents

A couple of weeks ago, I asked for people to weigh in on a security concern for temporary files. This was for Zoltan's attach functionality.

The code is now implemented and I would love if security experts could do a source code audit of our implementation. And validate whether our assumptions are safe.

Here is the source code.

In C# the defaut access level for members in classes and structs is "private".

There is no need to pepper the source code with "private" everywhere. It only introduces noise and makes your code more difficult to read.

Chris Anderson is wearing a Mono T-Shirt on this PDC interview and IronRuby.com is hosted on DekiWiki, a Mono-powered Wiki site.

This is clearly awesome.

In other news, the awesome hackers at Imendio have officially released Gtk+ for OSX packages.

Funkists, why does System.IO.Stream not have a CopyStream method, it seems like everyone ends up implementing this.

Discuss.

Kudos to Ben and the rest of the Captcha team.

Today I downloaded the binaries for reCaptcha.NET for a web page that I was setting up and it worked out of the box. No recompiles necessary:

• Drop the Recaptcha.dll in the bin directory of the ASP.NET app.
• Get your public/private key.
• Insert this in your page:

  		<recaptcha:RecaptchaControl
                          ID="recaptcha"
                          runat="server"
                          Theme="red"
                          PublicKey="YourKey"            
                          PrivateKey="YourOtherKey"/>
  		

• Hit refresh. Ta da!

I just found out about encfs, a user-space encrypted file system that runs on top of FUSE.

To use it, just type:

	$ encfs ~/.encryptedstorage ~/secure
	

And follow the directions. You then will have a ~/secure directory where you can stash all the material that you would not want to become public if you were to lose your laptop.

But it gets better than this. You can use sshfs to store files on a remote server, and then use the sshfs-backed file system as your encrypted storage, you can then keep your files stashed on a remote server, completely encrypted.

Hey folks, it is that time of the year when we are looking to hire some developers to work on Mono.

We are looking for people with experience in Linux, with experience building software from source code and good C# or Java skills. Experience with ASP.NET and ADO.NET is a plus.

The positions this time are only available in Boston, if you are interested, send me an email.

Update: we have filled this position.

There is a cool hack that we want to introduce in Mono that would allow a remote process to debug a examine data in a running Mono instance. The hack uses the embeddable compiler.

The proposed extension to Mono would use a socket on /tmp/mono-USER/.mono-PID created by the Mono process and set the permissions to read/write for the owner and nothing for the group or other users.

What can go wrong security-wise with the above setup? What should we check that is not immediately obvious?

So far:

• Create directory with 0600 permissions, bail if mkdir returns anything but 0.
• Create socket in directory; It should be safe at this point, and change the permissions of the socket (is this really needed?).

The csharp interactive shell and the embeddable compiler hacks could only have been possible thanks to the work that Marek Safar has put into implementing C# 3.0, cleaning up the compiler and laying down the foundation for a reusable compiler.

Some time ago, our regression test suite for the compiler was taking 20-30 minutes to run. Marek refactored the compiler so that we could reuse the same instance without launching a new process for each of our 1,500 individual tests.

Without Marek's dedication and love for the compiler, we would never have gotten here.

Hello Internets and C# Lovers!

After the C# Interactive Shell started working, it became obvious that we now had C#'s eval and that it would be incredibly interesting to enable people to embed a C# Eval in their application.

I did a little bit of refactoring on the codebase last night, and now we expose a few methods that allow developers to evaluate C# expressions and statements dynamically.

Usage is very simple, you must reference the `gmcs.exe' assembly, this contains the C# Eval, then you invoke the CSharpEvaluator.Evaluate method, and get the results back.

The following is a small command line calculator that takes C# expressions:

using Mono.CSharp;
using System;

class MyCalculator {
	static void Main (string [] args)
	{
		Console.WriteLine (CSharpEvaluator.Evaluate (args [0] + ";"));
	}
}
	

Compile the above, and execute:

$ mono calculator.exe '1+2'
3
	

There are a number of overload methods for Evaluate that give more control over how things are evaluated. The csharp command is now written entirely using the CSharpEvaluator API. The code is now pretty simple: pretty printing of values, dealing with partial input, loading startup files (if any) and import the default namespaces.

A few uses that come to mind:

• Use C# as a scripting language for your application.
• Use it for startup scripts of your own application (similar to .emacs, but with C# instead).
• Use it to explore the state of your application live.
• Use it to debug your application as it is running.
• Embed the compiler on Silverlight and run C# on your web browser, similar to how Jimmy did it for Ruby.
• Embed it into your app and expose it as a service that you can connect to, and do it live!

The four or five public methods of the API are documented in the source using C# doc tags. When Mono 2.2 goes out, we will publish docs for them.

As we announced back in April, Mono's C# compiler is now licensed under the MIT X11 license, so you can embed at will.

Manhole

Joe Shaw mentioned a few days ago that it would be nice to support something like Twisted's Manhole where your application is listening on a port and then you can connect to it and debug it. It should take a dozen lines to write it, but I am heading out for dinner now.

But if I were to do it, I would probably make the csharp command (all 240 lines of it) support a --server command line option and have it connect to the server and send strings over the wire instead of executing locally. That way you get to use the awesome getline.cs command line editor and history locally and let the server run the code.

Getting Your Hands on This

To enjoy this hack, you have to either wait for Mono 2.2 or install Mono from SVN, all of the code is checked in.

Update: Clarification

Since it was not obvious, we do support C# 3.0, which includes the entire LINQ stack in the above expression evaluator:

using Mono.CSharp;
using System.Collections;
using System;

class X {
	static void Main (string [] args)
	{
		Evaluator.Run ("using System; using System.Linq;");
		bool ress;
		object res;
		string s = Evaluator.Evaluate (
			"from x in System.IO.Directory.GetFiles (\"/etc\") where x.Length < 10 select x;",
			out res, out ress);

		foreach (var v in (IEnumerable) res){
			Console.Write (v);
			Console.Write (' ');
		}
	}
}

The above uses a LINQ expression, and this is the result in my system (files whose full pathname is less than 10 characters):

/etc/motd /etc/mtab /etc/raw /etc/rpc 

Hello Internets!

Some very good news: As part of the ongoing discussion on the MS Limited Permissive License, I was doing some Google searches on it and came across the Enterprise Library 4.0 release.

Enterprise Library was a project that came out of the Patterns and Practices group at Microsoft, and they had historically used licenses that were not open source friendly. This new release is now licensed under the terms of the open-source friendly MS-PL license.

In the past, various people have asked us to implement an open source version of it, so they could use those building blocks on Unix with Mono. We never had any spare cycles to look into this, so the blocks provided by Enterprise Library were always beyond Mono users.

As it turns out, since May 2008 it has been possible to use the library with Mono on non-Windows platforms.

Now all there is left to do is to package it in a convenient form.

During the last HackWeek, I had a chance to work on a project that we had been discussing in the Mono team: an interactive C# shell.

The idea was simple: create an interactive C# shell by altering the compiler to generate and execute code dynamically as opposed to merely generating static code.

The result is the csharp command. The command provides an read-eval-print loop for entering C# statements and expressions:

	csharp> 1;
	1
	csharp> "Hello, World".IndexOf (",");
	5
	csharp> Console.WriteLine ("Hello");
	Hello
	

Statements are executed; Expressions are evaluated, and the result displayed. There is support for rendering arrays, IEnumerables and dictionaries specially, consider the following C# 3 declaration:

	csharp> new Hashtable () { { "/bin", "executable files" }, { "/etc", "configuration files" } };
	

You will get this back when rendered:

	{{ "/tmp", "Temporary files" }, { "/bin", "executable files" }}
	

Statements can span multiple lines; In those cases the interactive shell will use a different prompt to indicate that more input is expected, for example:

	csharp> 1 +
	      > 2;
	3
	csharp>
	

One of the main advantages of this shell is that you can try out your LINQ expressions directly on the shell, for example, the following query works on the result from Directory.GetFiles:

	csharp> using System.IO;
	csharp> var last_week = DateTime.Now - TimeSpan.FromDays (7);
	csharp> from f in Directory.GetFiles ("/etc") 
	      >    let fi = new FileInfo (f)
	      >    where fi.LastWriteTime < last_week
              >      select f;
        { "/etc/adjtime", "/etc/asound.state", "/etc/ld.so.cache",
	"/etc/mtab", "/etc/printcap", "/etc/resolv.conf" }
	

The LINQ expressions are not limited to working on IEnumerables, you can also use LINQ to XML or LINQ to any database provider supported by Mono by loading the assembly:

	csharp> LoadLibrary ("System.Xml.Linq");
	csharp> using System.Xml.Linq;
	csharp> var xml = new XElement("CompilerSources",
	      >   from f in Directory.GetFiles ("/cvs/mcs/mcs")
	      >   let fi = new FileInfo (f)
	      >   orderby fi.Length
	      >  select new XElement ("file", new XAttribute ("name", f), new XAttribute ("size", fi.Length)));
	csharp> xml;
	<CompilerSources>
	  <file name="/cvs/mcs/mcs/mcs.exe.config" size="395" />
	  <file name="/cvs/mcs/mcs/gmcs.exe.config" size="464" />
	  <file name="/cvs/mcs/mcs/OPTIMIZE" size="498" />
	  <file name="/cvs/mcs/mcs/lambda.todo" size="658" />
	  <file name="/cvs/mcs/mcs/smcs.exe.sources" size="726" />
	  [...]
	</CompilerSources>
	

A differences between csharp and the C# language is that I felt that for interactive use, it would be important to change the type of a variable, so the following is allowed:

	csharp> var a = 1;
	csharp> a;
	1
	csharp> a.GetType ();
	System.Int32
	csharp> var a = "Foo";
	csharp> a;
	"Foo"
	csharp> a.GetType ();
	System.String
	csharp>  
	

To load code interactive I added two methods: LoadAssembly and LoadPackage.

LoadAssembly is the equivalent of passing the -r command line argument to csharp or mcs, this example shows System.Xml.Linq in use:

csharp> LoadAssembly ("System.Xml.Linq");
csharp> using System.Xml.Linq;
csharp> XDocument doc = new XDocument(
      >     new XDeclaration("1.0", "utf-8", "yes"),
      >     new XComment("Sample RSS Feed"),
      >     new XElement("rss", 
      >         new XAttribute("version", "2.0"),
      >         new XElement ("channel",
      >             new XElement("title", "RSS Channel Title"),
      >             new XElement("description", "RSS Channel Description."),
      >             new XElement("link", "https://tirania.org"),
      >             new XElement("item",
      >                 new XElement("title", "First article title"),
      >                 new XElement("description", "First Article Description"),
      >                 new XElement("pubDate", DateTime.Now.ToUniversalTime()),
      >                 new XElement("guid", Guid.NewGuid())),
      >             new XElement("item",
      >                 new XElement("title", "Second article title"),
      >                 new XElement("description", "Second Article Description"),
      >                 new XElement("pubDate", DateTime.Now.ToUniversalTime()),
      >                 new XElement("guid", Guid.NewGuid()))
      >             )
      >         )
      >      );
	

The variable doc is then rendered:

csharp> doc;
<?xml version="1.0" encoding="utf-16" standalone="yes"?>
<!--Sample RSS Feed-->
<rss version="2.0">
  <channel>
    <title>RSS Channel Title</title>
    <description>RSS Channel Description.</description>
    <link>https://tirania.org</link>
    <item>
      <title>First article title</title>
      <description>First Article Description</description>
      <pubDate>9/8/2008 5:13:34 PM</pubDate>
      <guid>bc6825ab-f1ab-4347-ad6e-3cf076011379</guid>
    </item>
    <item>
      <title>Second article title</title>
      <description>Second Article Description</description>
      <pubDate>9/8/2008 5:13:34 PM</pubDate>
      <guid>a474b2bb-deba-4973-9581-762857b24b53</guid>
    </item>
  </channel>
</rss>

	

LoadPackage is the equivalent of invoking the compiler with the -pkg: flag. This is a Mono-ism that integrates Mono libraries with Unix's pkg-config. Packages allow definitions and multiple assemblies to be loaded in a single call, for example, this loads the Gtk# 2.0 package:

	csharp> LoadPackage ("gtk-sharp-2.0");
	csharp> using Gtk;
	csharp> Application.Init ();
	csharp> var b = new Button ("Hello Interactive Shell");
	csharp> var w = new Window ("So cute!");
	csharp> b.Clicked += delegate { Application.Quit (); };
	csharp> w.Add (b);
	csharp> w.ShowAll ();
	csharp> Application.Run ();
	

Pleasure

This shell has been incredibly useful to debug things in the last few weeks, as it avoids the tedious typing to try out APIs and to see what some function might return. Launch csharp and test things away.

To improve the experience of the command line editing, I wrote a managed readline replacement, it provides history, keyboard navigation and searching.

Also the System, System.Linq, System.Collections, and System.Collections.Generic namespaces are imported by default.

Additionally, the csharp command will load any assemblies and C# scripts that you have in the ~/.config/csharp directory. If you are working constantly say on LINQ, you can put there all of your using and LoadLibrary statements.

Availability

The csharp command will be available in Mono 2.2, which is still a few months away, or it can be obtained by compiling Mono from SVN today.

Interactive Commands

Currently there are a few static methods and properties that you can invoke from the command line, like the "help" property that will display some help, and the "quit" property that terminates the shell as well as things like LoadLibrary and LoadPackage that I have described before. A complete list is available on the CsharpRepl page.

I am interested in hearing which other features should be added to the shell. I think we need some special commands to describe types and objects, like monop.

Additionally, currently values are displayed by using ToString(), but perhaps we need a method Inspect(object) that would show the values of all public fields and properties, like a debugger would.

Which other commands should be added?

Update: Microsoft has now changed the license to the MEF to the open source MS-PL license. Please see the follow up post for details.

As a .NET developer, you should avoid using the newly released Managed Extensibility Framework as its license prevents its use beyond the Windows platform. This will prevent your .NET software from running on Linux or MacOS in the future.

Luckily, there is a cross platform solution available today that has no platform limitations: Mono.Addins, a technology inspired by Eclipse's own plugin system. We have tutorials, reference manuals, API docs, our FAQ our public groups and multiple large applications with source code available that you can learn from (MonoDevelop, Banshee, F-Spot and Gnome-Do).

The rule obviously applies to any new APIs that are built for .NET as they are not immediately available for Mono. But unlike the binary-only APIs, these half-open source code releases pose additional problems for the open source CLI:

• More people are exposed to the source code, preventing them from working on a fully open source implementation of it.
• There is a smaller desire from the community to reimplement the code, as it is relatively easy to just use the existing implementation or ignore the issue for the time being.
• Some folks might not care about the license restriction, and ignore it altogether. A practice I do not endorse.

There are two additional issues that are worth pointing out. Should non-open source libraries be hosted on CodePlex to begin with? CodePlex is branded as the "Open Source Project Hosting" from Microsoft, yet, this license is clearly not open source and does not qualify as open source according to the definition and is a clear violation of Codeplex.Com's requirements:

The above link points to http://en.wikipedia.org/wiki/Open_source_licenses.

MEF should be pulled out of the CodePlex site along with all other platform-limiting software, like ASP.NET MVC (h/t to gblock for pointing out that MVC is also violating the terms). Unless CodePlex cooks up a special exception that some software and restrictions are more equal than others.

The second point that is worth making is that picking licenses like the MS-LPL for .NET software is shooting the .NET community in the foot. The LPL license is obviously an effort to tie things into the Windows platform, putting company first, community and developers second.

The MS-LPL is a poisonous license, do not use it (do not confuse with the MS-PL which is a decent license, the extra "L" makes a big difference).

Update: gblock points out that CodePlex technically only requires a "license" to be picked, but does not require it to be OSI compliant. But even if the wording on the FAQ does allow for that interpretation, it is misleading as there is a link to OSI licenses next to it, and the main page clearly states that Codeplex is an "Open Source Project Hosting" site. Capital "O" and capital "S".

The "CodePlex Basics: Getting Started" page also points to the Open Source License Definition, there are no words about custom licenses or about hosting proprietary code with source code available.

Update2: Glenn has reached out to the community and clearly wants to improve the status quo when it comes to MEF, open source, CodePlex and the official policies. Good luck to Glenn.

It is also worth pointing out that Glenn was one of the early adopters and advocates of the MS-PL at Microsoft (to clarify: the MS-PL is an open source license).

The following screenshots shows Visual Studio debugging a remote Mono process running on a Linux box.

Breakpoints, current line, and stack traces.

The setup works like this: you run a debugging server on Linux. On Windows you install a Visual Studio extension that provides a Debugging Engine and the configuration tools to start your application.

Then you start your application from Visual Studio (Tools/Launch with Mono) which will send the binaries and debug information over to Linux and execute it.

Showing the disassembled code generated by Mono's JIT

The core of the above work is to get the Windows to Linux debugging functionality enabled. We will have more news when the debugging experience is more complete and we are approaching the time to test drive it.

If you are interested in trying it out, sign up for our preview on our web site.

Debugging Support

Mono 2.0 will for the first time include a debugger, the command line mdb command.

Beyond this, we want to offer GUI debugging. The native solution is being built on top of MonoDevelop and will be available when we release MonoDevelop 2.0. It will work on both Linux and MacOS hosts.

The second GUI engine will be the above Visual Studio plugin for developers that use Windows as their main OS and Visual Studio as their IDE.

More Visual Studio Integration

In addition to this work to integrate Visual Studio with Mono for remote debugging, thanks to the Summer of Code (Ed Ropple) and Jonathan Pobst's work in this hack week we got some support to run applications on Mono/Windows and run our Mono Migration Analysis tool.

Jonathan's work is available today as an MSI, check out his post for more details.

You can also check Ed's summer of code report on CloverLeaf.