Big Day in MonoLand

Mark Probst found and squashed one of the hardest bugs in our upcoming garbage collector. Pablo from Codice has been testing the new GC under stress and posted a nice message to the list.

Plenty of great feedback on deprecating old libraries and tools from Mono. We will have a lighter distribution. As things are coming together so fast and we are now well into the features we had planned for 3.0, we might end up just calling the next version 3.0 instead o 2.8.

Andreia got Moonlight running on Chrome.

Simplified User Interfaces on the iPhone with MonoTouch.Dialog

At the core of the iPhone UI lies the UITableView, a powerful table rendering widget that is used by pretty much every application on the iPhone. The UITableView is a powerful widget that can render its data in many different ways based on how you configure the widget, these are all variations of the UITableView:

This is one powerful widget.

The contents of the UITableView are rendered by calling into a developer supplied set of routines that provide the data on demand. The protocol includes queries like "How many sections?", "How many rows in section N?", "What is the title for section N?" as well as callbacks to provide the actual contents of a cell. Although the widget is pretty powerful, creating UIs with it is a chore and a pain to maintain. The pain to maintain and the repetitive nature of the process leads to developers either spending too much time customizing each view, bare minimum configuration and lack of polish on certain configurations. As I ported many of the Objective-C samples to C# I found myself repeating the same process over and over.

My fingers developed calluses, and at night I kept thinking that there should be a better way. But at the time, I was just doing a line-by-line port, I was not really ready to build a new API on top of it.

Recently, when my favorite twitter client on the iPhone butchered its UI I figured I would write my own Twitter client. The first step was to create the account configuration for my twitter account. As you can imagine, the configuration is done with a variation of the UITableView. And once again I found myself setting up the model, responding to view events, sprinkling switch and if statements liberally all through my source code and just plainly not having any fun writing the code. This is how MonoTouch.Dialog was born.

I wanted to reflection to allow a class to be mapped to a dialog box, something that would allow me to write a C# class and have it mapped to a UITableView, something like this:

class TwitterConfig {
    [Section ("Account")]
      [Entry] string Username;
      [Password] string Password;

   [Section ("Settings")]
      bool AutoRefresh;
      bool AutoLoad;
      bool UseTwitterRetweet;
}

Instead of starting with the Reflection code to deal with this, I first created an in memory representation of the entire dialog. The idea was that the Reflection code would then be a bridge that could use the engine code.

The engine code is built on the idea that each row could be a specific kind of widget. It could contain text, a switch box, a text editing line, a slider control, a calendar picker or any other kind of user created control. I call these "Elements" and I created a bunch of these:

• BooleanElement: rendered with a UISwitch
• FloatElement: rendered with a slider.
• HtmlElement: when tapped, starts the web browser on a url.
• StringElement: used to render plain strings.
• MultilineElement: A multi-line string element.
• RadioElement: a radio-item, to select one of many choices.
• CheckboxElement: like the BooleanElement, but uses a checkbox instead of a UISwitch.
• ImageElement: Allows the user to pick an image or take a photo.
• EntryElement: text entry element.
• DateTimeElement, DateElement, TimeElement: pickers for dates and times, dates and times.

The MonoTouch.Dialog follows the Apple HIG for the iPhone and implements as much of the UI policy allowing me to focus on the actual application instead of focusing on the administrivia.

Although the UITableView is built on the powerful Model/View/Controller setup that would allow it to scale efficiently to large data sets, most configuration pages and data entry pages do not require this complexity.

Another feature is that it takes care of all the bookkeeping required to do text entry without any work from the programmer: accepting keyboard input, automatically switching to the next entry line on return, aligning all entry lines in a section, dismissing the keyboard with the last input is reached.

Here is a sample of the API in action:

var root = new RootElement ("Settings") {
  new Section (){
    new BooleanElement ("Airplane Mode", false),
  new RootElement ("Notifications", 0, 0) { Notifications }
  new Section (){
    new RootElement ("Sound"), { Sound },
    new RootElement ("Brightness"){ Brightness },
    new RootElement ("Wallpaper"){ Wallpaper }
  },
  new Section () {
    new EntryElement ("Login", "Your login name", "miguel"),
    new EntryElement ("Password", "Your password", "password", true),
    new DateElement ("Select Date", DateTime.Now),
    new TimeElement ("Select Time", DateTime.Now),
  }
}
	

Once the RootElement has been created, this can be passed to a DialogViewController to manage it:

var dv = new DialogViewController (root);
navigation.PushViewController (dv, true);

Reflection API

The reflection API inspects a class and looks for fields that have some special attributes on them.

Here is a sample class and how it is rendered:

class AccountInfo {
    [Section]
    public bool AirplaneMode;

    [Section ("Data Entry", "Your credentials")]

    [Entry ("Enter your login name")]
    public string Login;

    [Caption ("Password"), Password ("Enter your password")]
    public string passwd;

    [Section ("Travel options")]
    public SeatPreference preference;
}
	

As you can see, enumerations (SeatPreference) are automatically turned into a radio selection that uses the UINavigationController to navigate and the captions are computed from the field names, a behavior that you can customize with the [Caption] attribute.

The attributes that are attached to each instance variable control how things are rendered and can be used to specify a variety of attributes like captions, images and overriding the defaults from MonoTouch.Dialog.

Some Samples

Code:

new RootElement ("Settings") {
  new Section (){
    new BooleanElement ("Airplane Mode", false),
    new RootElement ("Notifications", 0, 0) { Notifications }
  new Section (){
    new RootElement ("Sound"), { Sound },
    new RootElement ("Brightness"){ Brightness },
    new RootElement ("Wallpaper"){ Wallpaper }
  },
  new Section () {
    new EntryElement ("Login", "Your login name", "miguel"),
    new EntryElement ("Password", "Your password", "password", true),
    new DateElement ("Select Date", DateTime.Now),
    new TimeElement ("Select Time", DateTime.Now),
  }
}
	

LINQ and MonoTouch.Dialog

Craig has written a great Conference application for the Mix 2010 conference. I helped him reduce the code size of the application by removing all of the repetitive code required to set up UITableViews for various pieces of the application with MonoTouch.Dialog. Since the conference application deals with a database schedule, I extended MonoTouch.Dialog to work better with LINQ.

In the same spirit of the System.Xml.Linq API that allows you to create XML documents with nested LINQ statements, you can use MonoTouch.Dialog to create your entire UIs.

For Craig's application, I wrote a SessionElement that allows sessions to be "starred" and shows both the title of the session as well as the location of the session.

This code constructs the entire UI for the "My Schedule" tab. The data is populated on demand (Apple recommends that all views are loaded lazily)

public class FavoritesViewController : DialogViewController {
  public FavoritesViewController () : base (null) { }

  public override void ViewWillAppear (bool animated)
  {
    var favs = AppDelegate.UserData.GetFavoriteCodes();
    Root = new RootElement ("Favorites") {
      from s in AppDelegate.ConferenceData.Sessions
        where favs.Contains(s.Code)
        group s by s.Start into g
        orderby g.Key
        select new Section (MakeCaption ("", g.Key)) {
          from hs in g
            select (Element) new SessionElement (hs)
        }
    };
  }
}
	

That is it. The entire source code.

So use any of the two models that you enjoy the most: Reflection for quick one-offs and quick data-entry or the Element API for more advanced user interfaces but without having to spend your life writing boilerplate code.

I hope that this helps guys spend more time improving their applications, and less time writing boilerplate code.

MonoTouch.Dialog is not perfect and does not have every possible bell and whistle that you might want to add. Although I do welcome patches for certain features, feel free to fork the code and patch at will to suit your needs.

MeeGo Support in MonoDevelop

We just landed the support on MonoDevelop's trunk to support developing applications that target MeeGo powered devices.

MeeGo is a Linux-based operating system designed to run on mobile computers, embedded systems, TVs and phones. Developers would not typically use a MeeGo device as a development platform, but as a deployment platform.

So it made sense for us to leverage the work that we have done in MonoDevelop to support the iPhone to support MeeGo. Unlike MonoTouch, we are not limited to running on a Mac, you can use MonoDevelop on Windows, Linux and OSX to target Meego Devices.

Developers would continue using their Linux Workstation, their Windows PC, or their Mac to develop and test and the resulting cross-platform binary can be deployed and debugged remotely over the wire using Mono's Soft Debugger.

In this video, I interview Michael Hutchinson as he demostrates the developer experience:

The MonoDevelop/Mono that we will be supporting is entirely Gtk# based, both during development as well as during deployment.

Using MonoDevelop to Debug Executables with no Solution Files

Every once in a while I need to debug some Mono program that does not come with a solution. Either a program that was compiled using a Makefile or an executable that I installed with RPM on my system.

Sometimes I would end up cretaing MonoDevelop solution that contained every source file, command line option and resource that I meticulously copied from the Makefile. In other words, I was in for a world of pain just to be able to use MonoDevelop's awesome debugger.

Lluis offered this solution, and I am blogging it hoping that it will help someone else in the future. Follow these steps to debug a Mono executable and set breakpoints on the source code or class libraries source code:

1. Create a Generic Solution

Select File/New Solution and select Generic Solution:

2. Open the Project Properties

Double click on your project, in my case I called the project "compiler":

3. Add an Execute Custom Command

In your project select Custom Commands:

Add a custom Execute command by selecting it from the "Select a Project Operation" option and point to your Mono executable:

4. Load source files

Use File/Open to load the source file where you want to set a breakpoint (the executable source or some class library source) and set your breakpoints:

Then use Run/Debug to start your program in debugging mode (Linux and Windows users can use F5, MacOS X users can use Command-Return).

What have we been up to?

Every once in a while I would run into someone that will ask me what exactly we are up to in Mono. As Mono becomes larger, and various big projects "land" into the trunk, we can no longer do releases on a monthly basis. Some of the work that we do is inherently very attractive, things like new features, new libraries, new UIs and new frameworks. But probably more important are the efforts to turn our code into programming system products: fixing bugs, testing it, packaging it, supporting it, writing documentation, test suites, improving error handling, scaling the software, making it faster, slimmer and backporting bug fixes.

I wanted to give my readers a little bit of an insight of the various things that we are doing at Novell in my team. This is just focused on the work that we do at Novell, and not on the work of the larger Mono community which is helping us fill in the blanks in many areas of Mono.

MonoDevelop work

We just released MonoDevelop 2.2, a major upgraded to our IDE story, and the backbone that allows developers on Linux to debug various kinds of Mono-based applications. With support for the new Soft debugging engine, it has allowed us to support debugging ASP.NET web sites, ASP.NET web services, Silverlight applications, Gtk# and Console applications with minimal effort. Since the soft debugger leverages Mono's JIT engine, porting the soft debugger to a new architecture is very simple.

MonoDevelop 2.2 major goal was to create a truly cross platform IDE for .NET applications. We are off to a solid start with Linux, Windows and OSX support as well as solid support for C#, VB, Vala and Python.

We are now turning our attention to MonoDevelop 2.4. This new release will incorporate many new UI touch ups which I will blog about separately.

MeeGo/Moblin Support

We have been working closely with the MeeGo (previously Moblin) team at Novell to offer a streamlined developer experience for developers on Windows, Mac and Linux to target MeeGo devices.

Developers will be able to develop, test and deploy from their favorite platform software for MeeGo devices.

Mono Service Pack

A service pack release of Mono's enterprise supported ASP.NET release is ahead of us and we will be upgrading Mono from release 2.4 to release 2.6.

This will bring to our customers support for all of the new features in Mono 2.6 with the added benefit that it has gone through four months of extra testing and polish.

As part of this effort we are also upgrading the MonoTools for Visual Studio to support the new Mono Soft Debugger.

Runtime Infrastructure

Mono's runtime is being upgraded in various ways: we continue to work on integrating LLVM [1], productize our new copying garbage collector that can compact the heap, and make Mono scale better on multi-core systems with the integration of ParallelFX into Mono as well as re-architecting thread management and thread pools in Mono.

A big upgrade for Mono 2.8 will be the support for obfuscated assemblies that insert junk in dead blocks. This is a feature that we never had, but with many Silverlight applications being deployed with these options we started work on this.

We are working to improve our support for F# and together with various groups at Microsoft we are working to improve Mono's compatibility with the CLR to run IronPython, IronRuby and F# flawlessly in Mono. Supporting F# will require some upgrades to the way that Mono works to effectively support tail call optimizations. [1] LLVM: better use LLVM to produce better code, use it in more places where the old JIT was still required and expand its use to be used for AOT code.

Mono for Mobile Devices

We recently shipped Mono for the iPhone and we continue to develop and improve that platform. Our goal is to provide developers with a great experience, so we are doing everything in our power to make sure that every wish and whim of the iPhone developer community is satisfied. We are working to expand our API coverage, write helper libraries to assist developers, tune existing .NET libraries to run on Mobile devices, reduce startup time, and reduce executable sizes.

But we have also just started an effort to ship MonoDroid: Mono for the Android platform. This will include a comprehensive binding to the Java APIs, but accessible through the JIT-compiled, 335-powered runtime engine.

Our vision is to allow developers to reuse their engine and business logic code across all mobile platforms and swapping out the user interface code for a platform-specific API. MonoTouch for iPhone devices and the Monodroid APIs for Android devices.

Head-less Tasks

A big part of Mono's effort is in the development kit: the compiler, the tools and the server side components.

Mono has now a complete C# 4.0 implementation that will be ready to ship with the next version of Mono. Anyone can try it today by building Mono from SVN. We are also porting our C# compiler to work with Microsoft's Reflection.Emit to enable us to run our C# Interactive Shell in Silverlight applications and to allow .NET developers to embed our compiler in their applications to support C# Eval.

Our MSBuild implementation is very robust these days, and it will be fully supported in Mono 2.8 (and we will be backporting it to Mono 2.6 as well).

On the ASP.NET front, we are working with third party vendors to certify that their controls work with Mono's ASP.NET (we will have some tasty announcements soon) and we are also catching up to the new features that are coming with .NET 4.0.

WCF has turned out to be one of the most requested features. We had historically only paid attention to WCF for its Silverlight/Moonlight use cases, but as time goes by, more and more users are moving to WCF. We are working on completing our WCF support.

On the ADO.NET front our major focus has been to complete the support for LINQ to SQL as part of the DbLinq project that we are contributing to. At this point we have no plans to implement Entity Frameworks due to the large scope of that project.

Moonlight 3

I do not need to say much about Moonlight 3. Moonlight 3 is one of our most visible projects right now due to the adoption of Silverlight and Smooth Streaming.

By the first week of Feburary there had been 610,000 downloads of Moonlight 2.0 for Linux. This is only counting the downloads since the official release on December.

Policy Changes

Mono 2.6 was the last release of Mono to support the .NET 1.0 API profile. With Mono 2.8 we are going to drop the class library support for 1.0 and ship both 3.5 and 4.0 assemblies.

With Mono 2.8 we are also switching the default tools and compiler to be 4.0-based as opposed to be based on the 3.5 profile. We will be doing a release of Mono 2.8 a couple of months after .NET 4.0 ships.

Ciao!

The above probably reflects the main focus of the team in the past three months. There are many community driven efforts that are very cool and that deserve their own space and a future blog post. Things like the amazing work that was done on Qyoto and the Synapse IM client come to mind.

We look forward to a great year ahead of us.

Valentine's Day Call to Action

Everyone knows that in this day an age a software product is not complete until it offers a Desktop UI, a Web UI, and a front-end on the Appstore.

We access beautiful web sites, we purchase 0.99 apps on our phones and install gorgeous native software on our desktops.

The sysadmin community, once the backbone of Linux adoption, keeps asking "but what about me?". Indeed. What about them?

What are we doing about these heroes? The heroes that ssh in the middle of the night to a remote server to fix a database; The heroes that remove a chubby log file clogging the web server arteries; The very same heroes that restore a backup after we drag and dropped the /bin directory into the trashcan?

They are a rare breed in danger of extinction, unable to transition into a GUI/web world. Trapped in a vt100 where they are forced to by conditions to a small 80x24 window (or 474 by 188 with 6 pixel fonts on a 21 inch flat screen display).

These fragile creatures need our attention.

Today I am doing my part, my 25 cents to help improve their lives.

I am releasing Mono Curses 0.4 a toolkit to create text-based applications using C# or your favorite CLR language.

The combination of C#'s high-level features, Mono's libraries, Mono/.NET third party library ecosystem and the beautifully designed gui.cs, we can bring both hope and change to this community. Hope and change in the form of innovative text-based applications that run comfortably in 80x24 columns.

What is gui.cs

We know that hardcore sysadmins will want full control over what gets placed on the screen, so at the core of mono-curses we expose a C# curses binding.

On top of this, we provide a widget set called "gui.cs". gui.cs was introduced in 2007 enjoying unprecedented public acclaim among a circle of five friends of mine. Eight months after its introduction, it experienced an outstanding 100% growth when a second application was written using it.

Today, gui.cs is the cornerstone of great work-in-progress applications that any decade now will see the light of day. Including a new and riveting version of the Midnight Commander:

With only 3% of features implemented progress is clearly unstoppable!

I believe in dogfooding my own software before I unleash it to the world:

On a typical 21" sysadmin setup: 474x188 with the "Sysadmin Choice" award winning 6 pixel font.

Valentine's Day

So in this Valentine's Day, after you are tired of spending quality time with your grandmother, making out with your significant other or a stranger you just met at the checkout line in Costco, consider donating. Donate some of your time towards building some nice applications for your favorite sysadmin. God knows that he spent the whole day monitoring the dmesg output, looking for a SATA controller failure and keeping an eye on /var/log/secure, waiting for your ex to deface your wordpress installation.

And you have a choice, you can use Boo, IronRuby, IronPython, F# for building your app. VB.NET is also available if you want to teach your sysadmin a lesson in humility.

Get inspired today with some of the old screenshots.

Winter Olympics on Linux

The amazing Moonlight Team lead by Chris Toshok just released Preview 2 of Moonlight 3 just in time for the Winter Olympics' broadcast:

The player has some nice features like live streaming, Tivo-like "jump-back", accelerated playback and slow motion and it does this using Smooth Streaming which adjusts the quality of your video feed based on your bandwidth availability.

Thanks to Tom Taylor, Brian Goldfarb and the rest of the team at Microsoft for assisting us with test suites and early access to some of the technologies in use at NBC Olympics. With their help we were able to make sure that Moonlight 3 would work on time for the event (with full 24 hours and 14 minutes still to burn!).

As usual, the team did a great job, considering that we had to implement in record time plenty of Silverlight 3 features for Moonlight.

	MOONLIGHT_OVERRIDES=effects=no firefox
	

Moonlight 3.0 Preview 1

We have just released our first preview of Moonlight 3.0.

This release contains many updates to our 3.0 support, mostly on the infrastructure level necessary to support the rest of the features.

In the release:

• MP4 demuxer support. The demuxer is in place but there are no codecs for it yet (unless you build from source code and configure Moonlight to pick up the codecs from ffmpeg).
• Initial work on UI Virtualization.
• Platform Abstraction Layer: the Moonlight core is now separated from the windowing system engine. This should make it possible for developers to port Moonligh to other windowing/graphics systems that are not X11/Gtk+ centric.
• The new 3.0 Binding/BindingExpression support is in.
• Many updates to the 3.0 APIs

The above is in addition to some of the Silverlight 3.0 features that we shipped with Moonlight 2.0.

For the adventurous among you, our SVN version of Moonlight contains David Reveman's pixel shader support:

From Silverlight Parametric Pixel Shader.

Mono at FOSDEM

I will be arriving in Brussels on Saturday Morning for the FOSDEM conference. We have an activity-packed day on Sunday of all-things-mono.

This is the current schedule, pretty awesome!

Feedback requested: My plan is to do a state-of-the-union kind of presentation on Mono, but if you have a specific topic that you would like me to present on, please leave a comment, I will try to prepare for that.

See you in Brussels!

iPad - Inspirational Hardware

iPad - Inspirational Hardware

As a software developer, I find the iPad inspirational.

Apple's iPad is not a new idea. They are not the first ones to think of a tablet and as many blogs have pointed out the Apple iPad is not everyone's dream machine, the hardware is lacking gadgets and the software is not that amazing.

Five elements come together to revolutionize software:

1. Price
2. Multi-touch centric development
3. Standard hardware available for consumers
4. Apple's AppStore
5. Large form factor.

The iPhoneOS is a multi-touch centric operating system. For years application developers have been subjected to the tyranny of the mouse and keyboard. This has been the only input technology that developers could reliably depend on and expect to be available on the user's system. Any software that requires different input mechanism sees its potential market reduced.

The mouse is a great device for certain class of desktop applications. But it has also led to applications that are incredibly frustrating to use. Software for editing music and audio is cumbersome. Find the target, drag it, move it, find the other button, click it, scroll, drag, click. Anyone that has used Garage Band to try to play along knows this. The same applies to software to paint or draw. The mouse and keyboard are poor substitutes for using your hands.

On the iPhone, and now the iPad, the situation is reversed. Multi-touch is the only input mechanism that developers can depend on. Apple's iPhone helped create a community of developers that think in terms of taps, pinches and twirls instead of clicks, double-clicks and right-clicks. It is no longer an after thought. It is no longer a feature that is added if there is enough time in the schedule or enough budget. It is the only option available.

Taps, pinches and twirls allow us to use the full expression of our hands to drive an application. And it is not just any multi-touch, it is multi-touch over the same surface where the application is providing feedback to the user. Software that respond to user input in the same way that a physical object responds to our physical contact is the key to create new user experiences.

This is a whole new space in which we can research, a new space that we can explore and where we can create a whole new class of computer/user interactions. With the new form factor, we can now create applications that just made no sense on the iPhone.

It is fascinating.

The standardized hardware means that software developers do not have face testing their software with dozens of combinatorial options. There are only a handful types of systems. If the software works on the core systems, they will work on all consumer devices. Standardized hardware is at the core of the success of the console gaming market, developers test and develop against a uniform platform. Price is the cherry on top of the cake, this device will be mass produced and the affordable price means that it will have a deep reach.

The possibilities for new computer/user interactions are no longer dampened by this market reality. As developers, a new door to invention and innovation has been opened for us. No longer will software developers have to cripple their user experiences based on the mouse and keyboard.

For the past couple of years I have had some ideas for some software that I wanted to build on a touch-based computer, but the specter of having a small user base for my experiments always discouraged me. Ever since I heard the rumors about Apple producing a tablet computer I have not cared about what the device looked like, or what the software stack for it was going to be. I wanted to try new touch-based UI ideas, I have dozens of ideas that I want to try out. And with Mono, I get to do it in my favorite language.

iPad Support for MonoTouch!

24 hours after we got the iPad SDK we completed the support for the iPad for MonoTouch!

To get started with iPad development, go to http://monotouch.net/iPad and follow the instructions.

Let the iPad# hacking begin!

Release-o-Rama

Nice new releases of software that I use in the last few days.

Banshee 1.5

A new Banshee release, now supports new device syncing options, audiobooks, eMusic and GIO for non-local files. Gabriel has more details as well.

Now with a fully self-contained Mono and Gtk+ stacks on OSX. On the OSX note, I recommend Michael Hutchinson's blog entries on how to package your Gtk# app for use in OSX as well as his article on how to make your Gtk# app integrate with OSX. Both based on the lessons of bringing MonoDevelop and MonoDoc to OSX.

Jeroen Frijters released his IKVM.Reflection API. His API could be very useful for Reflection-Emit compiler writers, perhaps we could even use it in Mono's C# compiler to solve our long standing issues with Reflection. More research is needed on this area.

Maurits Rijk has published a new version of GIMP# his Mono-based plugin engine that lets you write plugins in any Mono supported language. There are samples in C# 3, F#, Boo, Nemerle, Oxygene, IronPython, Java/IKVM and Visual Basic.

Sandy released a new version of Tomboy, now supports exporting data in HTML format to the clipboard and jump Lists on Windows 7.

24 hour race

Another Mono-race, in 24 hours we are aiming to:

• Support the iPad SDK from Apple (freshly baked and published).
• Add MonoDevelop support for it.

Preordering the Apple Tablet

MVP Award

Thanks to everyone that participated in the campaign to nominate me for a C# MVP award, when I got back to Boston I found on my piles of email that I am now part of the program.

This is Pretty Sweet(tm). This will be a great opportunity to build more bridges with Windows developers and show them that there is an ECMA CLI life in the other side of the OS spectrum.

Looking forward to the group picture!

Mono at FOSDEM

This year we will have a Mono Room at the FOSDEM Conference in Brussels. The FOSDEM conference is held on the weekend on February 6th and 7th.

Ruben and Stephane organized the room and the speakers for the it. has posted the finalized schedule for the Mono activities at FOSDEM on Sunday.

Here is the schedule, there are some pretty interesting talks:

09:00 - 09:15	Opening (Ruben Vermeersch, Stéphane Delcroix)
09:15 - 10:00	MonoDevelop (Lluis Sanchez Gual)
10:00 - 11:00	The Ruby and .NET love child (Ivan Porto Carrero)
11:00 - 12:00	Mono Edge (Miguel de Icaza)
Lunch Break
12:45 - 13:15	The evolution of MonoTorrent (Alan McGovern)
13:15 - 13:45	Image processing with Mono.Simd (Stéphane Delcroix)
13:45 - 14:15	ParallelFx, bringing Mono applications in the multicore era (Jérémie Laval)
Coffee Break
14:30 - 15:30	Building The Virtual Babel: Mono In Second Life (Jim Purbrick)
15:30 - 16:00	Moonlight and you (Andreia Gaita)
16:00 - 16:30	OSCTool - learning C# and Mono by doing (Jo Shields)
16:30 - 16:45	Smuxi - IRC in a modern environment (Mirco Bauer)
16:45 - 17:00	Closing (Ruben Vermeersch, Stéphane Delcroix)

Pixel Shaders for Moonlight

David Reveman has just posted a fascinating patch that debuts the support of pixel shaders in Moonlight.

David's patch uses Gallium, and he says:

The current implementation uses gallium's softpipe driver but hooking up the llvm driver as well should be a minor task and give significantly better software performance.

[...]

My current approach is to focus on getting all these things working in software first. By using a OpenVG backend for cairo we can incrementally move to using gallium and hardware for all rendering.

Moonlight: Platform Abstraction Layer Lands

Chris Toshok has landed the changes necessary to abstract Moonlight's engine from the platform.

The platform abstraction layer lives in moon/src/pal and the Gtk+ interface lives in moon/src/pal/gtk.

This is a necessary step to bring Moonlight to non-X11 powered GUI systems.

C# Support for Tuples

More Mono proof of concept extensions to C#.

As part of the list of things I would like to see in C# is support for tuples in the language. They would show up in a few places, for example, to return multiple values from a function and assign the results to multiple values at once.

In recent versions of the framework there is a new datatype called Tuple, it is used to hold N values, the Tuple for N=2 looks like this:

	public class Tuple<T1, T2> {
		public Tuple (T1 v1, T2 v2);
		T1 Item1 { get; set; }
		T2 Item2 {get; set; }
	}
	

The tuple patch extends the C# language to allow multiple variables to be assigned from any Tuple, like this:

	(user, password, host, port, path) = ParseUri (url);
	

The above would assign the four values to user, password, host, port and path from the call to ParseUri. ParseUri would be declared like this:

	Tuple<string, string, string, int, string> ParseUri (string url);
	

Future Work and Ideas

In addition to handling Tuples, I would like to extend this to support collections and IEnumerables as well, for example:

	(section, header) = my_array;
	

The above would store my_array [0] in section, and my_array [1] in header.

If the last element of a tuple is a collection, it could store the rest of the values from the collection or enumerable in the last element:

	(query, page, other_options) = Request.QueryString;
	

The above would store the first item in the QueryString into query, the second into page, and the rest into the other_options array.

Tuple creation syntax:I would like to add nicer support for creating Tuples as return values, it could just mirror the assignment syntax.

	ParseUri ()
	{
		...
		return (user, password, host, port, path);
	}
	

Handling well-known types: In addition to Tuple, ICollections and IEnumerables, perhaps the compiler should know about older versions of Tuple like DictionaryEntry.

Using statements: Today the using statement is limited to a single resource, with multi-valued return types, it could handle multiple resources at once, like this:

	using (var (image, audio, badge) = iphoneApp.GetNotifications ()){
	    // use IDisposable image
	    // use IDisposable audio
	    // use trivial int badge
	}
	

New Moonlight Covenant has been posted

As I mentioned a few days ago, there is a new covenant form Microsoft for Moonlight, it has been posted.

Cena Linuxera en el DF, hoy

Cena Linuxera/Monera hoy (Diciembre 22) en el bar/restaurante del Covadonga a las 7pm. Para todo p�blico (incluso talibanes).

Direcci�n: Puebla 121 cerca de el Metro Insurgentes.

C# String Interpolation

We have discussed in the past adding support to C# to support string interpolation. I have cooked a patch that allows C# developers to embed expressions inside strings, like this:

	var a = 'Hello {name} how are you?';
	

Single quotes are used for strings that will have expressions interpolated between the braces. The above sample is equivalent to:

	var a = String.Format ("Hello {0} how are you?", name);
	

Currently the patch supports arbitrary expressions in the braces, it is not limited to referencing variables:

	var a = 'There are {list.Count} elements';
	

This notation can be abused, this shows a LINQ expression embedded in the string:

	var a = 'The {(from x in args where x.StartsWith ("a") select x).FirstOrDefault ()} arguments';
	

I am not familiar with what are the best practices for this sort of thing in Python, Ruby and other languages. Curious to find out.

Update: after the discussion on the comments the syntax was changed to use $" instead of the single quote to denote a string that will be interpolated. Now you will write:

	var a = $"There are {list.Count} elements";
	var greeting = $"Hello {name} how are you?";
	

The updated patch is here.

Debugging Silverlight/Moonlight Apps on Linux

A little hidden feature from our release of MonoDevelop 2.2 and Mono 2.6 earlier this week was MonoDevelop's support for debugging Moonlight applications:

Moonlight debugging is a feature that came together very recently, but we delayed Mono and MonoDevelop's release to make sure that we shipped with it.

To debug, merely open your Moonlight/Silverlight project, set some breakpoints, and run your program (F5). Your app will be debugged.

I did a quick screencast and annotated it:

Releasing Moonlight 2, Roadmap to Moonlight 3 and 4

Today we are making a few of announcements:

• Moonlight 2 is complete: Moonlight 2, our open source implementation of Silverlight 2 is done.
• An updated collaboration agreement between Microsoft and Novell to bring Silverlight 3 and 4 to open source Unix.
• Microsoft has an updated patent covenant that will covers third party distributions.

Update: Sean Michael Kerner covers the announcement and talks to Brian Goldfarb from Microsoft.

Update 2: New covenant from Microsoft has been posted.

2.5 API

Moonlight 2 is a superset of Silverlight 2. It contains everything that is part of Silverlight 2 but already ships with various features from Silverlight 3:

• Silverlight 3 Pluggable Pipeline, this allows developers to hook into the media decoding pipeline at various points:
  • Data fetching (this is how adaptive streaming and smooth streaming are supported). Test it out at Experience Smooth Streaming.
  • Demuxing (this is how the Moonlight Ogg container can be fed into Moonlight and Microsoft's Silverlight).
  • Codecs (this is how our Vorbis and Dirac support work on both Moonlight and Microsoft Silverlight).
• Easing animation functions
• Partial out-of-browser support
• Writable bitmaps
• Some of the new databinding features of XAML in Silverlight 3

We are moving quickly to complete our 3 support. Microsoft is not only providing us with test suites for Moonlight but also assisting us in making sure that flagship Silvelright applications work with Moonlight.

When it comes to prioritization of Silverlight 3 features, we are going to focus on getting the major applications that users want to use first. Sunday Night Football, the Winter Olympics and Bing's Photosynth support.

Smooth streaming works really well. Visit the site and test the immediate seek, and play with the bandwidth limiter to see how Silverlight/Moonlight can adapt the video quality based ont he bandwidth available:

Moonlight 2

Moonlight 2 is the result of love and passion to bring the Silverlight runtime to Linux.

Moonlight 2 engine consists of 142,000 lines of C/C++ code and 320,000 lines of C# code (125,000 lines of code came from Microsoft's open source Silverlight Controls).

Moonlight is built on top of Mono 2.6 runtime, Cairo and Gtk+ and today supports Firefox on Linux. We are hard at work to support Google Chrome on Linux as well.

Updated Patent Covenant

We worked with Microsoft to make sure that Moonlight was available to everyone on Linux and BSD.

Culturally, we started on two opposite ends of the software licensing spectrum. The covenant that was issued for Moonlight 1 and 2 covered every user that used Moonlight, but only as long as the user obtained Moonlight from Novell. This is a model similar to how Flash is distributed: there is a well-known location where you get your plugin.

The open source world does not work that way though. In the open source world, the idea is to release source code and have distributions play the role of editors and curators and distribute their own versions of the software.

Microsoft's intention was to expand the reach of Silverlight, but the original covenant was not a good cultural fit. We worked with the team at Microsoft (Brian Goldfarb and Bob Muglia's teams) to make sure that the covenant would cover the other Linux distributions.

The new patent covenant ensures that other third party distributions can distribute Moonlight without their users fearing of getting sued over patent infringement by Microsoft.

There is one important difference between the version of Moonlight that will be available from Novell and the version that you will get from your distribution: the version obtained from Novell will have access to licensed media codecs.

Third party distributions of Moonlight will be able to play unencumbered media using Vorbis, Theora and Ogg inside Moonlight (and Silverlight), but for playing back other formats, they will have a few options:

• Negotiating directly with the media codec owners a license (MPEG-LA, Fraunhofer).
• Negotiate access to Microsoft's Media Pack with Microsoft.
• Plug-in GStreamer or another commercial codec license into their Moonlight implementations.
• Update: Use a hardware provided decoder like VDPau.

Moonlight 3 and Moonlight 4 Collaboration Agreement

As readers of my blog know, the Silverlight 4 feature set is something that is very interesting to me.

If our experience with the positive feedback that we have gotten from MonoDevelop is of any indication Silverlight 4 will enable a whole new class of cross-platform .NET application development to take place. Like nothing we have seen before.

We are thrilled to be working with Microsoft to make sure that we can improve, fix and fine tune Moonlight to meet those requirements and to do so in a purely open source fashion.

Update: Team Silverlight blogs.

Nine Months Later: Mono 2.6 and MonoDevelop 2.2

About nine months ago we released MonoDevelop 2.0 and Mono 2.4. Today we are releasing the much anticipated upgrades to both. Mono 2.6 and MonoDevelop 2.2.

For those in a hurry, binaries and source are available from:

• Mono Download page.
• MonoDevelop Download page.

And if you want a quick mnemonic to remember this release, just think debugger! and cross platform.

The Mono team and contributors worked on this release like we have never worked before. Thanks to everyone that reported bugs, filed feature requests, contributed code and helped newcomers with Mono.

Mono 2.6 highlights:

• WCF client and server, the subset exposed by Silverlight 2.0.
• LLVM support, to improve performance on server/computational loads.
• Continuations/Co-routine framework Mono.Tasklets (background info)
• LINQ to SQL using DbLinq.
• New Soft Debugger, integrated with MonoDevelop on Unix and OSX (background).
• System.IO.Packaging.
• csharp shell now supports auto-completion (press tab to complete)
• xbuild can now build most msbuild projects.
• Mono debuts a verifier and security sandbox (used by Moonlight).
• More complete 3.5 API coverage.
• This release includes Microsoft's open sourced ASP.NET MVC, ASP.NET AJAX and Microsoft's Dynamic Language Runtime.
• Faster and slimmer.

MonoDevelop 2.2 highlights (screenshots here and here):

• MonoDevelop code is now LGPLv2 and MIT X11 licensed. We have removed all of the GPL code, allowing addins to use Apache, MS-PL code as well as allowing proprietary add-ins to be used with MonoDevelop (like RemObject's Oxygene).
• User interface improvements: the first thing that MonoDevelop users will notice is that we have upgraded the UI to fit modern ideas. We borrowed ideas from Chrome, Firefox, Visual Studio, Eclipse and XCode.
• ASP.NET MVC support, you can now develop, debug and build ASP.NET MVC applications from MonoDevelop.
• New T4 Macro processor (Text Template Transformation Toolkit) integrated directly into the IDE (Mono's T4 is also available as a reusable library for use and abuse in your own programs).
• Moonlight Project Support: you can now build, debug and run Moonlight applications using MonoDevelop.
• New MacOS and Windows support. Check our feature matrix for details.
• New Debugger support allows debugging Console, Gtk#, ASP.NET, iPhone and Moonlight applications.
• Extensive text editor improvements:
  • Dynamic abbrev (Just like Emacs' Alt-/)
  • Code generator (Alt-Insert)
  • Acronym matching
  • Code templates
  • Block selection
  • C# Formatter
• New refactoring commands:
  • Inline Rename (see screenshot).
  • Resolve Namespace
  • Rename Refactoring with Preview
  • Extract Method
  • Declare Local Variable
  • Integrate Temporary Variable
  • Introduce Constant
  • Move Type to Own File
  • Remove Unused Usings
  • Sort Usings
  • Create/Remove Backing Store
  • Keybindable Commands
• Python add-in has graduated to be a supported plugin, includes code completion, syntax checking, method and class locator and code folding.
• iPhone development plugin.

The team is on #mono, #monodev and #monodevelop on irc.gnome.org fielding any questions you might have.

Update: the diffstat results for Mono 2.4 to 2.6 on a 2 million line patch:

 7208 files changed, 1392400 insertions(+), 440016 deletions(-)

About a million lines of new code in Mono.

For MonoDevelop the patch is 750,000 lines and:

 2427 files changed, 464284 insertions(+), 120124 deletions(-)

Roughly 300k lines of new code.

Apple's Reply to Nokia

I felt like an archaeologist trying to formulate a theory of what had happened there. I loved the feeling of trying to put together the story from a partial puzzle.

The patent infringement lawsuit against Apple was a list of accusations and patent lists that Nokia claims that Apple infringes with their iPhone. But behind the background information provided in the legal document and the list of ways in which Nokia felt Apple had wronged them, it was difficult to put together a narrative. Scanning the discussion forums for clues did not lead to anything significant beyond the superficial analysis.

As a software developer, and in particular a Linux software developer, I have mixed feelings about this lawsuit. Apple has not been exactly a model citizen when it comes to interoperability between Apple and Linux products while Nokia has embraced Linux, embraced open source development and contributed to the universal pool of software. But I also found myself enjoying using my iPhone and building software for the iPhone.

I wanted to give both companies the benefit of the doubt. What had happened between these two companies that had forced Nokia to sue Apple?

There were various possibilities.

The lack of immediate response from Apple suggested that they were caught unprepared, but that was just a small chance. Probably the companies had been on negotiations and these negotiations broke off when they could not reach an agreement. The iPhone had taken the world by surprise, nobody had seen it coming and nobody had envisioned that Apple would not merely do an incrementally better phone, but it would be many times better than anything available at the time.

When Apple launched the iPhone, Steve Jobs wanted everyone to know that iPhone's innovations were patented and that Apple planned to prevent others from copying those ideas.

Apple's response to Nokia is a very educational document. It reads as a crash course on patent litigation when it lays out Apple's strategy for their defense. It is also a crash course on the patent system and how corporation work with international bodies to develop technology. But most importantly for me, it fills some the gaps of what happened behind the scenes.

We do not know yet which company approached the other first about patent infringement. It could have been someone on Nokia's board that decided to extract some revenue from their patents to compensate for their business losses or it could have been initiated by Apple's team notifying Nokia that their new phones used some idea from their phones.

What does emerge from Apple's reply is that Nokia tried to use the patents that they had pledged to license under reasonable terms to get themselves rights to Apple's juicier iPhone innovations. Nokia's pledged patents might be formidable patents and key to the implementation of certain cellular and WiFi communications, but by being pledged under F/RAND terms to various industry consortia they lost a significant amount of value. But what they lost in value, they made up in volume. This is in stark contrast with Apple's un-pledged, pristine, fully proprietary patents that Nokia and everyone but China are trying to get rights to.

Marek Announces Mono's C# 4.0 has been completed

Marek has just announced that Mono's C# compiler 4.0 is complete.

To try it out, get the modules mono and mcs from our Anonymous Subversion Repository and build Mono like this:

	$ mkdir $HOME/mono4
	$ ./autogen.sh --prefix=$HOME/mono4 --with-profile4=yes
	$ make && make install
	

Mexico 2009

Voy a Mexiquito lindo estas vacaciones de Diciembre.

Si quieren que nos reunamos para hablar de software libre, Mono, Linux, Moonlight, Silverlight, C# o para discutir por que La Mancha ya no escribe o el ultimo blog del Jetas mándenme un correo para ponernos de acuerdo.

First MonoTouch Book is out!

This was fast! Wallace B. McClure has written the first e-book on getting started with Mono on the iPhone with MonoTouch.

This is a short e-book, 42-pages in size, but it is also very cheap, it is only 6.99 USD and will help you get started in no time with MonoTouch.

Here is the table of contents:

Table of Contents

iPhone Requirements 2

Development Strategies 3

Web Development with ASP.NET 3

MonoDevelop and MonoTouch 4

Visual Studio .NET ➪MonoDevelop 4

Classes in MonoTouch 4

What Is MonoTouch? 4

Namespaces and Classes 5

Introduction to Development on the Mac with MonoDevelop 6

Interface Builder 8

Outlets 10

Actions 14

Deploying to an iPhone 15

Mapping 17

MKMapView 17

The Application 18

Annotating the Map 20

Debugging 21

Interacting with Other Applications 22

UIPicker 22

NSUrl 24

UIAlertView 26

UITableView 26

DataSource 27

Binding Data to a UITableView 29

Customizing UITableView 30

Accelerometer 33

Settings 34

Things to Watch Out For 37

Resources Used 38

Mono Developer Room at FOSDEM

Stephane and Ruben are chairing the first Mono-developer room at the upcoming FOSDEM 2010.

Ruben writes:

As of now, you can submit your talk proposals! We want to make this a fun room and we want to accomodate all kinds of talks. For that reason, one thing we're experimenting with is having dynamic timeslots. Only want 15 minutes? That's okay! Need an hour? We'll see if we can squeeze it in! The most important factor is that it's interesting and fun.

So send in your proposals, be it large earth-shaking projects, or little hackery experiments that make you giggle with hacker joy, we want to hear it. We have the complete Sunday to schedule. Still have questions? Email me: ruben @ savanne be.

The submission form is here, go fill it in now! (Send in your proposals before December 20)

See you there!

Silverlight: Universal GUI toolkit

The most important piece of news from last week's PDC was Microsoft's decision to turn Silverlight into the universal platform for building cross platform applications.

The upcoming version of Silverlight will no longer be a Web-only technology. It will now be possible to build full desktop applications with Silverlight.

Desktop Silverlight applications differ from the standard Silverlight in a few ways:

• Full access to the host file system, like any other .NET application would have.
• None of the socket connectivity limitations that are present on the sandboxed versioned of Silverlight. Full network access (we should build a MonoTorrent UI for it!)
• Built-in Notifications API to show up bubbles to the user when they need to interact with the application.

Although Moonlight has supported this mode of operation since day one, turning this into a standard way to develop applications was going to take a long time. We would have needed to port Moonlight to Windows and OSX and then we would have to bootstrap the ecosystem of "Silverlight+" applications.

But having Microsoft stand behind this new model will open the gates to a whole new class of desktop applications for the desktop. The ones that I was dreaming about just two weeks ago.

This was a big surprise for everyone. For years folks have been asking Microsoft to give Silverlight this capability to build desktop apps and to compete with Air and it is now finally here. This is a case of doing the right thing for users and developers.

Desktop Tools in Silverlight?

Now that this technology is available, perhaps it is a good time to start a movement to create a suite of Silverlight-based desktop applications.

The benefits to me are many:

• .NET applications that actually look good. In the past your choices were basically of Gtk# or Winforms, neither one really designed for this graphic-designer driven world.
• We can join forces with Windows/MacOS developers to create the next generation of desktop applications.
• Developers can tap into the large ecosystem of third-party controls that exists for Silverlight.

For the Moonlight team, this means that there is a lot of work ahead of us to bring every Silverlight 3 and 4 feature. I think I speak for the whole Mono team when I say that this is exciting, fascinating, challenging and feels like we just drank a huge energy boost drink.

If you want to help, come join us in the #moonlight or #mono channels on the IRC server at irc.gnome.org.

Silverlight 4

There are many other great features in Silverlight 4, but none as important as Silverlight becoming a universal runtime for the CLR. This is a revolution.

If you are curious about all the new tactical features of the revolution, check Tim's Complete Guide to the new Silverlight Features.

If you have the time, watch Scott's keynote's presentation where he introduced the new features (he starts at 1:02). I loved the use of HTML as a Silverlight brush (paint with HTML and even Flash). If you have time, these are some great sessions on Silverlight:

• Microsoft Silverlight 4 Overview: A good intro, repeats a lot from Scott's keynote.
• Improving and Extending the Sandbox with Silverlight 4: Joe Stegman's presentation on the revolution.
• RIA Services: The technique that lets you "remove four acronyms" from your development, mindblowing.

Droolingly yours,
Miguel de Icaza.

Concurrency with DREAM

Arne's presentation on concurrent programming at Monospace ago has been published.

Apps

Although making changes to Moonlight might be a very fun thing to do, for the longest time I have had this entry in my TODO list: "Blog about writing a video editor in Moonlight".

As much as I love Gtk+ and the Gnome desktop, our contributions for our desktop applications and for Gtk+ come mostly from from folks developing on Linux for Linux (with a handful of exceptions). And we are a small fraction of desktop developers.

In my mind what is interesting to me about building applications with Silverlight is that we can create an ecosystem of free software applications that run on all three major platforms: Windows, Linux and MacOS.

A few years ago, as part of the Google Summer of Code for Mono we created a project that could have had a great future, the Diva project (by MDK). Sadly, Michael moved on to other things, but in the back of my mind, I always wanted to have a nice video editing application for Linux.

I like to think that with Silverlight we have a new opportunity: we can create a community of open source developers that goes beyond the Linux-desktop community, but will pull developers interested about such a project from the Windows and MacOS worlds. I know that various members of the Moonlight team are passionate about Moonlight because it is this next generation API for building GUI applications.

Which applications do you think are needed nad could be built with Moonlight?

I say video editing, and I have some ideas of how it should work.

The future of Moonlight

With Moonlight quickly approaching its first official 2.0 release (which has feature parity with Silverlight 2.0 and has a handful of 3.0 features in place) we have been thinking about the work ahead of us, 3.0 and beyond.

Although there is much work left to do to achieve full parity with Silverlight 3.0, a solid 2.0 core means that we can start thinking about new innovative ideas.

Here are some ideas of things that Moonlight could prototype:

• Video camera and audio support (Chris has a working prototype).
• Native menu support for out-of-browser: we feel strongly that out-of-browser apps should get native menus and not be done with XAML. To allow proper MacOS, Windows and Linux integration.
• Right-click support for taking screenshots of Moonlight apps, record a screencast of a video, support for saving individual images, XAML and video from the plugin.
• Print the current canvas.
• User opt-in for full network access and full file system access (This is possible with Moonlight if compiled for the desktop APIs, but perhaps we should extend this to the browser version).
• Make the "Moonlight" widget ubiquitous. Allow any existing Gtk+, Win32, OSX, iPhone and Android to create content from a XAML file:

  		var splash = Moonlight.LoadFrom ("splash.xaml");
  		container.Add (sphash);
  		[...]
  
  		var mainMenu = Moonlight.LoadFrom ("mainMenu.xaml");
  		mainMenu.FindObject ("Start").Clicked += {
  			StartGame ();
  		};
  		window.Add (mainMenu);
  		

We could use Silverlight to build the next wave of cross-platform desktop applications.

I think of the Moonlight relationship to Silverlight as the Firefox relationship to IE four years ago. It is a chance to try out new ideas in the Silverlight-o-sphere, we can try those ideas out, and if the ideas have merit, they could become part of the official Silverlight.

This echoes to some extent what Bob Muglia said on an interview a few days ago: "[it] will allow for Silverlight to move into all sorts of places where we can't - we can't see."

For example, we are hard at work to make Silverlight run on the native PlayStation 3 and the Wii.

In any case, we have been working on Moonlight for a long time. With our security system in place, and both our smooth streaming media engine and GUI engine in place I feel that open source innovation can begin.

BitTorrent in Silverlight

A few years ago, thanks to the Google Summer of Code, I got to meet a brilliant programmer that wrote an entire BitTorrent implementation as a reusable library called bitsharp. Today, I am lucky enough to work with Alan in the team team at Novell.

Alan wrote a Gtk+ UI on top of his library called Monsoon:

With Mono running in the browser as part of Silverlight, I have always felt that we needed to build a UI for bitsharp.

The most important advantage is that every web browser can be turned into a BitTorrent client without having to install extra software.

Some other advantages: you could connect to a web page that contains the BitTorrent client, it would turn your machine into a seeding/downloading host, it could be shut down with minimal fuzz and it would be trivial to spice up the UI and add all kinds of visualizations for the download.

Silverlight 2 now provides direct socket/network access and supports storing files in your local file system. Most of the pieces are already in place to make this happen.

There are a few challenges though. Silverlight does not allow a client to listen on a port, and for security reasons it has a limited set of ports it can connect to.

The first problem could be addressed by having the client initiate connections to third parties requesting and providing data. This might require a protocol extension.

The second problem is easy to solve, but would require other Bittorrent clients to listen in a new port to explicitly grant access to Silverlight clients by exposing a clientaccesspolicy.xml file.

Finally, at least for Moonlight's case, we could start doing some changes to the core and grant the out-of-browser plugin to have full network access.

Moonlight Branches

Chris posted the new rules for the moonlight-2-0 branch and how to start working on 3.0 stuff.

Another intesting post is David's initial take on bringing pixel shaders, perspective transforms and hardware acceleration to Moonlight.

Mono Tools for Visual Studio

Today is a big day for the Mono team, we just released the Mono Tools for Visual Studio. The goal of this release is to make it simpler for Visual Studio developers to deploy their applications on Linux. ASP.NET, Windows.Forms, server and console applications are supported. These are the major features in the release:

Deploy to Linux: You can deploy from Visual Studio to a Linux machine running Mono your software. We are using Universal Plug and Play to detect Linux machines on your local network, or you can enter the IP address of your favorite hosting provider.

Debug Linux system remotely from Visual Studio: Developers complained that our debugging story left much to be desired, that deploying to Linux was possible, but debugging cumbersome. We now allow you to debug your application running on Linux without ever having to leave Visual Studio.

You can continue to edit, build and debug the way you have always done it, but the software will be running on a Linux machine:

We hooked the Mono debugging engine to Visual Studio, so all the regular debugging tasks are available (watches, locals, breakpoints, stepping) that you come to expect from Visual Studio debugging are available.

Review your code for portability: We have integrated Mono's Migration Analysis (MOMA) tool into Visual Studio so you can check your software for Windows/Linux differences right from your IDE and work around APIs that do not exist in Linux or Mono and refactor your code accordingly.

Package your software for Linux: Right from Visual Studio you can package your .NET application in Linux native installation format. Just create a packaging project in your solution, configure your package and produce an RPM that you can distribute:

And finally, one of the most exciting features in this tool: go from shipping applications into shipping appliances in minutes:

You use our wizard to prepare your appliance, select a base operating system template (Server, Client, and base operating system) and off you go.

Once your appliance is built, you can test it and apply the finishing touches over the web (using a Flash applet that connects to our virtual machines in our data center) and when you are happy with the results, you can download and redistribute your appliance to your users.

Appliances

Earlier this year I wrote about Novell's SUSE Studio our hosted service that helps developers create ready-to-run appliances.

Today, as a .NET developer, when you distribute your software to your users and customers, you probably have a list of requirements that goes like this:

• Install Windows XXX, Reboot.
• Install security updates A through Z, reboot as many times as needed.
• Install .NET runtime, reboot as needed.
• Install SQL server, reboot as needed.
• Populate database, reboot as needed.
• Install third party tool, reboot as needed.
• Just to be sure, reboot.

Then your users can start installing your software. At that point, you initiate a series of support calls that go like this:

• Did you make sure that .NET xxx.yy was installed? Ah, so do that and resume the steps.
• Wrong databases, uninstall, reboot, reinstall database, reboot.

And repeat the above process for every single one of your users.

Basically, every user has to repeat the same steps. Everyone has to assemble the solutions made up of the operating system, various pieces of dependencies that you have and your software, like this:

The experience today is like trying to buy a car by buying the individual parts:

We believe that for a class of developers there is a better way.

We believe that we can help you put together the full car, and deliver the car in a single piece.

With appliances you can ship a pre-configured operating system, pre-configured database and every service pre-configured and installed together with your software to deliver a full package, so you prepare your software for distribution once, you configure the database once, and then you give your users a ready-to-run virtual machine, CD-ROM or USB stick:

SUSE Studio has been used to build Linux based appliances (over four thousand per week), and now we are making it easy for .NET developers to take advantage of one of Linux's strengths: it is free, it is open source, you can shrink it, you can grow it and you can ship your own version (and yes, we do provide the updates for all of the core components that you pick).

Getting Started

Check out our getting started document.

Cool use of MonoVS

This is a cool use of Mono for Visual Studio debugging the Axiom 3D engine from the amazing Michael Cummings:

Visual Studio remotely debugging Axiom on an Ubuntu virtual machine hosted in Sun's VirtualBox.

Update: How to get it to work with Ubuntu

Third Party MonoTouch controls

This is pretty sweet in a couple of levels.

A couple of hackers at RemObjects have released some reusable controls for use with MonoTouch that simplify some common tasks with UITableViews, they are part of their "Project Plateau":

Not only this is sweet in that these are the first set of 3rd party controls for MonoTouch, they are also sweet because the authors built these controls using Oxygene, their Object Pascal compiler.

Update: Some documentation can be found here: introducing Plateau for MonoTouch.

Mono's new Compacting GC

I missed this post from Friday.

Mark Probst has posted a call for testers for Mono's new garbage collector:

Our new garbage collector, SGen, is now at the point where it's ready
for early testing.  It's still a long way from being ready for
production use - it's still slow and consumes too much memory,
especially for long-running processes - but if you'd like to help us
find bugs, please consider giving it a try.  I'm especially interested
in cases where Mono crashes or hangs with SGen (other than for memory
exhaustion).  The simpler the test cases the better.

To use SGen instead of Boehm, configure mono with the switch
"--with-gc=sgen".  At this point it's only known to run on x86 and
AMD64 Linux.

This collector is a precise-heap, conservative stack, moving, copying collector and will return unused memory back to the operating system. The details of this collector are available in our Compacting GC page.

I am rebuilding my own Mono with Compacting GC now as we speak.

MonoTouch application for the PDC

Craig Dunn has turned his Monospace application for the iPhone into a Microsoft PDC application that conference attendees can use at the upcoming conference.

Mandatory screenshot:

Since this app is not on the AppStore, you need to compile it and use Ad-Hoc deployment to run it into your phone.

Update: If you do not have a Mac, come to our booth at the Microsoft PDC, and we will ad-hoc deploy the app for you.

Alleviating Memory Fragmentation in Mono

Currently, Mono's Garbage Collector does not compact the heap and one of the problems that users have is that certain allocation patterns might result in a heap fragmentation that is hard to recover from. The observable side effect is that your process memory usage grows, but the memory is mostly unused:

You can read more about this in the Compacting GC page at the Mono web site.

Although a new GC is being developed to address this, the code is still not switched on by default in Mono as we do not consider it ready for production use (you must compiled Mono with --with-gc=sgen).

Avoid allocating very large buffers, instead allocate smaller buffers, and if you need to provide the illusion of a large buffer, hide this behind an interface.

If you must use large buffers, you might want to consider using MindTouch's ChunkedMemoryStream which exposes the same interface as MemoryStream, except that it uses 16k chunks instead of a very large buffer.

This will avoid the fragmentation for a very common use case.

Pig Farm Tour Oaxaca 09

April IM logs:

Introducing Debugging for MonoTouch

Today we released MonoTouch 1.2.

Perhaps the most important new feature in MonoTouch 1.2 is that it now has a debugger that supports debugging on both the iPhone simulator and on the iPhone/iPod Touch.

The debugger is integrated directly into MonoDevelop/OSX. All you have to do is select one of the debugging configurations (either Debug|iPhone or Debug|iPhoneSimulator):

and run your application:

To set breakpoints, you use the usual MonoDevelop UI, just click on the left-hand side of the editor, or use Command-/:

The debugger offers the usual watch windows, and also allows you to navigate object state by hovering over the value in the IDE:

The Technology

Although this was developed for the iPhone, I should probably have said that this is Mono's new soft debugger engine.

The iPhone is once again a challenging platform to get debugging working on. Since Apple has not published the information necessary to implement something like GDB or Mono's own MDB and we are not going to reverse engineer the protocol, instead we created a new way of debugging Mono applications.

The Mono Debugger for the iPhone platform uses a soft-debugger. Unlike traditional debuggers that act like a fully controlling entity over the Mono process, the soft-debugger is actually a debugger that is built into the Mono process itself. MonoDevelop communicates with this debugger over a compact protocol, similar to what has been done in the past with Java debuggers:

We are providing a new library, the Mono.Debugger.Soft.dll that encapsulates the protocol and offers an API that developers can use to debug a remote Mono process.

On systems where we have access to breakpoints the soft debugging engine will use the standard operating system facilities to single step and set breakpoints.

But on systems like the iPhone and some video game consoles where there is no way to modify memory without special privileges we had to resort to a different technique. The Mono static compiler inserts special code at every sequence point that checks for single stepping or breakpoints. The code generated during these debug builds is larger, but it allows us to support the iPhone without having to resort to undocumented APIs in any form or shape.

MonoDevelop and the iPhone

When the user selects an application for debugging, MonoDevelop configures the application to contact MonoDevelop on startup and link the debugger to it, starts listening on a couple of ports (one for the debugging protocol, and one for redirecting standard output/standard error) and waits for the application on either the iPhone simulator or the iPhone to contact it.

Upon contact, the debugger handshake takes place and operation continues. For the simulator, this takes place with a local socket; for the device, this happens over a TCP/IP socket over WiFi.

One of the nice side effects of this approach to debugging is that it is possible to distribute binaries to testers (using the Ad-Hoc distribution model) and debug problems on a user's iPhone over the network.

To support this scenario, when you build applications with the "Debug|iPhone" configuration, MonoDevelop will modify your application's Settings file on the iPhone.

This allows your beta-testers to enable debugging and connect to your debugger for inspection. This is what the settings looks like:

The first port is the port where MonoDevelop will be listening to. The second port is the port where the standard output and standard error will be redirected to.

All of the Console.Output and Console.Error output will be sent to this port when debugging is enabled on the application.

Pros and Cons

There are pros and cons to the soft debugger approach.

Soft debuggers are unable to do low-level debugging or debug Mono itself (mixed-mode debugging) and they are unable to stop applications at will, all it can do is request politely that the application stops, and when the Mono runtime decides that it is a good time to stop, it will stop.

On the pro side, the soft debugger is easier to port and is more robust as it eliminates some dead-lock situations. These can happen when the runtime has been forcefully stopped by the debugger, and then the debugger invokes methods on the target. These methods in turn might require an internal lock that is currently held by a different thread.

The Protocol

The protocol used between the debugging agent running inside the Mono process and a debugger is implemented in the debugger-agent.c file.

Availability

MonoTouch customers will be offered the update the next time they start MonoDevelop or if they manually "Check for Updates" on the MonoDevelop IDE.

Users of MonoTouch for the iPhone simulator can get the it from the trial page.

The source code for the soft-debugger is available on SVN. The API to communicate with the Mono runtime is available in the Mono.Debugger.Soft assembly and the debugger itself has already been integrated into the Mono 2.6 branch and Mono trunk on SVN.

This should prove useful to other users of Mono that might benefit from a soft debugger like Unity3D or Second Life.

Screenshots

Brent has a nice gallery of screenshots of the debugger in action on his MonoTouch 1.2 with debugging released! post.

10 years of Ximian

Today is the ten year anniversary of the incorporation of Ximian, Inc. A company founded by Nat Friedman and myself whose goal was to advance the state of the Linux desktop. It was also an excuse for Nat and myself to hang out as we had become best friends forever in the previous two years.

Our conversations over the years have always been a little surreal. I have megabytes worth of logs that look like this:

Ximian was made up of friends that shared this vision, and its founders had no startup, business or management experience when we launched the company. We learned on the job and we were advised by friends that believed in our goals, and by people that cared about our mission.

From the archive: Ettore Perazzoli, Raph Levien, Nat and myself in Summer of Linux.

Ximian hired 90% of its employees from open source contributors in the community and folks that we had met over mailing lists or IRC channels.

Nat was a Computer Science and Math graduate; I was a math drop out and we had no management experience. This means that we made every possible management mistake in the book, but all of our friends and employees stuck with us as we learned and as we worked to get the company off the ground.

This is an interesting time to reflect on 10 years of hacking adventures. Writing, funding and advancing the state of the art of open source. In the next few weeks, Nat and myself will be publishing some short stories from Ximian.

Today Ximian has been incorporated into Novell. Our goals have been expanded, but we still continue to work to advance the state of the art in Linux.

Looking forward to another 10 years of joy and hacking.

iPhone application for MonoSpace Conference

I woke up this morning and found that the amazing Craig Dunn put together an iPhone app for the Monospace Conference using MonoTouch!

The app provides the schedule, access to the twitter feeds for the speakers and the conference, access to the MonoSpace blog, speaker profiles and access to their blogs and twitter feeds and a map to find the event.

You can learn more, see larger screenshots and download the code to run on your iPhone from Craig's Post.

MonoSpace Conference in Austin - October 27 through 30

Over the past few weeks, the final details of the program for the Monospace Conference have been announced, and now the event is just less than two weeks away. Some key details you may have missed:

• The Monspace conference features 2 days of workshops and 2 days of Open Space sessions.
• The full conference agenda has been posted, and now includes a full day of MonoTouch training (Mono for the iPhone).
• The two-day open space sessions will be kicked off on Thursday with an Open Source Panel featuring Sam Ramji (CodePlex Foundation), Ayende Rahien (NHibernate and Castle Contributor), and Glenn Block (Managed Extensibility Framework) and moderated by Rod Paddock, Editor of CoDe Magazine.
• The famous, prolific and thorough Ayende Rahien will be speaking on NHibernate.
• Microsoft's own Glenn will be talking about the Managed Extesibility Framework.
• M David Peterson will be discussing Mono on Amazon EC2.
• The ThoughtWorks crew will be talking about F# on Mono.
• Eric Hexter from Headsprings will be covering ASP.NET MVC.

From the Mono team, various engineers will be presenting on special topics:

• Larry Ewing will talk about the Moonlight, the open source implementation from
• Aaron Bockover will talk about cross platform GUI development with Gtk#.
• Geoff Norton will discuss development with MonoTouch on the iPhone.
• Jackson Harper will talk about ASP.NET MVC development with Mono and Joseph Hill will discuss Mono Tools for Visual Studio.
• Rodrigo Kumpera from our JIT team will do a low-level talk on Mono's Virtual Machine covering some of the innovations on the VM that are Mono-specific.
• And I will focus on the schmoozing.

The Monospace conference will be held in Austin, TX on Tuesday, October 27 through Friday, October 30. To register for the event, visit the Monospace Conference Registration Page.

Check the program for more information.

Registered Monospace attendees will receive a $150 discount on MonoTouch, if they purchase in the month of October.

Event mapping in MonoTouch

When we were designing the MonoTouch APIs we had to map some of the constructs used by CocoaTouch and Objective-C to what C# developer expect.

Like other modern toolkits, the behavior of a control can be customized by the programmer by hooking up to events and the responses to those events.

This is a common pattern used in other toolkits. In Gtk+ objects emit signals, and the programmer can connect event handlers to individual signals, for example this is how you would connect

void pause_player ()
{
	// pause the player.
}

void configure_pause ()
{
	button = gtk_button_new_with_label ("pause");
	gtk_signal_connect (button, "clicked", pause_player, NULL);
}
	

In the ECMA CLI world developers hook up to events by connecting to events directly in the object, it is similar in spirit:

void configure_pause ()
{
	button = new Button ("Pause");
	button.Clicked += delegate {
		// Pause the player here.
	};
}

In both the Gtk+ and the ECMA CLI worlds there can be multiple subscribers to the event.

In CocoaTouch, instead of having an event per action they use a pattern where the components emits all of its messages to an instance of an object. If a button were to emit messages like "clicked", "clicked_violently" and "caption_changed" all of those messages will be sent to an object that implemented the required interface.

I am going to give myself a literary license and use some terms loosely and write a little bit of imaginary C#.

A button implementation would look like this:

interface ButtonDelegate {
	optional void clicked ();
	optional void clicked_violently ();
	optional void caption_changed ();
}

class Button {
	public ButtonDelegate Delegate { get; set; }

	public Button (string text) { ... }

	public string Text {
		get { return text; }
		set {
			text = value;
			DoSomeRepainting ();
			Delegate.caption_changed ();
		}
	}

	public EventHandler ()
	{
		...
		if (event_this_is_the_user_clicking){
			// Send the message to the Delegate
			Delegate.clicked ();
		}
		

The user of the Button class then needs to implement the interface ButtonDelegate to respond to messages. This in Objective-C is made simple by allowing this interface to have optional methods.

This would not work with C# interfaces as those require that all the methods on the interface are implemented.

So we came up with a design that would blend those worlds together.

Supporting the Objective-C approach

We can support the Objective-C model where a class can satisfy the contract by just registering methods to respond to the actions. A user could respond to some events like this:

class Foo : NSObject { 
	void ConfigureButton ()
	{
		button = new Button ("pause");
		button.WeakDelegate = this;
	}
	
	[Export ("clicked")]
	void pause_clicked ()
	{
		// Pause the button.
	}
}

In MonoTouch we expose the "WeakDelegate" property as the weakly-typed version of the delegate, that means that you can assign to it anything that derives from the NSObject class, and respond to messages by just annotating your method with the [Export ("name")] attribute.

But weakly typing could introduce errors. What if the parameters of the method or the return value do not match the actual signature of the selector. If the developer introduce a typo, the compiler would not be able to point this out, and you would get a nice crash or some corrupt memory.

So we introduced strongly typed delegates, these require the programmer to implement a class and override methods that have a very specific signature. The code would then look like this:

class Foo : ButtonDelegate {
	void ConfigureButton ()
	{
		button = new Button ("pause");
		button.Delegate = this;
	}

	// strongly typed: override ensures that this method exists in
	// the base class, or the contract is not satisfied. 
	public override void pause_clicked ()
	{
		// Pause the button.
	}
}

The ButtonDelegate class is flagged with a special attribute, the [Model] attribute which informs the MonoTouch runtime that only methods that are overwritten from this class will be dispatched, any other methods will be considered optional and treated like optional methods.

The problem of course is that if you wanted to respond to multiple buttons, you would have to actually distinguish them somewhere in your pause_clicked with some sort of closure value, or use helper classes, one for each button that you want to respond to. But that is a minor detail.

So Objective-C developers that like the separation between model, view and controller can continue to use those patterns.

That being said, although there is a lot of literature and discussion about the clean separation of Models, Views and Controllers, most samples I run across are a hodgepodge of Controller code for all sorts of things. At least the production code and the samples I have seen make it obvious that the separation of concerns is mostly academic, and in practice an instance object becomes a general purpose responder to all events, very much like you see in Gtk+, Qt, or Winforms.

Our approach also has a limitation: the hodgepodge approach does not really work with the strongly-typed signatures as you can only derive from one class at a time. The solution is to use separate classes for each controller. Although that cleans things up and has some aesthetics associated with it in terms of clean separation of concerns, in my opinion, it is not very practical.

Supporting the C# style

With MonoTouch we wanted to preserve the C# style of programming where you could attach code to these notifications one-by-one.

With C# you can use anonymous methods or lambda functions. These are incredibly powerful as they are able to capture the environment in which they were created. This means that you can trivially pass information and share information between multiple handlers.

Additionally, you more than one method can "subscribe" to the same event, you are not limited to a single method being the receiving of the events.

This means that in MonoTouch you configure buttons like this:

void configure_button ()
{
	button = new UIButton (dims) { Text = "Pause" };
	button.Clicked += delegate {
		// Pause the video
	};
}

What MonoTouch does internally is that the first time that you attach to a C#-like event, it create an instance of the ButtonDelegate that merely raises the event in the C# style.

This means that you only pay for the memory usage if you use the feature. But when you do, you can subscribe to individual events and you can have more than one listener attached to the event.

The Pattern

Although we brought the pattern to most places in MonoTouch where a delegate was used, it is notably missing from the UITableView as we felt that the number of methods that had to be overwritten were too big for the model to make sense.

In a few instances like UITableView, we suggest that developers just use the strongly typed version of the delegate classes and override the methods accordingly.

One of the things that I would like to see is a UITableView derived class that can present the data and style the data entirely based on properties discovered at runtime with System.Reflection.

Another thing that I want to see is support for System.Data data binding style versions of the UITableView and other UI controls in a single page.

More details on the low-level implementation are available on the API design document on the web site.

Git# - First Public Release

Meinrad Recheis has lead the effort to bring a full managed GIT implementation to the .NET and Mono worlds and he just announced the release of GitSharp 0.1.3, the first public release of Git#.

The code is based on the original work from Kevin Thompson who ported the JGit Eclipse plugin to C#. Meinrad has put together a great community of contributors that integrated every JGit improvement, the credits for the first release are as follows:

• Dave Cameron: for fixing bugs
• Björn Carlson: for killing bugs that crept into the core
• caytchen: for porting huge parts of the transport layer, and hunting down nasty bugs. btw, what's your real name?
• Emeric Fermas: for eliminating some of the hardest to find bugs and for verifying the complete test suite against jgit
• Martinho Fernandes: for fixing bugs
• Andriano Machado: for porting massive amounts of code and tests, also fixing many bugs
• Jim Radford: for the continuous integration server account and the support
• Gil Ran: for porting and fixing lots of tests and initial efforts on Mono
• Dan Rigby: for porting and fixing tests
• rolenun: for the command line interface framework. hey, what is your real name?
• Mauricio Scheffer: for the testing and build server expertise and CI server trouble shooting
• Neil Stalker: for holding up the flag for git# on Mono compatibility and squashing bugs
• Kevin Thompson: for initially porting large amounts of code and letting me rise the baby

An easy to use API is being developed on top of the GIT core, check out some of the examples.

Your anthropological tidbit of the day

In 1991 or 1992, I wrote a curses-based file manager for Unix, a clone of the DOS Norton Commander.

Unlike the Norton Commander, curses, xterms and Linux did not support the mouse. So I called my file manager "The MouseLess Commander".

Linux at that time was a cauldron of hot ideas. And Alessandro Rubini created a library for console applications to get mouse support on Linux. We worked to add mouse support to the MouseLess Commander and by the end of this exercise we renamed the file manager to "The MouseLess Commander With Mouse Support".

Somehow I was talked into changing the name of the Mouseless Commander with Mouse Support. Since I was pretty bad at naming, we organized a vote on the mailing list and eventually the current name "Midnight Commander" was picked.

I miss the previous name.

World Views

Richard Stallman does not seem to have anything better to do than launch personal attacks against me. In his last piece he has decided to call me a Microsoft apologist because I do not participate in his witch hunt.

I merely happen to have a different perspective on Microsoft than he has. I know that there are great people working for the company, and I know many people inside Microsoft that are steering the company towards being a community citizen. I have blogged about this for the last few years.

At the end of the day, we both want to see free software succeed. But Richard, instead of opening new fronts to promote his causes, attacks his own allies for not being replicas of himself. To him, ridiculous statements like Linus "does not believe in Freedom" are somewhat normal [1].

A Matter of Perspective

A few years ago, I had the chance to listen to Benjamin Zander in person talk about his world of possibilities. His book "The Art of Possibility" had a profound effect on me.

Benjamin tells this story in the book:

Two shoe salesmen were sent to Africa in the early 1900's to scout the territory.

One telegraphed back: "Situation hopeless. Stop. No one wears shoes."

The other telegraphed: "Business opportunity. Stop. They have no shoes."

Since we only have a limited time on earth, I have decided to spend my time on earth as much as I can trying to be like the second salesman. Looking at opportunities where others see hopelessness.

This is why I love my work, because I work with friends that have consistently beat the odds and we have consistently proved our critics wrong. Because everyone that I work with wants to change the world and nobody I work with is dominated by fear.

Far too many times, fear has prevented people from coming up with creative solutions. I rather work on constructive solutions to problems than moan and complain.

Fear mongering is a vibrant industry, it is the easy way out for those that can not lead with the example. Create a bogeyman, make things up, lie, or tell half-truths, and demonize and you got a recipe to energize your base.

Richard's Fear Mongering

The documentary "The Power of Nightmares" is a fascinating view at the industry of selling "safety" to the scared population and how leaders like to scare people to push their own agendas. Richard's fear mongering is no different.

Richard Stallman frequently conjures bogeymen to rally his base. Sometimes it is Microsoft, sometimes he makes up facts and sometimes he even attacks his own community [2]. His language is filled with simple, George W Bush-eque terms like Good vs Evil, Us vs Them.

The creation of the CodePlex foundation was an internal effort of people that believe in open source at Microsoft. They have been working from within the company to change it. Working at CodePlex is a great way of helping steer Microsoft in the right direction.

But to Richard, this simply does not compute.

To Richard, the state of the world is hopeless. I on the other hand see possibility and opportunity.

Not only you attract more bees with honey than with vinegar, there are lots of shoes to sell.

[1] Plenty of attacks on other open source/free sofftware advocates/developers are readily googable.

[2] the "open source" vs "free software" non-issue and the "Linux" vs "GNU/Linux" wars of the 90's that haunt us to this day.

Comments

I am now closing the comments for this post. Feel free to move this to the Slashdot forum.

Feedback wanted: StackOverflow DevDays in Boston

I will be presenting at StackOverflow DevDays in Boston, on October 7th at 4.20pm.

As usual, I will be talking about Mono. But since Mono is an giant universe, I would like to know what the audience would like to hear about.

Please fill in my small survey to provide feedback.

See you there!

Pictures: ECMA 334/335 January 2003

At the ECMA meeting in January of 2003 Anders was presenting C# 2.0:

Todd Proebsting had just developed iterators for C# and was presenting, it was inspired by his previous work on Icon's iterators. Sam Ruby (from IBM), Thomas Plum (from Plum Hall) and Todd:

Anonymous methods, the foundation for lambdas in C#, used a beautiful code generation technique. Anders who has a beautiful hand writing was explaining the design to the committee:

Jim Hogg presented the design for generics in the ECMA CIL. Here he is discussing with Arch (at the time with Intel, and the guy behind ECMA's System.Parallelization), Jon Jagger and Emmanuel Stapf (ISE Eiffel):

Joel Marcey, then at Intel and Jim Miller:

Todd and Chris Fraser. I was fanboying them. Mono's original JIT design was based on a lot of his work. At dinner Chris recommened Csikszentmihalyi's book on Flow, which I loved:

Comment Policy on my Blog

Here is a rough draft of the comment policy for my blog.

When you come to my blog to comment, you are a guest in my blog and your opinions are hosted there in the same way that you would be hosted if you showed up at my place to chat.

So standard rules of social engagement are required. If you are unable to operate under regular social situations and you feel the need to insult others or myself in my blog, not only you are not welcome, but your comment will not show up and will be promptly deleted.

Constructive criticism is always welcome on my blog, now

If you feel the need to be rude, offensive, lie or you are intentionaly trying to start a fight, I encourage to do that in your blog.

If you can not articulate your thoughts without attacking and you can not keep your temper in check, I encourage to do that on your own blog.

Trolling will be removed.

Branching for Mono 2.6

Today Andrew branched Mono trunk for the Mono 2.6 release.

Mono 2.6 is the last release that will support the mscorlib 1.1 profile and the C# 1.0 compiler. We are now moving to a pure generics world with Mono.

Update: Specifically, we will be removing the 1.0 assemblies from the build, the mcs.exe command will now point to gmcs.exe, all of the xxx1 command versions will be eliminated and all of the conditional code in the Mono class libraires that depended on NET_1_0 and NET_1_1 will be removed.

As for the Mono 2.6 release notes: we will put those together in the next few days. Please be patient.

On Richard Stallman

I want to say that God loves all creatures. From the formidable elephant to the tiniest ant. And that includes Richard Stallman.

As for me, I think that there is a world of possibility, and if Richard wants to discuss how we can improve the pool of open source/free software in the world he has my email address.

Love, Miguel.

MonoTouch 1.0 goes live.

MonoTouch is a commercial product based on Mono and is made up of the following components:

• MonoTouch.dll The C# binding to the iPhone native APIs (the foundation classes, Quartz, CoreAnimation, CoreLocation, MapKit, Addressbook, AudioToolbox, AVFoundation, StoreKit and OpenGL/OpenAL).
• Command Line SDK to compile C# code and other CIL language code to run on the iPhone simulator or an iPhone/iPod Touch device.
• Commercial license of Mono's runtime (to allow static linking of Mono's runtime engine with your code).
• MonoDevelop Add-in that streamlines the iPhone development and integrates with Interface Builder to create GUI applications.

The MonoTouch API is documented on the Mono site. The MonoTouch API is a combination of the core of .NET 3.5 and the iPhone APIs.

We have created some tutorials on how to use MonoTouch and you can read the design documentation on the MonoTouch framework.

Some History

Almost a year ago when we released Mono 2.0 for OSX, I asked the readers of this blog to fill a survey to help us understand where we should take Mono.

Many developers asked us to provide Mono for the iPhone.

A year ago, it was possible to use Mono's static compiler to compile ECMA 1.0 CIL bytecode for the iPhone. This is the technology that powered Unity3D's engine for the iPhone.

Users of Unity3D have published more than 200 applications on the AppStore.

We debated internally what exactly Mono for the iPhone would be. We considered doing a Windows.Forms port, to bring Compact Framework apps to the iPhone, but that would have required a complete new backend for Windows.Forms and would have required also a "theme" engine to make it look like the iPhone. We discarded this idea early on.

We decided instead to go with a native binding the iPhone APIs. It would not be a cross platform API and it would tie developers to the iPhone platform, but it would also mean that applications would look and feel native, and developers would get closer to the iPhone.

Creating a binding for the Objective-C APIs with .NET in the past has relied extensively on dynamic code generation, and this feature is not available on the iPhone (the kernel prevents JIT compilation from taking place). As we started doing the work to bind the low-level C APIs, Geoff started prototyping a new Objective-C/C# bridge that was specifically designed to work within the iPhone requirements.

By the time we had sorted out the foundation and the Objective-C bridge we had gained some fresh experience from binding the C++ PhyreEngine to .NET and we decided to create a new binding generator to simplify our life (I will blog about this new approach to binding creation in another post).

At this point, we started investing into supporting not only the device, but support the simulator. This would prove to be incredibly useful for quickly testing the code, without having to wait a long time to compile and deploy on the device.

The Developer Experience

At this point, we had enough to create mix-mode GUI applications (they were mostly Objective-C apps hosting Mono) and the pipeline was horrible: it involved using some 3 compilers, having 3 Mono checkouts and applying individual patches to all trees.

We were aware of how easy Unity had made things for their users, they had set the bar pretty high in terms of usability for us. Unity had some advantages, they could "debug" without deploying the code to the device, something that we could not really do ourselves.

We were still far from this.

We introduced a command that would isolate all of the complexities of creating simulator and ARM executables, the mtouch command.

With the mtouch command and the bindings complete, we started trying out the API by porting the Apple iPhone samples from Objective-C to C#. And in the process finding two things: C# 3.0 constructor initializers are a thing of beauty:

UIControl SliderControl ()
{
	var slider = new UISlider (new RectangleF (174f, 12f, 120f, 7f)){
		BackgroundColor = UIColor.Clear,
		MinValue = 0f,
		MaxValue = 100f,
		Continuous = true,
		Value = 50f,
		Tag = kViewTag
	};
	slider.ValueChanged += delegate {
		Console.WriteLine ("New value {0}", slider.Value);
	};
	return slider;
}
	

And also that the samples ported were half the size of the equivalent Objective-C programs.

As we were doing this work to iron out the kinks in the API design, Michael got started on designing the MonoDevelop support for the iPhone. This included the regular bits that you would expect like templates, building, running and deploying, but most importantly the Interface Builder integration.

Interface Builder produces an XML file that describes your UI, and it is able to generate some header files for you, but in general developers that want to refer to components in the UI and implement their backing store have to repeat the same stuff over and over. For example, controlling the contents of a label on the screen requires the developer to declare the same variable three times: 2 in the header file, and one in the Objective-C file (if you are lucky).

We took a different approach. We made it so that MonoDevelop would automatically generate a code-behind partial C# class for each Interface Builder file. This meant that to users the entire process would be transparent.

They would define outlets or messages, and those would just become part of their class. Intellisense and code completion would become available to them without any extra coding.

CodePlex Foundation

Microsoft today launched the CodePlex Foundation a non-profit organization to promote the open source collaboration between companies and open source communties, projects and individuals.

This is another step in the right direction for Microsoft. I have been working on open source software since 1992, and back then a world where open source played a major role was a dream to many of us. Microsoft adopting open source licenses, releasing some of their code under open source licenses and being a better community member was the stuff of fiction.

There are still many things that I would like to see Microsoft do, and many things that I believe Microsoft has to change to become a full member of the open source community, but it is encouraging to me to see Microsoft evolve. I hope that the CodePlex foundation will help us continue to build bridges between our communities.

I am glad that I was asked to be part of the board of directors of the foundation. And to work together with some great advisory board.

I hope that I can last more on this foundation than I lasted at the FSF, where I was removed by RMS after refusing to be an active part of the campaign to rename Linux as GNU/Linux.

MonoDevelop 2.2 Beta 1: We go cross platform.

MonoDevelop goes cross platform.

Since the beginning of time, man has yearned to get a cross platform .NET IDE. Homer's Odyssey described one man's effort to achieve such a thing. And it was not until today, September 9th of 2009 that the world can test out such a tool.

With this release MonoDevelop leaves its cozy Linux nest and embarks on a wild adventure into the hearth of MacOS and Windows. The MonoDevelop team made this one of their major goals for this release: to turn our loved IDE into a cross platform IDE.

If you are curious about the details, check out the What is new in MonoDevelop 2.2 page.

MonoDevelop on Windows

We are not only bringing MonoDevelop to OSX and Windows as a plain GUI port, but we are also providing installers, deep operating system integration and support for native debugging on each platform.

MonoDevelop on MacOS X

In addition to becoming a cross platform IDE, there are many new features in MonoDevelop.

For instance, MonoDevelop can be used to develop ASP.NET MVC applications on OSX and Linux and Silverlight applications on OSX and Linux.

Debugger

MonoDevelop now has integrated debugger support. Not only it is able to debug Mono applications, it also can work as a frontend to GDB to debug native applications.

In addition, on Linux it is possible to debug ASP.NET pages.

New Add-ins

New exciting add-ins: ASP.NET MVC, Silverlight and iPhone (for use with MonoTouch).

Policies

A common problem that we face as open source developers is that not every project uses the same coding style. Different teams use different coding conventions. MonoDevelop now supports policies to describe how files should be edited and what defaults should be used in each:

Editor Improvements

My favorite new feature is Dynamic Abbrev (Alt-/) a feature that we brought from Emacs and that fills me with joy. That being said, for the non-Emacs lovers there are plenty of features that you asked for, and that we implemented:

• Extensive refactoring support. And I mean, it is extensive.
• Code templates.
• On the fly formatting.
• Acronym Matching in Code Completion
• XML Documentation Shown in Code Completion
• VI mode for those users hooked up on VI commands, they can now use those within MonoDevelop.

Another pretty cool feature is the code generation support that is triggered with Alt-Insert. When you press Alt-insert it will popup a context sensitive dialog box that offers a few options of code that could be generated at this point: ToString methods, Equals/GetHashCode methods all based on existing fields and properties.

Why go Cross Platform?

Going cross platform means that developers will have the same tool across all of the operating systems they use: Windows, Mac and Linux.

.NET developers that have been enchanted by OSX will be able to continue developing software with their favorite programming languages while enjoying OSX and will be able to go back and forth between Windows, OSX and Linux as needed. This also means that they can work with developers in other platforms, regardless of the personal choices of other team members.

As many of you know, the number of contributors to a project is linked to the number of users of that project. By expanding our market presence from Linux, we expect to get contributions, fixes, improvements, bug reports, code and add-ins from developers in other platforms.

We intend to make MonoDevelop the Eclipse of the .NET community. Just like Eclipse became the foundation for Java development, we hope that MonoDevelop will become the foundation for .NET development, and hopefully for much more than that.

A multi-system IDE

We are not religious when it comes to supporting other programming languages [1]. We want to embrace not only .NET-based projects like Gtk#, Silverlight, ASP.NET, Boo, C#, F#, Visual Basic and Windows.Forms. We are also embracing other developer platforms like Python, C/C++, Vala, and we want to expand our presence to work with the Flash, PHP, Ruby, Rails, Flex and any other communities that need a cross platform IDE.

[1] we are just religious about the fact that C# is a better programming language to build an IDE than Java is.

Thanks!

This release could not have been possible without the endless nights and the collaborations of our contributors and all of the end users that reported bugs and gave us feedback.

Mono Survey Time!

Is it that time of the year when I unleash Google Docs' Survey platform on the users of my blog.

Every once in a while people ask us "When will you support Mono on..." and our answer is usually "thanks for your feedback, we will carefully consider it".

The reality is that all we have at this point is a lot of anecdotal data. Since my previous surveys on Mono, Mono for OSX, Mono for the iPhone and Mono for Visual Studio turned out to be great tools to understand what people actually want, I have created a new survey.

In this survey, I want to know how much people are willing to pay for (or not pay) for Mono commercial support on a variety of platforms: Red Hat Enterprise Linux, HP-UX, AIX, Solaris on a variety of architectures.

Go to my survey

Mono/.NET CodeCamp in Catalonia, Spain

The .NET and Mono user communities are working together in Spain to organize a joint .NET/Mono Code Camp. This Code Camp is going to be interesting because it is putting together two communities with very different points of view, but between which there is a lot of synergy. It will be a good chance to talk about Mono, Moonlight, MonoDevelop, MonoTouch and all the stuff we are working on.

Lluis just announced that the call for papers is ending this Friday, so not much time is left if you want to participate with a talk. There are more details in his blog post

GitSharp making progress

Meinrad Recheis emailed me to inform me that he has forked the Gitty project (a project that was doing a line-by-line port of a Java version of GIT) and that the project is making progress on having a full C# based GIT implementation.

GitSharp source code can be downloaded from github and they are making sure that GitSharp works on both .NET and Mono.

I can not wait for a MonoDevelop UI for it!

MonoTouch Resources

While I was gone, a pretty nice site called MonoTouch.infowas created that is aggregating all kinds of MonoTouch samples, posts, and discussions.

From the MonoTouch Bill Splitter page.

It is aggregating many of the samples being discussed in the tweetosphere and from the MonoTouch mailing list.

The samples are particularly useful as MonoTouch merges two worlds: .NET development and Objective-C programming on the iPhone/iTouch.

MonoTouch supports the Objective-C like setup where classes can respond to messages and you have to structure your code with MVC patterns. But it also supports the C# event-style programming mode, where developers can use lambda functions to hook up to interesting events.

You can also follow @monotouchinfo on twitter.

Silverlight/C#, Flash/ActionScript 3 and Canvas/Javascript

Been following some of the discussion around Action Script 3 on the tweetosphere. Here are three implementations of the same model from Joa Ebert (he works on a fabulous Flash product for doing music that I absolutely love).

From his Flirting with Silverlight post:

Lately a lot of discussion had the ActionScript language as its focus. People start complaining about complex language features but I think they are great because the end user will benefit from that. Yesterday evening I had my very first test with Silverlight — and I am really impressed.

It took me a short amount of time to port the strange attractor to Silverlight. I agree that this is may be not a fair comparison because I know C# already but have a look at the source code. I make heavy use of type inference and the Matrix4x4 class has the plus, multiply and array-index operator overloaded. The code is more readable. And besides: it executes really fast. Faster than my heavily optimized ActionScript version. Imagine I would write var bleh = 1.0 in ActionScript. Framerate would drop to something like zero. But this is sad since there is no reason for me to write var bleh: Number = 1.0. A modern compiler should be able to figure this out. haXe can do it, C# can do it, OCaml can do it and lots of others as well.

Remember: This was my first time using Silverlight. To achieve the same result with Flash you have to be kind of an expert in strange player and language “features”. Now tell me again that the end user will not benefit.

Here are the three implementations:

• Silverlight/C#.
• Canvas/Javascript.
• Flash/ActionScript 3

C# 4's Dynamic in Mono

C# 4.0 introduces the dynamic type into the language. The language will ship in Visual Studio 2010.

Once a variable is declared as having type dynamic, operations on these value are not done or verified at compile time, but instead happen entirely at runtime.

Marek has been working on Mono's C# 4 implementation and it is coming along nicely. Zoltan wrote a small "PInvoke" dynamic class using Mono's C# compiler that allows you to call C library methods like this:

	dynamic d = new PInvoke ("libc");
	d.printf ("I have been clicked %d times", times);
	

In this case "printf" is resolved at runtime to be a method in the "libc" library. This is similar to Python's ctype library in about 70 lines of C# leveraging LINQ Expression, the Dynamic Language Runtime and C#'s dynamic:

The code is:

using System;
using System.Dynamic;
using System.Reflection;
using System.Reflection.Emit;
using System.Runtime.InteropServices;
using System.Threading;
using System.Linq.Expressions;

class PInvokeMetaObject : DynamicMetaObject {

    public PInvokeMetaObject (Expression parameter, object o) :
        base (parameter, BindingRestrictions.Empty, o) { }

    public override DynamicMetaObject BindInvokeMember (InvokeMemberBinder binder, DynamicMetaObject[] args) {

        var self = this.Expression;
        var pinvoke = (PInvoke)base.Value;

        var arg_types = new Type [args.Length];
        var arg_exps = new Expression [args.Length];

        for (int i = 0; i < args.Length; ++i) {
            arg_types [i] = args [i].LimitType;
            arg_exps [i] = args [i].Expression;
        }

        var m = pinvoke.GetInvoke (binder.Name, arg_types);
        var target = Expression.Block (
			    Expression.Call (m, arg_exps),
			    Expression.Default (typeof (object)));
        var restrictions = BindingRestrictions.GetTypeRestriction (self, typeof (PInvoke));

        return new DynamicMetaObject (target, restrictions);
    }
}

public class PInvoke : DynamicObject {
    string dll;

    AssemblyBuilder ab;
    ModuleBuilder moduleb;
    int id_gen;

    public PInvoke (string dll) {
        this.dll = dll;
    }

    public override DynamicMetaObject GetMetaObject (Expression parameter) {
        return new PInvokeMetaObject (parameter, this);
    }

    public MethodInfo GetInvoke (string entry_point, Type[] arg_types) {
        if (ab == null) {
            AssemblyName aname = new AssemblyName ("ctype");
            ab = AppDomain.CurrentDomain.DefineDynamicAssembly (aname, AssemblyBuilderAccess.Run);
            moduleb = ab.DefineDynamicModule ("ctype");
        }

        // Can't use DynamicMethod as they don't support custom attributes
        var tb = moduleb.DefineType ("ctype_" + Interlocked.Increment (ref id_gen) + "_" + entry_point);

        tb.DefinePInvokeMethod ("Invoke", dll, entry_point,
			    MethodAttributes.Static|MethodAttributes.PinvokeImpl,
			    CallingConventions.Standard, typeof
			    (void), arg_types,
			    CallingConvention.StdCall, CharSet.Auto);

        var t = tb.CreateType ();
        var m = t.GetMethod ("Invoke", BindingFlags.Static|BindingFlags.NonPublic);

        return m;
    }
}

public class Tests
{
    public static void Main (String[] args) {
        dynamic d = new PInvoke ("libc");

        for (int i = 0; i < 100; ++i)
            d.printf ("Hello, World %d\n", i);
    }
}    
	

You can download the C# 4.0 Future document for details.

Mono's C# 4 compiler so far implements optional parameters, named parameters, has support for the new covariant and contravariant specifiers and the dynamic keyword is evolving very quickly (as shown by the example above).

If you want to test drive Mono's C# 4.0 compiler, you need Mono from SVN and you must pass the "--enable-profile4=yes" argument. Once you do this, the new compiler will be installed as dmcs on your system.

Nikhil has a nice sample that shows JSon integrated directly into C#.

C# Sqlite

Noah Hart did a line-by-line port of Sqlite to C# and has uploaded the code to code.google.com/p/csharp-sqlite.

The frequently asked questions on the web site talks about performance (it is about 5 times slower than native code, but still, it can do 1.5 million selects per second, 300k inserts per second).

Still, not bad for a line-by-line port; Folks on the #mono channel on irc.gnome.org a couple of days ago were commenting on the possible ways of tuning it up by not forcing the C way where the C# way would work just fine.

The potential for this library is huge. It could be used for ASP.NET servers running in medium trust mode, or it could be used by Silverlight applications.

It seems like Tim Anderson is attempting to remove all the P/Invokes from it to make it run on Silverlight (the P/Invokes are mostly used for doing file-level locking; Again part of the line-by-line history of it).

MonoTouch: Mono on iPhone closed preview

We have reached feature completion status for our MonoTouch project and we are looking for some adventurous iPhone developers that might be interested in trying out Mono for the iPhone and the MonoTouch APIs (A C# API for building Cocoa applications).

We are doing a closed preview to gather feedback on MonoTouch, if you would like to be part of this preview/beta, please fill out the signup form.

You can browse the source code for the sample widget catalog built using the MonoTouch.UIKit .NET binding to Cocoa's UIKit.

MonoTouch: Mono on iPhone closed preview

We have reached feature completion status for our MonoTouch project and we are looking for some adventurous iPhone developers that might be interested in trying out Mono for the iPhone and the MonoTouch APIs (A C# API for building Cocoa applications).

We are doing a closed preview to gather feedback on MonoTouch, if you would like to be part of this preview/beta, please fill out the signup form.

You can browse the source code for the sample widget catalog built using the MonoTouch.UIKit .NET binding to Cocoa's UIKit.

Bridging C# and C++ on Unix

Unlike Microsoft's .NET, Mono lacks a C++ compiler that can generate CIL code or mixed CIL/native code. This makes it harder for C++ codebases to integrate with C# code.

One solution to the problem is to write C wrappers for every C++ method and then use P/Invoke to call into each wrapper. Dealing with virtual methods then becomes challenging.

Andreia Gaita explains on Binding C++ APIs a technique that can be used to bridge the C++ and C# worlds using Mono's built-in support for COM objects.

IronRuby Progres

Antonio Cangiano from IBM compares the performance of the recently released IronRuby with Ruby 1.8 and 1.9 on Windows.

On his blog post he looks at both the micro benchmarks and overall benchmarks. The summarized results:

Antonio comment:

IronRuby went from being much slower than Ruby MRI to considerably faster across nearly all the tests. That’s major progress for sure, and the team behind the project deserves mad props for it.

One final warning before we get too excited here. IronRuby is not faster than Ruby 1.9.1 at this stage. Don’t let that first chart mislead you. While it’s faster in certain tests, it’s also slower is many others. Currently, it’s situated between Ruby 1.8.6 and Ruby 1.9.1, but much closer to the latter. The reason why this chart is misleading is that it doesn’t take into account any tests that timed out, and several of such timeouts were caused by IronRuby (more than those caused by Ruby 1.9.1). If you were to add, say, 300 seconds to the total, for each timeout for the two implementations, you’d quickly see that Ruby 1.9.1 still has the edge. The second chart that compares macro-benchmarks does a better job at realistically showing how IronRuby sits between Ruby 1.8.6 and Ruby 1.9.1 from a performance standpoint. If you were to plot every single benchmark on a chart, you’d find a similar outcomes for a large percentage of the tests.

Whether it’s faster than Ruby 1.9 or not, now that good performances are staring to show up, it’s easier to see IronRuby delivering on it’s goal of becoming the main implementation choice for those who both develop and deploy on Windows. This, paired with the .NET and possible Visual Studio integration, the great tools available to .NET developers, and the ability to execute Ruby code in the browser client-side thanks to projects like Silverlight/Moonlight and Gestalt, make the project all the more interesting.

SUSE Studio is out - The Linux Appliance Builder

Update: Nat Friedman has written two great blog posts on SUSE Studio, check them out here:

• SUSE Studio 1.0
• What is a software appliance?

Today Novell announced the release of Nat's latest project: SUSE Studio. Nat has been working on this for a very long time.

Studio is a tool that allows anyone with a web browser to create Linux-based systems and appliances. We believe that this is one of Linux' major strengths and we wanted to turn this potential into a reality.

In addition to custom Linux builds, Nat's team has put a strong focus on Linux-based appliances.

The goal of an appliance is:

• Perform a single task well.
• Requires minimal assembly and setup.
• It comes as a standalone unit.

The idea is that software developers can combine into a single deployable unit (a) the operating system; (b) your software plus any dependencies requires; (c) the proper configuration and data files required to operate.

When you build your appliance with Studio, you can select the kind of system you want, the packages you want to install, you can preconfigure the system and you can even test-drive the result over the web (it uses a Flash applet) and even fine tune or do some last minute touches over the web (inside the Flash applet) before producing your final version:

Test your Linux over the web.

The resulting Linux distribution can be downloaded as an ISO image, you can update it, you can create a bootable USB Linux system with your software, generate a VMWare image or VPC image and next week you should be able to deploy them in Amazon EC2.

We have been using Studio for building the Mono downloaded VMware/VPC images for a while. We bundle the latest Mono, MonoDevelop and various ASP.NET and Windows.Forms samples for developers to try out.

As a software developer, you can move from a world where you let the end users assemble the solution themselves:

Into a setup where you deliver the entire package to your end user:

The above scenario is common in the Linux world, non-existent in the OSX world, and only large OEMs are able to do the same on the Windows world.

What we wanted to do was enable anyone, with little Linux expertise to setup their own operating systems. You do not have to be an expert on kernels, boot loading, dependencies or anything else to produce your own operating system.

In addition to Unix software, we are going to make it very easy for Windows developers to bring their applications to Linux. Our Mono for Visual Studio project will now allow developers to go straight from Visual Studio into a Linux powered appliance:

Mono Visual Studio Plugin allows .NET to Appliance compilation.

Proposed C# extensions

Scott Peterson has written a proposal to extend C# to support parameterless anonymous methods.

Scott's proposal includes a patch for Mono's C# compiler that implements the suggestion.

The short story is that with this patch allows constructor initializers to also initialize events in an object. So you can now write code like this:

var button = new Button () {
    Label = "Push Me",
    Relief = ReliefStyle.None,
    Clicked +=> Console.WriteLine ("ouch!")
};
	

Both Bockover and myself love the 3.0 initializer syntax for objects and the only place where the awesomeness broke was the event initialization.

Improving Mono's compatibility with .NET CLR

For as long as we remember, most new versions of IronPython, IronRuby or the Dynamic Language Runtime exposed new missing functionality in Mono.

After a major release of any of these tools that deeply exercise the internal Reflection.Emit support and the runtime we would implement the missing features and ship a new Mono version.

As part of the Microsoft/Novell collaboration on Silverlight for Linux, a couple of months ago, we received from Microsoft the test suite for the Silverlight class libraries.

We have been working on fixing all of the issues exposed by the test suite. The test suite exposed bugs in Mono and missing features that are necessary for Silverlight in general, as well as the DLR and the DLR-based languages like IronRuby and IronPython.

We are pretty excited about the progress and the increased compatibility of Mono with Microsoft's .NET.

MonoDevelop Refactoring

The other day I did a quick tweet-vey asking folks what they wanted to see in MonoDevelop the most. It turned out that what they wanted the most matched pretty much what we had been working on.

About 50% of the features requested will be in our upcoming MonoDevelop 2.2. One cool refactoring tool is bound to the F2 key. You press F2 in a parameter or local variable, and as you edit the local/variable, all the references in your method are updated accordingly live.

ObScreenshot:

There are many more features on the pipeline, but they deserve more than a quick post-and-run.

Using Ruby and Python on Web Clients

Project Gestalt allows developers to use Ruby and Python to script their web UIs.

The project is powered by Silverlight's Dynamic Language Runtime and the IronPython and IronRuby. This means that they run under a sandboxed JIT engine.

Using it in your web pages is very simple. All you need to do is include in your web page the "gestalt.js" javascript. Once you do that, you can then embed Python or Ruby source code using <script> language="python"<script> or <script> language="ruby"<script>

<body>

<button id="say_hello">Say, Hello!</button>

<script language="python">
def OnClick(s,e):
  document.say_hello.innerHTML = "Hello, World!"
    
document.say_hello.AttachEvent("onclick", 
                   EventHandler[HtmlEventArgs](OnClick))
</script>
</body>
</html>
	

This addresses a part of Silverlight's story that I always felt was less than ideal. Without the Gestalt script, developers using Ruby or Python had to package their software on a ZIP file before sending down to the client.

Jimmy covers in more detail this important development. We both disliked the ZIP-file deployment model that Silverlight comes out of the box with.

With Gestalt developers can treat Python and Ruby on the client in the same way they have been treating HTML and Javascript. Write code, hit save, refresh page.

Check the samples.

All four pieces (Gestalt, the Dynamic Language Runtime, IronRuby and IronPython) are open source technologies that run on either Microsoft's Silverlight or our own open source Moonlight.

Popular Topics on ServerOverflow

In the comments from my previous post someone pointed out ServerFault's question clouds:

The StackOverflow tags are:

I love to be working, contributing and participating with fabulous people that are advancing the hottest topics of discussion at Stack/Server/Overflow.

Popular Topics on StackOverflow

I am probably the last person to find about this, but I find the number of questions/tagged on StackOverflow fascinating:

TreemapViewer: Building Silverlight Applications on Unix

A few months ago, to get an idea of what contributed to the size of Mono libraries I wrote a small Treemap visualizer using Moonlight:

Moonlight's assemblies; area represents the codesize.

You can browser the code or get a copy from our AnonSVN repository.

There are a couple of tools here:

• A tool that extracts the sizes of methods for a collection of assemblies (built using Cecil).
• A Silverlight control to render treemaps. The code is based on a paper on how to do treemaps, and I borrowed a couple of routines from some code I found for WPF.

  A Silverlight-based front-end to host this on the web.

  An out-of-browser Gtk#-based front-end to run the application locally.

My Treemap is not very ambitious, and it is nowhere as complete as the new Treemap control that is part of the open source Silverlight Toolkit. But it was a fun learning experience for me.

Reusable Engine

I did not really know where we would use this control, on the web or on the desktop when I started. So I split the actual engine that does the heavy lifting from the actual chrome for the application.

This is why I ended up with a Silverlight user interface and a Gtk# user interface. This idea in general might be useful for other developers as well.

The Custom Control

The custom control is very simple, it is called TreemapRenderer and derives from UserControl. The code overwrites two methods: MeasureOverride and ArrangeOverride. These methods are used to allow the control to participate in the Silverlight layout system (for example, the control can be embedded in a table that can auto-stretch). Silverlight invokes your MeasureOverride to find out the desired size that you control would like to consume:

public class TreemapRenderer : UserControl {
	protected override Size MeasureOverride(Size availableSize)
	[...]

	protected override Size ArrangeOverride(Size finalSize)
	[...]
}
	

Silverlight will invoke your control's ArrangeOverride to layout its contents once it has determined the size that the control will use.

This is where my TreemapRenderer lays out its contents.

This code renders an actual region on the treemap, I like the C# 3.0 initializer syntax for the text created:

public void SetRegion (Rect newRegion)
{
        region = newRegion;
        content.Children.Clear ();
        content.Width = region.Width;
        content.Height = region.Height;
        
        if (caption != ""){
                int max;
                string formatted = MakeCaption (caption, out max);
                double w = region.Width * 1.60;
                double s = w / max;

                var text = new TextBlock () {
                        FontSize = s,
                        Text = formatted,
                        Foreground = new SolidColorBrush (Color.FromArgb (255, 0x5c, 0x5c, 0x5c))
                };

                Canvas.SetTop (text, (region.Height-text.ActualHeight)/2);
                Canvas.SetLeft (text, (region.Width-text.ActualWidth)/2);
                content.Children.Add (text);
        }
 
        Rect emptyArea = region;
        Squarify (emptyArea, root.Children);
 
        Plot (root.Children);
}
	

To provide feedback to the user, I change the background color of the treemap on enter/leave. I use anonymous two anonymous methods, one for MouseEnter, one for MouseLeave.

Notice that state is shared by these two blocks of code using C# variable capturing. A key feature of the language:

	bool inside = false;
	host.MouseEnter += delegate {
	        host.Background = new SolidColorBrush (Colors.Yellow);
	        if (text != null)
	                text.Foreground = new SolidColorBrush (Colors.Black);
	        inside = true;
	};
	
	host.MouseLeave += delegate {
	        host.Background = transparentBrush;
	        if (text != null)
	                text.Foreground = borderBrush;
	        inside = false;
	};
	

One thing that proved very useful was the ability to zoom-into an area. If you click on an assembly, you will get a rendering of the code size used by a type; If you click on that, you get a method breakdown on a class:

drill-down into mscorlib's sizes.

To transition, I added a cute animation where I render the new image and animate it from the area occupied in the larger view into the final size. This is the code that queues the animation:

        // Render a child
        void Clicked (Node n)
        {
		// This is the child rendered, configure it to its final size
                TreemapRenderer c = new TreemapRenderer (n, n.Name);

                Size ns = new Size(region.Width, region.Height);
                c.Measure (ns);
                c.Arrange(region);

		//
		// Scale it and position on the spot that it currently
		// occupies on the screen
		//
                var xlate = new TranslateTransform () {
                        X = n.Rect.X,
                        Y = n.Rect.Y };

                var scale = new ScaleTransform () {
                        ScaleX = n.Rect.Width / region.Width,
                        ScaleY = n.Rect.Height / region.Height };

                c.RenderTransform = new TransformGroup { Children = { scale, xlate } };
                c.Opacity = 0.5;
                content.Children.Add (c);
                activeChild = c;

                //
		// Animate this to its final location
		//
                TimeSpan time = TimeSpan.FromSeconds (0.3);

                var s = new Storyboard () {
                        Children = {
                                Animate (time, xlate, "X", 0),
                                Animate (time, xlate, "Y", 0),
                                Animate (time, scale, "ScaleX", 1.0),
                                Animate (time, scale, "ScaleY", 1.0),
                                Animate (time, c, "Opacity", 1.0),
                        }};

                s.Begin ();
	}

	// Helper method;   
        static Timeline Animate (TimeSpan time, DependencyObject target, string path, double to)
        {
                var animation = new DoubleAnimation () {
                        Duration = time,
                        To = to
                };

                Storyboard.SetTarget (animation, target);
                Storyboard.SetTargetProperty (animation, new PropertyPath (path));

                return animation;
        }
	

The Silverlight Application

Using the control is fairly simple, but since it loads a large XML files (feel free to change this to use a web service) I use the downloader and a callback to render the control.

        private void Application_Startup(object sender, StartupEventArgs e)
        {
	    [...]
            var webclient = new WebClient();
            webclient.DownloadStringCompleted += delegate (object sender2, DownloadStringCompletedEventArgs ee){
                try {
                    RootVisual.Dispatcher.BeginInvoke(() => LoadNodesFromString(ee.Result));
                }
                catch (Exception ex) {
                    main.BackButton.Content = ex.ToString();
                }
            };

            webclient.DownloadStringAsync(new Uri("../mscorlib.xml", UriKind.Relative));
        }
	

The actual creation of the control happens in LoadNodesFromString:

        void LoadNodesFromString(String s)
        {
	    // Use Size for the area.
            Node n = LoadNodes(s, "Size", "Extra");
            while (n.Children.Count == 1)
                n = n.Children[0];
            
            treemap = new TreemapRenderer(n, "");
            Grid.SetColumn(treemap, 0);
            Grid.SetRow(treemap, 1);
            main.grid.Children.Add(treemap);
        }

Building a Gtk# Out-of-Browser Client

The above works great for Silverlight, but I like my application on the desktop, so I created a Gtk# version of it.

Moonlight provides a Gtk# widget that can be embedded into C# desktop applications. This is what it looks like when running on the desktop. I know this screenshot is not too exciting as I did not do much with the Gtk+ side of things:

Gtk# is rendering, seriously.

The core of embedding Moonlight as a Gtk# widget is very simple, here it is:

using Moonlight.Gtk;
using Moonlight;

        public static void Main(string[] args)
        {
                n = LoadNodes (args [0], "Size", "Foo");

                Gtk.Application.Init ();
                MoonlightRuntime.Init ();

		// My container window.
                Gtk.Window w = new Gtk.Window ("Foo");
                w.DeleteEvent += delegate {
                        Gtk.Application.Quit ();
                };
                w.SetSizeRequest (width, height);

		// The Moonlight widget that will host my UI.
                MoonlightHost h = new MoonlightHost ();

		// Add Moonlight widget, show window.
                w.Add (h);
                w.ShowAll ();

                // Make it pretty, skip all levels that are just 1 element
                while (n.Children.Count == 1)
                        n = n.Children [0];

                // Render
                TreemapRenderer r = new TreemapRenderer (n, "");
                Size available = new Size (width, height);
                r.Measure (available);
                r.Arrange (new Rect (0, 0, width, height));

		// This informs the widget which widget is the root
                h.Application.RootVisual = r;
                Gtk.Application.Run ();
        }

There are Visual Studio and MonoDevelop solutions on the SVN for folks to try out.

You can also try a sample live on the web.

MiggyTour 2009

In the third and fourth weeks of August, Laura and I will be spending a couple of days in Madrid visiting friends.

Then a couple of days hanging out with Nat, Stephanie and a bunch of awesome friends and family friends in Florence.

And then we are spending a week in Morocco, most likely in Marrakesh.

If you live in Madrid or Marrakesh and would like to get together to meet and talk, send me an email.

StackOverflow DevDays in Boston

Joel Spolsky and Jeff Atwood have invited me to participate on StackOverflow's DevDays in Boston on October 7th.

I will be talking about Mono, Mono and Visual Studio and Mono on iPhone. Come equipped with questions.

Shocking, I know.

LLVM powered Mono

Mono from SVN is now able to use LLVM as a backend for code generation in addition to Mono's built-in JIT compiler.

This allows Mono to benefit from all of the compiler optimizations done in LLVM. For example the SciMark score goes from 482 to 610.

This extra performance comes at a cost: it consumes more time and more memory to JIT compile using LLVM than using Mono's built-in JIT, so it is not a solution for everyone.

Long running desktop applications like Banshee and Gnome-Do want to keep memory usage low and also would most likely not benefit from better code generation. Our own tests show that ASP.NET applications do not seem to benefit very much (but web apps are inherently IO-bound).

But computationally intensive applications will definitely benefit from this. Financial and scientific users will surely appreciate this performance boost.

Taking it out for a spin

You need to install both LLVM and Mono from SVN.

Get and install LLVM like this:

$ svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
$ cd llvm
$ ./configure --prefix=/mono --enable-optimized
$ make && make install

Then get Mono, and you need to apply a tiny patch to configure.in.

$ wget http://tirania.org/tmp/m7a9da378.txt
$ svn co svn://anonsvn.mono-project.com/source/trunk/mcs 
$ svn co svn://anonsvn.mono-project.com/source/trunk/mono 
$ cd mono
$ patch -p1 < m7a9da378.txt
$ ./autogen.sh --prefix=/mono --enable-llvm=yes
$ make && make install

Now you have an LLVM-powered Mono.

LLVM is not able to support some of the features that Mono needs, so in those cases the JIT compiler will still fall back to Mono's JIT engine (methods that contain try/catch clauses or methods that do interface calls).

This backend was written by Zoltan.

Banshee as a Platform?

Aaron Bockover recently discussed Banshee's modular architecture. Aaron discusses using Banshee as a foundation for managing not only music, and videos but merging with F-Spot to manage photos as well:

The slides from his talk go into a bit more detail.

Jonathan Pobst takes the platform idea one step further and suggests that Banshee's core could also be used to replace YAST's Software Installer UI:

Jonathan says:

Every time I use YaST's Software Manager, I wonder if it would be better implemented using Banshee. Banshee's interface has been tuned for usability, both on its own, and what it borrows from iTunes. Software management is a naturally scary operation, and using an interface that the user is already familiar with could help reduce user fear.

Of course, it would just be the interface pieces of Banshee in a new app, you wouldn't actually start Banshee for software installation.

Another feature I would like to see taken from iTunes/Banshee is downloading/installing in the background. Once I hit Install, go ahead and download the application in the background, and install it in the background. I can click on the "Downloading/Installing.." menu item if I want to see what's going on. Most of the time, I'd rather be looking at other things to download.

I think this is a brilliant idea.

From Microsoft: C# and CLI under the Community Promise

First the big news: Microsoft will be applying the Community Promise patent licensing to both C# and the CLI.

The announcement was done by Peter Galli at Microsoft over at Port25 and it states (emphasis is mine):

I have some good news to announce: Microsoft will be applying the Community Promise to the ECMA 334 and ECMA 335 specs.

ECMA 334 specifies the form and establishes the interpretation of programs written in the C# programming language, while the ECMA 335 standard defines the Common Language Infrastructure (CLI) in which applications written in multiple high-level languages can be executed in different system environments without the need to rewrite those applications to take into consideration the unique characteristics of those environments.

"The Community Promise is an excellent vehicle and, in this situation, ensures the best balance of interoperability and flexibility for developers," Scott Guthrie, the Corporate Vice President for the .Net Developer Platform, told me July 6.

It is important to note that, under the Community Promise, anyone can freely implement these specifications with their technology, code, and solutions.

You do not need to sign a license agreement, or otherwise communicate to Microsoft how you will implement the specifications.

The Promise applies to developers, distributors, and users of Covered Implementations without regard to the development model that created the implementations, the type of copyright licenses under which it is distributed, or the associated business model.

Under the Community Promise, Microsoft provides assurance that it will not assert its Necessary Claims against anyone who makes, uses, sells, offers for sale, imports, or distributes any Covered Implementation under any type of development or distribution model, including open-source licensing models such as the LGPL or GPL.

You can find the terms of the Microsoft Community Promise here.

I told you this was good news!

A few months ago we approached Bob Muglia and Brian Goldfarb (@bgoldy) at Microsoft with a request to clarify the licensing situation for the ECMA standards covering C# and the CLI (also ISO standards, for the ISO loving among you).

Previously Microsoft had detailed the patent license plans and today they have delivered on those plans.

Astute readers will point out that Mono contains much more than the ECMA standards, and they will be correct.

In the next few months we will be working towards splitting the jumbo Mono source code that includes ECMA + A lot more into two separate source code distributions. One will be ECMA, the other will contain our implementation of ASP.NET, ADO.NET, Winforms and others.

Depending on how you get Mono today, you might already have the this split in house or not.

Thanks to everyone at Microsoft that worked to get this approved and released. We appreciate that they made this a priority when we approached them, and we know that everyone in the .NET team was also incredibly busy with various betas: .NET 4, Visual Studio 2010, Silverlight, MVC, MEF and much more.

I am overflowing with joy right now. Cheers!

Update: Send your thanks to @bgoldy on tweeter, who crossed all the t's and dotted all the i's to make this happen.

Update: Moderation of comments is taking place. Off topic messages will be removed immediatly. Trolling, as mild as it might be will be deleted. If you want to argue what language is the best one take the debate to a newsgroup.

PhyreSharp runs on the PS3

Mono on the PS3 has been making some nice progress. Supporting the PS3 requires some special features in Mono, for instance, a static compiler for .NET code for the PowerPC/Cell processor.

PhyreSharp, our .NET binding to Sony's PhyreEngine game engine ran on Thursday for the first time.

MonoSpace Conference Announced

Scott Bellware has announced the MonoSpace Conference in Austin Texas on October 27-30th.

Scott has made a Call for Speakers:

The Monospace Conference is looking for teachers to give tutorials on the Mono framework, tools, languages, and platforms supported by Mono.

Some tutorials are aimed at .NET developers with little experience with operating systems other than Windows, and others are geared to experienced Mono developers with exposure to the various Mono platforms.

The tutorials are two hour to three hour interactive sessions that can be any combination of follow-along examples, labs, and lecture.

We're looking for tutorials on subjects such as Linux, Mac, Windows, web, desktop, servers, message queues, databases, iPhone, Android, Amazon's EC2, among others.

You can track the progress of the conference at the MonoSpace Conf Blog.

You can also follow the progress on twitter.

Scott was one of the founders of the Alt.Net series of conferences.

Some Cool Mono Announcements

Yesterday we shipped Mono 2.4.2, our long-term supported version of Mono. It ships Microsoft's opensourced ASP.NET MVC stack for the first time (you could get it before on your own, but now it is integrated) and fixes over 150 reported bugs.

Chris Toshok announced M/Invoke a tool to port applications that use P/Invokes on Win32 to Linux and MacOS.

What Chris does not talk about on his post is that he was trying to use some .NET software that interfaces via USB to his glucose meter and was trying to get this to run on Linux. The tool is mostly .NET with the usual handful of P/Invokes to Win32. And this is how M/Invoke was born: a tool to retarget P/Invoke happy applications into becoming pure managed applications.

This opens new doors to forcefully port more apps to Linux.

Alan McGovern released a new version of Mono.Nat one of the libraries used by MonoTorrent.

Jordi Mas released a new version of Mistelix a DVD authoring tool for Linux:

Jordi's GBrainy brain teaser game was picked up by MoLinux, a regional Linux distribution, and shipped it translated to Spanish:

Joe Audette's mojoPortal was being installed four times as much when it got included in in Microsoft's Web Platform Installer site (more stats here).

For years I have loved the Joel on Software rules for software engineering. And one of those rules is "Build in one step". We have not always succeeded, but we have always tried. Lluis delivers the one step to build and run for MonoDevelop on Windows: Load solution, Hit F5, up and running.

Google Chrome really lead the way here, and I want very badly to have all of Mono building in Visual Studio with one keystroke, but we are not there yet.

Stephane reports on some nice startup performance improvements for F-Spot. Loading time for 10 images from Stephane's own image collection went from 1.2 seconds to .5 seconds.

MonoDevelop got some enhanced support for autoconf integration.

Jeremy Laval released another version of ZenComic a desktop Comic reader:

David Siegel announced a new release of Gnome Do on behalf of the Gnome Do team. In particular, it is now easier to write "Docklets" for the Gnome Do panel and for those of us that like the Emacs keybindings, it is now possible to use C-N and C-P for navigation

And of course the Google Summer of Code is in full swing:

And we have various very exciting projects brewing.

Jonathan Pobst has been exploring integration points for Mono and Visual Studio 2010:

Guadec: I will sadly not be attending the Guadec/Akademy conference in Canaria next week. This is going to be a busy summer for us as we are shipping a lot of code in the next few months: Moonlight 2.0, Mono for Visual Studio, MonoTouch 1.0 and Mono 2.6.

Mono

PhyreEngine, Mono, cool Mono uses in Gaming, and more.

Last week there was a little Mono surprise. It can be found on this Novell-hosted web page web page (scroll a little bit).

It has been a few very busy weeks at Novell's Eastern Research and Development Facility (Novell NERD Facility) here in Cambridge and we have been incredibly busy polishing some nice toys.

A few weeks ago we learned about Sony's developer event in the West Coast. Michael, Zoltan and myself worked very hard to put together a demo to show the virtues of C# and the CIL to developers. So we cranked on some record time some code:

• PhyreEngine#
• Static compiler for PowerPC for Mono on PS3
• A yield-based co-routine framework.

We picked Sony's PhyreEngine to demostrate how to use Mono to write the high-level code for a game using Sony's finely tuned engine. We figured this was better than showing a for loop printing the numbers 1 to 10 on the screen.

PhyreEngine# wraps PhyreEngine using the same techniques that we used in Gtk# and Moonlight. The resulting API is glorious and by letting PhyreEngine do all the heavy lifting while driving all the high-level from C# there is no way of telling that the driving force is not C++. All you get is pure unadultered productivity.

To make our demos a little more interesting, Michael wrote a minimalistic yield-based co-routine framework inspired by some of the ideas that our friend Lucas gave us. It is a tiny toy, but we used it to illustrate the concept of using C# iterators as the foundation for game logic development and how a cooperative scheduler would work (Unity game logic works just like this).

We were also working on completing Mono's port to the PlayStation 3's native operating system (this is different than running Mono on Linux on the PS3: that already works, and it was used for developing CellDotNet, a JIT for the PS3's SPUs). Zoltan developed the static compiler for PowerPC and I did the platform support.

Mono can now run "Hello World" on the PS3 native OS. There are still lots of ins, lots of outs and lots of whathaveyous that need to be tied up before this fully works and before we are able to run PhyreEngine# on the PS3.

Developing Cross Platform application with MonoDevelop

Yesterday Lluis announced the last missing piece from our strategy to make MonoDevelop a full cross-platform IDE: MonoDevelop now runs on Windows as well:

When we started the planning for MonoDevelop 2.2, the major goal of that release was to get feature parity on Linux, MacOS and Windows.

We want to grow the community of developers that contribute to MonoDevelop and we wanted to attract add-in developers that wanted to bring their IDE extensions to all three platforms.

MonoDevelop has recently been getting some nice community contributed plugins like Flash/Flex development support, Vala language support, Mono debugger for OSX (thanks to the nice folks at Unity for this!), VI editing mode and of course our own Silverlight and ASP.NET MVC add-ins.

My theory is that supporting MonoDevelop on all the three major operating systems will have a multiplication effect in terms of contributions to MonoDevelop: it will help both users and will enable developers that extend MonoDevelop with add-ins to reach more users.

I secretly want Unity to adopt MonoDevelop as the code editor for Unity; for the FlashDevelop guys on Windows to adopt MonoDevelop as their cross-platform foundation (their users want a cross platform Flashdevelop); for Flashbang to bring their UnityScript framework to MonoDevelop

Developing an add-in for MonoDevelop now brings your enhancements to a much larger community.

Look and Feel

Although the IDE is built using Gtk#, but we are aware that developers want to get things integrated with their operating system as much as possible. This is why we have invested in properly integrating MonoDevelop with the Mac and Windows.

The Look of MonoDevelop still has a heavy feel of the Linux Gtk+, but we are bluring the lines by making the theme and style match the operating system. Development in Gtk native themes will also continue to improve things.

Feel wise, we make MonoDevelop follow the conventions of the host platform. For example, on the Mac, MonoDevelop uses the Mac system menu, it uses an entirely different keybinding style that follows what every Mac developer expects (Command-KEY operations that match X-Code for example) and even text selection in the editor behaves differently:

More work will come, because we want MonoDevelop to feel native on each platform.

On Windows for example, MonoDevelop runs on top of the .NET Framework and uses the .NET managed debugger instead of using Mono's runtime and Mono's debugger, so there is no dependency on Mono to be installed on the system.

Moonlight 2 Preview 3

Another week of excellent work on the Moonlight universe and we bring you our Preview 3 release of Moonlight. Alan McGovern said it best though.

This week stats:

This is what the Silverlight Toolkit Sample page looked with Preview 2:

Moonlight 2 Preview 2

This is what the Silverlight Toolkit page looks with Preview 3:

Moonlight 2 Preview 3

You should be able to update directly from Firefox, or if you are trying it for the first time, go to our http://go-mono.com/moonlight-preview/ page.

Now, although Preview 2 was able to run the IronPython mini-Web IDE I am still going to try to pass that as a new feature.

And now you can try the IronPython mini-Web IDE!

Three Melon's Quest for R2-D2

I had the honor to meet the Three Melon developers at Unity's party this year at the Game Developer Conference. Three Melons is an Argentinian-based company that builds online games, and most recently they have been using Unity to build their new online games.

Today they just announced that their latest online game powered by Unity and Mono. "Quest for R2-D2" is now live at Lego.com:

Pato Jutard, co-founder of Three Meleons announced the launch and posted the Making of Lego Star Wars game by Three Melons:

Congratulations to the Three Melon developers for their launch!

You can follow them on twitter.

Moonlight 2 Preview 2

As we promised last week (threatened?) now that the foundation for Moonlight 2 is in place, we will be doing weekly releases for folks to try out, and to increase the feedback that we have received.

Thanks to everyone that provided bug reports and contacted us about the problems with last week's preview. In particular the issue affecting Ubuntu and SLED 10 users has been fixed and the plugin should work.

If you installed Moonlight already, you can update either by restarting Firefox or by following these steps:

Click "Find Updates":

Micro Focus Video

My friend Stephen sent me an interesting video they just cooked up at Micro Focus giving some hints as to what is coming up on the keynote of Micro Focus Live.

They recently bought Borland.

Video: Micro Focus Live Developers Keynote.

Developing Silverlight Apps on Linux and MacOS with Moonlight

Earlier this week I promised I would blog about how to build Silverlight apps in Linux. Michael beat me to this and did a couple of screencasts.

In Moonlight Development in Linux with MonoDevelop he walks us through the steps necessary to install the Moonlight SDK on top of Mono 2.4 and using MonoDevelop to create your app. Once you get these installed, here is how you get started with development:

MonoDevelop will provide auto-complete for the Silverlight APIs and also provide auto-complete in XAML files.

In addition to Linux, you can also use MonoDevelop on OSX to do the same thing. We shipped Moonlight's SDK as part of the MonoDevelop/OSX release, the result runs with Microsoft's Silverlight.

Michael again talks about it and produced a nice screencast:

I, for one, welcome our new SH4 embedded overlords.

ST Microelectronics, the maintainers of GCC-CIL (the GCC code generator backend that produces ECMA CIL bytecodes for Mono/.NET) have announced their port of Mono to the SH4 platform.

The code is currently available from http://code.google.com/p/mono-sh4/.

In addition to the MIPS64 port that I mentioned last week from SiCortex/nIX is now being merged into Mono's repository.

MIPS apparently is not only used in super computers, but apparently is also powering $169 dollar netbooks.

Innovation

Dana Blankenhorn comments on the the Moonlight 2.0 release and says:

It's the kind of open source success story Microsoft wants publicized. Microsoft innovates, open source copies.

It's not the kind of open source story open source needs, however.

What open source needs is real innovation, created by teams who may or may not represent Microsoft's fierce competitors. This can be hard to deliver, and Microsoft would like us all to know resistance in this case is futile.

A few points.

First: Moonlight's core is designed to ensure that Linux users get access to content that is produced for Silverlight on the web.

This is about making sure that Linux (and for that matter any other system where Moonlight can be compiled) does not become a second class citizen on the web.

Folks will argue all day whether the Silverlight model is the right one; whether it is gaining adoption; whether it is necessary; whether it is part of the open web.

But none of that matters when trying to access content on Linux: it is either possible to use it, or not. And having a working relationship with Microsoft allows us to bring it to Linux.

Second: Dana is looking in all the wrong places for innovation. If he wants to see my team's work that deviates from the set of APIs that Microsoft has created, he could look at our work on SIMD; our interactive C#; Static compilation technology to support things like the iPhone; our cross platform MonoDevelop (Linux, OSX, Windows); Our Gtk# API for building the above; He could look at all of our Mono.* classes, or all of the libraries and APIs produced by our community (Mono.Addins, Mono.Nat, Mono.ZeroConf, BitSharp, Cecil, CocoaSharp, MonoObjc, Crimson, and some forty others; I just got tired of going through the list here and here).

All of these created to solve a particular problem with the tools that we had on the platform we used.

Or for that matter, even reading the announcements on my blog.

Or he could look elsewhere in the vast universe of open source projects for ideas that match his definition of innovation. Not everything that is built in the open source world has to be about innovating in a completely new direction.

Third, and most important one: The definition of Innovation.

Innovation

Most people that discuss innovation have not even bothered to actually think about what this means in the first place. And I am particularly bothered when people claim that open source does not innovate, but can only copy.

Google's define:innovation is a good starting point.

Are Ideas Innovations? Everyone has ideas, even great ideas. Every day you go to lunch, every day you are taking a shower, every day you are walking alone and thinking you are having new ideas.

You can have a million ideas, and these might be innovative, but if they do not reach the world, did they matter?

For an idea or an innovation to have a practical effect, they need to go beyond the discussion at the lunch table with your friends and become a reality.

Bringing an Idea to Life Once I sit down and turn my idea into an actual tangible result there are a number of hurdles in my path.

The idea must be good enough for people to try out, I must get it distributed, and I must get people to use it.

Being first versus being to market first: It does not matter that many great ideas originate in the open source world or at the lunch table with your friends. You must bring the ideas to the public and the public must be in a position to adopt it.

For instance, Compaq/Digital were showing portable MP3 players based on Linux years before the iPod took the world by storm. Yet, nobody remembers these devices anymore and Apple gets the credit for bringing digital audio to the masses.

Or tagging and searching your email. GMail uses it today, but few people remember that the idea had been implemented before in Digital's Pachyderm.

Many ideas might originate as personal prototypes or even open source prototypes, but without a distribution channel and an ecosystem that would sustain the innovation many of those ideas exist merely to be replaced by folks in a better position to market/distribute it.

Claiming "I had the idea first" or "we were the first ones" is of little consolation if someone out-executes and out-markets you.

Definitions of Innovation: Wikipedia (as of 10 seconds ago) defines Innovation as:

The term innovation means a new way of doing something. It may refer to incremental, radical, and revolutionary changes in thinking, products, processes, or organizations. A distinction is typically made between invention, an idea made manifest, and innovation, ideas applied successfully.

I also like this one (from Google's define: too):

Innovation is the process that translates knowledge into economic growth and social well-being. It encompasses a series of scientific, technological, organizational, financial and commercial activities.

I think that Moonlight fits this definition perfectly well in that case.

Moonlight and Innovation

Sure, we do follow the APIs that Microsoft set for Silverlight.

But we have innovated in a number of ways:

• We innovate in the development process: we are an open source project, taking contributions and consuming other open source components.

  We dogfood other FOSS components, and we iterate to improve them.

  In 1999, the MIT Technology Review magazine named me the innovator of the year. In the award ceremony, Bob Metcalfe said that I was receiving the award for the work on Gnome, not because Gnome was a ground-breaking system, but because the goals and processes of Gnome were.

• We innovate in our cross-platform stance: Moonlight runs on more platforms (although admittedly, with a smaller market share than Windows and MacOS).
• We had out-of-browser support before Silverlight did: Sure it was trivial, but we had it before (see above note on futility of claiming "we were here first").
• Working with Microsoft: Ok, this is a bit of a stretch to claim as innovation, but here it goes: I am very proud that we are finding ways of working with a company that for many years resisted Linux and open source as well as helping Microsoft with all of its inertia become more open source friendly.
• Update: Mats on the comments points out something that I wish I had thought of when I wrote this: "Innovation is also, and I'd say mostly if history teaches us anything, about building and improving upon ideas - not just making new ones from scratch."

We (the Mono/Moonlight team) are not Dana's beacon of revolutionary change. But it is no secret that we are fans of the CLI virtual machine, and we believe that giving developers this platform will help them turn their ideas into innovations by giving them the best technologies available.

Users of Mono and Moonlight have already demonstrated that they have way better ideas than I have ever had. And they already have used Mono in brilliant ways. Dana might want to check my blog more periodically to take note of those innovations.

Mono talk at the Beantown.NET Users Group

Tomorrow, Joseph, Michael, Gonzalo and myself will be talking at the Beantown.NET Users Group about Mono, Moonlight and MonoDevelop at 6pm.

If you are in the Boston area, come join us for these open source festivities at Microsoft NERD center in Cambridge.

MonoDevelop on MacOS X

Good news for all OSX users, a build of MonoDevelop that integrates with OSX is now available:

MonoDevelop OSX-ified.

For the impatient among you, click and run MonoDevelop.app.zip.

It requires Mono 2.4 for OSX, on the upside, you can use this to develop ASP.NET MVC apps on the Mac.

Some Background

Back in February I showed a screenshot of MonoDevelop 2.0 for the Mac, it looked like this:

Super-Alpha-Preview of MonoDevelop on OSX.

It was basically the Gtk# based MonoDevelop IDE binary executing on the Mac. There was no porting involved, just the same executable running under Mono/OSX. The code works, but it did not feel like a Mac app:

• The Menu bar was embedded in the window.
• The keyboard accelerators were the Linux ones. They felt unnatural for OSX users, and also did not take advantage of the spare key (Command) to liberate the control key for other uses.
• The editor behaved like a Linux editor.

There is a vibrant Mono on OSX community out there, and it will only grow larger. We wanted to make sure that all of the work that is going into creating a great IDE is available for folks on the Mac in a way that is actually comfortable to use.

So working with folks in the Mac Community and with folks at Unity Michael has been working on tuning up MonoDevelop on the Mac to become an editor that does not get in the way of Mac users and developers and integrates better from the "feel" perspective with other tools in the OS.

For instance, not only does the new MonoDevelop for MacOS use the Mac menu bar, and the Mac accelerators (a combination of XCode and Textmate accelerators), but even the text editor has been altered to support the way selection and navigation works on the Mac.

I figured that for every 100 users of MonoDevelop one of them will contribute patches back to the effort. If you happen to be that 1% hacker that will contribute back, you might want to look at a list of ideas to improve MonoDevelop on the Mac.

MonoDevelop on Windows.

MonoDevelop on Windows is on a similar boat: the 2.0 release works "out of the box" on Windows, but again, it is a GNOME IDE in a Windows land.

Stay tuned for news from the MonoDevelop community as to what will happen there.

Update: Lluis posted an update on the progress of MonoDevelop on Windows.

Smooth Streaming with Moonlight

I never tested Smooth Streaming before, just the more basic media tests, but the Smooth Streaming stack is running with our Moonlight preview:

You can see a few rendering glitches for the controls on the screen. But this is really good news for our new media pipeline.

First Moonlight 2.0 Preview is Out

After a loving incubation period, the Moonlight 2.0 preview, an open source implementation of Microsoft's Silverlight for Linux has been released:

• source code (tar.bz)
• Firefox/Linux binaries for x86 and x86/64
• Release Notes
• Sites that we are tracking.
• Chris Toshok's blog with more details (Chris is the Moonlight team lead).

This is really the release I have been looking for since Microsoft first introduced Silverlight 1.1 and ever since our 21-day hack-a-thon to bring Silverlight to Linux.

This is the ECMA VM running inside the browser and powering C# and any other CIL-compatible languages like Ruby, Python and Boo. You can use Moonlight/Silverlight as a GUI (this is what most folks do) or you can use it as the engine to power your Python/Ruby scripting in the browser.

Installing Moonlight 1.9

Go to our preview page, select the platform and hit the download icon.

That will download and install the plugin in your Firefox installation. You can then restart the browser, and you should see this:

Then you can try out some of the test web sites we have been working on. This is CNN's The Moment that uses Silverlight/Photosynth:

If instead of binaries you want to build Moonlight in the comfort of your own living room while sipping margaritas, fetch the source code for mono, mcs, mono-basic and moon from the branch and build them in this order: mono, mono-basic and moon.

While one hand holds your margarita, use the other one to follow the instructions on how to compile Mono from SVN.

Some Notes on this Release

Now some qualifications to this release:

This is a preview release. By this we mean that we are not yet feature complete with Silverlight 2.0 feature-to-feature but we are relatively close. For example, we do not yet pass the entire Silverlight GUI 2.0 test suite that was provided to us by Microsoft and you can spot glitches in various web sites.

Security Sandbox: One of the reasons we delayed the first preview of Moonlight for public consumption was that we did not want to release Moonlight without the security sandbox. In the pre-Moonlight days there was no reason for Mono to implement a security sandbox, so we never had it. With Moonlight a security sandbox is mandatory so we implemented it.

Moonlight 2.0 ships with the new CoreCLR Sandbox that was introduced in Silverlight 2.0. This security system is very easy to understand, it is pretty straightforward and is a lot easier to secure and audit than something like CAS. I will blog about the security stack in another post.

But even if we now have a security sandbox , we have not completed the security audit.

Weekly Releases: Our current plan is to update the plugin once a week during this preview/alpha period hoping that we can get good bug reports and to ensure that we work in as many Linux distributions as possible.

Debug Builds: During the preview/alpha cycle we are shipping our code with debugging symbols hoping that this will improve the quality of the bug reports that we receive. This means that the plugin size instead of being 3.9 megs is 8.8 megs on average. This will change when we do the final release.

The Cool Toys

There are a number of cool toys on this release, the foundation for many things to come. Here are some:

Silverlight Unix SDK: If you install Mono 2.4 and Moonlight SDK (not the browser plugin, but the -devel package) you can now develop Silverlight applications entirely in Unix.

In fact when you install Eclipse4SL (a Microsoft sponsored project) you need Mono 2.4 to build Silverlight apps. With the Moonlight SDK you can skip an entire step by having the SDK assemblies present at installation time.

I will do another blog on how to build Silverlight apps from the ground up on Unix using the Moonlight SDK.

Microsoft MS-PL Controls: Instead of reimplementing the high-level controls (buttons, Checkboxes, listboxes, containers, calendars, datepickers, sliders) or the very advanced controls (like a full database bound datagrid) Moonlight reuses Microsoft's open sourced Silverlight controls.

Iron* Languages: In addition to C# you can run code written in a variety of programming languages that target the ECMA CLI. In particular dynamic languages.

IronRuby and IronPython are open source implementations of Ruby and Python done by Microsoft that can be used in Silverlight but you can also use a variety of other languages in the browser like Visual Basic or PHP (Phalanger).

Visual Basic Runtime: This is just a plug for the work that Mainsoft did a few years ago. One of the things that Silverlight ships with is a Visual Basic class library for all the VB helper functions.

Mainsoft contributed a few years ago a VB runtime written entirely in VB

We ship a "tuned" version of their assemblies as part of the Moonlight release.

Adaptive Streaming: This also deserves a blog entry of its own. In addition to the support for HTTP-Streaming (to support seeking and stream quality selection) Silverlight allows developers to create their own transports to fetch media and not be limited by HTTP.

For instance, a developer could write a transport that fetches different bits of the media from different servers. Or use bittorrent to fetch the media instead of depending on a single server. More in an upcoming blog entry.

DeepZoom: with all of the bells and whistles that you expect.

Hard Rock Memorabilia on Moonlight/Linux.

Silverlight 3.0 APIs: As we were implementing the 2.0 APIs a handful of features from 3.0 fit naturally into our design. So instead of going the extra mile to limit things in 2.0, we just expose the 3.0 APIs in a forward-compatible fashion.

This Moonlight preview includes a few 3.0 features:

• Out-of-browser support (although this is currently a manual process, not yet automated, and without a GUI).
• WritableBitmap class.
• 3.0 pluggable media pipeline.
• SaveDialog support.

There are more details see Chris Toshok's blog entry.

The pluggable media framework is very exciting to us, because it means that developers can author their own codecs without waiting for Silverlight or Moonlight to add support for it.

We have developed a handful of open source codecs for Dirac, Vorbis and ADPCM that can be used with Silverlight 3/Moonlight Preview based on existing C# and Java implementations. Hopefully someone will help us fill in the blanks with more codecs (like Theora).

For up-to-date news check out our README file.

In the words of Paris Jobs, this release is nothing short of hawt.

Moonlight Twitteristas

If you want to follow the progress of various Moonlight activities on Twitter, you can follow these folks:

• moonproject: the twitter account for project news.
• kumpera: in charge of the Mono metadata and IL verifier and creator of Mono.SIMD.
• spouliot: Mono's security and cryptography Czar and graphics hacker.
• lewing: Moonlight core hacker, in charge of Moonlight's layout and performance and also the creator of the Linux logo and co-developer of F-Spot.
• toshok: Moonlight team leader and maintainer of the core of Moonlight. ex-Winforms team lead.
• jbevain: In charge of the Silverlight 2.1 API, Mono's Linker and Cecil.
• kangamono: Codec hacker, hard-problem solver, iPhone support, Mono on Mac.
• atsushi_eno: XML, LINQ to XML, WCF and pluggable codec author.
• sh4na: Recovering winformista, in charge of the new toggle ref implementation and the Moonlight/Javascript bridge.
• rolfkvinge: media pipeline, visual basic runtime, vbnc compiler (Mono's VB compiler written in VB).
• jstedfast: text layout engine, and ex-Evolution hacker.
• jacksonh: All things XAML, a recovering Winformista/Ilasm-ista. A must-follow twitterista, considering that even Aimee_b_loved follows him.
• rustyhowell: release engineering.

Some of the team members are not twitteristas yet.

Alternatively, if you are not really into twitter, you can always check our aggregated blogs at monologue.

Interactive C# Code Completion

Last month I introduced code completion in Mono's Interactive C# shell. You can use the TAB key to auto-complete at the current cursor position. Pressing the TAB key twice will list all possible completions.

This should make the csharp more pleasurable to use and for bash junkies like me a more natural fit.

This is particularly useful to explore an API like Gtk#:

	csharp> LoadPackage ("gtk-sharp-2.0");
	csharp> using Gtk;
	csharp> Application.Init ();
	csharp> var w = new Window ("Hello");
	csharp> w.SetF[tab]
	SetFlag SetFocus SetFrameDimensions
	csharp> w.SetFo[tab]
	csharp> w.SetFocus ();
	

This comes in quite handy for completing namespaces, types and valid methods. It works with the C# 3.0 initializer syntax as well, that one is useful in Silverlight for those of us that can not stand to type XAML instead of C#:

	csharp> new TextBlock () { Bac[tab]
	

Does the nice:

	csharp> new TextBlock () { Background
	

Bonus points: another tab at that point inserts the equal sign to assign the value.

This was done by extending the Mono.CSharp.Evaluator API to provide code completion.

The API is fairly simple:

public static string [] GetCompletions (string input, out string prefix)
	

This will provide possible completions (methods, properties, fields) that are valid at that point in the string.

A discussion that details the implementation of how the compiler supports code completion is in the mailing list and our compiler documentation has been updated to include a tutorial on expanding code completion.

The next step is to implement this for the interactive GUI shell.

Gtk# 2.12.8 Installer for Windows

Mike Kestner yesterday announced the availability of the new Gtk# installer for Windows.

A few good news: the entire stack (Gtk+, Cairo and Gtk#) comes in a nice 8 meg download, it is packaged as an MSI and it is now signed by Novell's certificate, so you no longer get a scary "Unknown Publisher" dingus on the screen.

This is the equivalent of the greek god Prometheus giving fire to humans.

We are giving Windows developers a nice cross-platform toolkit that is nicely integrated into Visual Studio. To try a sample application using it, you can download Tomboy, load the Tomboy.sln solution, hit F5 and enjoy.

PseudoTerminal class

As a follow up to yesterday's post I did the "hard work" of cut-and-pasting the VTE pseudo-terminal support+gnome-pty-helper into an independent module and wrote a managed binding for the code, autoconf-ified it and put it on SVN.

Code lives in the pty-sharp module, or you can get a tarball.

Now someone needs to do the trivial hack of writing the Mono terminal emulator.

Pseudo Terminals and Terminal Emulators

There was a discussion about how to host REPLs in applications like MonoDevelop recently and some of the discussion was centered around how to host something like a shell into a program like MD.

Since I have been thinking about building a Silverlight-based version of the Midnight Commander (OH NOES!) I figured I should share some thoughts that I had on this matter.

Widgets like ZVT and VTE today bundle a number of things in a single widget:

• A Unix Pseudo terminal with login/logout support (Unix flavor-specific).
• A terminal emulator that turns escape sequences generated by a stream into rendering commands.
• A binding to the Gtk+ toolkit to send input events to the child process and render the terminal results.

I would like to see some nice C# libraries for doing each one of those tasks independently of each other. Think of it as the MVC of terminal widgets. Like this:

Reusable Blocks for Terminal Emulation.

Pseudo-terminal support: the functionality to create pseudo terminals is very OS-specific, it is hard to get right and getting the more advanced features like registering your session is even harder. Very few applications get this right (mc, zvt and vte all use the same code that took me years to fine tune but has never been made reusable for other applications).

This can be used beyond terminal emulators, it can be used to script or control programs expect a real terminal to run. These are typically interactive Unix console applications.

Those applications either break or refuse to run when their standard input or standard output are redirected. Expect is a system built on top of this functionality

Terminal Emulation: A terminal emulator class that supports the vt100/xterm command set and render it into some internal buffer; can take high-level keystrokes and encode them as byte streams (for example turning Alt-X into ESC-x) and supports terminal resizing.

This terminal emulator should not be bound to Gtk+ it should merely render into a text buffer.

Gtk#, Silverlight and Curses Bindings: Once the underlying terminal emulator exists we will need to write a handful of bindings to the terminal emulator.

The Gtk# is an obvious choice for embedding terminal emulators inside things like MonoDevelop.

The Silverlight binding would allow people to create full fledged SSH clients on the web.

The curses binding could be used to implement an application like GNU Screen or it could be used in an application like the Midnight Commander (the Midnight Commander plays some Unix tricks to avoid having to emulate a terminal, and this has been a small weakness).

Banshee and Tomboy over the weekend

Over the weekend a couple of interesting post were made:

In "Fitting the Kitchen Sink into a CD" Jo from the the Debian/Ubuntu Mono developers and describes how the way they have split Mono on Debian/Ubuntu makes it so that replacing Rhytmbox with Banshee Media Player ends up consuming less space on Ubuntu's LiveCD (6 megs) and brings more features.

Nice to see that using managed code consumes less space and delivers more features. There is a heated debate on the comments as well.

Sandy Armstrong also posted an update on the desktop note-taking application Tomboy that now runs on Windows and MacOS X.

Sandy was just saying a few weeks ago that porting Tomboy to Windows brought new developers to the project. Although some people have historically been against the idea of making Linux software available on other platforms, it is nice to see day-to-day validation that by expanding the scope of our open source software to other platforms it directly improves the software in our own platform (as many predicted).

Common Compiler Infrastructure and Cecil.

Last week at Lang.NET 2009 conference the Common Compiler Infrastructure was open sourced under the terms of the MS-PL license.

This library was developed used internally at Microsoft some years ago to support some internal projects. This provides a set of services similar to our own Cecil library. Despite this, it is nice to see Microsoft open source more code.

Cecil, in addition to the low-level APIs for reading and writing ECMA CIL files has a few niceties layered on top of it like a Flowanalysis engine (it is used to decompile byte codes into ASTs) and a full fledged decompiler.

Additionally, Marek is currently replacing the backend in our C# compiler to move away from System.Reflection.Emit into using Cecil so that we can bring our C# REPL to Windows developers.

Mono's C# REPL currently works in a very limited mode in .NET (no generics, no LINQ) because .NET's System.Reflection has several limitations for building full-fleged compilers. To work around this issue Mono has over the years extended the Reflection stack to provide the features that were missing. We were never quite happy with this and we are now dropping it in exchange for Cecil.

JB, the creator of Cecil shares with us his take on Cecil and the CCI.

Job Posting: Mono Hacker Looking for a Job

A good friend of mine that has extensive experience with C# and has written significant portions of the Banshee media player, has contributed to our C# 3.0 and C# 4.0 support and to our runtime is looking for a programming job.

If you want to hire him, drop me an email and I will get you in touch with him.

Continuations in Mono: Embrace and Extending.NET, Part 3

Update: I accidentally published an incomplete version of this blog entry the other day before the actual content and samples were ready.

As part of our ongoing Embrace and Extend.NET series (SIMD, supercomputing), today I wanted to talk about the Mono.Tasklets library that has just landed on Mono's repository.

This library will become part of the Mono 2.6 release.

Mono.Tasklets is the Mono-team supported mechanism for doing continuations, microthreading and coroutines in the ISO CLI. It is based on Tomi Valkeinen's excellent work on co-routines for Mono.

Unlike the work that we typically do in Mono which is pure C# and will work out of the box in .NET (even our Mono.SIMD code will work on .NET, it will just run a lot slower) Mono.Tasklets requires changes to the VM that are not portable to other ISO CLI implementations.

History

Unity Early Experiments

Back in 2004 when Unity first started to explore Mono, Joachim Ante brought up the topic of coroutines in Mono. On an email posted to the mono-devel-list he stated:

I want to be able to write scripts like this:

	void Update ()
	{
	    Console.WriteLine ("Starting up");
	    //Yields for 20 seconds, then continues
	    WaitFor (20.F);
	    Console.WriteLine ("20 seconds later");
	}
	

The WaitFor function would yield back to unmanaged code. The unmanaged code would then simply go on, possibly calling other C# methods in the same class/instance. After 20 seconds, the engine would resume the Update method.

The idea here is to have multiple execution paths running on a single thread using a sort of cooperative multitasking. GUI programmers are already used to this sort of work by using callbacks: event callbacks, timer callbacks and idle callbacks. In Gnome using C or C# 1.0 you use something like:

	void do_work ()
	{
		printf ("Starting up\n");
		g_timeout_add (20 * msec, do_work_2, NULL);
	}

	void do_work_2 ()
	{
		printf ("20 seconds later\n");
	}
	

Although lambdas help a little bit in C# 2.0 if the core of your application needs to chain many of these operations the style becomes annoying:

	DoWork ()
	{
		Console.WriteLine ("starting up");
		Timeout.Add (20 * msesc, delegate {
			Console.WriteLine ("20 seconds later");
		});
	}
	

In event-based programming everything becomes a callback that is invoked by the main loop. The developer registers functions to be called back later in response to a timeout or an event.

Another alternative is to build a state machine into the callbacks to put all of the code in a single method. The resulting code looks like this:

	void do_work (void *state)
	{
		MyState *ms = (MyState *) state;
	
		switch (ms->state){
		case 0:
			printf ("starting up\n");
			ms->state = 1;
			g_timeout_add (20 * msec, do_work, state);
			break;
		case 1:
			printf ("20 seconds later");
		}
	}
	

It is worth pointing out that Joachim and in general the gaming world were ahead of our time when they requested these features. This style of threading is commonly referred as microthreading or coroutines.

At the time, without runtime support, Rodrigo suggested that a framework based on a compiler-supported generator-based system using the yield instruction would satisfy Joe's needs for coroutines in the gaming space.

This is what Unity uses to this day.

C# Yield Statement in Mono

The yield statement in C# works by building a state machine into your method. When you write code like this:

	IEnumerable DoWork ()
	{
		Console.WriteLine ("Starting up");
		yield return new WaitFor (20 * msec);
		Console.WriteLine ("After resuming execution");
	}
	

The compiler generates a state machine for you. In the above example there are two states: the initial state that starts execution at the beginning of the function and the second state that resumes execution after the yield statement.

A year later we used a variation of the above by employing nested yield statements in C# to implement Mono's HttpRuntime pipeline stack.

Cute screenshot from my blog at the time:

Yield statements can be used to solve this class of problems, but they become annoying to use when every method participating in suspending execution needs to become an iterator. If any part of the pipeline is not built with explicit support for yield, the system stops working. Consider this:

void Attack()
{
	 DoTenThings ();
}

void DoTenThings()
{
	  for (int i=0; i < 10; i++){
	      C();
	  }
}

IEnumerable C()
{
	   yield WaitForIncomingMessageFromNetwork();
}
	

Here, even if the WaitForIncomingMessageFromNetwork uses yield the callers (DoTenThings and Attack) are not participating, they merely discard the return from yield, so the execution does not return to the main loop.

Using a yield-based framework is not much of a problem if you only need to use this every once in a while. For example we use this in our ASP.NET engine but it is only used in a handful of places.

Unity used an approach built on top of the yield framework to suspend and resume execution. For example this is invoked by the Update() function on an enemy script:

function Patrol() {
	while(true) {
		if (LowHealth ())
			RunAway();
		else if (EnemyNear ())
			Attack();
		else
			MoveSomewhere();
		yield; // done for this update loop!
	}
}

function Attack () {
	while (!LowHealth () && EnemyNear ()) {
		DoTheAttack ();
		// done with this update, and wait a bit
		yield WaitForSeconds (attackRate);
	}
	// return to whatever invoked us
}
	

The same can be done in Unity with C#, but your functions should be declared as returning an IEnumerable.

Microthreading in SecondLife

In 2006, Jim from LindenLabs introduced the work that they had done in SecondLife to support microthreading.

Jim's work was a lot more ambitious than what both Joe had requested. SecondLife required that code be suspended at any point in time and that its entire state be serializable into a format suitable for storage into a database. Serialized state could then be restored at a different point in time or on a different computer (for example while moving from node to node).

For this to work, they needed a system that would track precisely the entire call stack chain, local variables and parameters as well as being able to suspend the code at any point.

Jim did this by using a CIL rewriting engine that injected the state serialization and reincarnation into an existing CIL instructions stream. He covered the technology in detail in his Lang.NET talk in 2006.

The technology went in production in 2008 and today this continuation framework powers 10 million Mono scripts on SecondLife.

Tomi's Microthreading Library

That same year Tomi posted a prototype of coroutines for Mono called MonoCo, inspired by the use of Stackless Python for EVE Online.

The Stackless Python on EVE presentation goes through the concepts that Tomi adopted for his Mono.Microthread library.

Tomi's approach was to modify the Mono runtime to support a basic construct called a Continuation. The continuation is able to capture the current stack contents (call stack, local variables and paramters) and later restore this state.

This extension to the runtime allowed athe entire range of operations described in the Stackless Python on Eve presentation to be implemented. It also addresses the needs of developers like Joachim, but is not able to support the SecondLife scenarios.

Mono.Tasklet.Continuation

The Mono.Tasklet.Continuation is based on Tomi's Microthreading library, but it only provides the core primitive: the continuation. None of the high-level features from Tomi's library are included.

This is the API:

	public class Continuation {
		public Continuation ();
		public void Mark ();
		public int Store (int state);
		public void Restore (int state);
	}
	

When you call Store the current state of execution is recorded and it is possible to go back to this state by invoking Restore. The caller to Store tells whether it is the initial store or a restore point based on the result from Store:

	var c = new Continuation ();
	...

	switch (c.Store (0)){
	case 0:
		// First invocation
	case 1:
		// Restored from the point ahead.
	}
	...
	// Jump back to the switch statement.
	c.Restore (1); 
	

Tomi implemented a Microthreading library on top of this abstraction. I ported Tomi's Microthreading library to Mono.Tasklet framework to test things out and I am happy to report that it works very nicely.

Tomi's patch and library were adopted by various people, in particular in the gaming space and we have heard from many people that they were happy with it. Not only they were happy with it but also Paolo, Mono's VM curator, liked the approach.

Frameworks

Speaking with Lucas, one of the advocates of Tomi's VM extensions, at the Unite conference it became clear that although the Mono.Microthreads work from Tomi was very useful, it was designed with the EVE scenario in mind.

Lucas was actually not using Mono.Tasklets on the client code but on the server side. And when used in his game the Stackless-like primitives were getting on his way. So he used the basic Continuation framework to create a model that suited his game. He uses this on his server-side software to have his server micro-threads wait for network events from multiple players. The Tomi framework was getting in Lucas' way so he created a framework on top of the Continuations framework that suited his needs. He says:

I found however, that what system you build on top of the core continuations tech, really depends on what kind of application you're building. For instance, I have a system where I send serialized "class ProtocolMessage" 's over the network. they have a questionID and an answerID, which are guids.

in my code I can say:

	// automatically gets questionID guid set.
	var msg = new RequestLevelDescriptionMessage();         
	someConnection.SendMessageAndWaitForAnswer (msg);
	

the last call will block, and return once a message with the expected type, and matching answerID has been received. This is made to work because the SendMessageAndWaitForAnswer<T>() call adds itself to a dictionary<GUID,MicroThread> that keeps track of what microthreads are waiting for which answer. A separate microthread reads messages of the socket, and reads their answerID. it then looks to see if we have any "microthreads in the fridge" that are waiting for this message, by comparing the guid of the message, to the guid that the microthread said it was waiting for. If this is it, it reschedules the microthreads, and provides the message as the return type for when the microthread wakes up.

This is very specific to my use case, others will have things specific to their use cases.

Going back to the Joachim sample from 2004, using Tomi's code ported to Mono.Tasklets, the code then becomes:

	void Update ()
	{
	    Console.WriteLine ("Starting up");
	    //Yields for 20 seconds, then continues
	    Microthread.WaitFor (20.F);
	    Console.WriteLine ("20 seconds later");
	}
	

The MicroThread.WaitFor will suspend execution, save the current stack state --which could be arbitrarily nested-- and transfer control to the scheduler which will pick something else to do, run some other routine or sleep until some event happens. Then, when the scheduler is ready to restart this routine, it will restore the execution just after the WaitFor call.

A sample from the game world could go like this:

	prince.WaitForObjectVisible (princess);
	prince.WalkTo (princess);
	prince.Kiss (princess);
	

The code is written in a linear style, not in a callback or state machine style. Each one of those methods WaitForObjectVisible, WalkTo and Kiss might take an arbitrary amount of time to execute. For example the prince character might not kick into gear until the princess is visible and that can take forever. In the meantime, other parts of the game will continue execution on the same thread.

The Continuation framework will allow folks to try out different models. From the Microthread/coroutine approach from Tomi, to Lucas' setup to other systems that we have not envisioned.

Hopefully we will see different scheduling systems for different workloads and we will see libraries that work well with this style of programming, from socket libraries to web libraries to file IO libraries. This is one of the features that Lucas would like to see: Networking, File and other IO classes that take advantage of a Microthreading platform in Mono.

Boston .NET Users Group Presentation

Today both Joseph and myself are doing a presentation on Mono, Moonlight at the Boston .NET Users Group meeting in Waltham.

We will be demoing the Visual Studio integration and remote debugging capabilities of Mono as well as SuseStudio to go from an ASP.NET application into an appliance with a handful of clicks.

Update

The actual post on Mono continuations and coroutines is available here.

Monday Mystery: Poetry Showing up on my Surveys.

Yesterday I was pondering whether I should go to Lang.NET 2009 and unable to make a decision, I ran an online poll and asked people to vote on twitter:

In my survey I listed the pros and cons to give people a feeling of what I was up against.

The first couple of votes were not very helpful, I got 2 votes, one said yes, one said no.

Then another twenty or so votes came and the balance started to shift towards "go". Here is the final result:

And then something very strange happened, the comment section for the online poll started getting people's opinion in the form of poems or Haikus.

I copy pasted some of the poems below:

In my heart you will always be now in heaven i cant wait again to see

When I decided not to roam
I just stayed home
Got a lot done
Enjoyed some time in the sun

---

Listen to good talks with your friends
Before all the fun ends!

Catch up and learn about Google's V8
--Maybe you'll even get a date!

You should go because it will be a blast
Life is short, so make it last!

---

a haiku for miguel:

Improve your cv
Boston is lovely right now
Drink with nerdy folks

---

I can’t make up my mind.
I really am in a bind.
I can go to Lang dot net,
And my day will be set.
Or stay at home and work,
Like a common store clerk.
This choice will be easy.
Lang is where I will be.

Should I stay,
or should I go?
For every con,
there is a pro.

I could see old friends,
and make some new.
Or stay at home,
and get work done too.

I could learn about Second Life,
and Google's V-8.
And actually accruing miles
would really be great.

It would take a whole week.
And delay my work,
But learning something new
Is definitely a perk.

A programmer I am,
and a programmer I will be.
I think this great poem
made my mind up for me.

---

New friends
Old friends
Learning new things
And when you get back to Boston
You can have some baked beans

---

destroyer of days seven
bringer of travel expenses
Lang.NET

provider of lectures
connector of colleagues
Lang.NET

delayer of projects
impeder of family time
Lang.NET

programmer of dynamic C#
granter of Second Life
Lang.NET

Why go across the country to have a v8?
Because that's what .NET is 4.
Lang.NET

---

Kill time
or make new friends?
Lose a little time from work
or catch up with those you've fallen out of touch with?
Work can always wait Miguel
opportunity cannnot.

---

Another dollar for yet another day,
That is what they all used to say.
For Lang.NET 2009 I pine away
The cost to do it makes me say “Hey!”
Give me Second Life with which to play
If only I could go to the conference that way
I’d program it if C was their programming thang’

---

Let me help Miguel on what to do.
He's undecided...can't decide between the two.
If he goes to Lang. NET, work at home will pile high.
If he stays home to work, perhaps he'd cry.
He'd miss the chance to learn Second Life,
Or make new friends, maybe even find a wife!

So it costs a few bucks to get to there.
He'd blow it on something else other than air fare!
Make the reservations today and don't delay.
You'll have a good time....that's what I say!

---

Divided I stand
The opportunity to advance ahead
A week of talks and new friends await me
On my way to Lang.NET 2009.

One week gone,
Trekking across the country.
Putting all tasks aside,
I delay work
On my way to Lang.NET 2009.

In the end,
The journey made
Is better than the journey dreamed.

---

Oh the choice, to go or to stay
that is Miguel's question of the day!

Should he go, he would have a great time
Should he stay...he might save a dime!

Going he will learn about new things out there
Staying he might...get more work done fair.

The choice looks easy to someone like me
but then again I'm not Miguel....I'm Susie

---

Once when Miguel was feeling weak,
A conference looked to kill his week.
But opportunity there it seemed was rife:
A chance to master Second Life!

---

Miguel's Dilemma
Should I stay or should I go
The trip could be for not
But my gut says I should go
Money,work,time and home
Oh, well bye bye I go.....

---

knowledge is calling me to fill him up
my friends are quite expecting
there are also some guys I don't know that i have to spy on

who cares about work!
who cares if I hitchhike to Lang.Net!

home will always wait
across the country I will enlighten myself

---

To Lang.NET or not to Lang.NET ... that is the question.
Whether 'tis nobler to learn about great stuff
Like Second Life and dynamic C#,
Or consider it the death of
An entire week away from home,
Thereby missing out on the catching up with old friends.
What value, then, shall be given
This knowledge and friendship?
Is it greater than the cost of transport and per diem?
Only Miguel can truly tell.

---

Lang.NET
Lang.NET
Oh this decision should not make you fret

Think about the things that are great
Second Life! .NET 4 and Google V8!

These things are worth the week away
your delay in work, and your hit in pay

Think about the old friends you'll greet
And all the new ones that you will meet

The knowledge base you'll gain and learn
will help you back help you back home and what you earn

Lang.NET
Lang.NET
Looks like going is really your best bet.

---

Haiku:

While traveling sucks
Experience should be worthwhile
Go enjoy Lang.NET

---

There are times we can't resist
Pondering the things we've missed.
We vow, "Next time I won't be slow.
I'll just pick up myself and go.".
And when we're back, our money spent,
We often think, "I'm glad I went".

---

Lang.Net
Not a bad gig to get
Sure it's a road trip
And a work skip
A budget breaker
But 'cmon,
Ain't it grand to be a taker?

---

Lang.Net
Not a bad gig to get
Sure it's a road trip
And a work skip
A budget breaker
But 'cmon,
Ain't it grand to be a taker?

---

Miguel should fly cross country,
And not worry about the money.
The trip will be great,
You must learn about Google V8.

Leaving home for a week,
Does sound rather bleak,
But you'll chat with old pals,
And may meet some gals.

---

There is much to learn,
but conferences return.
If you stick to your guns,
keep your money in your pocket,

you'll have more in store
when next year rolls around.
So stand on solid ground!,
and in 2010 launch rocket.

Not quite sure why I got so many replies in the form of poems, and Google surveys do not track the IP address of the submissions, so I have no idea if these were all submitted by the same person or not.

Need to get to the bottom of this mystery.

Supercomputing Mono

Last year we did some work in Mono together with Luis Ortiz to support 64-bit arrays in the VM.

What was interesting about this work is that even though the ECMA standard allows the index of arrays to be a long .NET on Windows64 does not support this and Java would require modifications to the bytecode format. Altering Mono became the natural choice for those looking to host very large arrays in an advanced and managed VM.

This means that you can continue to use your existing tools to build applications, but when running under Mono you will get to use those arrays without that pesky 2,147,483,648 upper boundary.

Today Ian Dichkovsky (from N-iX)announced on the mailing list their work to bring Mono to the MIPS64 from SiCortex. This is based on the excellent work from Mark Mason that did the MIPS32 port.

SiCortex has an entry-level desktop computer with 72 MIPS processors and if you have the budget you can get modules with 5,832 processors. The MIPS processors helps them stay eco-friendly.

Telerik Announces Support for their ASP.NET controls on Mono!

Telerik is one of the most famous provider of controls for .NET. We have been working for the past few months with Telerik to make sure that their RadControls for ASP.NET AJAX product worked out of the box with Mono:

Today Telerik announced the availability of their product officially for Mono-based customers on Linux systems. From their press release:

Telerik, the leading vendor of development tools and components for the Microsoft .NET platform announced that RadControls for ASP.NET AJAX fully supports the Mono runtime environment, an open source .NET framework sponsored by Novell, tailored for development of Linux applications. The Telerik AJAX UI components is the first major commercial user interface (UI) suite to go cross-platform and allows developers to build rich .NET applications in a Linux environment. “This has been a long-awaited feature, which we have been quietly working on for quite some time. Over the past few months, we have been actively testing the compatibility of our RadControls for ASP.NET AJAX offering with Mono", said Hristo Kosev, Telerik CTO. "We are extremely happy that our joint work with Novell will allow customers to build compelling high-performance ASP.NET AJAX-based applications and run them on Linux using Mono 2.4.” The decision to work with Novell to extend the capabilities of RadControls over other platforms is in direct response to customer feedback and interest in Mono. Telerik and Novell are optimistic about the effect their partnership will have on the industry and the benefits it will bring to .NET developers.

Telerik is a major player in the control space in the .NET world and many developers turn to them for ready to use controls for their applications. Developers that were previously using Telerik products can now host their products on Linux servers.

Special thanks go to Marek Habersack in the Mono team who worked tirelessly to fix Mono's ASP.NET stack. Working with the Telerik folks was a pleasure. Telerik helped us by providing us access to their source code, their test suite and their QA team that made sure that their thousands of tests ran equally well on Mono as they did on Microsoft's .NET.

You can try the Telerik controls running on Mono at http://mono.telerik.com/.

MonoDevelop Support for ASP.NET MVC

Michael Hutchinson blogs about how to use the recently open sourced ASP.NET MVC framework with MonoDevelop. Go from installing MonoDevelop 2.0 to your first ASP.NET MVC application 3 minutes:

There are a few very simple steps:

• Go to Tools/Add-in Manager.
• Click "Install Add-ins".
• In ASP.NET select "ASP.NET MVC".
• Install.
• Select File/New Solution
• Select ASP.NET MVC project.
• Hit F5 (run) to run your first app:

This will give you basic templates and dialog boxes for solutions, views, controllers and master pages. The code uses Michael's recent implementation of the T4 engine.

Check Michael's Blog for a complete step-by-step setup.

The Add-in bundles Microsoft's recently open sourced ASP.NET MVC engine to run on top of Mono 2.4.

Kudos to Michael that created this add-in in his copious spare time. And kudos to the MonoDevelop team that created such a pleasant platform to extend.

Microsoft releases ASP.NET MVC under the MS-PL License

Microsoft's ASP.NET MVC is an extension built on the core of ASP.NET that brings some of the popular practices and ease of development that were popularized by Ruby on Rails and Django to the .NET developers.

Scott Guthrie ---the inventor of ASP.NET--- just announced that Microsoft is open sourcing the ASP.NET MVC stack under the MS-PL license:

I’m excited today to announce that we are also releasing the ASP.NET MVC source code under the Microsoft Public License (MS-PL). MS-PL is an OSI-approved open source license. The MS-PL contains no platform restrictions and provides broad rights to modify and redistribute the source code. You can read the text of the MS-PL at: http://www.opensource.org/licenses/ms-pl.html

These are incredibly good news. Worth dancing for!

I know that a lot of developers inside Microsoft worked to get this important piece of code released under the MS-PL to ensure that the users of ASP.NET could benefit from the code being open source. I know that at least Phil Haack, Scott Guthrie, Scott Hanselman, Dimitry Robsman, Rob Conery and Brian Goldfarb pushed for this.

I am psyched, not only because ASP.NET MVC is usable in Mono and the code is licensed under open source terms, but also because I strongly believe that the same innovation, rapid adoption and experimentation that has happened with the new wave of web stacks will come to ASP.NET MVC across all platforms.

The source code is available for download and we are hoping to integrate this into Mono shortly. Scott Hanselman has a nice blog entry on how ASP.NET MVC went from price-free to open-source free.

In Scott's PDF tutorial he discussed how to build applications with ASP.NET MVC using Visual Studio and how the Rails practices of not repeating yourself and convention over configuration are used by ASP.NET MVC.

We have developed a MonoDevelop add-in that provides a set of templates, dialog boxes and the tooling necessary to take advantage of ASP.NET MVC on Linux and MacOS X as well. Hopefully the experience will be very similar to Visual Studio.

It was only two weeks ago that we were sipping virgin pina coladas at Mix09:

CoreCLR Security Model

Mono is quickly approaching having a complete implementation of the CoreCLR security model for Mono. This is being developed primarily for use in Moonlight.

This new and simplified security model allows Moonlight to download and execute untrusted code and run it inside a sandbox. A full implementation requires Mono to have an executable image verifier (making sure the binary that we download follows all of the rules and does not try some funny business), an IL verifier that ensures that the code does not contain any unsafe operations and the sandbox system that ensures that the downloaded code only calls methods that it has permission to call.

Click for passable illustration of how the sandbox works.

MSDN has a short introduction to the sandbox and I blogged a long list of links to the original blog entries that documented it.

CoreCLR security can be customized using a handful of attributes. Instead of sprinkling our source code with the attributes and a gazillions #ifdefs we are using our Mono Linker and a few tools and configuration files to reshape our libraries to contain the necessary attributes required to secure the sandbox. We use a number of tools to automate this process and a manual auditing process to audit the results.

This is cool because this is a much simpler sandbox system than CAS ever was and our tools make it very simple for third parties embedding Mono into their applications to create their own sandboxes and reshape what is allowed or not allowed by the sandbox based on their specific needs.

The bad news: this sandbox is only available from trunk right now and will not be easily available until Mono 2.6.

Mono 2.4 and MonoDevelop 2.0 released

We just released two big projects we have been working on for quite a while.

Mono 2.4 is a much faster, scalable and tuned version of Mono, like you have never seen before. Major highlights from the previous release are documented in our release notes.

And MonoDevelop 2.0

And a brand new web site

I previously blogged about the list of all the new MonoDevelop 2.0 features. The most visible one is the integrated debugger both for Mono applications and for C-based applications (using GDB).

Dogfooding: In addition to all the nice features in MonoDevelop 2.0, Lluis migrated the web site for MonoDevelop from MediaWiki to the Mono-powered MindTouch Deki content management system.

Game Developers Conference

I am heading out to the Game Developers Conference in San Francisco as an attendee after some strong endorsments from some friends on tweeter.

If you are at the GDC or in San Francisco and would like to get together at some point drop me an email (miguel at gnome dot org). Or if there are any great hacker get-togethers for game developers, I would love to hang out with them.

I would not want to dissapoint, and as a one trick pony kind of person, I will likely be talking about Mono, Moonlight and the virtues of managed code to anyone willing to listen.

Looking forward to see what my friends have been up to. I can not wait to see the C# repl in a Unity/Web app.

Moonlight 1.9 and Ogg

As I mentioned on a previous post Silverlight 3 opens the doors for developers to plug their own Codecs into the Silverlight media pipeline.

Only a few hours later I read on twitter that Atsushi and Rolf has implemented not only the Ogg/Vorbis Codec for Silverlight 3, but also implemented the Silverlight 3 API in Moonlight:

This means that you can now use your Silverlight-based players to playback Ogg/Vorbis content. Theora and Dirac are still missing, but with the sample code that we now have, it is going to be merely a weekend hack to get it done. Fluendo has a nice implementation of both already in Java.

Update on May 6th, 2009: open source implementations of Dirac, Vorbis and adpcm now live in the mooncodecs module.

Update: link fixed.

You can see the sample in action in Atsushi's test page.

Like Jo said on IRC:

it also works on SL3 though. that's the bit that excites me, since it means we have proper cross-platform playback with Free codecs *today* working in most browsers that matter

In the words of Annie Hall: La de da.

Go Moonlight Go!

Hot Hot Hot: Silverlight 3 Pluggable Codec Architecture (OGG, Theora, Vorbis and Dirac).

Burried in the list of what is new in Silverlight 3 there is this gem:

Extensible media format support: With the new Raw AV pipeline, Silverlight can easily support a wide variety of third-party codecs. Audio and video can be decoded outside the runtime and rendered in Silverlight, extending format support beyond the native codecs.

What the above means is that with Silverlight 3 in addition to the built-in codecs for VC-1 and H.264 and the built-in containers (ASF and MOV) developers can plug an arbitrary audio or video codec and containers into the pipeline to support other formats like Dirac, vorbis, theora and the OGG container.

Both the codecs and the container parsers are authored using C# (or any other .NET supported language).

It would be nice to use Mono.SIMD where appropriate for these codecs. Mono.SIMD works out of the box on .NET, but it is hardware accelerated in Mono.

Atsushi at Novell has done some of the work to get an old C#-based Vorbis codec working with Silverlight 3. We will post more details when we have more information (the fix is on SVN).

Mono and the Google Summer of Code 2009

Once again, the Mono project will be participating in the fabulous Google Summer of Code.

This is a great opportunity for students that want to get involved with open source to contribute, learn and get paid for their work during the summer.

We have been very lucky in recruiting some great students in the past years and these students have taken on some very sophisticated tasks over the years. MonoTorrent, ParallelFX, FastCGI for mod_mono, WinForms designer and theming, Gendarme development, Gtk# widgets and much more.

We have posted some ideas for students to get started, but students that are passionate about Mono should feel free to pitch their own ideas.

We tend to pick students for advanced projects over the milder, simpler tasks.

This year, I am excited about a few special projects:

• Making the Mono VM use LLVM as a code generation backend.
• Porting Mono.SIMD to new architectures.
• ParallelFX updates.
• Using NVidia's VDPAU library to provide a new codec implementation in Moonlight to support VC-1/H.264.
• Managed (C#) implementations of Dirac and Vorbis as Silverlight 3 codecs.
• A Database designer Add-in for MonoDevelop to support DBML editing.

There are many more of course, but the above are the ones that are making me drool.

BareFTP

Christian just pointed me to BareFTP a graphical file transfer client that supports FTP, FTPS, SSH and SFTP protocols to transfer files.

I am a command line kind of person, but many of my friends like to use GUI clients for this.

Moonlight brings Playboy archives to Linux

Since yesterday's announcement that the Playboy archives would be hosted using Silverlight's DeepZoom folks have been hard at work getting the remaining Silverlight 2 features implemented in Moonlight.

Click for screenshot.

Hot Off the Presses: Unity Goes to Windows

Unity has announced that their Unity 2.5 IDE is now cross platform and now works Windows in addition to MacOS.

Unity rebuilt the entire Cocoa-based UI that they had previously with a Unity-powered UI. The entire UI is now built in C# using the Unity built-in APIs (all the controls, views, widgets).

This is a little bit like a compiler compiling itself. This time it is an IDE built using the IDE itself

Lucas integrates csharp REPL into Unity

Lucas Meijer has integrated Mono's C# REPL into Unity.

Visit his post and check out the flash demo of the C# REPL in action.

Voices from Post-Saddam Iraq

My friend Victoria Fontan who works at the UN's University for Peace in Costa Rica just published the book from her research work on Iraq.

The book is Voices from Post-Saddam Iraq: Living with Terrorism, Insurgency, and New Forms of Tyranny. From the editorial reviews:

Even today, most Americans can not understand just why the fighting continues in Iraq, whether our nation should be involved there now, and how we could change our tactics to help establish a lasting peace in the face of what many fear will become a full-fledged civil war. In the book at hand, Victoria Fontan - a professor of peace and conflict studies who lived, worked and researched in Iraq - shares pointed insights into the emotions of Iraq's people, and specifically how democratization has in that country come to be associated with humiliation. Including interviews with common people in Iraq this work makes clear how laudable intentions do not always bring the desired result when it comes to international conflict and cross-cultural psychology. For example, Fontan explains, one might consider the comment of a young Shiite: "The greatest humiliation of all was to see foreigners topple Saddam, not because we loved him, but because we could not do it ourselves." This gripping text is focused on a new and growing area of human psychology - humiliation studies.

Please vote to have the book available on Kindle. I got a hardcopy, but I would love to travel with it instead.

Mono and Qt

The KDE folks have created some brilliant bindings for Mono and .NET called Qyoto.

But there is nothing like a polished application to really test the bindings. This week Eric Butler announced Synapse: an advanced Instant Messaging platform.

This is the first large application built with Qt/Qyoto/Mono and it is a beautiful application:

I had a chance to see Synapse live a couple of weeks ago in Seattle when we met Eric for dinner. Eric has written a very polished application. This is what love does to software.

Congratulations to Eric for the release of his app, to the Qymono crowd for creating these polished applications and Nokia/Trolltech for releasing Qt under the LGPL license.

Developers interested in doing Qyoto development with MonoDevelop can take advantage of the QyotoDevelop add-in that Eric created as well. This add-in generates code from the Qt Designers UI files (click for a screenshot).

Mono's Text Template Transformation Toolkit (T4)

At the ALT.NET Seattle conference I was introduced for the first time to the Text Template Transformation Toolkit. Also known as T4. T4 is built into Visual Studio and developers use TT to generate code from all kinds of data sources. This tutorial covers the basics.

T4 Support in MonoDevelop, with error reporting and document outline.

T4 is very much like ASP.NET in that code is mixed with output code. Additionally TT has access to data on its "host". This allows for some creative data extraction from the environments before it generates output.

To my surprise T4 thing is wildly used by lots of people. Daniel Cazzulino's company has a product just to improve Visual Studio's support for editing .tt files.

What really got me interested in T4 were the templates that Damien wrote to convert from DBML files into C# code that is suitable for use with Linq. A nice replacement for the SQLMetal tool.

I mentioned this -in passing- to Michael Hutchinson as he had been working on ASP.NET MVC support for MonoDevelop and there are some nice ASP.NET MVC T4 files out there.

In a week he implemented: the T4 command line tool, the MonoDevelop host (to support ASP.NET MVC) and he even added syntax highlighting to it (see the above screenshot).

We have also started using it to migrate the code that previously used assorted WriteLines to generate RPM files from Visual Studio/MonoDevelop projects into a nice T4 template:

Packaging Template

Gnome Do

Gnome Do got a new and shiny web site.

ALT.NET this weekend

My friend تلة جوزيف (Youssef) and myself will be heading out to Redmond for the ALT.NET Seattle event this weekend.

This will be my first ALT.NET conference and I do not know quite what to expect. Youssef keeps telling me "You should not prepare for this", but I feel like I should at least prepare something exciting to get the juices flowing for discussion.

Thoughts?

CoyoteLinux uses Mono for syadmin tools

Interesting find: Coyote Linux -a firewall in a box- configuration of Linux is using Mono and ASP.NET for its admin tools.

Web Admin

Here is the rationale for the switch to C#:

One of the biggest changes to this release of Coyote Linux is the use of C# as the primary development language used for most of the administration, configuration, and maintenance utilities. Previous implementations of Coyote Linux made heavy use of C, Pascal (namely Delphi), and Bash shell scripting for this purpose. The change is being made to C# after nearly 2 years of working with the language in a cross-platform setting which involved the use of both Red Hat Linux and Windows 2003/2008 servers. The ability to use a single development environment (in my case, Visual Studio 2008) and produce executables that will execute in unmodified form on both Linux and Windows has seriously put the “R” in RAD programming. I am still actively involved in projects that require the development of cross-platform utilities and am already paying for all of the necessary licenses to provide my company with a full array of software and hardware to develop applications that work in a mixed server OS environment.

I have spent a great deal of time testing C# applications under Linux using Mono as the executing environment. While this is not necessarily the best choice for small, embedded hardware (486 / ARM class processing power) it works very well for anything using i686 or better technology. Another wonderful advantage of using this technology is the ability to run the same set of executables on both 32 and 64 bit hardware without the need for compatibility libraries to be installed. The installation of Mono dictates the 32/64 bit execution environment, preventing the need to recompile the full Coyote Linux software package.

Traffic

System.Shell.CommandLine does not belong in System.Core

Update: Justin Van Patten at Microsoft clarifies that the System.Shell.CommandLine API that was on the CTP for VS2010 will not be part of the final .NET 4.0. Instead better versions (similar in spirit to Mono.Options) will be made available in CodePlex in the future. Relief.

Update 2: Justin gave me permission to quote from his private email, which I include:

We are not shipping System.Shell.CommandLine in .NET 4. This was based on an intern project from a couple of years back that was mistakenly public in the .NET Framework 4.0 CTP. It wasn't a design that we were happy with and has been removed and will not be present in the next preview release.

We have a *much better* command line parsing API, along the lines of Mono.Options, that we're planning to release on CodePlex later this year.

Today I was alarmed by a new API being introduced into .NET 4.0, the System.Shell.CommandLine which is being dumped into System.Core.

An introductory blog post shows a bloated, over-engineered, too rich in the OO, too poor in the taste look at the API. Not only it is a terrible API, it is being dumped right in the core of the framework on the System.Core assembly.

This is the kind of API that you get when the work is commissioned as opposed to be created as an act of love. This is what you get from a culture of process. Some PM figured out "We need command line parsing". And since it does not look like rocket science they assigned this to someone that had absolutely no interest in this task. And clearly nobody involved (the PM, the developer and the tester) fought back. They were forced to do this, and they came up with this aberration, hoping that the sooner they were done with this, the sooner they could move on with their lives.

Compare this with the labor of love from Jonathan Pryor and his Mono.Options library. This API was discussed publicly, it was adopted in a few applications and tried out, it was morphed to support Windows, Unix and GNU style command line options and the result shows what passion and love deliver when it comes to APIs.

Compare and contrast. This is the sample posted for on the blog above:

	using System;
	using System.Shell.CommandLine;

	class Program
	{
	static void Main()
	        {
	            Console.WriteLine("Checks a disk and displays a status report.\n");

	            CommandLineParser cmd = new CommandLineParser();
	            cmd.AddParameter(CommandLineParameterType.String, "volume", ParameterRequirement.Required,
	            ParameterNameRequirement.NotRequired, "Specifies the drive letter.");
	            cmd.AddParameter(CommandLineParameterType.Boolean, "F",      ParameterRequirement.NotRequired,
	            ParameterNameRequirement.Required,    "Fixes errors on the disk.");
	            cmd.AddParameter(CommandLineParameterType.Int32, "L",      ParameterRequirement.NotRequired,
	            ParameterNameRequirement.Required,    "Changes the log file size to the specified number of kilobytes.");

	            try
	            {
	                cmd.Parse();
	            }
	            catch (ParameterParsingException ex)
	            {
	                Console.WriteLine(cmd.GetHelp());
	                Console.WriteLine(ex.Message);
	                return;
	            }

	            string volume = cmd.GetStringParameterValue ("volume");
	            bool fix      = cmd.GetBooleanParameterValue("F");
	            int? logSize  = cmd.GetInt32ParameterValue  ("L");

	            Console.WriteLine("Checking volume {0}...", volume);

	            if (fix)
	                Console.WriteLine("Fixing errors...");

	            if (logSize != null)
	                Console.WriteLine("Changing log size to {0}...", logSize);
	            else
	                Console.WriteLine("Current log size: {0}", 1024);
	        }
	    }
	

This is the equivalent in Mono.Options. With Mono.Options you can take advantage of C# 3 features and make the code more succinct. An important style difference is that the policy is not limited by what can be poorly expressed by an enumeration but can be anything that can be expressed with a real language.

What is the point for "ParameterNameRequirement.Required" for example? This is essentially a bool option with a long name. The fundamental problem is not that it looks like someone left a turd on my source code or the fact that it is a glorified bool value. The problem is that enumerations and an OO structure will not give you the flexibility that is required for command line handling. This API would not be able to cope magically with conflicting options (either -a or -b can be used) or with options that are required if another option is set (if -v set, then -log is needed) or with custom parsing required after the basic command line options are parsed (consider a C compiler, -I and -L and -l options can be specified multiple times).

This is the equivalent code in Mono.Options. Notice that the policy can be enforced either outside of the parameter parsing (after the basic parsing has been done) or as each one of the delegates for the options:

	using System;
	using Mono.Options;

	class Program {
	    static void ShowHelp (string msg, OptionSet p)
	    {
	        p.WriteOptionDescriptions (Console.Error);
	        Console.Error.WriteLine (msg);
	    }

	    static void Main (string [] args)
	    {
	        Console.WriteLine("Checks a disk and displays a status report.\n");

	        bool fix = false;
	        int logSize = 1024;
	        string volume = null;

	        OptionSet p = new OptionSet ()
	            .Add ("volume=", "Specifies the drive letter.", v => volume = v)
	            .Add ("f|F",  "Fixes error on the disk", v => fix = true)
	            .Add ("l=", "Changes the log file size to the specified number of kilobytes",
	                  v => logSize = int.Parse (v));

	        try {
	            p.Parse (args);
	        }
	        catch (OptionException)
	        {
	            ShowHelp ("Error, usage is:", p);
	        }
	        if (volume == null)
	            ShowHelp ("Error: must specify volume", p);

	        Console.WriteLine("Checking volume {0}...", volume);

	        if (fix)
	            Console.WriteLine("Fixing errors...");

	        if (logSize != null)
	            Console.WriteLine("Changing log size to {0}...", logSize);
	        else
	            Console.WriteLine("Current log size: {0}", 1024);
	    }
	}
	

The number of lines is roughly the same, but one is an eye sore and limited. The other is both beautiful and extensible.

Both APIs are capable of more. But System.Shell.CommandLine will merely give you more enumerations with limited functions and you will end up rolling out your own if you want to do anything remotely interesting.

Mono.Options is more of an open ended, future-proof and extensible API. It is implemented as a single C# source file (1,112 lines of code) that you can use in your own projects (and even tune/modify), you can start using today and will work on .NET 2.0 and up, it is nicely documented. More examples: here, here and here.

System.Shell.CommandLine does not belong in System.Core. System.Core is a fundamental assembly that will be everywhere .NET is, and it will soon enough run into the same upgrade and maintenance restrictions that mscorlib has.

This API needs to be moved into CodePlex or be available as an unsupported "PowerOverEngineeredPack.dll".

MonoDevelop 2.0 Beta 1

Earlier this week we released the first beta of MonoDevelop 2.0.

MonoDevelop 2.0 is a very ambitious release in terms of the new functionality available since the MonoDevelop 1.0 release back in March 2008. It is ambitious, but also very stable, we have had hundreds of people dogfood this new release of MonoDevelop during the entire development cycle.

There are a number of new features since MonoDevelop 2.0 that are worth calling out:

Built-in Debugger

MonoDevelop now has a built-in debugger. The debugger supports both debugging Mono-based applications as well as native applications using GDB.

While hovering over variables, you can explore the values of complex data structures:

Breakpoints and Tooltips

You can debug both at the source code level, or the generated assembly code:

Debugging the Mono Runtime

Auto-complete on the watch window:

Auto-complete in the watch window.

You can also attach to running processes, both native or Mono processes and debug them:

Process selector

For more information see our list of supported features.

Improved ASP.NET support

Our ASP.NET story is getting better. web projects are now compatible with Visual Studio 2008 and Visual Web Developer 2008 SP1.

Our ASP.NET text editor now offers code completion of tag, attributes, attribute values and event handlers is now supported for ASP.NET and various HTML DTDs. For example:

We also now have DOM breadcrumbs in the editor as you edit your file, and a nice DOM/Document outline for navigating your HTML and DOM documents.

The beta does not contain this feature, but we will be publishing an Add-in that will help you get your ASP.NET MVC projects up and running with minimum hassle by the time ASP.NET MVC ships.

New Text Editor

A new text editor, this text editor is written entirely in C# and replaces the GtkSourceView widget. This has allowed us to more easily add features to the editor and bring MonoDevelop to the 21st century. Some of the features in the new text editor include:

• Flagging errors with underlines on the fly.
• Change bars (track what changes are in your buffers).
• Incremental search.
• Code folding: Support for collapsing regions of text (#region), methods, classes.
• Themes for the editor.
• Source code split views.
• Transparent popups. Holding down the Control key when a completion popup is shown will make the popup transparent to make it easier to see the text that is underneath the popup.
• VI-mode support. We understand that some of our users would rather use VI keybindings than Emacs keybindings. VI support is here.

Source Code Editing

Intellisense now works for pretty much every piece of the C# 3 language. I am not supposed to use the word "Intellisense", but I just did.

Also, sagacious readers will have noticed that I sneaked in "3.0" in the above statement. MonoDevelop now understands the C# 3.0 syntax. A great companion to our award-winning C# 3.0 compiler.

Now, technically speaking we have not received any awards for our C# 3.0 compiler, but we should have, because we are awesome. And in fact, I will be arranging a dinner at my place this coming weekend where we will award prizes to the best pieces of technologies and our C# compiler is a nominee.

Notice how it also supports nice automatic generations of methods when you declare an event:

MonoDevelop is also aware of types, so for example, if you type "override" when entering a method, it will offer a list of methods that can be overwritten. O-M-G.

There are other cute features like MonoDevelop can stub out interface methods for you. O-M-G.

There are also a few features that we liked from editors like TextMate that should make it more suitable for managing web projects like the revamped "Go to File" dialog (invoke it with control-alt-o). It is now able to do acronym searches.

New XML Editor

The XML Editor from SharpDevelop has been fully integrated into MonoDevelop and improved. It supports code completion of tags, attributes and attribute values (available for registered XSD schemas). A range of schemas are supplied with MonoDevelop.

XML files can be validated using the built in schemas, and can have XSL transforms applied. in addition, XSD schemas can be generated from XML files.

For instance, it is currently used to allow editing of Silverlight XAML files and have auto-completion of XAML tags that are valid for Silverlight/Moonlight.

Project Improvements

We have switched to msbuild-style project files to increase interoperability with Visual Studio.

Support for opening multiple solutions at once, and support for Workspaces.

We now have cascading project policies. This is useful for example to use different coding styles depending on the project that you are working on.

Cascading Project Policies

Gtk# GUI Designer

You can now make your custom widgets available on the toolbox, by just adding the [ToolboxItem] to your widget.

My favorite one is that now constructed dialogs and windows expose the Gtk.UIManager as a field. It was previously hidden, and not possible to adjust the UI dynamically without much work.

Assembly Browser

There is no better way of learning an API than browsing the data types exposed and their relationships. A new Assembly Browser has now been included.

Assembly Browser

Switching

A cute little window pops-up when you press Control-Tab these days:

Vala Support

Support for the Vala programming language has been integrated:

Vala Support

The Future: MonoDevelop, the Cross Platform IDE.

We are very excited about this release, and there are a few areas in which we would like to improve MonoDevelop for future releases.

We want to bring MonoDevelop to Windows to be able to reach into more users and to help developers doing Gtk# development on Windows.

MonoDevelop on Vista.

We are also currently shipping a preview of MonoDevelop for the Mac, it is not yet ready as there are a few kinks that need to be sorted out on that platform, but we are working to resolve those issues. For example, we want to integrate with the Mac menu system, and to provide bindings that are familiar for Mac users. Here is a preview:

Super-Alpha-Preview of MonoDevelop on OSX.

Unity has stated that they will be making MonoDevelop the standard editor for Unity3D on MacOS.

Mono Runtime Debugging

We now have better integration of GDB with Mono. Information on how to use this is in Debugging with GDB in XDEBUG mode in our wiki.

This will be useful if you are debugging the Mono runtime, or debugging Mono embedded into an application new versions of Mono (for example debugging Moonlight).

This will give you symbols for managed code in stack traces, for example the bold text in this example are the managed frames. These would previously just be rendered as "????????" by gdb.

(gdb) xdb
(gdb) bt
#0  0x0000000040cd707e in Tests:pass_floats_doubles (a=100, b=101, c=102, d=103, e=104, f=105, g=106)
#1  0x0000000040cd6fd8 in Tests:test_721_sparc_float_argument_passing ()
#2  0x0000000040a6228a in (wrapper runtime-invoke) Tests:runtime_invoke_int (param0=0x0, param1=0x7fc05e5b0e00,
    param2=0x0, param3=0x40cd6f80)
#3  0x00000000004219f7 in mono_jit_runtime_invoke (method=0x9daa70, obj=0x0, params=0x0, exc=0x0) at mini.c:4253
#4  0x00000000005c1d2c in mono_runtime_invoke (method=0x9daa70, obj=0x0, params=0x0, exc=0x0) at object.c:2399
#5  0x00000000005c39d7 in mono_runtime_invoke_array (method=0x9daa70, obj=0x0, params=0x0, exc=0x0) at object.c:3488
#6  0x00000000005cdc31 in ves_icall_InternalInvoke (method=0x7fc05c371be0, this=0x0, params=0x0, exc=0x7fff66729368)
    at icall.c:3038
#7  0x0000000040cd6bee in (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (
    this=0x7fc05c371be0, param0=0x0, param1=0x0, param2=0x7fff66729368)
#8  0x0000000040cd690c in System.Reflection.MonoMethod:Invoke (this=0x7fc05c371be0, obj=0x0, invokeAttr=0x0,
    binder=0x0, parameters=0x0, culture=0x0)
#9  0x0000000040cd683b in System.Reflection.MethodBase:Invoke (this=0x7fc05c371be0, obj=0x0, parameters=0x0)
#10 0x0000000040a6275c in TestDriver:RunTests (type=0x7fc05e5b6dc8, args=0x0)
#11 0x0000000040a62380 in TestDriver:RunTests (type=0x7fc05e5b6dc8)
#12 0x0000000040a62354 in Tests:Main ()
#13 0x0000000040a6228a in (wrapper runtime-invoke) Tests:runtime_invoke_int (param0=0x0, param1=0x7fc05e5b0e00,
    param2=0x0, param3=0x40a62340)
#14 0x00000000004219f7 in mono_jit_runtime_invoke (method=0x9576f0, obj=0x0, params=0x7fff66729600, exc=0x0)
    at mini.c:4253
	

For people running the archer branch of gdb, there is a "mono mode" implemented in python for gdb, which offers additional features: http://anonsvn.mono-project.com/viewvc/trunk/mono/data/gdb/

Mono on Android, update

Koushik Dutta has posted two great updates on his blog about Mono running on the Android powered G1 phone. The code necessary to build Mono on Android is available at the androidmono page.

The first bit is his work to allow Mono code to call Dalvik code as well as the bridge to allow Dalvik/Java to call into .NET code. This is necessary to allow Mono-based applications to call into the phone API and integrate with the existing stack.

The second bit is a cute show of DLR-based IronPython running on the phone.

This of course means that with the bridge in place, any CLR or DLR-based language can access the functionality exposed in Dalvik.

Mono 2.0 - .NET Tool/Add-in of the Year

Jeff just pointed out to me that Mono 2.0 won the ".NET Tool/Add-in of the Year" award from the guys at developer.com.

Sweetness!

Linus does 25 things

Linus Torvalds does 25 things about me.

Moonshine

Now that Moonlight 1.0 is out, I should talk a little bit about Aaron Bockover's amazing Moonshine plugin.

Moonshine's About Box while playing HBO.com

Moonshine is a tiny plugin that registers itself with Firefox to render Windows Media streams and emulates the Windows Media Javascript API by redirecting it to Moonlight.

So when you visit a page that in the past used to embed the Windows Media Player (for example at HBO.com or at C-SPAN), instead you get a Moonlight-based rendering engine and uses Microsoft's Media Pack for doing the video and audio playback.

It is trivial to install it, just go to this page and click on the moonshine that is right for your platform.

Internally we referred to this project as "Pornilus" a homage to the Roman senator and patron of the arts from the 3rd century. And like Pornilus, we want to bring the arts to the people.

Cute fact: Pornilus/Moonshine will pick-up your Gnome theme and theme itself accordingly.

Oddly enough, there is no Wikipedia page for Pornilus yet. Someone needs to correct this.

Moonlight 1.0 goes live

Moonlight, the open source implementation of Silverlight for Unix systems has officially reached its 1.0 level. We are feature complete, we pass all the Microsoft regression test suites and we shipped support for Microsoft's Media Pack for x86 and x86-64 architectures.

Moonlight is available as a Firefox plugin that can be installed with a single click from the moonlight download page.

What is in Moonlight 1.0

Moonlight 1.0 contains our plugin that can be used in Firefox 2 and 3 on Unix systems using the X11 windowing system.

Moonlight 1.0 (and Silverlight 1.0) both come with a graphics pipeline, video and audio frameworks and a javascript bridge and neither one of them contains an actual execution environment. The execution environment is the browser's own Javascript engine. When developers build 1.0-based plugins they script all of the functionality using the browser's own Javascript engine.

The browser Javascript engine communicates with Silverlight (or Moonlight) through the Javascript API exposed by the plugin.

With Silverlight 2.0 and Moonlight 2.0 in addition to this model where the browser's Javascript drives the interaction a new model is available: the ECMA CLI execution system powers the actual execution of the code and will deliver performance anywhere between 20 to 300 times faster execution speed than even the most modern Javascript implementation if you use a strongly typed language like C# or Boo.

CNN's The Moment on Moonlight.

It is worth pointing out that Moonlight is provided both for 32 bit systems and 64 bit systems on the launch date.

We are also hoping to expand our reach to other Unix variants that use X11 like the various BSD systems and Solaris and make codecs available for those.

How we got here

The development of Moonlight has been a fascinating adventure. It all started at the Mix conference in May 2007 when Scott Guthrie introduced Silverlight 1.1. It was a bold move for Microsoft to embed the ECMA CLI into their Silverlight 1.0 plugin.

In my blog post called "Mix 07, Silverlight, Dynamic Languages Runtime and OpenSource". From that post you can see that I was already excited about the technology, and I could not wait to get this technology to Linux. The talk on the DLR at Mix 07 was also fascinating and got me interested in bringing this to Linux.

A few weeks after the DLR had been announced and open sourced, our team had it working on Linux with Mono and by the end of May I had cooked up enough to render a spinning video on the screen.

IronPython 3D visualization on Moonlight

It was during the dynamic language workshop at Microsoft that I had a chance to have dinner with Jason Zander and Scott Guthrie in an Indian restaurant in downtown Redmond. In this dinner they discussed some of the design tradeoffs in Silverlight and these would become part of our own implementation a few days later.

At Mix 2007 I had the chance to meet Marc Jalabert from Microsoft France. Marc invited me to the Remix event in Paris but did not take the invitation seriously until he offered us to demo Moonlight on Jun 21st.

Other than a spinning video and the DLR we did not really have much code so on May 31st I sent an email to the team and asked them to work on an intense 21-day hackaton to bring Silverlight 1.1 to life on Linux. By Jun 21st we had a demo working and we showed Silverlight 1.1 applications (with the CLR) running on Linux.

A few weeks passed by, and Jeff Jaffe from Novell asked me to present our Moonlight to Bob Muglia as part of the regular Microsoft/Novell interoperability meetings. After struggling with the video projector for what seemed like an eternity the Silverlight Chess and the Silverlight Airlines demo came up on the screen on Linux.

In the meantime, we were in love with our Moonlight engine, and we used to build desktop applications in addition to web applications.

After this meeting, I do not remember exactly how things happened as too much happened too quickly, but Microsoft and Novell agreed to collaborate on bringing Silverlight to Linux. We announced the collaboration on September 5th.

It was early on, at that dinner with Jason and Scott that the issue of how to properly license codecs for MP3, WMV and VC-1 had been discussed. We knew that we could implement the engine, but the question remained: how to get codecs to end-users in a fully licensed way. This and other problems had been already discussed and agreed on the collaboration agreement. Microsoft would develop, distribute and maintain their own Media Pack for Linux users and other Unix operating systems.

The entire media work involved hard work at every level, but it was worth the effort. We now have one of the best open source media pipelines implemented. And it will only get better with all the new features in Silverlight 2 for adaptive streaming.

The Immediate Future

We are now hard at work on Moonlight 2, and those of you interested in trying it out can do so by following the build instructions on our web site.

Silverlight 2.0 was a major upgrade from its original announcement Silverlight 1.1. It is more complete, more polished and has been future-proofed.

Microsoft has continued to help us all along in creating an open source implementation of Silverlight. They have open sourced the Microsoft DLR, the Microsoft MEF framework and the crown jewels: the Microsoft Silverlight Control Library and the Control Toolkit under the OSI-approved MS-PL licenses. Without this it would have taken years for us to catch up.

Jimmy Schementi's IronRuby + DOM + Flickr sample.

Up until two weeks ago we could not see much in the screen as a lot of Moonlight had inter-dependencies between various subsystems. But once Larry Ewing's layout system landed in our tree, magically many things started to come together.

You can try out yourself Moonlight with some very hot demos including CNN's The Moment, the Photosynth-based 3D browsing engine for Obama's Inauguration and of course the always amazing DLR demostrations.

Silverlight 3

Silverlight 2 is incredibly exciting, it is delicious and mindblowing. There is a lot of excitement about it, my favorite three sites on Silverlight 2 include:

• Silverlight News on twitter.
• Infragistics Quince: UX Patterns Explorer
• Information Visualization with Descry: a project do visualize information in interesting ways using Silverlight. The project is now open source.

Microsoft will be announcing the details about Silverlight 3 at their Mix conference in March in Las Vegas.

My wish list

I love Silverlight and the use of the CLR for building web applications. That is just how I am wired up.

I still personally wish that Silverlight 2.0 had a JSon interface to XAML, like the prototype that Chris Toshok did, or that Silverlight had a more fluent model for application deployment. I would like the XAP model to be entirely optional or non-existent for IronRuby or IronPython.

It is that time of the Quarter! Traveling to Microsoft.

When Joseph and myself head out to Redmond to meet with some folks at Microsoft about Moonlight.

This is a call for all cool cats at Microsoft that would like to get together and talk shop to drop me an email (miguel at gnome dot org) and we can schedule something.

XBox Division at Microsoft

For about a year I have been trying to find someone in the XBox360 division at Microsoft that we can talk to about bringing Mono to the XBox360 to allow C/C++ developers to script their applications with the high performing C#, Boo or the Iron* languages as opposed to interpreters.

A year ago Mono could not target the XBox360 as apparently this platform, like the iPhone, does not support JITing. Mono now supports full static compilation of .NET code into native code before deployment and we would very much like to bring this to the XBox360.

If you are a Microsofty and you know how to get a hold of someone on the XBox360 group in the Middleware division and you could hook us up, I would love if you could arrange an introduction.

Linux Outlaws Podcast

Last week the folks at Linux Outlaws interviewed me about Mono.

The idea was that someone on a previous episode apparently did not quite like Mono and they wanted to hear my take. In the end I am not sure that we even talked about their concerns, but it was a fun interview.

IronClad

The guys at Resolver Systems have released IronClad. IronClad is a library that allows IronPython to use any existing compiled CPython extension.

The new version has matured to the point that it is able to use CPython's numpy and pass its test suite.

It is lovely to see third parties start to test their code with Mono in addition to .NET as part of the release process. The code has been tested with Mono, and comes with Unix makefiles.

One Month of Email Gone

If you sent me an email in the last month, and you are waiting for me to reply, please resend your email.

I accidentally deleted all email since December 18th, 2008.

DekiWiki powers WhoRunsGov.Com

Aaron Fulkerson and his team at Mindtouch have done it again. This time they landed the Washington Post new http://whorunsgov.com project:

WhoRunsGov.com provides a unique look at the world of Washington through its key players and personalities. The site features concise profiles of influential political officials who shape government policy, including members of the new presidential administration, Pentagon officials, lawmakers, senior congressional aides and committee staff. The first several hundred profiles are being crafted by a newly created editorial team at the Washington Post Company, as well as a group of experienced outside contributors. Each profile provides in-depth information on an official’s policy experience, involvement in government decision-making, major policy positions, key associates, political affiliations, voting records, campaign and personal finance information, plus relevant news articles from around the Web.

Their Deki project has gone from the cutest Wiki system to a full collaboration platform.

Their press release has the details.

And as my readers have come to expect, yes, this is also built on top of Mono. Deki is not built with ASP.NET --Microsoft's web platform-- instead the engine is built on top of Mindtouch's Dream framework and the presentation layer is built on top of PHP.

Congraulations to Mindtouch on this important launch!

Cartoon Network's Kid's MMO and Mono.

The amazing Joachim Ante from Unity3D wrote me to tell me that Cartoon Network's new browser-based MMO for kids FusionFall has finally launched to the public.

Fusion Fall takes advantage of many new features in Unity3D for creating large worlds. I live blogged some of the details as Joachim presented them at the Unite Conference.

Unity uses the Mono runtime on both Windows and MacOS and it might become one of the largest deployment vehicles for the Mono VM.

There is an air of coolness in the fact that Mono is being used on Windows instead of .NET. And part of it has to do with the fact that Mono's open source engine allowed Unity to modify it to suit their very specific needs.

As I mentioned at my PDC talk, the .NET engine is fantastic, but up until Mono only Microsoft was in a position to reshape .NET into different forms (Silverlight and Mesh both use a special trimmed-down .NET called CoreCLR). I would love to see a world where people take Mono (or chunks of Mono) tune it and shape it to suit their needs.

Congratulations to the team at Unity for a job well done, and to the team that produced FusionFall. You can see the introduction video:

One thing that stands out in FusionFall is that it shows what a big creative budget can do with Unity.

Go Mono gaming, Go!

Linux and the Inauguration

Not only was the inauguration transmitted using Silverlight, but also on each window for everyone watching the inauguration from pic2009.org the following caption was at the bottom of every page:

Every viewer could see "Linux-compatible Silverlight Player".

Watching the Obama Official Inauguration on Linux with Moonlight.

I just wanted to confirm that you can watch today's Barack Obama Official Inauguration video stream using Moonlight on Linux/x86 and Linux/x86-64 systems.

All you need to do is to go to the Moonlight Download page, install Moonlight and restart your browser. Then you can visit www.pic2009.org in a few hours and watch the event from Linux.

Microsoft worked late last night to get us access to the code that will be used during the inauguration so we could test it with Moonlight.

Thanks to: Joseph, Larry, Geoff, Rusty and specially Aaron who all worked tirelessly to implement and get everything tested tonight and ready to go on the Novell side. To Brian, Eric, Ben at Microsoft and Mio at IStreamPlanet to make sure that Linux users will be able to watch Obama's inauguration.

Ben Waggoner has posted an update on the Microsoft side..

Aaron's code will also be powering MacOS/PPC streaming.

Now everyone say at once: O-ba-ma! O-ba-ma! O-ba-ma!

Mono's New Code Generation Engine

Three years ago, in November of 2005 we started a project to upgrade Mono's code generation engine as the engine started to age and it became increasingly difficult to improve code generation and extend the JIT engine in meaningful ways.

The new code generation engine is based on a linear intermediate representation as opposed to the tree-based intermediate representation that we had used up to Mono 2.0.

Switching the code generation engine is a pretty significant effort and we did not want to switch it shortly before we had to ship Mono 2.0, so we decided to ship 2.0 with the engine that had been in wide use.

Shortly after we branched Mono's tree for the 2.0 release Zoltan merged his work from the linear branch into the main tree.

We have now shipped all of this as part of Mono 2.2, you can get it here.

Some Benchmarks

Mono's new engine generates much better code than the version found in Mono 2.0.

Speed: The engine will mostly benefit computationally intensive code, usually between 10% and 30% performance increase, with some cases going up as high as being 50% faster.

Code size: the new engine generates slimmer code, typically 12% to 20% smaller code generated.

Check out some of the benchmark results.

Debugging the Transition

Although we had our test suite, and we regularly tested the code against most apps, we were still afraid that something might go wrong. The new code could miss-compile something, and it would be hard in a large project to pin point exactly what went wrong.

For example, the problem might not appear while compiling a small test program like `hello world', but could appear when running a web site under heavy load or when running MonoDevelop.

Zoltan came up with a very interesting solution: for a period of time Mono had two JIT engines built into it, the new and the old one. Here is where the clever trick comes in: an environment variable contained the number of methods that should be compiled with the new JIT engine. After the Nth method had been compiled, the engine would switch code generators.

This was used to bisect regressions and failures.

A couple of months after we had done the switch and both our unit tests and our system tests passed the old JIT engine was eliminated from Mono.

SIMD

Using SIMD for accelerating certain floating point operations had been in the back of our minds for a while. We looked into implementing that in our old engine, but that turned out to be very difficult.

With the new engine, Rodrigo was able to put together a prototype in a weekend (the legend goes that Rodrigo's wife was busy that weekend).

This prototype was later turned into Mono.SIMD an API for accelerating vector operations.

Mono 2.2 is the first release to officially support and distribute it. To learn more about Mono.SIMD support, you can see this blog entry.

Full Generics Sharing

With this release, the generics code sharing engine has been completely debugged and is now enabled not only for code that lives in mscorlib, but for all generics code written by the user.

The Technical Details

We have provided A complete description of Mono's new engine design and the the various code generation stages.

More Mono-based games on the iPhone

Update: as of September there are at least 258 games on the AppStore build with Unity3D according to this interview. And 5 out of the Top100 applications are Unity based.

Randy Edmonds pointed out in my previous post that FlashBang Studios's RaptorCopter is not the first or the only Unity3D/Mono-based game on the Apple AppStore.

I counted almost 40 apps on the AppStore based on Mono, from the thread here.

These are a few other games available today from the AppStore that are powered by Mono:

• Downhill Bowling (5 stars), (screenshots and video).
• Billiards.
• SpacePig.
• Age of Curling.
• Dusktreaders.
• X-Razer.
• Tapball (video).
• InvinciBall (video).
• SlidePop.
• FuguMaze.
• Monkey Diving (video).
• Ball-X (with the dash in the middle, or you wont find it).
• FuguTilt.
• Pizza Dash (you deliver pizzas in a car).
• Debris (you do controlled demolitions).
• Trash it!
• Rotunda.
• Asteroid Strike.
• Crazy Snowboard (and 2.0).
• Bubble Bang (video).
• iStronaut Willy.
• Bounce Pop.
• Labyrinth 3D.
• FuguBall.
• SpaceRace (video).
• iDrone.
• Mars Explorer.

Word games:

• Christmas spell.
• Alpha Blocks: Brain Freeze.

Not really games, but cool hacks:

• Moobox 3D, cute!
• Widget Monkey.
• Butterflies.
• Dice 3D.
• Night Divine.
• Rainbow Day.
• Zen of Snow and Zen of Snow 2.
• Jingle Bells.
• ArtiFISHal Life (3D Aquarium).
• Leaves.
• iFeathers.
• Bobblehead Santa.

Update:

• Cricket (from SpinFast).
• Rock'em Blocks (video).
• Pelagic Tones.

First Mono-game hits the Apple AppStore

Blurst's Raptor Copter game built using Unity3D and Mono just hit the Apple AppStore.

From the announcement:

Raptor Copter has become our first Unity-made iPhone game to hit the App Store! We’re making it available for a limited-time price of $0.99. The game is a loose follow-up to Off-Road Velociraptor Safari. Instead of a jeep, you have a Chinook helicopter, but the basic game loop is the same: Capture raptors, drop them into factories, and teleport their sweet meats to the future.

You can get it for your iPod Touch or iPhone from this Raptor Copter iTunes Link.

Cute video:

Unity3D is using Mono's full static compilation to allow the game to run JIT-less and interpreter-less on the iPhone.

First Mono-based Wii Game on the Shelves

Christian Lassmann from Weltenbauer Software Entwicklung GmbH, who I had the pleasure of meeting at the Unite Conference in Denmark in October, just wrote to tell me that "My Animal Center", a game built with Unity3D's Wii Edition and Mono hit the shelves on December 20th in Germany.

The game uses C# extensively. It was a joy to hear Christian explain how the various effects were created, I wish he blogged about it.

Cute trailer (text is in German):

The game is coming to a Wii near you in the US soon.

Mono goes Accessible!

Brad Taylor has announced the first release of the Mono Accessibility stack:

UI Automation provides programmatic access to most user interface (UI) elements on the desktop, enabling assistive technology products such as screen readers to provide information about the UI to end users and to manipulate the UI by means other than standard input. UI Automation also allows automated test scripts to interact with the UI.

Mono's Accessibility Framework is an implementation of UI Automation. The same API that is available for WPF and the framework is used by Silverlight and Windows.Forms.

Client Code: The initial launch of Mono Accessibility adds accessibility support to applications built with Windows.Forms to be accessible.

Backend Code: The code has a bridge that talks to the existing ATK framework on Linux.

In the future the Mono Accessibility framework will be used in our own Moonlight 2.0.

Check the release notes, install from source or use OpenSUSE's 1-click install.

Mono Goes to Android

Koushik Dutta got Mono running on the Android-based G1 phone.

He posted a video of the phone compiling "Hello World" (he points out that it is slower due to Mono running from the SD card):

He also posted some performance and memory usage comparisons between Dalvik, Mono and Java/ARM. Short story: Mono does great!

There are some caveats on running Mono on the G1, see the comments on this post. Still, these are encouraging news.

F-Spot Extensions

Today I upgraded my F-Spot, I had not upgraded it since before the PDC.

It is now has a Picassa-like toolbar on the left:

And third-party extensions are starting to come out:

Mono brings Plastic's Windows.Forms UI to Linux and MacOS

Pablo has sent me these two screencasts of their Plastic SCM product running on Linux and MacOS using Mono 2.0's Windows.Forms support:

Plastic on Linux, the GUI toolkit is Windows.Forms with custom widgets and a nice color scheme.

Plastic has a nice graphical diff tool:

A preview of Plastic on MacOS X.

Cute graphs

More screenshots here.

Plastic is one of the finalists for this year's Jolt Awards.

Evolution wish-list: IMAP server built into the client

For a while I wanted to be able to get programmatic access to my email store in Evolution, just like it is possible to have programmatic access to the contacts and calendar through the Evolution Data Server.

The advantages of using IMAP as the protocol to talk to Evolution are simple: I can use any existing IMAP client library, or any other IMAP client to connect to my Evolution store. The protocol is well known, documented and the large ecosystem of IMAP clients makes it a natural feature.

There is also an application that I have in mind for it. I keep all of my email in Evolution, I download all of my email into my local hard disk so I can have all my information with me even when I am disconnected from the net. This means I can always check patches, review comments, discussions even when I am disconnected or with poor network connectivity.

But when I go on vacation, I do not want to bring my laptop or Evolution with me. Instead I end up using internet cafes to read my gmail and all of the other email addresses end up in Novell's server. Novell provides a convenient Web UI that I can use to read my email.

But the problem is that I end up reading emails twice: once in the road with the web UIs, and another time when I get back home and import all my email into Evolution.

By having Evolution expose an IMAP interface, I could use any IMAP client on the road, or ssh into my box and use mutt to read from the same email store that Evolution is keeping track of.

Visiting Microsoft

Joseph, Chris and myself are visiting Microsoft this week to learn more about Silverlight 3.0

If you are in town and have some time to meet to discuss open source, Mono, .NET, the CLI, the DLR or and whatever else you think we might have a fun conversation about, please drop me an email.

Moonlight goes 3D

Argiris Kirtzidi (one of the developers behind Managed OGRE) modified Moonlight to run inside the Ogre3D engine. You can render Moonlight applications or XAML files inside Ogre3D.

The Moonlight Calculator Example.

Your standard XAML tiger.

We are merging his patches to make it simpler for Moonlight to be compiled by Windows users.

Update:For more details about how this was done, and how he modified Cairo to be hardware accelerated check Argiris's post.

Groupwise Calendar to Google Calendar Exporter

I wrote a small tool that exports my Groupwise Calendar to Google Calendar.

This tool only runs on Windows as it is using the Groupwise COM APIs to fetch the calendar data. I would love to have this work on Linux if someone knows how to get to these from Unix.

You will need the Google Calendar assemblies (Google.GData.AccessControl, Google.GData.Calendar, Google.GData.Extenions) and the Groupwise Assemblies (GroupwiseTypeLibrary, GroupWiseCommander) and a text file that contains your passwords (called `passwords').

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Google.GData.Calendar;
using Google.GData.Client;
using Google.GData.Extensions;
using Google.Accounts;
using System.Threading;
using System.IO;

namespace CalendarExporter
{
    class Program
    {
	const string google_email = "YOUR_EMAIL@gmail.com";
	const string groupwise_login = "YOURNAME";
	
        static void Main(string[] args)
        {
            var f = File.OpenRead("passwords");
            var reader = new StreamReader(f);
            var google_passowrd = reader.ReadLine();
            var groupwise_password = reader.ReadLine();

            new Thread(delegate() {
                Thread.Sleep(1000 * 120);
                Console.Error.WriteLine("Timing out");
                Environment.Exit(1);
            }).Start();

            ClientLoginRequest login = new ClientLoginRequest();
            login.AccountType = "GOOGLE";
            login.Email = google_email;
            login.Password = google_password;
            login.Service = "cl";
            login.Source = "YOURNAMECo-CalendarPush-1";

            var token = login.Login();

            CalendarService cs = new CalendarService("YOURNAMECo-CalendarPush-1");
            cs.SetAuthenticationToken(token.Auth);


            CalendarQuery cq = new CalendarQuery();
            cq.Uri = new Uri("http://www.google.com/calendar/feeds/default/owncalendars/full");
            CalendarFeed resultFeed = cs.Query(cq);
            CalendarEntry gw_at_google = null;
            foreach (CalendarEntry entry in resultFeed.Entries) {
                if (entry.Title.Text == "Groupwise Calendar") {
                    entry.Delete();
                    break;
                }
                
            }
            gw_at_google = new CalendarEntry();
            gw_at_google.Title.Text = "Groupwise Calendar";
            gw_at_google.Summary.Text = "This is the syncrhonized calendar at Novell's Groupwise server";
            gw_at_google.TimeZone = "America/New_York";
            gw_at_google.Hidden = false;
            gw_at_google.Color = "#2952a3";
            gw_at_google.Location = new Where("", "", "Boston");
            gw_at_google.Selected = true;
            Uri postUri = new Uri("http://www.google.com/calendar/feeds/default/owncalendars/full");
            CalendarEntry cal = (CalendarEntry)cs.Insert(postUri, gw_at_google);

            string calurl = cal.EditUri.Content;
            int p = calurl.LastIndexOf ('/');
            string code = calurl.Substring (p);

            
            Uri edit_uri = new Uri ("http://www.google.com/calendar/feeds" + code + "/private/full");
            GroupwareTypeLibrary.GWSession2Class gsc = null;
            try
            {
                gsc = new GroupwareTypeLibrary.GWSession2Class();
            }
            catch
            {
                Console.WriteLine("Did not regsvr the file c:\novell\groupwise\gwcma1.dll and is this program x86-only?");
                return;
            }
            var account = gsc.Login(groupwise_login, "", groupwise_password, null, null);
            var path_to_host = account.PathToHost;

            //alendar calendar = new iCalendar();

            
            int count = 0, skipped =0;
            foreach (GroupwareTypeLibrary.Message m in account.Calendar.Messages)
            {
                if (!m.ClassName.StartsWith ("GW.MESSAGE.APPOINTMENT"))
                    continue;

                GroupwareTypeLibrary.Appointment2 app = (GroupwareTypeLibrary.Appointment2) m;

                // Ignore appointments that are older than 15 days.
                if (app.EndDate < DateTime.Now - TimeSpan.FromDays(7)) {
                    skipped++;
                    continue;
                }

                var ee = new EventEntry();
                ee.Title.Text = app.Subject.PlainText;
                ee.Content.Content = app.BodyText.PlainText;
                ee.Locations.Add (new Where () { ValueString = app.Place });
                ee.Times.Add(new When(app.StartDate, app.EndDate));
                ee.EventVisibility = app.Private ?
                    EventEntry.Visibility.PRIVATE : EventEntry.Visibility.PUBLIC;

                cs.Insert (edit_uri, ee);
            }
            
            Console.WriteLine("Done2");
            Environment.Exit(0);
        }
    }
}
	

You will also need the Login.cs which is some sample code that I found on the tubes for doing Google Account authentication.

Moonlight 1.0 Beta 1

We have released the first beta of Moonlight 1.0.

This release supports the Microsoft Media Pack for playing back video and audio files. These are the same video and audio decoders that Microsoft uses in Silverlight 2.0.

Check our Moonlight roadmap for details on upcoming versions.

You can try some of the sites tests that we used to test Moonlight.

Here are some Silverlight 1.0 materials:

• How To Videos on Silverlight 1.0
• Essential Silverlight Training.
• Silverlight 1.0 Unleashed. This is my favorite Silverlight 1.0 book, one of the books that we used to get some of the Moonlight team members up to speed.

You can also read the Silvelright's XAML vocabulary description and its XAML Foundation Specification.

Moonlight's Media Stack

As part of the Moonlight Beta release, I wanted to devote a few blog posts to exploring the features in Moonlight and how we implemented those Silverlight features in Moonlight.

Before I get started on today's topic, we would like to get some feedback from our users to find out which platforms they would like us to support with packages and media codecs. Please fill out our completely platform and media codec survey.

Moonlight 1.0 is an implementation of the Silverlight 1.0 API. It is an entirely self-contained plugin written in C++ and does not really provide any built-in scripting capabilities. All the scripting of an embedded Silverlight component is driven by the browser's Javascript engine. This changes with the 2.0 implementation, but that is a topic of a future post.

The Silverlight/Moonlight Developer View.

One of the most important features of the Silverlight/Moonlight web plugin is to support audio and video playback.

Silverlight takes an interesting approach to video and audio playback. In Silverlight the video can be treated like any other visual component (like rectangles, lines) this means that you can apply a number of affine transformations to the video (flip, rotate, scale, skew), have the video be composed with other elements, add a transparency layer to it or add a clipping path to it.

This is the simplest incarnation of a video player in XAML:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <MediaElement x:Name="ExampleVideo"
		  Source="TDS.wmv"
		  Width="320" Height="240"
		  AutoPlay="true"/>
</Canvas>
	

The result looks like this when invoked with when embedded in a web page (or when using the mopen1 command that I am using to take the screenshots):

The MediaElement has a RenderTransform property that we can use to apply a transformation to it, in this case, we are going to skew the video by 45 degrees:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Source="TDS.wmv" Width="320" Height="240">
	   <MediaElement.RenderTransform>
	     <SkewTransform CenterX="0" CenterY="0" AngleX="45" AngleY="0" />
	   </MediaElement.RenderTransform>
	</MediaElement>
</Canvas>
	

The result looks like this:

But in addition to the above samples, MediaElements can be used as brushes to either fill or stroke other objects.

This means that you can "paint" text with video, or use the same video source to render the information in multiple places on the screen at the same time. You do this by referencing the MediaElement by name as a brush when you paint your text.

This shows how we can fill an ellipse with the video brush:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Opacity="0.0" Source="TDS.wmv" Width="320" Height="240"/>

	<Ellipse Width="320" Height="240" >
	   <Ellipse.Fill>
 	      <VideoBrush SourceName="ExampleVideo"/>
	   </Ellipse.Fill>
	</Ellipse>
</Canvas>

This looks like this:

You can also set the stroke for an ellipse. In the following example we use one video for the stroke, and one video for the fill. I set the stroke width to be 30 to make the video more obvious.

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Opacity="0.0" Source="TDS.wmv" Width="320" Height="240"/>
	<MediaElement x:Name="launch" AutoPlay="true" Opacity="0.0" Source="launch.wmv" Width="320" Height="240"/>

	<Ellipse Width="320" Height="240" StrokeThickness="30">
	   <Ellipse.Fill>
 	      <VideoBrush SourceName="ExampleVideo"/>
	   </Ellipse.Fill>
	   <Ellipse.Stroke>
 	      <VideoBrush SourceName="launch"/>
	   </Ellipse.Stroke>
	</Ellipse>
</Canvas>
	

Notice that in the examples above I have been using AutoPlay="true". Silverlight provides fine control over how the media is played as well as a number of events that you can listen to from Javascript for various events (for example, you get events for buffering, for the media being opened, closed, paused, or when you hit a "marker" on your video stream).

Streaming, Seeking and Playback

Depending on the source that you provide in the MediaElement, Moonlight will determine the way the video will be played back.

The simplest way of hosting a video or audio file is to place the audio or video file in a web server, and then have Moonlight fetch the file by specifying the Source attribute in the MediaElement. You do not need anything else to start serving videos.

Seeking on the standard Web server scenario: When the programmer requests the media to be played (either by calling the Play method on the MediaElement, or because the MediaElement has AutoPlay set to true) Moonlight will start buffering and play the video back.

If the user wants to seek backwards, or forward, Moonlight will automatically take care of this. In the case where the user fast-forwards to a place in the file that has yet to be downloaded, playback will pause until then.

Seeking with an enhanced media server: If your server implements the Windows Media Streaming HTTP extension if the user seeks to a point in the file beyond the data that has been downloaded, it will send a special message to the server to seek. The server will start sending video from the new position. The user will get the playback started immediately without having to wait. The details of the protocol are documented in the MS-WMSP specification. This is enabled by using the "mms://" prefix for your media files instead of using the "http://" prefix.

Notice that even if it says "mms", Silverlight and Moonlight do not actually speak to an MMS server, they merely replace that with "http" and speak http/streaming to the server.

The extension is pretty simple, it is basically a "Pragma" header on the HTTP requests that contains the stream-time=POSITION value. Our client-side implementation is available here.

You can use IIS, or use the mod_streaming to enhance the video experience for your end users.

This basically means that you can stream videos on the cheap, all you need is a Linux box, two wires, and a 2HB pencil.

Adaptive Streaming

Another cool feature of the Adaptive Streaming support in Moonlight is that the server can detect the client throughput, and depending on the bandwidth available, it can send a high bitrate video, or a low bitrate video. This is a server side feature.

This feature was demoed earlier this year at Mix 08:

I am not aware of an adaptive streaming module for Apache.

Supported Media Formats in Moonlight 1.0

Although Moonlight 1.0 exposes the Silverlight 1.0, Moonlight 1.0 ships a 2.0 media stack (minus the DRM pieces). This means that Moonlight ships with support for the media codecs that are part of Silverlight 2.0 and supports adaptive streaming. This is what we are shipping:

Video:

• WMV1: Windows Media Video 7
• WMV2: Windows Media Video 8
• WMV3: Windows Media Video 9
• WMVA: Windows Media Video Advanced Profile, non-VC-1
• WMVC1: Windows Media Video Advanced Profile, VC-1

Audio:

• WMA 7: Windows Media Audio 7
• WMA 8: Windows Media Audio 8
• WMA 9: Windows Media Audio 9
• WMA 10: Windows Media Audio 10
• MP3: ISO/MPEG Layer-3
  • Input: ISO/MPEG Layer-3 data stream
  • Channel configurations: mono, stereo
  • Sampling frequencies: 8, 11.025, 12, 16, 22.05, 24, 32, 44.1, and 48 kHz
  • Bit rates: 8-320 kbps, variable bit rate
  • Limitations: "free format mode" (see ISO/IEC 11172-3, sub clause 2.4.2.3) is not supported.

We also support server-side playlists.

For more information see the Silverlight Audio and Video Overview document on MSDN.

Media Pipeline

When we first prototyped Moonlight we used the ffmpeg media pipeline. A media pipeline looks like this:

Charts by the taste-impaired.

Originally ffmpeg handled everything for us: fetching media, demultiplexing it, decoding it and scaling it.

Since we needed much more control over the entire pipeline, we had to write our own, one that was tightly integrated with Moonlight.

Today if you download Moonlight's source code you can build it with either the ffmpeg codecs or you can let Moonlight fetch the binary Microsoft Media Pack and use Microsoft's codecs on Linux.

Microsoft Media Pack

The Microsoft Media Pack is a binary component that contains the same code that Microsoft is using on their Silverlight product.

The Moonlight packages that we distribute do not actually have any media codecs built into them.

The first time that Moonlight hits a page that contains media, it will ask you whether you want to install the Microsoft Media Pack which contains the codecs for all of the formats listed before.

Today Microsoft offers the media codecs for Linux on x86 and Linux x86-64 platforms. We are looking for your feedback to find out for which other platforms we should ship binary codecs.

Tests

No animals were harmed in the development of the Moonlight Video Stack. To ensure that our pipeline supported all of the features that Microsoft's Silverlight implementation supports we used a number of video compliance test that Microsoft provided us with as part of the joint Microsoft-Novell collaboration.

In addition to Microsoft's own tests, we created our own set of open source tests. All of these tests are available from the moon/test/media module. This includes the videos that are specially encoded with all the possible combinations and features used as well as XAML files and associated javascript.

Mono on PowerPC 64

As part of SUSE 11, Mono needs to run on the PowerPC in 64 bit mode. The effort was bootstrapped with some early work from Andreas Faerber.

It was fun to watch Mark's daily commits progress of the port, the tests referenced here are the basic runtime tests that we use to check for regressions and to get a port up and running, it is a good roadmap for how a port comes to life:

	* mini-ppc64.c, cpu-ppc64.md: Fixed some opcodes.  PPC64
	passes basic.exe now.

	---

	* cpu-ppc64.md: Fixed a few instruction lengths.

	* mini-ppc64.c: Don't emit SETLRET.  Now PPC64 passes
	basic-float.exe.

	---


	* decompose.c: Decompose carry and overflow add on PPC64 like
	on other 64 bit archs.  Don't decompose sub at all on PPC64.

	* mini-ppc64.c, exceptions-ppc64.c, tramp-ppc64.c,
	cpu-ppc64.md: 	Several fixes and new opcodes.  Now PPC64 runs (but doesn't
	pass) basic-long.exe.

	---
	
	* ppc/ppc-codegen.h: Use ppc_load_reg instead of ppc_ld in
	ppc_load_func to fix the 2 bit shift.

	---

	* mini-ppc64.c, mini-ppc64.h, cpu-ppc64.md: Several fixes.
	Now PPC64 passes basic-long.exe.

	---

	* ppc/ppc-codegen.h: Make ppc_is_[u]imm16() work with 64 bit
	values.

	---

	* mini-ppc64.h, cpu-ppc64.md: Fixed caller/callee saved
	floating point regs.  Now PPC64 passes basic-calls.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, exceptions-ppc64.c,
	tramp-ppc64.c, cpu-ppc64.md: Several fixes.  PPC64 now runs objects.exe.

	---

	* mini-ppc64.c, tramp-ppc64.c: Small fixes.  PPC64 now runs
	arrays.exe and basic-math.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, exceptions-ppc64.c,
	cpu-ppc64.md: Several fixes.  PPC64 now runs exceptions.exe and
	devirtualization.exe.

	---

	* mini-ppc64.c: Several fixes.  PPC64 now runs iltests.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, tramp-ppc64.c: Disable generic
	code sharing.  PPC64 now passes generics.exe.

	---

	* basic-long.cs: New test case.

	---

	* mini-ppc64.c, mini-ppc64.h, tramp-ppc64.c, cpu-ppc64.md:
	Several	fixes.  PPC64 now passes most of the runtime regressions.
	
	

Followed by today's tweet:

The bootstrap means that the Mono JIT is actually doing a full build of Mono's compilers and class libraries and can be built on the target platform.

Update: Mark has posted a great picture of Jim Purbrick from Second Life, the man behind Mono running on Second Life.

Unity on Linux, First Screenshots

The first Unity3D on Linux screenshot:

The above program was built on MacOS, the result copied to Linux and then executed using the LinuxPlayer. This is still very basic, the port is yet far from done.

I followed Joachim's advise and added a tiny script to update the cube on the screen. See the video of the cubes in action: ogg and wmv.

Framework Design Guidelines, 2nd Edition

A couple of years ago I wrote an enthusiastic review of Brad Abrams and Krzysztof Cwalina's Framework Design Guidelines, a book that I absolutely love.

The book is a great compendium of best-practices for building software, traps and pitfalls to avoid.

But most importantly, it is the best source to learn the idioms and patterns used in the .NET Frameworks. Learning these idioms will have you writing code like the native C# speakers in no time.

I was incredibly honored when Brad asked me earlier this year to write the foreword for the second edition of the Framework Design Guidelines.

The second edition tracks the evolution of .NET and they apply as well to Mono. For instance, it now contains LINQ design patterns, extension methods patterns and DependencyProperties (used in WPF and Silverlight).

Silverlight Toolkit, now MS-PL

Update: Fixed some links, corrected some text.

Shawn Burke announced the Silverlight Toolkit and it is licensed under the open source MS-PL. The code is available here complete with unit tests (check Ning's blog on the unit testing framework).

With the Silverlight Toolkit they are taking a new approach to shipping new controls in an effort to move swiftly and deliver the controls to people at the right time. Their previous approach was to ship the Toolkit when every component was ready, and completely fleshed out.

Now they will be shipping the Toolkit with controls that might have different levels of quality (and they are clearly flagged in the documentation). Shawn explains the new "Quality Bands" model that they are using in his post.

You can try the components on the web. The charting control can be tried out with the ChartBuilder (check David's blog for details on the ChartBuilder):

The source code for the Toolkit and the Controls is great to learn how to use Silverlight and it is great for people that need to tweak them for their own applications. When it comes to these controls, you no longer need Microsoft to make small changes for you or the small bug fixes that impact your application.

Themes: An interesting control container in Silverlight is the theming control. You wrap your code around this, and it will let you skin your control with XAML and define the animations and interactions with XAML and the Visual State Manager:

Some of these themes reminded me of the Gtk+ themes from 1998. Back in the days of Enlightenment and the "Cheese Pixmap" theme were hot. Mehdi explains how the themes work and Jafar explains the ImplicitStyleManager, the foundation for themes.

Shawn's Talk

Shawn's talk at the PDC was very interesting. I did not get to see it during the conference, but I watched it in the comfort of home (wmv, mp4 and slide deck).

Moonlight Updates

Last week we branched Moonlight for the 1.0 release, full with the licensed Microsoft Codecs and started our release process for Moonlight Beta 1 to be available in the next few days. This release is not yet published on our web site, watch this space.

The Moonlight engine team has now resumed our work on Moonlight 2.0, the version that will track Silverlight 2.0.

In the meantime, while the GUI team was busy completing 1.0, the Mono core team has been working on the security framework for Moonlight, the networking stack (Silverlight allows Socket connections using policy files) and web services (System.ServiceModel, a subset of WCF).

The security system is the trickier and is the one that has received the most attention. We started early on last year in to implementing this, as we knew it would end up burning a lot of cycles to get it right.

Our hero has posted the initial work partition for the upcoming GUI work on Moonlight 2.0.

Moonlight is a blast, and who knows, maybe with our static compilation engine we might be able to deliver Silverlight on the iPhone.

Change.Gov

I wanted to thank everyone that helped get Barack Obama elected. Those that endorsed Obama passionately, those that videocasted, blogged, improved Obama's web site, donated to his campaign, wrote, discussed and voted on Tuesday to get him elected.

Barack does not only represent a change of direction for public policy, he is a truly brilliant candidate.

Some cool links on Barack:

• Larry Lessig: 20 minutes or so on why I am for Barack Obama.
• Tim O'Reilly's Why I Support Barack Obama.
• The Obama I know gives a glimpse of how Obama works.
• The Letter of Endorsement from 76 American Nobel Prize winners. winners.
• For the most cynics among us, you can watch the Chomsky on Obama interview which is as good as an endorsement as you can get from Noam.
• Beautiful pictures from the campaign trail from the Boston Globe.

I was surprised that the Obama campaign already launched their Change.Gov (thanks Nat) web site. You can now see how the team operates in real life, and you can share your story and you can share your vision of where America should go. The blog is here.

The above starts to deliver on the promise he had made during the campaign.

Got a cool collection of pictures about Obama or the reaction to the results? Please post it in the comments.

Inflamatory or misinformed comments will be deleted pronto.

Static Compilation in Mono

Another nice piece of technology that we showed at the PDC was static compilation, the feature behind allowing Mono to run on the iPhone in a fully legit way (no jail-breaking):

Screenshot from the Unity IDE.

Although Mono has supported batch compilation from CIL into native code in the past there was a small amount of code that was still generated dynamically. The intention originally was to help reduce startup and increase code sharing across multiple processes.

Code sharing is important once you have a handful of Mono processes running on the system. Instead of having to JIT the same code once per Mono instance for say "System.Console.WriteLine", the code laid out in essentially a shared object.

Our generated code uses many of the concepts found on the ELF file format to ensure that we can share code and that the code pages are read-only and not written to. This means that methods are invoked through a program linkage table instead of directly (to cope with the shared libraries being loaded at different addresses).

The Extra Mile

Although we are not certified XBox360 developers yet (we have yet to find the right person at Microsoft to talk to) we know from some of our users that have ported Mono to the XBox360 that JITing is not an option on that platform.

The XBox360 seems to have the same security-imposed limitations that the iPhone has, it is not possible for a Just-in-Time compiler to run in the platform as either the license terms or the kernel do not allow writable pages to become executable pages.

During the last few months we developed a static compilation mode for Mono. First we did this for the 1.0 profile, and now we are working on the 2.0 profile (so that we can support static compilation of generics). The work to support the 2.0 profile is reusing Mark's work on generic code sharing, which I found out to be a very nice synergy of efforts internally.

This means that it is now possible compile code from CIL to native code and not even ship the JIT compiler in your final program (saving some precious kilobytes from the final executable).

To do this, you must:

• Use Mono 2.0.1 at least.
• Request that Mono performs a full AOT compilation by using: mono --aot=full program.exe. That will generate your static executable. This executable still needs the runtime for things like garbage collection, threading and other runtime services.
• You then run your executable with Mono: mono --full-aot program.exe
• Optionally: build a new Mono on a separate location that removes the JIT engine by configuring Mono like this: configure --enable-minimal=jit. This will reduce your deployment by a few hundred Ks as the code generation and JIT engines are stripped out.
• Optionally: build a smaller set of libraries by using the Mono Linker (this is the tool that we use for turning Mono's 3.5 APIs into the Silverlight 2.0 APIs).
• Optionally: strip out the CIL code from the assemblies. We still require the assemblies for their rich metadata, but the actual CIL instructions can be safely removed. The new cil-strip tool built on top of Mono.Cecil can further shrink your deployed executables.

Developers interested in trimming down Mono can look into our documentation for more features that can be removed by using the --enable-minimal option.

Of course, once you remove the JIT you will not be able to use any dynamically generated code. This means no Reflection.Emit dynamically and at least for the time being or no IronPython/IronRuby.

John Lam told me at the PDC that they are looking into bringing static compilation for IronPython/IronRuby/DLR back, so this might just be a short-lived limitation.

For those interested in using Mono on the iPhone today the situation is a bit painful right now. You must run Mono on the target system to do the batch compilation and send the data back to assembly it on the host before you send the code back to the iPhone to run.

If you are wondering how did the demo go so smoothly at the PDC, the reason is that I was using Unity. Unity modified their local copy of Mono to be hardwired to do cross compilation to that exact platform. A more general purpose solution is needed to allow arbitrary platform-to-platform cross compilation, and we hope that this will be available in the future.

If you must quench your thirst for C# on the iPhone today your best choice is to use Unity's product and start building games instead of the enterprise application you always dreamed of.

From the Unity's Video Sample

If your boss demands that line of application running on the iPhone, you have a perfect excuse to learn the Unity gaming APIs and deliver the most glorious multi-touch, 3D-transformed line of business application to ever grace this world full with spinning AI for your "Sort By Customer Last Name" button.

C# 4.0: var, object and dynamic

Anders presentation on C# 4 was as usual great to listen to. He continues to evolve the language with solid steps, and the presentation was quite fun.

You can watch his presentation or just read the slide deck.

With C# 4 the new "dynamic" keyword has been introduced to flag a variable as a dynamic variable.

This is slightly different than var and object, the differences are as follows:

• "object x" is a shorthand for "System.Object x". This declares the variable x to have the type System.Object, this is strongly typed. And since C# provides autoboxing, you can assign anything you want to this variable.
• "var x = E" declares a variable x to be of the type of the expression E. The E is required, not optional. This is a strongly typed declaration, and you can only assign values whose type is typeof(E) to it.
• "dynamic x" declares the variable x to have dynamic semantics. This means that the C# compiler will generate code that will allow dynamic invocations on x. The actual meaning of "x.M" is deferred until runtime and will depend on the semantics of the IDynamicObject implementation.

Mono's SIMD Support: Making Mono safe for Gaming

This week at the Microsoft PDC we introduced a new feature in the Mono virtual machine that we have been working on quietly and will appear in our upcoming Mono 2.2 release (due in early December).

I believe we are the first VM for managed code that provides an object-oriented API to the underlying CPU SIMD instructions.

In short, this means that developers will be able to use the types in the Mono.Simd library and have those mapped directly to efficient vector operations on the hardware that supports it.

With Mono.Simd, the core of a vector operations like updating the coordinates on an existing vector like the following example will go from 40-60 CPU instructions into 4 or so SSE instructions.

Vector4f Move (Vector4f [] pos, ref Vector4f delta)
{
	for (int i = 0; i < pos.Length; i++)
		pos [i] += delta;
}
	

Which in C# turns out to be a call into the method Vector4f.operator + (Vector4f a, Vector4f b) that is implemented like this:

Vector4f static operator + (Vector3f a, Vector3f b)
{
	return new Vector4f (a.x+b.x, a.y+b.y, a.z+b.z, a.w+b.w);
}
	

The core of the operation is inlined in the `Move' method and it looks like this:

movups (%eax),%xmm0
movups (%edi),%xmm1
addps  %xmm1,%xmm0
movups %xmm0,(%eax)
	

You can see the details on the slides that I used at the PDC and look at the changes in the generated assembly, they are very large.

Ideally, once we tune the API based on our user feedback and contributions, it should be brought to ECMA for standardization. Hopefully we can get Microsoft to implement the SIMD support as well so all .NET developers have access to this.

Making Managed Code Viable for Gaming

Many developers have to resort to C++ or assembly language because managed languages did not provide the performance they needed. We believe that we can bring the productivity gains of managed languages to developers that seek high performance applications:

But even if you want to keep using your hand-tuned C++ game engine, the SIMD extensions will improve the performance of your scripting code. You can accelerate your ray casting operations by doing all the work in the managed world instead of paying for a costly managed to unmanaged transition and back.

You can avoid moving plenty of code from C# into C++ with this new functionality.

Some SIMD Background

Most modern CPUs contain special instructions that are able to perform arithmetic operations on multiple values at once. For example it is possible to add two 4-float vectors in one pass, or perform these arithmetic operations on 16-bytes at a time.

These are usually referred to as SIMD instructions and started showing up a few years ago in CPUs. On x86-class machines these new instructions were part of MMX, 3DNow or the SSEx extensions, on PowerPC these are called Altivec.

CPU manufacturers have been evolving the extensions, and newer versions always include more functionality and expand on the previous generations.

On x86 processors these instructions use a new register bank (the XMM registers) and can be configured to work on 16 bytes at a time using a number of possible combinations:

• byte-level operations on 16 elements.
• short-level operations on 8 elements.
• single precision or integer-level operations on 4 elements.
• double precision or long-integer operations on 2 elements.

The byte level operations are useful for example when doing image composition, scaling or format conversions. The floating point operations are useful for 3D math or physics simulations (useful for example when building video games).

Typically developers write the code in assembly language to take advantage of this feature, or they use compiler-specific intrinsic operations that map to these underlying instructions.

The Idea

Unlike native code generated by a compiler, Common Intermediate Language (CIL) or Java class files contain enough semantic information from the original language that it is very easy to build tools to compute code metrics (with tools like NDepend), find bugs in the code (with tools like Gendarme or FxCop, recreate the original program flow-analysis with libraries like Cecil.FlowAnalysis or even decompile the code and get back something relatively close to the original source code.

With this rich information, virtual machines can tune code when it is just-in-time compiled on a target system by tuning the code to best run on a particular system or recompiling the code on demand.

We had proposed in the past mechanisms to improve code performance of specific code patterns or languages like Lisp by creating special helper classes that are intimately linked with the runtime.

As Mono continues to be used as a high-performance scripting engine for games we were wondering how we could better serve our gaming users.

During the Game Developer Conference early this year, we had a chance to meet with Realtime Worlds which is using the Mono as their foundation for their new work and we wanted to understand how we could help them be more effective.

One of the issues that came up was the performance of Vector operations and how this could be optimized. We discussed with them the possibility of providing an object-oriented API that would map directly to the SIMD hardware available on modern computers. Realtime Worlds shared with us their needs in this space, and we promised that we would look into this.

The Development

Our initial discussion with Realtime Worlds was in May, and at the time we were working both towards Mono 2.0 and also on a new code generation engine that would improve Mono's performance.

The JIT engine that shipped with Mono 2.0 was not a great place to start adding SIMD support, so we decided to postpone this work until we switched Mono to the Linear IL engine.

Rodrigo started work on a proof-of-concept implementation for SIMD and after a weekend he managed to get the basics in place and got a simple demo working.

Beyond the proof of concept, there was a lingering question: were the benefits of Vector operations going to be noticeably faster than the regular code? We were afraid that the register spill/reload would eclipse the benefits of using the SIMD instructions or that our assumptions had been wrong.

Over the next few weeks the rest of the team worked with Rodrigo to turn the prototype into something that could be both integrated into Mono and would execute efficiently (Zoltan, Paolo and Mark).

For example, with Mono 2.2 we will now align the stack conveniently to a 16-byte boundary to improve performance for stack-allocated Mono.SIMD structures.

So far the reception from developers building games has been very positive.

Although today we only support x86 up to SSE3 and some SSE4, we will be expanding both the API and the reach of of our SIMD mapping based on our users feedback. For example, on other architectures we will map the operations to their own SIMD instructions.

The API

The API lives in the Mono.Simd assembly and is available today from our SVN Repository (browse the API or get a tarball). You can also check our Mono.Simd documentation.

This assembly can be used in Mono or .NET and contains the following hardware accelerated types (as of today):

Mono.Simd.Vector16b  - 16 unsigned bytes
Mono.Simd.Vector16sb - 16 signed bytes
Mono.Simd.Vector2d   - 2 doubles
Mono.Simd.Vector2l   - 2 signed 64-bit longs
Mono.Simd.Vector2ul  - 2 unsigned 64-bit longs
Mono.Simd.Vector4f   - 4 floats
Mono.Simd.Vector4i   - 4 signed 32-bit ints
Mono.Simd.Vector4ui  - 4 unsigned 32-bit ints
Mono.Simd.Vector8s   - 8 signed 16-bit shorts
Mono.Simd.Vector8us  - 8 unsigned 16-bit shorts
	

The above are structs that occupy 16 bytes each, very similar to equivalent types found on libraries like OpenTK.

Our library provides C# fallbacks for all of the accelerated instructions. This means that if your code runs on a machine that does not provide any SIMD support, or one of the operations that you are using is not supported in your machine, the code will continue to work correctly.

This also means that you can use the Mono.Simd API with Microsoft's .NET on Windows to prototype and develop your code, and then run it at full speed using Mono.

With every new generation of SIMD instructions, new features are supported. To provide a seamless experience, you can always use the same API and Mono will automatically fallback to software implementations if the target processor does not support the instructions.

For the sake of documentation and to allow developers to detect at runtime if a particular method is hardware accelerated developers can use the Mono.Simd.SimdRuntime.IsMethodAccelerated method or look at the [Acceleration] atribute on the methods to identify if a specific method is hardware accelerated.

The Speed Tests

When we were measuring the performance improvement of the SIMD extensions we wrote our own home-grown tests and they showed some nice improvements. But I wanted to implement a real game workload and compare it to the non-accelerated case.

I picked a C++ implementation and did a straight-forward port to Mono.Simd without optimizing anything to compare Simd vs Simd. The result was surprising, as it was even faster than the C++ version:

Based on the C++ code from F# for Game Development

The source code for the above tests is available here.

I use the C++ version just because it peeked my curiosity. If you use compiler-specific features in C++ to use SIMD instructions you will likely improve the C++ performance (please post the updated version and numbers if you do).

I would love to see whether Johann Deneux from the F# for Game Development Blog could evaluate the performance of Mono.Simd in his scenarios.

If you are curious and want to look at the assembly code generated with or without the SIMD optimizations, you want to call Mono's runtime with the -v -v flags (yes, twice) and use -O=simd and -O=-simd to enable or disable it.

Presentation

You can watch the presentation to get some color into the above discussion or check it in the Silverlight player, Get it as PDF, or PPTX.

Mono and .NET talk at PDC video.

The PDC 2008 was a blast. It was a privilege to be able to present Mono to all of these developers.

Joseph Hill helped me prepare my presentation for the PDC. Our goal was to explore how Mono could help .NET developers, but we did not want to go down a laundry list of features, or a list of APIs, or rehash what the advanced audience at the PDC already knew.

player, pdf, pptx

The idea was to pick a couple of interesting innovations from Mono and try stitch a story together. I discussed our embeddable C# compiler (which we need to start calling "Compiler Service"), some applications of Mono in gaming, our recent SIMD extensions and using Mono on the iPhone.

As for me, I am catching up on all the sessions I missed this weekend. All of the videos and slide decks are available for free from the Microsoft PDC site, and republished in Channel9.

In the next few days I will blog in more detail about each topic.

Interactive GUI Shell

This week at the Microsoft PDC I showed gsharp, our GUI repl for the C# 3.0 language, a tool that I had previously talked about.

Before the PDC we copied an idea from Owen's great reinteract shell where we provide our REPL with a mechanism to turn objects into Gtk.Widgets which we then insert.

Out of the box we support System.Drawing.Bitmap images, we turn those into Gtk Widgets that then we render:

I also added a toy Plot command that can take a number of lambdas that return floats to do some cute plots. The plots are just rendered into a System.Drawing.Bitmap so they get painted on the screen automatically:

But you can add your own handlers for any data types you want, all you have to do is call RegisterTransformHandler with a function that can return a Gtk.Widget based on an input object, or null if it does not know how to render it.

The implementation to render images is very simple, this is the implementation:

using System;

public class MyRenderHelpers {
	public static object RenderBitmaps (object o)
	{
		System.Drawing.Bitmap bitmap = o as System.Drawing.Bitmap;
		if (bitmap == null)
			return null;

		return new BitmapWidget (bitmap);
	}
}
	

You can put your own library of helper methods in a compiled assembly in ~/.config/gsharp, and then register all of your types from a file ending with the extension .cs in ~/.config/gsharp:

RegisterTransformHandler (MyRenderHelpers.RenderBitmaps);
	

And you are done.

The above could be used for example to create all kinds of information visualizers for the GUI REPL. I would love to see a LINQ query navigator, similar to the one in LinqPad.

Update: A one line change that brings gsharp into the new millenium by rendering `true' and `false' with icons instead of text:

In LA for the Microsoft PDC

After a great week in Copenhagen with the Unity community I spent 14 hours on a high-tech tin can flying to LA for the Microsoft PDC conference.

Mono Talk

I am doing a talk on Wednesday at 4:45 in room 515B.

Unlike previous talks that I have done about Mono, this is an advanced talk and will skip over all the generalities and go straight to Mono CLI/C# features and innovations.

I decided against talking about Moonlight or APIs, as information about can be better learned elsewhere.

Speculation

There has been enough leaked information that we know some bits about C# 4. Some guess it includes dynamic support, other that it will be more F#-like and add immutability, others that it will introduce some Alef-like threading capabilities.

Then there is talk about .NET 4, and I just have no clue what they will announce.

So what do you think they are announcing this week?

Speculate away in the comments.

Live Blogging from Unite

I am live blogging from Unite, the Unity3D conference, one of the most fun users of Mono.

Announcements

Next UnityEditor will run on Windows, and its rewritten in C# (it originated on MacOS, and is now moving to Windows).

Unity as of today ships for building games on the iPhone. These are fully legit binaries, no need to crack your iPhone, they are using Mono's batch compilation to generate static binaries with no JITing involved (per Apple licensing requirements).

Side Note

Since I am a Linuxista, you might be wondering why I am so excited about Unity. Of course I am excited because they use Mono, but I am also excited because Novell is working with Unity to bring this to Linux:

erandi$ uname -a
Linux erandi 2.6.25.16-0.1-pae #1 SMP 2008-08-21 00:34:25 +0200 i686 i686 i386 GNU/Linux
erandi$ ls -l unity-linux/build/LinuxPlayer 
-rwxr-xr-x 1 miguel users 45735629 2008-10-12 00:37 unity-linux/build/LinuxPlayer*
	

We do not have a timeline yet, please do not spam the Unity guys with requests, stay tuned to this blog for updates.

FusionFall pre-mortem

12:17am Joachim explains their strategy with the web plugin and how they will cope with multiple versions of it as time goes by.

12:06am Cartoon Network will drive a lot of the plugin penetration.

11:50am Joachim is showing profiling information (particle, physics, the top scripts, time taken per frame).

The game went from 2gigs to 300megs by using some interesting compressions fo meshes and animations.

All of the features that were added for FusionFall are being folded back into the future Unity 2.5.

Pics: I wish I had brought my USB cable to post pictures from the camera. I will try to spice up this post tonight with the photos.

11:45am Runtime World Streaming: scenes are dynamically loaded and unloaded base on the player position in the world, the world is made up of a 16x16 grid. Scene loading happens when a player approaches a boundary in the world.

11:40am The Unity guys are talkng about the challenge of converting the assets from Gamebryo to Unity, the volume was large (25k files of Gamebryo data, which was constantly changing and growing).

They added support to Unity to interoperate with the Gamebryo and Cartoon Network data and wrote plenty of C# importing scripts and tools.

11:30am FusionFall talk by Joachim Ante.. FusionFall is a project that was done for the Cartoon Network.

It is an MMO with platform game elements.

It has a huge streaming world, so there are no pauses as you navigate the world. The game is targetted at kids (8-14).

The game was produced by Cartoon Network and developed by Grigon Entertainment, a team of 40 developers, 10 of them programmers, and has been under development for 3 years. Originally this was Windows-only standalone executable, developed with Gamebryo.

The cycle at some point included the prototyping done in Unity, shipping the executable to engineering, and engineering reimplementing in Gamebryo. They realized that they could turn the prototype as the actual client and that they could just communicate to their backend server. This allowed them to switch the entire game from Gamebryo to Unity.

Originally the standalone game was 2gb in size (lots of music, voice overs, terrains). This was a problem for kids, since they are not going to wait for 2 gigs to download, this was a big barrier to entry.

Unity's web based distribution and the strong world-streaming features were a good match for this project. This allowed Cartoon Network to give a great experience.

The entire MMO was ported in four months by a team Grigon developers and four Unity engineers working with them. They were on a very tight schedule. Four developers at Grigo ported the game client from C++ to C#.

Keynote

10:52am Interesting overview of the challenges of the game industry. Where does the game industry go next? A discussion on integrarting games with the web and delivering games as services.

On one end there is the flashy stuff, on the other end lots of talk about the enterprise components of gaming. I had no idea.

Between the low-barrier to entry and the high-barrier to entry markets, there is a large space for gaming and where 3D games on the browser will make a difference.

Phil sees Unity as an agent that will help transform the game industry.

10:31am Phil Harrison president of Atari is now on stage, "First time I have done a presentation in a Planetarium". Atari has no commercial relationship with Unity or investing-wise. He is here because "I wanted to be here, what David and his team are doing is transformational for the industry. I had an Eureka moment early this year, I had just joined Atari, and someone told me `check out Unity3d.com', I had heard about it, but never used it. Using the Web player demo was eye opening for me. [...] This is something similar to what I saw at Sony in 1993, when we first got the dinosaur demo on the PS1 [...] The island demo I believe is a game changer in this industry".

"I have become an unpaid evangelist of Unity".

Phil is now going to talk about the game industry, wont blog that.

10:29am Introspection, why we want to do this? Goals: we want people to build games for the web, the iPhone, the Wii, and for everything else.

10:29am Announcing Indie version, 199 dollars, but has some watermark/splashscreen at the beginning.

10:26am Windows Vista logo on screen.

"This is true, I have to admit it", they are demostrating the new Unity3D IDE on Unix. The same Unity3D tool but now running on Windows.

"We are going completely cross platform", every script that you write will run on both platforms (Woohoo! Go CIL! Go!).

10:21am Nicholas is going to talk about "Secret Labs". He talks about "Jus t Press Play", "Buliding for multiple targets" and the script property editor.

They wanted to improved upon th eUnity editor.

They rewrote the Unity editor from zero. Created entirely on top of the Unity APIs themselves - EditorWindows, unityGUI, GameObjects and Components. Unity is built on Unity now. "It is way faster than cocoa".

He is showing Unity 2.5, looks like Unity 2.1; They now use tabs and the various windows can be dragged around, very much like MonoDevelop. He is showing the editor by dragging a lightpost into a paradise island and showing the new GUI tools like snapping, rotation and UIs that are closer to Maya's tools.

He shows "Snap to surface" so you can easily position stuff on the 3D terrain. People like it.

The UI is a lot easier to use. "We have been focused on the tiny details, but we are not shipping this yet". Everything in the IDE can now be scripted.

He shows how a few lines of code a developer can attach a camera view when clicking on a property.

130 new API entry points, Unity developers can now do everything that the Unity3D GUI can do.

10:20am David talks about the pricing; Two pricing: cheap and expensive.

10:13am Joachim Ante is introducing Unity for the iPhone. "With Unity we have always focused on iteration time". He goes into some of the technical details, "With all the new input mechanisms for theiphone, how do we provide a quick feedback system, we wanted to keep the experience of develop, hit run under a second".

They provide a "Unity remote" that runs on the iPhone, you use the iPhone to control the game, but the debugging actually happes on the PC. So you hit "run" and there is no wait at all.

Joachim is explaining how they are using AOT to run native code on the iPhone without having any JIT on the iPhone.

Joachim shows "Build and Run" that does the cross compiler and sends the code to the iPhone. It takes about 10 seconds and he now has the game running on the iPhone and shows how both Javascript and C# code running on the system.

10:10am Some stats: sold out event; 180 attendees; community doubled in the last year (2x employees, 2x posts, 2x users);

Last year they released Unity 2.0.

Unity 2.1 was released, should have been 3.0, but they did not charge more.

They announced Unity for Wii.

Big games using Unity: Sony, Disney. Virtual worlds like Hangout.Net is out (very pretty!) A new online community was created (Blurst). SeriousGames released a new title "Global Conflicts" Latin America.

Hard to keep up with the list of users.

10:04am The Unity founders Joachim, David and Nicholas are on stage to start Unite'08.

They moved us from the room downstairs to the iMax auditorium whihc is packed with game guys. During the moments of panic that ensued after they moved us from the downstairs room into the switched rooms, I ended up in the front row, which in retrospect was a big mistake, you cant take pictures with the standard lens of this massive screen.

Registration

9:48am Waiting for the keynote at the Unite conference. There are about two hundred people packed in the Tycho Planetarium. The Blurst guys are all wearing matching shirts and have taken over the first line of the auditorium, From here I can see a them debugging a Sonic-like game that he is prototyping. Then he switched to some game that has an angry minotaur breaking dishes in a museum.

The minotaur looks angry and just scored 3,000 points of some kind.

Everyone in this conference seems to be using a Mac, and I could swear this computer is the only Linux machine in the audience.

Pre-Conference

Last night I had dinner at the Unity headquarters and got a chance to meet some of the Unity hackers and users before the conference started.

Gained a deeper insight into what we can do to improve Mono's VM for games. Lots of good ideas.

Phil Harrison brought up "the debugger" issue ;-).

Hopefully Rasmus from CellDotNet will show up for the Unity Mingle tonight.

Mono 2.0 OSX Installer Ready

We released an updated installer for Mono 2.0 on MacOS X.

This release got delayed because we wanted to upgrade our bundled Gtk# stack to contain the latest release of Imendio's Gtk+ for MacOS X.

Banshee coming to an OSX near you this week.

Mono OSX Survey!

We are trying to understand how we can improve Mono on the OSX space. Help us figure this out by filling out our Mono on OSX Survey.

Relocatable Applications

If you have followed our Guidelines for Application Deployment your software should be easy to be packaged and distributed for MacOS X as a relocatable application.

Eoin Hennessy worked on integrating Banshee into the OS, and packaging it into a bundle that runs out of the box on MacOS. The following are some screenshots from Aaron's box:

The Banshee open source media player.

Sandy has ported Tomboy and Tasque to MacOS and Windows and provided installers for both.

Tomboy integrates into the dock and menus.

Aaron Bockover from the Novell Desktop Team has promised that Eoin's work will be part of Friday's Banshee release. From this point on, Banshee will be released both for Linux and MacOS X at the same time.

Maybe F-Spot is not too far behind?

The Small Print

• We downgraded the bundled Gtk+ from 2.15 to 2.14.3, as 2.15 was a development version and 2.14 is the officially supported Gtk+.
  This means that applications that linked directly against Gtk+ 2.15 from Mono 1.9 will fail to run. Please re-link those binaries.
• We removed MonoDevelop from this distribution, so our package only contains the Mono SDK and Mono runtime.
  A MonoDevelop installation package will come later, we apologize for this delay.
  On the upside: now that the distribution is split, we will be doing MonoDevelop Beta 2 previews as DMGs after the PDC.

Help us improve Mono on OSX by completing the Mono on OSX Survey and providing comments at the end.

Parallel Programming

As much as I personally dislike the use of threads for user code, multi-cores systems are here to stay. They are becoming increasingly popular (most laptops now ship with dual core systems, home computers ship with 3 cpus and gaming consoles ship with multiple general purpose cpus as well).

Developers will need new frameworks for developing software that is ready to take advantage of multiple CPUs. But most importantly they will need to learn the traps and pitfalls of writing parallel/threaded code.

Here are two fantastic articles on MSDN that cover these topics:

• Design Considerations For Parallel Programming by David Callahan.
• Solving 11 Likely Problems In Your Multithreaded Code by the excellent blogger and hacker Joe Duffy.

J�r�mie Laval worked on an ParallelFX implementation for Mono over the summer as part of the Google Summer of Code.

The implementation currently lives on the student repository at code.google.com. I can not wait for the API to be stabilized so we can move it into the main Mono distribution.

Going to Copenhagen

Next week I will be in Copenhagen for Unity3D's Unite conference.

Unity3D is one of the most fun users of Mono as they create IDEs for Game Developers and they are driving the adoption of Mono, C#, Boo and their own UnityScript in the gaming space.

As a newcomer into this industry, there are various sessions from actual Unity user on how they have built their games from start to finish. Other sessions include details on publishing, production (ArtPlant), Physics (Flashbang), Shader Programming (Unity), developing on the iPhone (Unity), a post-morterm on FusionFall's work for Cartoon Network and the hands-on lab.

Some cool stuff from the agenda includes a keynote participation from Atari's President.

If you want to meet up, drop me an email. I will likely be going to the Unity Mingle events at night and departing early on Friday to fly to the Microsoft PDC in LA.

Alan "BitSharp" McGovern Joins Novell

Alan McGovern, who created BitSharp during a Google Summer of Code for Mono has joined the Moonlight team at Novell.

Imagine the possibilities! Bittorrent clients, servers, trackers all running from Silverlight 2.0 Web Applets!

Discuss.

Mono 2.2 Feature: Mono.Options

In the upcoming Mono 2.2 release we will be distributing Jonathan Pryor's most excellent Mono.Options library.

Mono.Options is a beautiful command line parsing library. It is small, succinct, a joy to use, easy and powerful, all in one.

It is one of those libraries that does more with less. Something that every programmer aspires to write, but that we seldom achieve.

It has also struck a good balance for Unix and Windows developers as options can be used on both systems, and map well to practices on both systems. It took a long time to get the right "blend" of parsing, but I think Jonathan has achieved it.

Consider this example:

using System;
using System.Collections.Generic;
using NDesk.Options;

class Test {
    static int verbosity;

    public static void Main (string[] args)
    {
        bool show_help = false;
        List names = new List ();
        int repeat = 1;

        var p = new OptionSet () {
            { "n|name=", "the name of someone to greet.",
              v => names.Add (v) },
            { "r|repeat=", "the number of times to repeat the greeting.",
              (int v) => repeat = v },
            { "v", "increase debug message verbosity",
              v => { if (v != null) ++verbosity; } },
            { "h|help",  "show this message and exit", 
              v => show_help = v != null },
        };

        List extra;
        try {
            extra = p.Parse (args);
        }
        catch (OptionException e) {
            Console.Write ("greet: ");
            Console.WriteLine (e.Message);
            Console.WriteLine ("Try `greet --help' for more information.");
            return;
        }

        if (show_help) {
            ShowHelp (p);
            return;
        }

        string message;
        if (extra.Count > 0) {
            message = string.Join (" ", extra.ToArray ());
            Debug ("Using new message: {0}", message);
        }
        else {
            message = "Hello {0}!";
            Debug ("Using default message: {0}", message);
        }

        foreach (string name in names) {
            for (int i = 0; i < repeat; ++i)
                Console.WriteLine (message, name);
        }
    }

    static void ShowHelp (OptionSet p)
    {
        Console.WriteLine ("Usage: greet [OPTIONS]+ message");
        Console.WriteLine ("Greet a list of individuals with an optional message.");
        Console.WriteLine ("If no message is specified, a generic greeting is used.");
        Console.WriteLine ();
        Console.WriteLine ("Options:");
        p.WriteOptionDescriptions (Console.Out);
    }

    static void Debug (string format, params object[] args)
    {
        if (verbosity > 0) {
            Console.Write ("# ");
            Console.WriteLine (format, args);
        }
    }
}
	

And here is an example of its use:

$ mono greet.exe --help
Usage: greet [OPTIONS]+ message
Greet a list of individuals with an optional message.
If no message is specified, a generic greeting is used.

Options:
  -n, --name=VALUE           the name of someone to greet.
  -r, --repeat=VALUE         the number of times to repeat the greeting.
  -v                         increase debug message verbosity
  -h, --help                 show this message and exit

$ mono greet.exe -v- -n A -name=B --name=C /name D -nE
Hello A!
Hello B!
Hello C!
Hello D!
Hello E!

$ mono greet.exe -v -n E custom greeting for: {0}
# Using new message: custom greeting for: {0}
custom greeting for: E

$ mono greet.exe -r 3 -n A
Hello A!
Hello A!
Hello A!

$ mono greet.exe -r not-an-int
greet: Could not convert string `not-an-int' to type Int32 for option `-r'.
Try `greet --help' for more information.

	

He has also documented it thoroughly.

Where possible (new tools being written, or tools that have a similar command line structure that is compatible) we will be switching to this command line parsing library.

The library is small, so developers should include a copy of the source code in their application, this is how you should include it in your makefiles:

Options.cs:
	cp `pkg-config --variable=Sources mono-options` .

	

Then your code can just include a reference to it.

MonoDevelop gets VI bindings

I grew up mostly with Turbo Pascal as my development environment. When I started to write C code in DOS, I used Turbo C briefly but for some reason I switched to the BRIEF text editor for a while.

Around 1989 my friend Max Mendizabal who used nothing but Epsilon told me "Unix is the future, if you learn Epsilon, you will be ready to switch to Emacs when the time comes".

Prophetic words.

When I eventually switched to Unix in 1992, having learned Epsilon was useful, but Emacs was too slow for quick edits. I still used Emacs for programming, but for quickly making changes to a file, I ended up learning vi.

When computers got faster, I tried to switch to Emacs for all my editing tasks, but my brain had been hardwired. I even added "alias vi=emacs" to by shell, and I would find myself typing subconsciously "/usr/bin/vi".

To this day, I use both editors interchangeably.

In any case, the above story was just an excuse to introduce VI Mode for MonoDevelop.

Mono 2.0 is out!

Today we released Mono 2.0 to the world. You can download sources and binaries from our download page. And our official release notes are up as well. This of course would not be possible without the open source contributors that worked tirelessly on Mono sending patches, fixing bugs, helping the community, answering questions, creating test cases and supporting us all these years.

Mono 2.0 is both a runtime for application and a kit for developers for writing applications with C# and other ECMA CLI languages for a wide spectrum of uses.

Big thanks go to the fantastic Mono team at Novell that has kept the excitement and the pace over all these years (we started in 2001), the large contributions from Mainsoft, Unity3D and our users that pushed us to fix bugs, implement new features and tune Mono. Also, we very much appreciate the work of the ECMA 334 and 335 committee members that worked on the CLI and C# specifications and everyone at Microsoft that answered our questions over the years and specially those that licensed code under open source licenses.

We originally started to work on Mono, because we wanted to make developers happier and more productive on Linux. We liked C#, we liked the CIL and we wanted to have those technologies available on our platform.

Since we have been active in the Linux Desktop world, it is not a surprise that the early use of Mono was mostly on Linux desktop applications, and Mono continues to shine there. Server-side use of Mono was a natural evolution and we soon were powering ASP.NET sites on Linux.

There is one area where we under-delivered in the past, and it has been a constant source of pain. Up until now, we did not have a working debugger. This has finally changed, and Mono 2.0 includes for the first time a debugger, the time for WriteLine() debugging is now behind us.

As the project matured, developers started taking advantage of Mono's open source nature: essentially .NET on their own terms. A platform that could be adapted, morphed, ported and modified to suit many different uses. Today Mono is embedded in portable mp3 players and powers Unity3D's game engine on the Apple iPhone, the Nintendo Wii, MacOS X and Windows (Some folks at Novell are working with Unity on bringing Unity3d to Linux!).

It has also been deployed to run code on large clusters of servers for SecondLife, powers our open source Silverlight implementation (Moonlight) and powers the popular DekiWiki: a Social Collaboration Tool.

Mono is a large project and it is hard to pick one feature to talk about as there are so many, so instead I put together a quick table of the major features that are part of this release:

Compiler Support	.NET APIs	Mono APIs
Mono's Open Source Compilers: C# 3.0 compiler. LINQ to Objects. LINQ to XML. Visual Basic 8 compiler. Open Source Compilers: Boo. Cobra. GNU's GCC with the CIL backend. Microsoft's IronPython. Jeroen's IKVM (Java on .NET). Microsoft's IronRuby. Phalanger (PHP on CLI). Commercial Compilers: ISE's Eiffel. Microsoft's C#. Microsoft's F#. Microsoft's VB.NET. RemObject's Oxygene (Object Pascal). And many more.	Core API: 2.0 core APIs. System, System.Xml. 3.5 System.Core. System.Drawing. System.DirectoryServices. System.Web.Services. Windows.Forms 2.0: Win32 driver. Quartz/OSX driver. Cairo/X11 Unix driver. ASP.NET 2.0: Core ASP.NET. ASP.NET AJAX. Apache and FastCGI integration. ADO.NET 2.0 plus providers for: Managed drivers: Postgresql, MS SQL Server, Sybase. Semi-managed drivers: Firebird, IBM DB2, Oracle, Sqlite. MySQL provides their own drivers.	GUI APIs: Gtk# (Unix, Windows, MacOS X). Cocoa# (MacOS X). Mono Core: Mono.Addins - Extensibility Framework. Mono.Cairo - Cairo Graphics Binding. Mono.Cecil - ECMA CIL Manipulation. Xml.Relaxng. Novell.Directory.Ldap C5 - Generics Library. Linux Specific: Mono.Posix/Mono.Unix. NDesk.DBus Other Ecosystem Libraries: Bit# - Bittorrent client/server library. Mono.Fuse - User-space file systems. Mono.ZeroConf - Bonjour stack. Mono.Nat - Network Address Translation. Mono.Upnp - Universal Plug and Play. Tao Framework - OpenGL, OpenAL, SDL and Cg bindings.

We have ported Mono to a wide variety of platforms and operating systems on this 1.0 to 2.0 cycle. These platforms include:

• Linux (x86, x86-64, PowerPC32, Itanium, SPARC, ARM, s390, s390x.
• Solaris (x86-64, SPARC).
• MacOS X (x86, PowerPC32).
• Windows (x86, support for x86-64 will come in Mono 2.2).
• Nintendo's Wii.
• iPhone/iPod Touch (ARM, limited functionality due to licensing requirements; I will blog later this week about this).
• *BSD (x86, x86-64).

Developing with Mono

Long time Linux developers will probably continue to use Emacs and VI, but some new Linux developers might want to use an IDE. New developers can use our open source MonoDevelop IDE on Linux, or alternatively the commercial X-Develop IDE or SlickEdit.

If you are a Windows developer, you can continue using Visual Studio or your IDE of choice to write the code and compile it. Your binaries will run just fine on Linux.

To assist Windows developers in porting their applications to Unix, we have provided the Mono Migration Analysis tool.

Runtime Changes

The Mono Virtual Machine gained plenty of features since Mono 1.2 was released. We have added:

• Generic Code Sharing and VTable Compression have been implemented: this significantly reduces the memory footprint of generic type instantiations, while still getting the speed benefits of using generics.
• AOT support: in addition to x86, we now also support ARM and x86-64.
• COM interop is now complete (works on Windows with "real COM" and can be used on Unix with Mainsoft's COM or Mozilla's XPCOM).
• AOT code can now AOT parts of 2.0 assemblies (assemblies that contain generics).
• Full AOT support (allows code to run JIT-less, this is limited to 1.0 code).
• CIL Verifier: Now Mono has a CIL verifier.
• CoreCLR Security: the security system used by Moonlight.
• Many optimizations that improve execution performance: New intrinsics operations (Math.Min/Max for example); various operations are now inlined by the JIT; managed allocations (no transition to unmanaged code for allocating memory); multi-array access is now tuned by the JIT; constant and block initializations are now handled by the JIT; Faster initialization and access to multi-dimensional arrays (4x faster).
• The runtime went on a diet, many runtime data structures are smaller making Mono lighter.

Tools

In addition the the Mono Debugger making its debut appearance on this release, we are very proud of our code analyzer Gendarme.

Gendarme is a extensible rule-based tool to find problems in .NET applications and libraries. Gendarme inspects programs and libraries that contain code in ECMA CIL format (Mono and .NET) and looks for common problems with the code, problems that compilers do not typically check or have not historically checked.

Feedback

Mono is not perfect, but we want to improve it. Like many other open source projects, we need your bug reports to improve Mono. If you have problems with Mono, help us by filing a bug report.

Special Thanks

Special thanks to Hacker Extraordinaire Aaron Bockover who not only brings us the best media player in the world, but created the new web site design and implemented and tuned it over very long extra hours up until 7am in the morning on his weekend.

And to our packaging and QA team that spent extra hours to get all the bits and pieces in place for the release.

Five Second Linux Boot

I loved this LWN article on the changes necessary to make Linux boot in 5 seconds on the Asus EEE PC (a relatively slow computer).

Hopefully all Linux distributions will adopt these changes for custom deployments.

Moonlight Update: Media Codecs

A couple of weeks ago we started the work on porting Microsoft's Media Codecs to Linux and we got the C version running.

Popfly in Firefox3/Linux/x86

Geoff, Fernando and Rolf have been hard at work on this, and have also added the infrastructure to download and install the codecs on demand.

The next step was getting all the assembly language supported in Linux, and today Geoff got the assembly optimized SSE1 audio decoder running (the first chunk of the decoders).

Of course, the rest of the team has been busy fixing bugs and improving the performance in preparation for the first public beta of Moonlight.

Microsoft changes the Managed Extensibility Framework License

A couple of weeks ago I suggested that developers interested in having their .NET software run in other platforms should avoid Microsoft's Managed Extensibility Framework (MEF) as it was not an open source library.

I had a chance to discuss with Glenn, Sam and Bob the benefits of using the MS-PL for this library first over twitter and then over email.

Representing .NET's loyal competitor, I did not think that we stood a chance of getting Microsoft to change the license, but I was pleasantly surprised. Glenn understood the value of open source, Sam wanted to do the right thing about this library and CodePlex and Bob argued that Mono already had Mono.Addins anyways.

Today Glenn announced that Microsoft has changed the license for MEF to the open source MS-PL license.

Thanks to everyone at Microsoft that helped change the license!

DbLINQ, LINQ to Databases and Mono

Atsushi Enomoto blogs about the work involved in bringing LINQ to Databases to Mono.

The effort was a joint collaboration between the awesome DbLINQ developers, Pablo Iñigo Blasco our Google Summer of Code Student and Novell's Atsushi Enomoto.

The DbLINQ developers had created a general purpose LINQ provider that could be used with database providers other than SQL Server. They also relicensed their code a few months ago to the MIT X11 license to allow for integration with Mono's code base.

Read Atsushi's description of how the effort was put together and how DbLINQ is being refactored to become a full System.Data.Linq implementation and still provide the foundation for third-parties to easily create database LINQ providers.

DbLINQ is a great project, and it needs your help to complete the effort.

Mono at the PDC 2008

Great News Mono Lovers!

Later this month I will be presenting the session "Mono and .NET" at the Microsoft Professional Developers Conference in LA.

Exciting times!

Update: My talk will cover new technologies that we have created as part of Mono. Some of them are reusable on .NET (we try to make our code cross platform) and some other are features that specific to Mono's implementation of the CLI.

Microsoft to incorporate jQuery into Visual Studio

This weekend the news came out that Microsoft was going to bundle and support John Resig's jQuery as part of Visual Studio and ASP.NET. From Scott's blog:

I'm excited today to announce that Microsoft will be shipping jQuery with Visual Studio going forward. We will distribute the jQuery JavaScript library as-is, and will not be forking or changing the source from the main jQuery branch. The files will continue to use and ship under the existing jQuery MIT license.

Beyond the obvious benefits to developers, what interests me is that Microsoft is now bundling an open source component into their commercial offerings.

This is a first time for Microsoft.

With IronPython they continued development of an open source project in house. With IronRuby, they were open to external contributions to the libraries of IronRuby. In both cases they were not taking external code or contributions directly into their core products.

Hopefully they will start using more open source code in their products. Maybe one day they will bundle Mono's Cecil or Mono's embeddable C# compiler.

Monovation: Assembly Injection into Live Processes

People are loving the C# Interactive Shell.

In the past people have embedded languages like Boo into their applications (MonoDevelop and Banshee for example both have options to embed Boo and a shell).

Zoltan designed a new system for Mono that allows developers to inject and execute code into running Mono applications. Paolo provided significant feedback and design guidelines for the code to be incorporated into Mono.

Thanks to both Zoltan and Paolo this functionality is now available through the Mono.Management assembly: you provide an executable and a PID, and the executable is injected and its Main method executed on the target process:

	using Mono.Attach;

	[..]

	// Get a handle to a running Mono process.
	VirtualMachine vm = new VirtualMachine (pid);

	// Load hello.exe into the target process, and run it
	vm.Attach ("/tmp/hello.exe", "");
	

You can use this for billions of things of course. You could patch running programs, you could attach logging software to a running program, or you could inject a C# evaluator into a live application and examine its state, or tweak it live.

Zoltan came up with a really cool extension to the the csharp command (this is the command-line C# Interactive Shell). The csharp command now takes an --attach command line argument and a PID.

The csharp shell can now use the Mono.Attach.VirtualMachine to injects itself into the remote process and then the client and server communicate through a couple of sockets.

For example, this is the sample that Zoltan used to pitch his idea for supporting attaching to the virtual machine. With the following you can pause a live Banshee process:

$ ps aux | grep banshee
miguel   12359 17.0  2.7 141372 55708 pts/5    Sl+  14:30   0:02 banshee-1 /usr/lib64/banshee-1/Nereid.exe
$ csharp --attach 12359
csharp> using Banshee.ServiceStack;
csharp> using Banshee.Gui;
csharp> var s = ServiceManager.Get ();
csharp> s.PlaybackActions ["PlayPauseAction"].Activate ();
	

All of the code is now on SVN, you need both the mono and mcs modules from trunk.

A GUI Shell

The above commands are a tiny bit risky and are also limited to the shell.

The above commands will execute on a separate thread from the application, and any commands that you execute would be executed on a separate thread which could corrupt the state of your application.

So this weekend, I wrote a tool that integrates with Gtk# applications, its called gsharp and you can find it in the mono-tools module (it borrows some code from Banshee).

gsharp is a Gtk# version of the csharp command. What is important is that it also supports injection of the code on other programs, but makes sure that all the code executes in the Gtk# thread, by issuing all of its commands from the Gtk# idle handler. This means that it is safe to use gsharp with your Gtk# applications.

GUI version of the tool.

Here you can select which project you want to inject the GUI into:

Needs some work, show process names.

This version shows the gsharp tool attached to F-Spot, as I am not very familiar with the codebase, I can not do very much with it:

We will need to implement one of these as well for Windows.Forms applications. This should luckily be easy to do as most of the smarts live in the Mono.CSharp assembly (our embeddable compiler).

Security of Agents

A couple of weeks ago, I asked for people to weigh in on a security concern for temporary files. This was for Zoltan's attach functionality.

The code is now implemented and I would love if security experts could do a source code audit of our implementation. And validate whether our assumptions are safe.

Here is the source code.

Public Service Announcement

In C# the defaut access level for members in classes and structs is "private".

There is no need to pepper the source code with "private" everywhere. It only introduces noise and makes your code more difficult to read.

Love World Domination!

Chris Anderson is wearing a Mono T-Shirt on this PDC interview and IronRuby.com is hosted on DekiWiki, a Mono-powered Wiki site.

This is clearly awesome.

In other news, the awesome hackers at Imendio have officially released Gtk+ for OSX packages.

Stream.CopyStream

Funkists, why does System.IO.Stream not have a CopyStream method, it seems like everyone ends up implementing this.

Discuss.

reCaptcha.NET

Kudos to Ben and th