Mono goes Accessible!

Brad Taylor has announced the first release of the Mono Accessibility stack:

UI Automation provides programmatic access to most user interface (UI) elements on the desktop, enabling assistive technology products such as screen readers to provide information about the UI to end users and to manipulate the UI by means other than standard input. UI Automation also allows automated test scripts to interact with the UI.

Mono's Accessibility Framework is an implementation of UI Automation. The same API that is available for WPF and the framework is used by Silverlight and Windows.Forms.

Client Code: The initial launch of Mono Accessibility adds accessibility support to applications built with Windows.Forms to be accessible.

Backend Code: The code has a bridge that talks to the existing ATK framework on Linux.

In the future the Mono Accessibility framework will be used in our own Moonlight 2.0.

Check the release notes, install from source or use OpenSUSE's 1-click install.

Mono Goes to Android

Koushik Dutta got Mono running on the Android-based G1 phone.

He posted a video of the phone compiling "Hello World" (he points out that it is slower due to Mono running from the SD card):

He also posted some performance and memory usage comparisons between Dalvik, Mono and Java/ARM. Short story: Mono does great!

There are some caveats on running Mono on the G1, see the comments on this post. Still, these are encouraging news.

F-Spot Extensions

Today I upgraded my F-Spot, I had not upgraded it since before the PDC.

It is now has a Picassa-like toolbar on the left:

And third-party extensions are starting to come out:

Mono brings Plastic's Windows.Forms UI to Linux and MacOS

Pablo has sent me these two screencasts of their Plastic SCM product running on Linux and MacOS using Mono 2.0's Windows.Forms support:

Plastic on Linux, the GUI toolkit is Windows.Forms with custom widgets and a nice color scheme.

Plastic has a nice graphical diff tool:

A preview of Plastic on MacOS X.

Cute graphs

More screenshots here.

Plastic is one of the finalists for this year's Jolt Awards.

Evolution wish-list: IMAP server built into the client

For a while I wanted to be able to get programmatic access to my email store in Evolution, just like it is possible to have programmatic access to the contacts and calendar through the Evolution Data Server.

The advantages of using IMAP as the protocol to talk to Evolution are simple: I can use any existing IMAP client library, or any other IMAP client to connect to my Evolution store. The protocol is well known, documented and the large ecosystem of IMAP clients makes it a natural feature.

There is also an application that I have in mind for it. I keep all of my email in Evolution, I download all of my email into my local hard disk so I can have all my information with me even when I am disconnected from the net. This means I can always check patches, review comments, discussions even when I am disconnected or with poor network connectivity.

But when I go on vacation, I do not want to bring my laptop or Evolution with me. Instead I end up using internet cafes to read my gmail and all of the other email addresses end up in Novell's server. Novell provides a convenient Web UI that I can use to read my email.

But the problem is that I end up reading emails twice: once in the road with the web UIs, and another time when I get back home and import all my email into Evolution.

By having Evolution expose an IMAP interface, I could use any IMAP client on the road, or ssh into my box and use mutt to read from the same email store that Evolution is keeping track of.

Visiting Microsoft

Joseph, Chris and myself are visiting Microsoft this week to learn more about Silverlight 3.0

If you are in town and have some time to meet to discuss open source, Mono, .NET, the CLI, the DLR or and whatever else you think we might have a fun conversation about, please drop me an email.

Moonlight goes 3D

Argiris Kirtzidi (one of the developers behind Managed OGRE) modified Moonlight to run inside the Ogre3D engine. You can render Moonlight applications or XAML files inside Ogre3D.

The Moonlight Calculator Example.

Your standard XAML tiger.

We are merging his patches to make it simpler for Moonlight to be compiled by Windows users.

Update:For more details about how this was done, and how he modified Cairo to be hardware accelerated check Argiris's post.

Groupwise Calendar to Google Calendar Exporter

I wrote a small tool that exports my Groupwise Calendar to Google Calendar.

This tool only runs on Windows as it is using the Groupwise COM APIs to fetch the calendar data. I would love to have this work on Linux if someone knows how to get to these from Unix.

You will need the Google Calendar assemblies (Google.GData.AccessControl, Google.GData.Calendar, Google.GData.Extenions) and the Groupwise Assemblies (GroupwiseTypeLibrary, GroupWiseCommander) and a text file that contains your passwords (called `passwords').

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Google.GData.Calendar;
using Google.GData.Client;
using Google.GData.Extensions;
using Google.Accounts;
using System.Threading;
using System.IO;

namespace CalendarExporter
{
    class Program
    {
	const string google_email = "YOUR_EMAIL@gmail.com";
	const string groupwise_login = "YOURNAME";
	
        static void Main(string[] args)
        {
            var f = File.OpenRead("passwords");
            var reader = new StreamReader(f);
            var google_passowrd = reader.ReadLine();
            var groupwise_password = reader.ReadLine();

            new Thread(delegate() {
                Thread.Sleep(1000 * 120);
                Console.Error.WriteLine("Timing out");
                Environment.Exit(1);
            }).Start();

            ClientLoginRequest login = new ClientLoginRequest();
            login.AccountType = "GOOGLE";
            login.Email = google_email;
            login.Password = google_password;
            login.Service = "cl";
            login.Source = "YOURNAMECo-CalendarPush-1";

            var token = login.Login();

            CalendarService cs = new CalendarService("YOURNAMECo-CalendarPush-1");
            cs.SetAuthenticationToken(token.Auth);


            CalendarQuery cq = new CalendarQuery();
            cq.Uri = new Uri("http://www.google.com/calendar/feeds/default/owncalendars/full");
            CalendarFeed resultFeed = cs.Query(cq);
            CalendarEntry gw_at_google = null;
            foreach (CalendarEntry entry in resultFeed.Entries) {
                if (entry.Title.Text == "Groupwise Calendar") {
                    entry.Delete();
                    break;
                }
                
            }
            gw_at_google = new CalendarEntry();
            gw_at_google.Title.Text = "Groupwise Calendar";
            gw_at_google.Summary.Text = "This is the syncrhonized calendar at Novell's Groupwise server";
            gw_at_google.TimeZone = "America/New_York";
            gw_at_google.Hidden = false;
            gw_at_google.Color = "#2952a3";
            gw_at_google.Location = new Where("", "", "Boston");
            gw_at_google.Selected = true;
            Uri postUri = new Uri("http://www.google.com/calendar/feeds/default/owncalendars/full");
            CalendarEntry cal = (CalendarEntry)cs.Insert(postUri, gw_at_google);

            string calurl = cal.EditUri.Content;
            int p = calurl.LastIndexOf ('/');
            string code = calurl.Substring (p);

            
            Uri edit_uri = new Uri ("http://www.google.com/calendar/feeds" + code + "/private/full");
            GroupwareTypeLibrary.GWSession2Class gsc = null;
            try
            {
                gsc = new GroupwareTypeLibrary.GWSession2Class();
            }
            catch
            {
                Console.WriteLine("Did not regsvr the file c:\novell\groupwise\gwcma1.dll and is this program x86-only?");
                return;
            }
            var account = gsc.Login(groupwise_login, "", groupwise_password, null, null);
            var path_to_host = account.PathToHost;

            //alendar calendar = new iCalendar();

            
            int count = 0, skipped =0;
            foreach (GroupwareTypeLibrary.Message m in account.Calendar.Messages)
            {
                if (!m.ClassName.StartsWith ("GW.MESSAGE.APPOINTMENT"))
                    continue;

                GroupwareTypeLibrary.Appointment2 app = (GroupwareTypeLibrary.Appointment2) m;

                // Ignore appointments that are older than 15 days.
                if (app.EndDate < DateTime.Now - TimeSpan.FromDays(7)) {
                    skipped++;
                    continue;
                }

                var ee = new EventEntry();
                ee.Title.Text = app.Subject.PlainText;
                ee.Content.Content = app.BodyText.PlainText;
                ee.Locations.Add (new Where () { ValueString = app.Place });
                ee.Times.Add(new When(app.StartDate, app.EndDate));
                ee.EventVisibility = app.Private ?
                    EventEntry.Visibility.PRIVATE : EventEntry.Visibility.PUBLIC;

                cs.Insert (edit_uri, ee);
            }
            
            Console.WriteLine("Done2");
            Environment.Exit(0);
        }
    }
}
	

You will also need the Login.cs which is some sample code that I found on the tubes for doing Google Account authentication.

Moonlight 1.0 Beta 1

We have released the first beta of Moonlight 1.0.

This release supports the Microsoft Media Pack for playing back video and audio files. These are the same video and audio decoders that Microsoft uses in Silverlight 2.0.

Check our Moonlight roadmap for details on upcoming versions.

You can try some of the sites tests that we used to test Moonlight.

Here are some Silverlight 1.0 materials:

• How To Videos on Silverlight 1.0
• Essential Silverlight Training.
• Silverlight 1.0 Unleashed. This is my favorite Silverlight 1.0 book, one of the books that we used to get some of the Moonlight team members up to speed.

You can also read the Silvelright's XAML vocabulary description and its XAML Foundation Specification.

Moonlight's Media Stack

As part of the Moonlight Beta release, I wanted to devote a few blog posts to exploring the features in Moonlight and how we implemented those Silverlight features in Moonlight.

Before I get started on today's topic, we would like to get some feedback from our users to find out which platforms they would like us to support with packages and media codecs. Please fill out our completely platform and media codec survey.

Moonlight 1.0 is an implementation of the Silverlight 1.0 API. It is an entirely self-contained plugin written in C++ and does not really provide any built-in scripting capabilities. All the scripting of an embedded Silverlight component is driven by the browser's Javascript engine. This changes with the 2.0 implementation, but that is a topic of a future post.

The Silverlight/Moonlight Developer View.

One of the most important features of the Silverlight/Moonlight web plugin is to support audio and video playback.

Silverlight takes an interesting approach to video and audio playback. In Silverlight the video can be treated like any other visual component (like rectangles, lines) this means that you can apply a number of affine transformations to the video (flip, rotate, scale, skew), have the video be composed with other elements, add a transparency layer to it or add a clipping path to it.

This is the simplest incarnation of a video player in XAML:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <MediaElement x:Name="ExampleVideo"
		  Source="TDS.wmv"
		  Width="320" Height="240"
		  AutoPlay="true"/>
</Canvas>
	

The result looks like this when invoked with when embedded in a web page (or when using the mopen1 command that I am using to take the screenshots):

The MediaElement has a RenderTransform property that we can use to apply a transformation to it, in this case, we are going to skew the video by 45 degrees:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Source="TDS.wmv" Width="320" Height="240">
	   <MediaElement.RenderTransform>
	     <SkewTransform CenterX="0" CenterY="0" AngleX="45" AngleY="0" />
	   </MediaElement.RenderTransform>
	</MediaElement>
</Canvas>
	

The result looks like this:

But in addition to the above samples, MediaElements can be used as brushes to either fill or stroke other objects.

This means that you can "paint" text with video, or use the same video source to render the information in multiple places on the screen at the same time. You do this by referencing the MediaElement by name as a brush when you paint your text.

This shows how we can fill an ellipse with the video brush:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Opacity="0.0" Source="TDS.wmv" Width="320" Height="240"/>

	<Ellipse Width="320" Height="240" >
	   <Ellipse.Fill>
 	      <VideoBrush SourceName="ExampleVideo"/>
	   </Ellipse.Fill>
	</Ellipse>
</Canvas>

This looks like this:

You can also set the stroke for an ellipse. In the following example we use one video for the stroke, and one video for the fill. I set the stroke width to be 30 to make the video more obvious.

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Opacity="0.0" Source="TDS.wmv" Width="320" Height="240"/>
	<MediaElement x:Name="launch" AutoPlay="true" Opacity="0.0" Source="launch.wmv" Width="320" Height="240"/>

	<Ellipse Width="320" Height="240" StrokeThickness="30">
	   <Ellipse.Fill>
 	      <VideoBrush SourceName="ExampleVideo"/>
	   </Ellipse.Fill>
	   <Ellipse.Stroke>
 	      <VideoBrush SourceName="launch"/>
	   </Ellipse.Stroke>
	</Ellipse>
</Canvas>
	

Notice that in the examples above I have been using AutoPlay="true". Silverlight provides fine control over how the media is played as well as a number of events that you can listen to from Javascript for various events (for example, you get events for buffering, for the media being opened, closed, paused, or when you hit a "marker" on your video stream).

Streaming, Seeking and Playback

Depending on the source that you provide in the MediaElement, Moonlight will determine the way the video will be played back.

The simplest way of hosting a video or audio file is to place the audio or video file in a web server, and then have Moonlight fetch the file by specifying the Source attribute in the MediaElement. You do not need anything else to start serving videos.

Seeking on the standard Web server scenario: When the programmer requests the media to be played (either by calling the Play method on the MediaElement, or because the MediaElement has AutoPlay set to true) Moonlight will start buffering and play the video back.

If the user wants to seek backwards, or forward, Moonlight will automatically take care of this. In the case where the user fast-forwards to a place in the file that has yet to be downloaded, playback will pause until then.

Seeking with an enhanced media server: If your server implements the Windows Media Streaming HTTP extension if the user seeks to a point in the file beyond the data that has been downloaded, it will send a special message to the server to seek. The server will start sending video from the new position. The user will get the playback started immediately without having to wait. The details of the protocol are documented in the MS-WMSP specification. This is enabled by using the "mms://" prefix for your media files instead of using the "http://" prefix.

Notice that even if it says "mms", Silverlight and Moonlight do not actually speak to an MMS server, they merely replace that with "http" and speak http/streaming to the server.

The extension is pretty simple, it is basically a "Pragma" header on the HTTP requests that contains the stream-time=POSITION value. Our client-side implementation is available here.

You can use IIS, or use the mod_streaming to enhance the video experience for your end users.

This basically means that you can stream videos on the cheap, all you need is a Linux box, two wires, and a 2HB pencil.

Adaptive Streaming

Another cool feature of the Adaptive Streaming support in Moonlight is that the server can detect the client throughput, and depending on the bandwidth available, it can send a high bitrate video, or a low bitrate video. This is a server side feature.

This feature was demoed earlier this year at Mix 08:

I am not aware of an adaptive streaming module for Apache.

Supported Media Formats in Moonlight 1.0

Although Moonlight 1.0 exposes the Silverlight 1.0, Moonlight 1.0 ships a 2.0 media stack (minus the DRM pieces). This means that Moonlight ships with support for the media codecs that are part of Silverlight 2.0 and supports adaptive streaming. This is what we are shipping:

Video:

• WMV1: Windows Media Video 7
• WMV2: Windows Media Video 8
• WMV3: Windows Media Video 9
• WMVA: Windows Media Video Advanced Profile, non-VC-1
• WMVC1: Windows Media Video Advanced Profile, VC-1

Audio:

• WMA 7: Windows Media Audio 7
• WMA 8: Windows Media Audio 8
• WMA 9: Windows Media Audio 9
• WMA 10: Windows Media Audio 10
• MP3: ISO/MPEG Layer-3
  • Input: ISO/MPEG Layer-3 data stream
  • Channel configurations: mono, stereo
  • Sampling frequencies: 8, 11.025, 12, 16, 22.05, 24, 32, 44.1, and 48 kHz
  • Bit rates: 8-320 kbps, variable bit rate
  • Limitations: "free format mode" (see ISO/IEC 11172-3, sub clause 2.4.2.3) is not supported.

We also support server-side playlists.

For more information see the Silverlight Audio and Video Overview document on MSDN.

Media Pipeline

When we first prototyped Moonlight we used the ffmpeg media pipeline. A media pipeline looks like this:

Charts by the taste-impaired.

Originally ffmpeg handled everything for us: fetching media, demultiplexing it, decoding it and scaling it.

Since we needed much more control over the entire pipeline, we had to write our own, one that was tightly integrated with Moonlight.

Today if you download Moonlight's source code you can build it with either the ffmpeg codecs or you can let Moonlight fetch the binary Microsoft Media Pack and use Microsoft's codecs on Linux.

Microsoft Media Pack

The Microsoft Media Pack is a binary component that contains the same code that Microsoft is using on their Silverlight product.

The Moonlight packages that we distribute do not actually have any media codecs built into them.

The first time that Moonlight hits a page that contains media, it will ask you whether you want to install the Microsoft Media Pack which contains the codecs for all of the formats listed before.

Today Microsoft offers the media codecs for Linux on x86 and Linux x86-64 platforms. We are looking for your feedback to find out for which other platforms we should ship binary codecs.

Tests

No animals were harmed in the development of the Moonlight Video Stack. To ensure that our pipeline supported all of the features that Microsoft's Silverlight implementation supports we used a number of video compliance test that Microsoft provided us with as part of the joint Microsoft-Novell collaboration.

In addition to Microsoft's own tests, we created our own set of open source tests. All of these tests are available from the moon/test/media module. This includes the videos that are specially encoded with all the possible combinations and features used as well as XAML files and associated javascript.

Mono on PowerPC 64

As part of SUSE 11, Mono needs to run on the PowerPC in 64 bit mode. The effort was bootstrapped with some early work from Andreas Faerber.

It was fun to watch Mark's daily commits progress of the port, the tests referenced here are the basic runtime tests that we use to check for regressions and to get a port up and running, it is a good roadmap for how a port comes to life:

	* mini-ppc64.c, cpu-ppc64.md: Fixed some opcodes.  PPC64
	passes basic.exe now.

	---

	* cpu-ppc64.md: Fixed a few instruction lengths.

	* mini-ppc64.c: Don't emit SETLRET.  Now PPC64 passes
	basic-float.exe.

	---


	* decompose.c: Decompose carry and overflow add on PPC64 like
	on other 64 bit archs.  Don't decompose sub at all on PPC64.

	* mini-ppc64.c, exceptions-ppc64.c, tramp-ppc64.c,
	cpu-ppc64.md: 	Several fixes and new opcodes.  Now PPC64 runs (but doesn't
	pass) basic-long.exe.

	---
	
	* ppc/ppc-codegen.h: Use ppc_load_reg instead of ppc_ld in
	ppc_load_func to fix the 2 bit shift.

	---

	* mini-ppc64.c, mini-ppc64.h, cpu-ppc64.md: Several fixes.
	Now PPC64 passes basic-long.exe.

	---

	* ppc/ppc-codegen.h: Make ppc_is_[u]imm16() work with 64 bit
	values.

	---

	* mini-ppc64.h, cpu-ppc64.md: Fixed caller/callee saved
	floating point regs.  Now PPC64 passes basic-calls.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, exceptions-ppc64.c,
	tramp-ppc64.c, cpu-ppc64.md: Several fixes.  PPC64 now runs objects.exe.

	---

	* mini-ppc64.c, tramp-ppc64.c: Small fixes.  PPC64 now runs
	arrays.exe and basic-math.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, exceptions-ppc64.c,
	cpu-ppc64.md: Several fixes.  PPC64 now runs exceptions.exe and
	devirtualization.exe.

	---

	* mini-ppc64.c: Several fixes.  PPC64 now runs iltests.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, tramp-ppc64.c: Disable generic
	code sharing.  PPC64 now passes generics.exe.

	---

	* basic-long.cs: New test case.

	---

	* mini-ppc64.c, mini-ppc64.h, tramp-ppc64.c, cpu-ppc64.md:
	Several	fixes.  PPC64 now passes most of the runtime regressions.
	
	

Followed by today's tweet:

The bootstrap means that the Mono JIT is actually doing a full build of Mono's compilers and class libraries and can be built on the target platform.

Update: Mark has posted a great picture of Jim Purbrick from Second Life, the man behind Mono running on Second Life.

Unity on Linux, First Screenshots

The first Unity3D on Linux screenshot:

The above program was built on MacOS, the result copied to Linux and then executed using the LinuxPlayer. This is still very basic, the port is yet far from done.

I followed Joachim's advise and added a tiny script to update the cube on the screen. See the video of the cubes in action: ogg and wmv.

Framework Design Guidelines, 2nd Edition

A couple of years ago I wrote an enthusiastic review of Brad Abrams and Krzysztof Cwalina's Framework Design Guidelines, a book that I absolutely love.

The book is a great compendium of best-practices for building software, traps and pitfalls to avoid.

But most importantly, it is the best source to learn the idioms and patterns used in the .NET Frameworks. Learning these idioms will have you writing code like the native C# speakers in no time.

I was incredibly honored when Brad asked me earlier this year to write the foreword for the second edition of the Framework Design Guidelines.

The second edition tracks the evolution of .NET and they apply as well to Mono. For instance, it now contains LINQ design patterns, extension methods patterns and DependencyProperties (used in WPF and Silverlight).

Silverlight Toolkit, now MS-PL

Update: Fixed some links, corrected some text.

Shawn Burke announced the Silverlight Toolkit and it is licensed under the open source MS-PL. The code is available here complete with unit tests (check Ning's blog on the unit testing framework).

With the Silverlight Toolkit they are taking a new approach to shipping new controls in an effort to move swiftly and deliver the controls to people at the right time. Their previous approach was to ship the Toolkit when every component was ready, and completely fleshed out.

Now they will be shipping the Toolkit with controls that might have different levels of quality (and they are clearly flagged in the documentation). Shawn explains the new "Quality Bands" model that they are using in his post.

You can try the components on the web. The charting control can be tried out with the ChartBuilder (check David's blog for details on the ChartBuilder):

The source code for the Toolkit and the Controls is great to learn how to use Silverlight and it is great for people that need to tweak them for their own applications. When it comes to these controls, you no longer need Microsoft to make small changes for you or the small bug fixes that impact your application.

Themes: An interesting control container in Silverlight is the theming control. You wrap your code around this, and it will let you skin your control with XAML and define the animations and interactions with XAML and the Visual State Manager:

Some of these themes reminded me of the Gtk+ themes from 1998. Back in the days of Enlightenment and the "Cheese Pixmap" theme were hot. Mehdi explains how the themes work and Jafar explains the ImplicitStyleManager, the foundation for themes.

Shawn's Talk

Shawn's talk at the PDC was very interesting. I did not get to see it during the conference, but I watched it in the comfort of home (wmv, mp4 and slide deck).

Moonlight Updates

Last week we branched Moonlight for the 1.0 release, full with the licensed Microsoft Codecs and started our release process for Moonlight Beta 1 to be available in the next few days. This release is not yet published on our web site, watch this space.

The Moonlight engine team has now resumed our work on Moonlight 2.0, the version that will track Silverlight 2.0.

In the meantime, while the GUI team was busy completing 1.0, the Mono core team has been working on the security framework for Moonlight, the networking stack (Silverlight allows Socket connections using policy files) and web services (System.ServiceModel, a subset of WCF).

The security system is the trickier and is the one that has received the most attention. We started early on last year in to implementing this, as we knew it would end up burning a lot of cycles to get it right.

Our hero has posted the initial work partition for the upcoming GUI work on Moonlight 2.0.

Moonlight is a blast, and who knows, maybe with our static compilation engine we might be able to deliver Silverlight on the iPhone.

Change.Gov

I wanted to thank everyone that helped get Barack Obama elected. Those that endorsed Obama passionately, those that videocasted, blogged, improved Obama's web site, donated to his campaign, wrote, discussed and voted on Tuesday to get him elected.

Barack does not only represent a change of direction for public policy, he is a truly brilliant candidate.

Some cool links on Barack:

• Larry Lessig: 20 minutes or so on why I am for Barack Obama.
• Tim O'Reilly's Why I Support Barack Obama.
• The Obama I know gives a glimpse of how Obama works.
• The Letter of Endorsement from 76 American Nobel Prize winners. winners.
• For the most cynics among us, you can watch the Chomsky on Obama interview which is as good as an endorsement as you can get from Noam.
• Beautiful pictures from the campaign trail from the Boston Globe.

I was surprised that the Obama campaign already launched their Change.Gov (thanks Nat) web site. You can now see how the team operates in real life, and you can share your story and you can share your vision of where America should go. The blog is here.

The above starts to deliver on the promise he had made during the campaign.

Got a cool collection of pictures about Obama or the reaction to the results? Please post it in the comments.

Inflamatory or misinformed comments will be deleted pronto.

Static Compilation in Mono

Another nice piece of technology that we showed at the PDC was static compilation, the feature behind allowing Mono to run on the iPhone in a fully legit way (no jail-breaking):

Screenshot from the Unity IDE.

Although Mono has supported batch compilation from CIL into native code in the past there was a small amount of code that was still generated dynamically. The intention originally was to help reduce startup and increase code sharing across multiple processes.

Code sharing is important once you have a handful of Mono processes running on the system. Instead of having to JIT the same code once per Mono instance for say "System.Console.WriteLine", the code laid out in essentially a shared object.

Our generated code uses many of the concepts found on the ELF file format to ensure that we can share code and that the code pages are read-only and not written to. This means that methods are invoked through a program linkage table instead of directly (to cope with the shared libraries being loaded at different addresses).

The Extra Mile

Although we are not certified XBox360 developers yet (we have yet to find the right person at Microsoft to talk to) we know from some of our users that have ported Mono to the XBox360 that JITing is not an option on that platform.

The XBox360 seems to have the same security-imposed limitations that the iPhone has, it is not possible for a Just-in-Time compiler to run in the platform as either the license terms or the kernel do not allow writable pages to become executable pages.

During the last few months we developed a static compilation mode for Mono. First we did this for the 1.0 profile, and now we are working on the 2.0 profile (so that we can support static compilation of generics). The work to support the 2.0 profile is reusing Mark's work on generic code sharing, which I found out to be a very nice synergy of efforts internally.

This means that it is now possible compile code from CIL to native code and not even ship the JIT compiler in your final program (saving some precious kilobytes from the final executable).

To do this, you must:

• Use Mono 2.0.1 at least.
• Request that Mono performs a full AOT compilation by using: mono --aot=full program.exe. That will generate your static executable. This executable still needs the runtime for things like garbage collection, threading and other runtime services.
• You then run your executable with Mono: mono --full-aot program.exe
• Optionally: build a new Mono on a separate location that removes the JIT engine by configuring Mono like this: configure --enable-minimal=jit. This will reduce your deployment by a few hundred Ks as the code generation and JIT engines are stripped out.
• Optionally: build a smaller set of libraries by using the Mono Linker (this is the tool that we use for turning Mono's 3.5 APIs into the Silverlight 2.0 APIs).
• Optionally: strip out the CIL code from the assemblies. We still require the assemblies for their rich metadata, but the actual CIL instructions can be safely removed. The new cil-strip tool built on top of Mono.Cecil can further shrink your deployed executables.

Developers interested in trimming down Mono can look into our documentation for more features that can be removed by using the --enable-minimal option.

Of course, once you remove the JIT you will not be able to use any dynamically generated code. This means no Reflection.Emit dynamically and at least for the time being or no IronPython/IronRuby.

John Lam told me at the PDC that they are looking into bringing static compilation for IronPython/IronRuby/DLR back, so this might just be a short-lived limitation.

For those interested in using Mono on the iPhone today the situation is a bit painful right now. You must run Mono on the target system to do the batch compilation and send the data back to assembly it on the host before you send the code back to the iPhone to run.

If you are wondering how did the demo go so smoothly at the PDC, the reason is that I was using Unity. Unity modified their local copy of Mono to be hardwired to do cross compilation to that exact platform. A more general purpose solution is needed to allow arbitrary platform-to-platform cross compilation, and we hope that this will be available in the future.

If you must quench your thirst for C# on the iPhone today your best choice is to use Unity's product and start building games instead of the enterprise application you always dreamed of.

From the Unity's Video Sample

If your boss demands that line of application running on the iPhone, you have a perfect excuse to learn the Unity gaming APIs and deliver the most glorious multi-touch, 3D-transformed line of business application to ever grace this world full with spinning AI for your "Sort By Customer Last Name" button.

C# 4.0: var, object and dynamic

Anders presentation on C# 4 was as usual great to listen to. He continues to evolve the language with solid steps, and the presentation was quite fun.

You can watch his presentation or just read the slide deck.

With C# 4 the new "dynamic" keyword has been introduced to flag a variable as a dynamic variable.

This is slightly different than var and object, the differences are as follows:

• "object x" is a shorthand for "System.Object x". This declares the variable x to have the type System.Object, this is strongly typed. And since C# provides autoboxing, you can assign anything you want to this variable.
• "var x = E" declares a variable x to be of the type of the expression E. The E is required, not optional. This is a strongly typed declaration, and you can only assign values whose type is typeof(E) to it.
• "dynamic x" declares the variable x to have dynamic semantics. This means that the C# compiler will generate code that will allow dynamic invocations on x. The actual meaning of "x.M" is deferred until runtime and will depend on the semantics of the IDynamicObject implementation.

Mono's SIMD Support: Making Mono safe for Gaming

This week at the Microsoft PDC we introduced a new feature in the Mono virtual machine that we have been working on quietly and will appear in our upcoming Mono 2.2 release (due in early December).

I believe we are the first VM for managed code that provides an object-oriented API to the underlying CPU SIMD instructions.

In short, this means that developers will be able to use the types in the Mono.Simd library and have those mapped directly to efficient vector operations on the hardware that supports it.

With Mono.Simd, the core of a vector operations like updating the coordinates on an existing vector like the following example will go from 40-60 CPU instructions into 4 or so SSE instructions.

Vector4f Move (Vector4f [] pos, ref Vector4f delta)
{
	for (int i = 0; i < pos.Length; i++)
		pos [i] += delta;
}
	

Which in C# turns out to be a call into the method Vector4f.operator + (Vector4f a, Vector4f b) that is implemented like this:

Vector4f static operator + (Vector3f a, Vector3f b)
{
	return new Vector4f (a.x+b.x, a.y+b.y, a.z+b.z, a.w+b.w);
}
	

The core of the operation is inlined in the `Move' method and it looks like this:

movups (%eax),%xmm0
movups (%edi),%xmm1
addps  %xmm1,%xmm0
movups %xmm0,(%eax)
	

You can see the details on the slides that I used at the PDC and look at the changes in the generated assembly, they are very large.

Ideally, once we tune the API based on our user feedback and contributions, it should be brought to ECMA for standardization. Hopefully we can get Microsoft to implement the SIMD support as well so all .NET developers have access to this.

Making Managed Code Viable for Gaming

Many developers have to resort to C++ or assembly language because managed languages did not provide the performance they needed. We believe that we can bring the productivity gains of managed languages to developers that seek high performance applications:

But even if you want to keep using your hand-tuned C++ game engine, the SIMD extensions will improve the performance of your scripting code. You can accelerate your ray casting operations by doing all the work in the managed world instead of paying for a costly managed to unmanaged transition and back.

You can avoid moving plenty of code from C# into C++ with this new functionality.

Some SIMD Background

Most modern CPUs contain special instructions that are able to perform arithmetic operations on multiple values at once. For example it is possible to add two 4-float vectors in one pass, or perform these arithmetic operations on 16-bytes at a time.

These are usually referred to as SIMD instructions and started showing up a few years ago in CPUs. On x86-class machines these new instructions were part of MMX, 3DNow or the SSEx extensions, on PowerPC these are called Altivec.

CPU manufacturers have been evolving the extensions, and newer versions always include more functionality and expand on the previous generations.

On x86 processors these instructions use a new register bank (the XMM registers) and can be configured to work on 16 bytes at a time using a number of possible combinations:

• byte-level operations on 16 elements.
• short-level operations on 8 elements.
• single precision or integer-level operations on 4 elements.
• double precision or long-integer operations on 2 elements.

The byte level operations are useful for example when doing image composition, scaling or format conversions. The floating point operations are useful for 3D math or physics simulations (useful for example when building video games).

Typically developers write the code in assembly language to take advantage of this feature, or they use compiler-specific intrinsic operations that map to these underlying instructions.

The Idea

Unlike native code generated by a compiler, Common Intermediate Language (CIL) or Java class files contain enough semantic information from the original language that it is very easy to build tools to compute code metrics (with tools like NDepend), find bugs in the code (with tools like Gendarme or FxCop, recreate the original program flow-analysis with libraries like Cecil.FlowAnalysis or even decompile the code and get back something relatively close to the original source code.

With this rich information, virtual machines can tune code when it is just-in-time compiled on a target system by tuning the code to best run on a particular system or recompiling the code on demand.

We had proposed in the past mechanisms to improve code performance of specific code patterns or languages like Lisp by creating special helper classes that are intimately linked with the runtime.

As Mono continues to be used as a high-performance scripting engine for games we were wondering how we could better serve our gaming users.

During the Game Developer Conference early this year, we had a chance to meet with Realtime Worlds which is using the Mono as their foundation for their new work and we wanted to understand how we could help them be more effective.

One of the issues that came up was the performance of Vector operations and how this could be optimized. We discussed with them the possibility of providing an object-oriented API that would map directly to the SIMD hardware available on modern computers. Realtime Worlds shared with us their needs in this space, and we promised that we would look into this.

The Development

Our initial discussion with Realtime Worlds was in May, and at the time we were working both towards Mono 2.0 and also on a new code generation engine that would improve Mono's performance.

The JIT engine that shipped with Mono 2.0 was not a great place to start adding SIMD support, so we decided to postpone this work until we switched Mono to the Linear IL engine.

Rodrigo started work on a proof-of-concept implementation for SIMD and after a weekend he managed to get the basics in place and got a simple demo working.

Beyond the proof of concept, there was a lingering question: were the benefits of Vector operations going to be noticeably faster than the regular code? We were afraid that the register spill/reload would eclipse the benefits of using the SIMD instructions or that our assumptions had been wrong.

Over the next few weeks the rest of the team worked with Rodrigo to turn the prototype into something that could be both integrated into Mono and would execute efficiently (Zoltan, Paolo and Mark).

For example, with Mono 2.2 we will now align the stack conveniently to a 16-byte boundary to improve performance for stack-allocated Mono.SIMD structures.

So far the reception from developers building games has been very positive.

Although today we only support x86 up to SSE3 and some SSE4, we will be expanding both the API and the reach of of our SIMD mapping based on our users feedback. For example, on other architectures we will map the operations to their own SIMD instructions.

The API

The API lives in the Mono.Simd assembly and is available today from our SVN Repository (browse the API or get a tarball). You can also check our Mono.Simd documentation.

This assembly can be used in Mono or .NET and contains the following hardware accelerated types (as of today):

Mono.Simd.Vector16b  - 16 unsigned bytes
Mono.Simd.Vector16sb - 16 signed bytes
Mono.Simd.Vector2d   - 2 doubles
Mono.Simd.Vector2l   - 2 signed 64-bit longs
Mono.Simd.Vector2ul  - 2 unsigned 64-bit longs
Mono.Simd.Vector4f   - 4 floats
Mono.Simd.Vector4i   - 4 signed 32-bit ints
Mono.Simd.Vector4ui  - 4 unsigned 32-bit ints
Mono.Simd.Vector8s   - 8 signed 16-bit shorts
Mono.Simd.Vector8us  - 8 unsigned 16-bit shorts
	

The above are structs that occupy 16 bytes each, very similar to equivalent types found on libraries like OpenTK.

Our library provides C# fallbacks for all of the accelerated instructions. This means that if your code runs on a machine that does not provide any SIMD support, or one of the operations that you are using is not supported in your machine, the code will continue to work correctly.

This also means that you can use the Mono.Simd API with Microsoft's .NET on Windows to prototype and develop your code, and then run it at full speed using Mono.

With every new generation of SIMD instructions, new features are supported. To provide a seamless experience, you can always use the same API and Mono will automatically fallback to software implementations if the target processor does not support the instructions.

For the sake of documentation and to allow developers to detect at runtime if a particular method is hardware accelerated developers can use the Mono.Simd.SimdRuntime.IsMethodAccelerated method or look at the [Acceleration] atribute on the methods to identify if a specific method is hardware accelerated.

The Speed Tests

When we were measuring the performance improvement of the SIMD extensions we wrote our own home-grown tests and they showed some nice improvements. But I wanted to implement a real game workload and compare it to the non-accelerated case.

I picked a C++ implementation and did a straight-forward port to Mono.Simd without optimizing anything to compare Simd vs Simd. The result was surprising, as it was even faster than the C++ version:

Based on the C++ code from F# for Game Development

The source code for the above tests is available here.

I use the C++ version just because it peeked my curiosity. If you use compiler-specific features in C++ to use SIMD instructions you will likely improve the C++ performance (please post the updated version and numbers if you do).

I would love to see whether Johann Deneux from the F# for Game Development Blog could evaluate the performance of Mono.Simd in his scenarios.

If you are curious and want to look at the assembly code generated with or without the SIMD optimizations, you want to call Mono's runtime with the -v -v flags (yes, twice) and use -O=simd and -O=-simd to enable or disable it.

Presentation

You can watch the presentation to get some color into the above discussion or check it in the Silverlight player, Get it as PDF, or PPTX.

Mono and .NET talk at PDC video.

The PDC 2008 was a blast. It was a privilege to be able to present Mono to all of these developers.

Joseph Hill helped me prepare my presentation for the PDC. Our goal was to explore how Mono could help .NET developers, but we did not want to go down a laundry list of features, or a list of APIs, or rehash what the advanced audience at the PDC already knew.

player, pdf, pptx

The idea was to pick a couple of interesting innovations from Mono and try stitch a story together. I discussed our embeddable C# compiler (which we need to start calling "Compiler Service"), some applications of Mono in gaming, our recent SIMD extensions and using Mono on the iPhone.

As for me, I am catching up on all the sessions I missed this weekend. All of the videos and slide decks are available for free from the Microsoft PDC site, and republished in Channel9.

In the next few days I will blog in more detail about each topic.

Interactive GUI Shell

This week at the Microsoft PDC I showed gsharp, our GUI repl for the C# 3.0 language, a tool that I had previously talked about.

Before the PDC we copied an idea from Owen's great reinteract shell where we provide our REPL with a mechanism to turn objects into Gtk.Widgets which we then insert.

Out of the box we support System.Drawing.Bitmap images, we turn those into Gtk Widgets that then we render:

I also added a toy Plot command that can take a number of lambdas that return floats to do some cute plots. The plots are just rendered into a System.Drawing.Bitmap so they get painted on the screen automatically:

But you can add your own handlers for any data types you want, all you have to do is call RegisterTransformHandler with a function that can return a Gtk.Widget based on an input object, or null if it does not know how to render it.

The implementation to render images is very simple, this is the implementation:

using System;

public class MyRenderHelpers {
	public static object RenderBitmaps (object o)
	{
		System.Drawing.Bitmap bitmap = o as System.Drawing.Bitmap;
		if (bitmap == null)
			return null;

		return new BitmapWidget (bitmap);
	}
}
	

You can put your own library of helper methods in a compiled assembly in ~/.config/gsharp, and then register all of your types from a file ending with the extension .cs in ~/.config/gsharp:

RegisterTransformHandler (MyRenderHelpers.RenderBitmaps);
	

And you are done.

The above could be used for example to create all kinds of information visualizers for the GUI REPL. I would love to see a LINQ query navigator, similar to the one in LinqPad.

Update: A one line change that brings gsharp into the new millenium by rendering `true' and `false' with icons instead of text:

In LA for the Microsoft PDC

After a great week in Copenhagen with the Unity community I spent 14 hours on a high-tech tin can flying to LA for the Microsoft PDC conference.

Mono Talk

I am doing a talk on Wednesday at 4:45 in room 515B.

Unlike previous talks that I have done about Mono, this is an advanced talk and will skip over all the generalities and go straight to Mono CLI/C# features and innovations.

I decided against talking about Moonlight or APIs, as information about can be better learned elsewhere.

Speculation

There has been enough leaked information that we know some bits about C# 4. Some guess it includes dynamic support, other that it will be more F#-like and add immutability, others that it will introduce some Alef-like threading capabilities.

Then there is talk about .NET 4, and I just have no clue what they will announce.

So what do you think they are announcing this week?

Speculate away in the comments.

Live Blogging from Unite

I am live blogging from Unite, the Unity3D conference, one of the most fun users of Mono.

Announcements

Next UnityEditor will run on Windows, and its rewritten in C# (it originated on MacOS, and is now moving to Windows).

Unity as of today ships for building games on the iPhone. These are fully legit binaries, no need to crack your iPhone, they are using Mono's batch compilation to generate static binaries with no JITing involved (per Apple licensing requirements).

Side Note

Since I am a Linuxista, you might be wondering why I am so excited about Unity. Of course I am excited because they use Mono, but I am also excited because Novell is working with Unity to bring this to Linux:

erandi$ uname -a
Linux erandi 2.6.25.16-0.1-pae #1 SMP 2008-08-21 00:34:25 +0200 i686 i686 i386 GNU/Linux
erandi$ ls -l unity-linux/build/LinuxPlayer 
-rwxr-xr-x 1 miguel users 45735629 2008-10-12 00:37 unity-linux/build/LinuxPlayer*
	

We do not have a timeline yet, please do not spam the Unity guys with requests, stay tuned to this blog for updates.

FusionFall pre-mortem

12:17am Joachim explains their strategy with the web plugin and how they will cope with multiple versions of it as time goes by.

12:06am Cartoon Network will drive a lot of the plugin penetration.

11:50am Joachim is showing profiling information (particle, physics, the top scripts, time taken per frame).

The game went from 2gigs to 300megs by using some interesting compressions fo meshes and animations.

All of the features that were added for FusionFall are being folded back into the future Unity 2.5.

Pics: I wish I had brought my USB cable to post pictures from the camera. I will try to spice up this post tonight with the photos.

11:45am Runtime World Streaming: scenes are dynamically loaded and unloaded base on the player position in the world, the world is made up of a 16x16 grid. Scene loading happens when a player approaches a boundary in the world.

11:40am The Unity guys are talkng about the challenge of converting the assets from Gamebryo to Unity, the volume was large (25k files of Gamebryo data, which was constantly changing and growing).

They added support to Unity to interoperate with the Gamebryo and Cartoon Network data and wrote plenty of C# importing scripts and tools.

11:30am FusionFall talk by Joachim Ante.. FusionFall is a project that was done for the Cartoon Network.

It is an MMO with platform game elements.

It has a huge streaming world, so there are no pauses as you navigate the world. The game is targetted at kids (8-14).

The game was produced by Cartoon Network and developed by Grigon Entertainment, a team of 40 developers, 10 of them programmers, and has been under development for 3 years. Originally this was Windows-only standalone executable, developed with Gamebryo.

The cycle at some point included the prototyping done in Unity, shipping the executable to engineering, and engineering reimplementing in Gamebryo. They realized that they could turn the prototype as the actual client and that they could just communicate to their backend server. This allowed them to switch the entire game from Gamebryo to Unity.

Originally the standalone game was 2gb in size (lots of music, voice overs, terrains). This was a problem for kids, since they are not going to wait for 2 gigs to download, this was a big barrier to entry.

Unity's web based distribution and the strong world-streaming features were a good match for this project. This allowed Cartoon Network to give a great experience.

The entire MMO was ported in four months by a team Grigon developers and four Unity engineers working with them. They were on a very tight schedule. Four developers at Grigo ported the game client from C++ to C#.

Keynote

10:52am Interesting overview of the challenges of the game industry. Where does the game industry go next? A discussion on integrarting games with the web and delivering games as services.

On one end there is the flashy stuff, on the other end lots of talk about the enterprise components of gaming. I had no idea.

Between the low-barrier to entry and the high-barrier to entry markets, there is a large space for gaming and where 3D games on the browser will make a difference.

Phil sees Unity as an agent that will help transform the game industry.

10:31am Phil Harrison president of Atari is now on stage, "First time I have done a presentation in a Planetarium". Atari has no commercial relationship with Unity or investing-wise. He is here because "I wanted to be here, what David and his team are doing is transformational for the industry. I had an Eureka moment early this year, I had just joined Atari, and someone told me `check out Unity3d.com', I had heard about it, but never used it. Using the Web player demo was eye opening for me. [...] This is something similar to what I saw at Sony in 1993, when we first got the dinosaur demo on the PS1 [...] The island demo I believe is a game changer in this industry".

"I have become an unpaid evangelist of Unity".

Phil is now going to talk about the game industry, wont blog that.

10:29am Introspection, why we want to do this? Goals: we want people to build games for the web, the iPhone, the Wii, and for everything else.

10:29am Announcing Indie version, 199 dollars, but has some watermark/splashscreen at the beginning.

10:26am Windows Vista logo on screen.

"This is true, I have to admit it", they are demostrating the new Unity3D IDE on Unix. The same Unity3D tool but now running on Windows.

"We are going completely cross platform", every script that you write will run on both platforms (Woohoo! Go CIL! Go!).

10:21am Nicholas is going to talk about "Secret Labs". He talks about "Jus t Press Play", "Buliding for multiple targets" and the script property editor.

They wanted to improved upon th eUnity editor.

They rewrote the Unity editor from zero. Created entirely on top of the Unity APIs themselves - EditorWindows, unityGUI, GameObjects and Components. Unity is built on Unity now. "It is way faster than cocoa".

He is showing Unity 2.5, looks like Unity 2.1; They now use tabs and the various windows can be dragged around, very much like MonoDevelop. He is showing the editor by dragging a lightpost into a paradise island and showing the new GUI tools like snapping, rotation and UIs that are closer to Maya's tools.

He shows "Snap to surface" so you can easily position stuff on the 3D terrain. People like it.

The UI is a lot easier to use. "We have been focused on the tiny details, but we are not shipping this yet". Everything in the IDE can now be scripted.

He shows how a few lines of code a developer can attach a camera view when clicking on a property.

130 new API entry points, Unity developers can now do everything that the Unity3D GUI can do.

10:20am David talks about the pricing; Two pricing: cheap and expensive.

10:13am Joachim Ante is introducing Unity for the iPhone. "With Unity we have always focused on iteration time". He goes into some of the technical details, "With all the new input mechanisms for theiphone, how do we provide a quick feedback system, we wanted to keep the experience of develop, hit run under a second".

They provide a "Unity remote" that runs on the iPhone, you use the iPhone to control the game, but the debugging actually happes on the PC. So you hit "run" and there is no wait at all.

Joachim is explaining how they are using AOT to run native code on the iPhone without having any JIT on the iPhone.

Joachim shows "Build and Run" that does the cross compiler and sends the code to the iPhone. It takes about 10 seconds and he now has the game running on the iPhone and shows how both Javascript and C# code running on the system.

10:10am Some stats: sold out event; 180 attendees; community doubled in the last year (2x employees, 2x posts, 2x users);

Last year they released Unity 2.0.

Unity 2.1 was released, should have been 3.0, but they did not charge more.

They announced Unity for Wii.

Big games using Unity: Sony, Disney. Virtual worlds like Hangout.Net is out (very pretty!) A new online community was created (Blurst). SeriousGames released a new title "Global Conflicts" Latin America.

Hard to keep up with the list of users.

10:04am The Unity founders Joachim, David and Nicholas are on stage to start Unite'08.

They moved us from the room downstairs to the iMax auditorium whihc is packed with game guys. During the moments of panic that ensued after they moved us from the downstairs room into the switched rooms, I ended up in the front row, which in retrospect was a big mistake, you cant take pictures with the standard lens of this massive screen.

Registration

9:48am Waiting for the keynote at the Unite conference. There are about two hundred people packed in the Tycho Planetarium. The Blurst guys are all wearing matching shirts and have taken over the first line of the auditorium, From here I can see a them debugging a Sonic-like game that he is prototyping. Then he switched to some game that has an angry minotaur breaking dishes in a museum.

The minotaur looks angry and just scored 3,000 points of some kind.

Everyone in this conference seems to be using a Mac, and I could swear this computer is the only Linux machine in the audience.

Pre-Conference

Last night I had dinner at the Unity headquarters and got a chance to meet some of the Unity hackers and users before the conference started.

Gained a deeper insight into what we can do to improve Mono's VM for games. Lots of good ideas.

Phil Harrison brought up "the debugger" issue ;-).

Hopefully Rasmus from CellDotNet will show up for the Unity Mingle tonight.

Mono 2.0 OSX Installer Ready

We released an updated installer for Mono 2.0 on MacOS X.

This release got delayed because we wanted to upgrade our bundled Gtk# stack to contain the latest release of Imendio's Gtk+ for MacOS X.

Banshee coming to an OSX near you this week.

Mono OSX Survey!

We are trying to understand how we can improve Mono on the OSX space. Help us figure this out by filling out our Mono on OSX Survey.

Relocatable Applications

If you have followed our Guidelines for Application Deployment your software should be easy to be packaged and distributed for MacOS X as a relocatable application.

Eoin Hennessy worked on integrating Banshee into the OS, and packaging it into a bundle that runs out of the box on MacOS. The following are some screenshots from Aaron's box:

The Banshee open source media player.

Sandy has ported Tomboy and Tasque to MacOS and Windows and provided installers for both.

Tomboy integrates into the dock and menus.

Aaron Bockover from the Novell Desktop Team has promised that Eoin's work will be part of Friday's Banshee release. From this point on, Banshee will be released both for Linux and MacOS X at the same time.

Maybe F-Spot is not too far behind?

The Small Print

• We downgraded the bundled Gtk+ from 2.15 to 2.14.3, as 2.15 was a development version and 2.14 is the officially supported Gtk+.
  This means that applications that linked directly against Gtk+ 2.15 from Mono 1.9 will fail to run. Please re-link those binaries.
• We removed MonoDevelop from this distribution, so our package only contains the Mono SDK and Mono runtime.
  A MonoDevelop installation package will come later, we apologize for this delay.
  On the upside: now that the distribution is split, we will be doing MonoDevelop Beta 2 previews as DMGs after the PDC.

Help us improve Mono on OSX by completing the Mono on OSX Survey and providing comments at the end.

Parallel Programming

As much as I personally dislike the use of threads for user code, multi-cores systems are here to stay. They are becoming increasingly popular (most laptops now ship with dual core systems, home computers ship with 3 cpus and gaming consoles ship with multiple general purpose cpus as well).

Developers will need new frameworks for developing software that is ready to take advantage of multiple CPUs. But most importantly they will need to learn the traps and pitfalls of writing parallel/threaded code.

Here are two fantastic articles on MSDN that cover these topics:

• Design Considerations For Parallel Programming by David Callahan.
• Solving 11 Likely Problems In Your Multithreaded Code by the excellent blogger and hacker Joe Duffy.

J�r�mie Laval worked on an ParallelFX implementation for Mono over the summer as part of the Google Summer of Code.

The implementation currently lives on the student repository at code.google.com. I can not wait for the API to be stabilized so we can move it into the main Mono distribution.

Going to Copenhagen

Next week I will be in Copenhagen for Unity3D's Unite conference.

Unity3D is one of the most fun users of Mono as they create IDEs for Game Developers and they are driving the adoption of Mono, C#, Boo and their own UnityScript in the gaming space.

As a newcomer into this industry, there are various sessions from actual Unity user on how they have built their games from start to finish. Other sessions include details on publishing, production (ArtPlant), Physics (Flashbang), Shader Programming (Unity), developing on the iPhone (Unity), a post-morterm on FusionFall's work for Cartoon Network and the hands-on lab.

Some cool stuff from the agenda includes a keynote participation from Atari's President.

If you want to meet up, drop me an email. I will likely be going to the Unity Mingle events at night and departing early on Friday to fly to the Microsoft PDC in LA.

Alan "BitSharp" McGovern Joins Novell

Alan McGovern, who created BitSharp during a Google Summer of Code for Mono has joined the Moonlight team at Novell.

Imagine the possibilities! Bittorrent clients, servers, trackers all running from Silverlight 2.0 Web Applets!

Discuss.

Mono 2.2 Feature: Mono.Options

In the upcoming Mono 2.2 release we will be distributing Jonathan Pryor's most excellent Mono.Options library.

Mono.Options is a beautiful command line parsing library. It is small, succinct, a joy to use, easy and powerful, all in one.

It is one of those libraries that does more with less. Something that every programmer aspires to write, but that we seldom achieve.

It has also struck a good balance for Unix and Windows developers as options can be used on both systems, and map well to practices on both systems. It took a long time to get the right "blend" of parsing, but I think Jonathan has achieved it.

Consider this example:

using System;
using System.Collections.Generic;
using NDesk.Options;

class Test {
    static int verbosity;

    public static void Main (string[] args)
    {
        bool show_help = false;
        List names = new List ();
        int repeat = 1;

        var p = new OptionSet () {
            { "n|name=", "the name of someone to greet.",
              v => names.Add (v) },
            { "r|repeat=", "the number of times to repeat the greeting.",
              (int v) => repeat = v },
            { "v", "increase debug message verbosity",
              v => { if (v != null) ++verbosity; } },
            { "h|help",  "show this message and exit", 
              v => show_help = v != null },
        };

        List extra;
        try {
            extra = p.Parse (args);
        }
        catch (OptionException e) {
            Console.Write ("greet: ");
            Console.WriteLine (e.Message);
            Console.WriteLine ("Try `greet --help' for more information.");
            return;
        }

        if (show_help) {
            ShowHelp (p);
            return;
        }

        string message;
        if (extra.Count > 0) {
            message = string.Join (" ", extra.ToArray ());
            Debug ("Using new message: {0}", message);
        }
        else {
            message = "Hello {0}!";
            Debug ("Using default message: {0}", message);
        }

        foreach (string name in names) {
            for (int i = 0; i < repeat; ++i)
                Console.WriteLine (message, name);
        }
    }

    static void ShowHelp (OptionSet p)
    {
        Console.WriteLine ("Usage: greet [OPTIONS]+ message");
        Console.WriteLine ("Greet a list of individuals with an optional message.");
        Console.WriteLine ("If no message is specified, a generic greeting is used.");
        Console.WriteLine ();
        Console.WriteLine ("Options:");
        p.WriteOptionDescriptions (Console.Out);
    }

    static void Debug (string format, params object[] args)
    {
        if (verbosity > 0) {
            Console.Write ("# ");
            Console.WriteLine (format, args);
        }
    }
}
	

And here is an example of its use:

$ mono greet.exe --help
Usage: greet [OPTIONS]+ message
Greet a list of individuals with an optional message.
If no message is specified, a generic greeting is used.

Options:
  -n, --name=VALUE           the name of someone to greet.
  -r, --repeat=VALUE         the number of times to repeat the greeting.
  -v                         increase debug message verbosity
  -h, --help                 show this message and exit

$ mono greet.exe -v- -n A -name=B --name=C /name D -nE
Hello A!
Hello B!
Hello C!
Hello D!
Hello E!

$ mono greet.exe -v -n E custom greeting for: {0}
# Using new message: custom greeting for: {0}
custom greeting for: E

$ mono greet.exe -r 3 -n A
Hello A!
Hello A!
Hello A!

$ mono greet.exe -r not-an-int
greet: Could not convert string `not-an-int' to type Int32 for option `-r'.
Try `greet --help' for more information.

	

He has also documented it thoroughly.

Where possible (new tools being written, or tools that have a similar command line structure that is compatible) we will be switching to this command line parsing library.

The library is small, so developers should include a copy of the source code in their application, this is how you should include it in your makefiles:

Options.cs:
	cp `pkg-config --variable=Sources mono-options` .

	

Then your code can just include a reference to it.

MonoDevelop gets VI bindings

I grew up mostly with Turbo Pascal as my development environment. When I started to write C code in DOS, I used Turbo C briefly but for some reason I switched to the BRIEF text editor for a while.

Around 1989 my friend Max Mendizabal who used nothing but Epsilon told me "Unix is the future, if you learn Epsilon, you will be ready to switch to Emacs when the time comes".

Prophetic words.

When I eventually switched to Unix in 1992, having learned Epsilon was useful, but Emacs was too slow for quick edits. I still used Emacs for programming, but for quickly making changes to a file, I ended up learning vi.

When computers got faster, I tried to switch to Emacs for all my editing tasks, but my brain had been hardwired. I even added "alias vi=emacs" to by shell, and I would find myself typing subconsciously "/usr/bin/vi".

To this day, I use both editors interchangeably.

In any case, the above story was just an excuse to introduce VI Mode for MonoDevelop.

Mono 2.0 is out!

Today we released Mono 2.0 to the world. You can download sources and binaries from our download page. And our official release notes are up as well. This of course would not be possible without the open source contributors that worked tirelessly on Mono sending patches, fixing bugs, helping the community, answering questions, creating test cases and supporting us all these years.

Mono 2.0 is both a runtime for application and a kit for developers for writing applications with C# and other ECMA CLI languages for a wide spectrum of uses.

Big thanks go to the fantastic Mono team at Novell that has kept the excitement and the pace over all these years (we started in 2001), the large contributions from Mainsoft, Unity3D and our users that pushed us to fix bugs, implement new features and tune Mono. Also, we very much appreciate the work of the ECMA 334 and 335 committee members that worked on the CLI and C# specifications and everyone at Microsoft that answered our questions over the years and specially those that licensed code under open source licenses.

We originally started to work on Mono, because we wanted to make developers happier and more productive on Linux. We liked C#, we liked the CIL and we wanted to have those technologies available on our platform.

Since we have been active in the Linux Desktop world, it is not a surprise that the early use of Mono was mostly on Linux desktop applications, and Mono continues to shine there. Server-side use of Mono was a natural evolution and we soon were powering ASP.NET sites on Linux.

There is one area where we under-delivered in the past, and it has been a constant source of pain. Up until now, we did not have a working debugger. This has finally changed, and Mono 2.0 includes for the first time a debugger, the time for WriteLine() debugging is now behind us.

As the project matured, developers started taking advantage of Mono's open source nature: essentially .NET on their own terms. A platform that could be adapted, morphed, ported and modified to suit many different uses. Today Mono is embedded in portable mp3 players and powers Unity3D's game engine on the Apple iPhone, the Nintendo Wii, MacOS X and Windows (Some folks at Novell are working with Unity on bringing Unity3d to Linux!).

It has also been deployed to run code on large clusters of servers for SecondLife, powers our open source Silverlight implementation (Moonlight) and powers the popular DekiWiki: a Social Collaboration Tool.

Mono is a large project and it is hard to pick one feature to talk about as there are so many, so instead I put together a quick table of the major features that are part of this release:

Compiler Support	.NET APIs	Mono APIs
Mono's Open Source Compilers: C# 3.0 compiler. LINQ to Objects. LINQ to XML. Visual Basic 8 compiler. Open Source Compilers: Boo. Cobra. GNU's GCC with the CIL backend. Microsoft's IronPython. Jeroen's IKVM (Java on .NET). Microsoft's IronRuby. Phalanger (PHP on CLI). Commercial Compilers: ISE's Eiffel. Microsoft's C#. Microsoft's F#. Microsoft's VB.NET. RemObject's Oxygene (Object Pascal). And many more.	Core API: 2.0 core APIs. System, System.Xml. 3.5 System.Core. System.Drawing. System.DirectoryServices. System.Web.Services. Windows.Forms 2.0: Win32 driver. Quartz/OSX driver. Cairo/X11 Unix driver. ASP.NET 2.0: Core ASP.NET. ASP.NET AJAX. Apache and FastCGI integration. ADO.NET 2.0 plus providers for: Managed drivers: Postgresql, MS SQL Server, Sybase. Semi-managed drivers: Firebird, IBM DB2, Oracle, Sqlite. MySQL provides their own drivers.	GUI APIs: Gtk# (Unix, Windows, MacOS X). Cocoa# (MacOS X). Mono Core: Mono.Addins - Extensibility Framework. Mono.Cairo - Cairo Graphics Binding. Mono.Cecil - ECMA CIL Manipulation. Xml.Relaxng. Novell.Directory.Ldap C5 - Generics Library. Linux Specific: Mono.Posix/Mono.Unix. NDesk.DBus Other Ecosystem Libraries: Bit# - Bittorrent client/server library. Mono.Fuse - User-space file systems. Mono.ZeroConf - Bonjour stack. Mono.Nat - Network Address Translation. Mono.Upnp - Universal Plug and Play. Tao Framework - OpenGL, OpenAL, SDL and Cg bindings.

We have ported Mono to a wide variety of platforms and operating systems on this 1.0 to 2.0 cycle. These platforms include:

• Linux (x86, x86-64, PowerPC32, Itanium, SPARC, ARM, s390, s390x.
• Solaris (x86-64, SPARC).
• MacOS X (x86, PowerPC32).
• Windows (x86, support for x86-64 will come in Mono 2.2).
• Nintendo's Wii.
• iPhone/iPod Touch (ARM, limited functionality due to licensing requirements; I will blog later this week about this).
• *BSD (x86, x86-64).

Developing with Mono

Long time Linux developers will probably continue to use Emacs and VI, but some new Linux developers might want to use an IDE. New developers can use our open source MonoDevelop IDE on Linux, or alternatively the commercial X-Develop IDE or SlickEdit.

If you are a Windows developer, you can continue using Visual Studio or your IDE of choice to write the code and compile it. Your binaries will run just fine on Linux.

To assist Windows developers in porting their applications to Unix, we have provided the Mono Migration Analysis tool.

Runtime Changes

The Mono Virtual Machine gained plenty of features since Mono 1.2 was released. We have added:

• Generic Code Sharing and VTable Compression have been implemented: this significantly reduces the memory footprint of generic type instantiations, while still getting the speed benefits of using generics.
• AOT support: in addition to x86, we now also support ARM and x86-64.
• COM interop is now complete (works on Windows with "real COM" and can be used on Unix with Mainsoft's COM or Mozilla's XPCOM).
• AOT code can now AOT parts of 2.0 assemblies (assemblies that contain generics).
• Full AOT support (allows code to run JIT-less, this is limited to 1.0 code).
• CIL Verifier: Now Mono has a CIL verifier.
• CoreCLR Security: the security system used by Moonlight.
• Many optimizations that improve execution performance: New intrinsics operations (Math.Min/Max for example); various operations are now inlined by the JIT; managed allocations (no transition to unmanaged code for allocating memory); multi-array access is now tuned by the JIT; constant and block initializations are now handled by the JIT; Faster initialization and access to multi-dimensional arrays (4x faster).
• The runtime went on a diet, many runtime data structures are smaller making Mono lighter.

Tools

In addition the the Mono Debugger making its debut appearance on this release, we are very proud of our code analyzer Gendarme.

Gendarme is a extensible rule-based tool to find problems in .NET applications and libraries. Gendarme inspects programs and libraries that contain code in ECMA CIL format (Mono and .NET) and looks for common problems with the code, problems that compilers do not typically check or have not historically checked.

Feedback

Mono is not perfect, but we want to improve it. Like many other open source projects, we need your bug reports to improve Mono. If you have problems with Mono, help us by filing a bug report.

Special Thanks

Special thanks to Hacker Extraordinaire Aaron Bockover who not only brings us the best media player in the world, but created the new web site design and implemented and tuned it over very long extra hours up until 7am in the morning on his weekend.

And to our packaging and QA team that spent extra hours to get all the bits and pieces in place for the release.

Five Second Linux Boot

I loved this LWN article on the changes necessary to make Linux boot in 5 seconds on the Asus EEE PC (a relatively slow computer).

Hopefully all Linux distributions will adopt these changes for custom deployments.

Moonlight Update: Media Codecs

A couple of weeks ago we started the work on porting Microsoft's Media Codecs to Linux and we got the C version running.

Popfly in Firefox3/Linux/x86

Geoff, Fernando and Rolf have been hard at work on this, and have also added the infrastructure to download and install the codecs on demand.

The next step was getting all the assembly language supported in Linux, and today Geoff got the assembly optimized SSE1 audio decoder running (the first chunk of the decoders).

Of course, the rest of the team has been busy fixing bugs and improving the performance in preparation for the first public beta of Moonlight.

Microsoft changes the Managed Extensibility Framework License

A couple of weeks ago I suggested that developers interested in having their .NET software run in other platforms should avoid Microsoft's Managed Extensibility Framework (MEF) as it was not an open source library.

I had a chance to discuss with Glenn, Sam and Bob the benefits of using the MS-PL for this library first over twitter and then over email.

Representing .NET's loyal competitor, I did not think that we stood a chance of getting Microsoft to change the license, but I was pleasantly surprised. Glenn understood the value of open source, Sam wanted to do the right thing about this library and CodePlex and Bob argued that Mono already had Mono.Addins anyways.

Today Glenn announced that Microsoft has changed the license for MEF to the open source MS-PL license.

Thanks to everyone at Microsoft that helped change the license!

DbLINQ, LINQ to Databases and Mono

Atsushi Enomoto blogs about the work involved in bringing LINQ to Databases to Mono.

The effort was a joint collaboration between the awesome DbLINQ developers, Pablo Iñigo Blasco our Google Summer of Code Student and Novell's Atsushi Enomoto.

The DbLINQ developers had created a general purpose LINQ provider that could be used with database providers other than SQL Server. They also relicensed their code a few months ago to the MIT X11 license to allow for integration with Mono's code base.

Read Atsushi's description of how the effort was put together and how DbLINQ is being refactored to become a full System.Data.Linq implementation and still provide the foundation for third-parties to easily create database LINQ providers.

DbLINQ is a great project, and it needs your help to complete the effort.

Mono at the PDC 2008

Great News Mono Lovers!

Later this month I will be presenting the session "Mono and .NET" at the Microsoft Professional Developers Conference in LA.

Exciting times!

Update: My talk will cover new technologies that we have created as part of Mono. Some of them are reusable on .NET (we try to make our code cross platform) and some other are features that specific to Mono's implementation of the CLI.

Microsoft to incorporate jQuery into Visual Studio

This weekend the news came out that Microsoft was going to bundle and support John Resig's jQuery as part of Visual Studio and ASP.NET. From Scott's blog:

I'm excited today to announce that Microsoft will be shipping jQuery with Visual Studio going forward. We will distribute the jQuery JavaScript library as-is, and will not be forking or changing the source from the main jQuery branch. The files will continue to use and ship under the existing jQuery MIT license.

Beyond the obvious benefits to developers, what interests me is that Microsoft is now bundling an open source component into their commercial offerings.

This is a first time for Microsoft.

With IronPython they continued development of an open source project in house. With IronRuby, they were open to external contributions to the libraries of IronRuby. In both cases they were not taking external code or contributions directly into their core products.

Hopefully they will start using more open source code in their products. Maybe one day they will bundle Mono's Cecil or Mono's embeddable C# compiler.

Monovation: Assembly Injection into Live Processes

People are loving the C# Interactive Shell.

In the past people have embedded languages like Boo into their applications (MonoDevelop and Banshee for example both have options to embed Boo and a shell).

Zoltan designed a new system for Mono that allows developers to inject and execute code into running Mono applications. Paolo provided significant feedback and design guidelines for the code to be incorporated into Mono.

Thanks to both Zoltan and Paolo this functionality is now available through the Mono.Management assembly: you provide an executable and a PID, and the executable is injected and its Main method executed on the target process:

	using Mono.Attach;

	[..]

	// Get a handle to a running Mono process.
	VirtualMachine vm = new VirtualMachine (pid);

	// Load hello.exe into the target process, and run it
	vm.Attach ("/tmp/hello.exe", "");
	

You can use this for billions of things of course. You could patch running programs, you could attach logging software to a running program, or you could inject a C# evaluator into a live application and examine its state, or tweak it live.

Zoltan came up with a really cool extension to the the csharp command (this is the command-line C# Interactive Shell). The csharp command now takes an --attach command line argument and a PID.

The csharp shell can now use the Mono.Attach.VirtualMachine to injects itself into the remote process and then the client and server communicate through a couple of sockets.

For example, this is the sample that Zoltan used to pitch his idea for supporting attaching to the virtual machine. With the following you can pause a live Banshee process:

$ ps aux | grep banshee
miguel   12359 17.0  2.7 141372 55708 pts/5    Sl+  14:30   0:02 banshee-1 /usr/lib64/banshee-1/Nereid.exe
$ csharp --attach 12359
csharp> using Banshee.ServiceStack;
csharp> using Banshee.Gui;
csharp> var s = ServiceManager.Get ();
csharp> s.PlaybackActions ["PlayPauseAction"].Activate ();
	

All of the code is now on SVN, you need both the mono and mcs modules from trunk.

A GUI Shell

The above commands are a tiny bit risky and are also limited to the shell.

The above commands will execute on a separate thread from the application, and any commands that you execute would be executed on a separate thread which could corrupt the state of your application.

So this weekend, I wrote a tool that integrates with Gtk# applications, its called gsharp and you can find it in the mono-tools module (it borrows some code from Banshee).

gsharp is a Gtk# version of the csharp command. What is important is that it also supports injection of the code on other programs, but makes sure that all the code executes in the Gtk# thread, by issuing all of its commands from the Gtk# idle handler. This means that it is safe to use gsharp with your Gtk# applications.

GUI version of the tool.

Here you can select which project you want to inject the GUI into:

Needs some work, show process names.

This version shows the gsharp tool attached to F-Spot, as I am not very familiar with the codebase, I can not do very much with it:

We will need to implement one of these as well for Windows.Forms applications. This should luckily be easy to do as most of the smarts live in the Mono.CSharp assembly (our embeddable compiler).

Security of Agents

A couple of weeks ago, I asked for people to weigh in on a security concern for temporary files. This was for Zoltan's attach functionality.

The code is now implemented and I would love if security experts could do a source code audit of our implementation. And validate whether our assumptions are safe.

Here is the source code.

Public Service Announcement

In C# the defaut access level for members in classes and structs is "private".

There is no need to pepper the source code with "private" everywhere. It only introduces noise and makes your code more difficult to read.

Love World Domination!

Chris Anderson is wearing a Mono T-Shirt on this PDC interview and IronRuby.com is hosted on DekiWiki, a Mono-powered Wiki site.

This is clearly awesome.

In other news, the awesome hackers at Imendio have officially released Gtk+ for OSX packages.

Stream.CopyStream

Funkists, why does System.IO.Stream not have a CopyStream method, it seems like everyone ends up implementing this.

Discuss.

reCaptcha.NET

Kudos to Ben and the rest of the Captcha team.

Today I downloaded the binaries for reCaptcha.NET for a web page that I was setting up and it worked out of the box. No recompiles necessary:

• Drop the Recaptcha.dll in the bin directory of the ASP.NET app.
• Get your public/private key.
• Insert this in your page:

  		<recaptcha:RecaptchaControl
                          ID="recaptcha"
                          runat="server"
                          Theme="red"
                          PublicKey="YourKey"            
                          PrivateKey="YourOtherKey"/>
  		

• Hit refresh. Ta da!

Encrypted File Systems

I just found out about encfs, a user-space encrypted file system that runs on top of FUSE.

To use it, just type:

	$ encfs ~/.encryptedstorage ~/secure
	

And follow the directions. You then will have a