The most important piece of news from last week's PDC was Microsoft's decision to turn Silverlight into the universal platform for building cross platform applications.

The upcoming version of Silverlight will no longer be a Web-only technology. It will now be possible to build full desktop applications with Silverlight.

Desktop Silverlight applications differ from the standard Silverlight in a few ways:

• Full access to the host file system, like any other .NET application would have.
• None of the socket connectivity limitations that are present on the sandboxed versioned of Silverlight. Full network access (we should build a MonoTorrent UI for it!)
• Built-in Notifications API to show up bubbles to the user when they need to interact with the application.

Although Moonlight has supported this mode of operation since day one, turning this into a standard way to develop applications was going to take a long time. We would have needed to port Moonlight to Windows and OSX and then we would have to bootstrap the ecosystem of "Silverlight+" applications.

But having Microsoft stand behind this new model will open the gates to a whole new class of desktop applications for the desktop. The ones that I was dreaming about just two weeks ago.

This was a big surprise for everyone. For years folks have been asking Microsoft to give Silverlight this capability to build desktop apps and to compete with Air and it is now finally here. This is a case of doing the right thing for users and developers.

Desktop Tools in Silverlight?

Now that this technology is available, perhaps it is a good time to start a movement to create a suite of Silverlight-based desktop applications.

The benefits to me are many:

• .NET applications that actually look good. In the past your choices were basically of Gtk# or Winforms, neither one really designed for this graphic-designer driven world.
• We can join forces with Windows/MacOS developers to create the next generation of desktop applications.
• Developers can tap into the large ecosystem of third-party controls that exists for Silverlight.

For the Moonlight team, this means that there is a lot of work ahead of us to bring every Silverlight 3 and 4 feature. I think I speak for the whole Mono team when I say that this is exciting, fascinating, challenging and feels like we just drank a huge energy boost drink.

If you want to help, come join us in the #moonlight or #mono channels on the IRC server at irc.gnome.org.

Silverlight 4

There are many other great features in Silverlight 4, but none as important as Silverlight becoming a universal runtime for the CLR. This is a revolution.

If you are curious about all the new tactical features of the revolution, check Tim's Complete Guide to the new Silverlight Features.

If you have the time, watch Scott's keynote's presentation where he introduced the new features (he starts at 1:02). I loved the use of HTML as a Silverlight brush (paint with HTML and even Flash). If you have time, these are some great sessions on Silverlight:

• Microsoft Silverlight 4 Overview: A good intro, repeats a lot from Scott's keynote.
• Improving and Extending the Sandbox with Silverlight 4: Joe Stegman's presentation on the revolution.
• RIA Services: The technique that lets you "remove four acronyms" from your development, mindblowing.

Droolingly yours,
Miguel de Icaza.

With Moonlight quickly approaching its first official 2.0 release (which has feature parity with Silverlight 2.0 and has a handful of 3.0 features in place) we have been thinking about the work ahead of us, 3.0 and beyond.

Although there is much work left to do to achieve full parity with Silverlight 3.0, a solid 2.0 core means that we can start thinking about new innovative ideas.

Here are some ideas of things that Moonlight could prototype:

• Video camera and audio support (Chris has a working prototype).
• Native menu support for out-of-browser: we feel strongly that out-of-browser apps should get native menus and not be done with XAML. To allow proper MacOS, Windows and Linux integration.
• Right-click support for taking screenshots of Moonlight apps, record a screencast of a video, support for saving individual images, XAML and video from the plugin.
• Print the current canvas.
• User opt-in for full network access and full file system access (This is possible with Moonlight if compiled for the desktop APIs, but perhaps we should extend this to the browser version).
• Make the "Moonlight" widget ubiquitous. Allow any existing Gtk+, Win32, OSX, iPhone and Android to create content from a XAML file:

  		var splash = Moonlight.LoadFrom ("splash.xaml");
  		container.Add (sphash);
  		[...]
  
  		var mainMenu = Moonlight.LoadFrom ("mainMenu.xaml");
  		mainMenu.FindObject ("Start").Clicked += {
  			StartGame ();
  		};
  		window.Add (mainMenu);
  		

We could use Silverlight to build the next wave of cross-platform desktop applications.

I think of the Moonlight relationship to Silverlight as the Firefox relationship to IE four years ago. It is a chance to try out new ideas in the Silverlight-o-sphere, we can try those ideas out, and if the ideas have merit, they could become part of the official Silverlight.

This echoes to some extent what Bob Muglia said on an interview a few days ago: "[it] will allow for Silverlight to move into all sorts of places where we can't - we can't see."

For example, we are hard at work to make Silverlight run on the native PlayStation 3 and the Wii.

In any case, we have been working on Moonlight for a long time. With our security system in place, and both our smooth streaming media engine and GUI engine in place I feel that open source innovation can begin.

Arne's presentation on concurrent programming at Monospace ago has been published.

Although making changes to Moonlight might be a very fun thing to do, for the longest time I have had this entry in my TODO list: "Blog about writing a video editor in Moonlight".

As much as I love Gtk+ and the Gnome desktop, our contributions for our desktop applications and for Gtk+ come mostly from from folks developing on Linux for Linux (with a handful of exceptions). And we are a small fraction of desktop developers.

In my mind what is interesting to me about building applications with Silverlight is that we can create an ecosystem of free software applications that run on all three major platforms: Windows, Linux and MacOS.

A few years ago, as part of the Google Summer of Code for Mono we created a project that could have had a great future, the Diva project (by MDK). Sadly, Michael moved on to other things, but in the back of my mind, I always wanted to have a nice video editing application for Linux.

I like to think that with Silverlight we have a new opportunity: we can create a community of open source developers that goes beyond the Linux-desktop community, but will pull developers interested about such a project from the Windows and MacOS worlds. I know that various members of the Moonlight team are passionate about Moonlight because it is this next generation API for building GUI applications.

Which applications do you think are needed nad could be built with Moonlight?

I say video editing, and I have some ideas of how it should work.

A few years ago, thanks to the Google Summer of Code, I got to meet a brilliant programmer that wrote an entire BitTorrent implementation as a reusable library called bitsharp. Today, I am lucky enough to work with Alan in the team team at Novell.

Alan wrote a Gtk+ UI on top of his library called Monsoon:

With Mono running in the browser as part of Silverlight, I have always felt that we needed to build a UI for bitsharp.

The most important advantage is that every web browser can be turned into a BitTorrent client without having to install extra software.

Some other advantages: you could connect to a web page that contains the BitTorrent client, it would turn your machine into a seeding/downloading host, it could be shut down with minimal fuzz and it would be trivial to spice up the UI and add all kinds of visualizations for the download.

Silverlight 2 now provides direct socket/network access and supports storing files in your local file system. Most of the pieces are already in place to make this happen.

There are a few challenges though. Silverlight does not allow a client to listen on a port, and for security reasons it has a limited set of ports it can connect to.

The first problem could be addressed by having the client initiate connections to third parties requesting and providing data. This might require a protocol extension.

The second problem is easy to solve, but would require other Bittorrent clients to listen in a new port to explicitly grant access to Silverlight clients by exposing a clientaccesspolicy.xml file.

Finally, at least for Moonlight's case, we could start doing some changes to the core and grant the out-of-browser plugin to have full network access.

Chris posted the new rules for the moonlight-2-0 branch and how to start working on 3.0 stuff.

Another intesting post is David's initial take on bringing pixel shaders, perspective transforms and hardware acceleration to Moonlight.

Today is a big day for the Mono team, we just released the Mono Tools for Visual Studio. The goal of this release is to make it simpler for Visual Studio developers to deploy their applications on Linux. ASP.NET, Windows.Forms, server and console applications are supported. These are the major features in the release:

Deploy to Linux: You can deploy from Visual Studio to a Linux machine running Mono your software. We are using Universal Plug and Play to detect Linux machines on your local network, or you can enter the IP address of your favorite hosting provider.

Debug Linux system remotely from Visual Studio: Developers complained that our debugging story left much to be desired, that deploying to Linux was possible, but debugging cumbersome. We now allow you to debug your application running on Linux without ever having to leave Visual Studio.

You can continue to edit, build and debug the way you have always done it, but the software will be running on a Linux machine:

We hooked the Mono debugging engine to Visual Studio, so all the regular debugging tasks are available (watches, locals, breakpoints, stepping) that you come to expect from Visual Studio debugging are available.

Review your code for portability: We have integrated Mono's Migration Analysis (MOMA) tool into Visual Studio so you can check your software for Windows/Linux differences right from your IDE and work around APIs that do not exist in Linux or Mono and refactor your code accordingly.

Package your software for Linux: Right from Visual Studio you can package your .NET application in Linux native installation format. Just create a packaging project in your solution, configure your package and produce an RPM that you can distribute:

And finally, one of the most exciting features in this tool: go from shipping applications into shipping appliances in minutes:

You use our wizard to prepare your appliance, select a base operating system template (Server, Client, and base operating system) and off you go.

Once your appliance is built, you can test it and apply the finishing touches over the web (using a Flash applet that connects to our virtual machines in our data center) and when you are happy with the results, you can download and redistribute your appliance to your users.

Appliances

Earlier this year I wrote about Novell's SUSE Studio our hosted service that helps developers create ready-to-run appliances.

Today, as a .NET developer, when you distribute your software to your users and customers, you probably have a list of requirements that goes like this:

• Install Windows XXX, Reboot.
• Install security updates A through Z, reboot as many times as needed.
• Install .NET runtime, reboot as needed.
• Install SQL server, reboot as needed.
• Populate database, reboot as needed.
• Install third party tool, reboot as needed.
• Just to be sure, reboot.

Then your users can start installing your software. At that point, you initiate a series of support calls that go like this:

• Did you make sure that .NET xxx.yy was installed? Ah, so do that and resume the steps.
• Wrong databases, uninstall, reboot, reinstall database, reboot.

And repeat the above process for every single one of your users.

Basically, every user has to repeat the same steps. Everyone has to assemble the solutions made up of the operating system, various pieces of dependencies that you have and your software, like this:

The experience today is like trying to buy a car by buying the individual parts:

We believe that for a class of developers there is a better way.

We believe that we can help you put together the full car, and deliver the car in a single piece.

With appliances you can ship a pre-configured operating system, pre-configured database and every service pre-configured and installed together with your software to deliver a full package, so you prepare your software for distribution once, you configure the database once, and then you give your users a ready-to-run virtual machine, CD-ROM or USB stick:

SUSE Studio has been used to build Linux based appliances (over four thousand per week), and now we are making it easy for .NET developers to take advantage of one of Linux's strengths: it is free, it is open source, you can shrink it, you can grow it and you can ship your own version (and yes, we do provide the updates for all of the core components that you pick).

Getting Started

Check out our getting started document.

Cool use of MonoVS

This is a cool use of Mono for Visual Studio debugging the Axiom 3D engine from the amazing Michael Cummings:

Visual Studio remotely debugging Axiom on an Ubuntu virtual machine hosted in Sun's VirtualBox.

Update: How to get it to work with Ubuntu

This is pretty sweet in a couple of levels.

A couple of hackers at RemObjects have released some reusable controls for use with MonoTouch that simplify some common tasks with UITableViews, they are part of their "Project Plateau":

Not only this is sweet in that these are the first set of 3rd party controls for MonoTouch, they are also sweet because the authors built these controls using Oxygene, their Object Pascal compiler.

Update: Some documentation can be found here: introducing Plateau for MonoTouch.

I missed this post from Friday.

Mark Probst has posted a call for testers for Mono's new garbage collector:

Our new garbage collector, SGen, is now at the point where it's ready
for early testing.  It's still a long way from being ready for
production use - it's still slow and consumes too much memory,
especially for long-running processes - but if you'd like to help us
find bugs, please consider giving it a try.  I'm especially interested
in cases where Mono crashes or hangs with SGen (other than for memory
exhaustion).  The simpler the test cases the better.

To use SGen instead of Boehm, configure mono with the switch
"--with-gc=sgen".  At this point it's only known to run on x86 and
AMD64 Linux.

This collector is a precise-heap, conservative stack, moving, copying collector and will return unused memory back to the operating system. The details of this collector are available in our Compacting GC page.

I am rebuilding my own Mono with Compacting GC now as we speak.

Craig Dunn has turned his Monospace application for the iPhone into a Microsoft PDC application that conference attendees can use at the upcoming conference.

Mandatory screenshot:

Since this app is not on the AppStore, you need to compile it and use Ad-Hoc deployment to run it into your phone.

Update: If you do not have a Mac, come to our booth at the Microsoft PDC, and we will ad-hoc deploy the app for you.

Currently, Mono's Garbage Collector does not compact the heap and one of the problems that users have is that certain allocation patterns might result in a heap fragmentation that is hard to recover from. The observable side effect is that your process memory usage grows, but the memory is mostly unused:

You can read more about this in the Compacting GC page at the Mono web site.

Although a new GC is being developed to address this, the code is still not switched on by default in Mono as we do not consider it ready for production use (you must compiled Mono with --with-gc=sgen).

Avoid allocating very large buffers, instead allocate smaller buffers, and if you need to provide the illusion of a large buffer, hide this behind an interface.

If you must use large buffers, you might want to consider using MindTouch's ChunkedMemoryStream which exposes the same interface as MemoryStream, except that it uses 16k chunks instead of a very large buffer.

This will avoid the fragmentation for a very common use case.

Today we released MonoTouch 1.2.

Perhaps the most important new feature in MonoTouch 1.2 is that it now has a debugger that supports debugging on both the iPhone simulator and on the iPhone/iPod Touch.

The debugger is integrated directly into MonoDevelop/OSX. All you have to do is select one of the debugging configurations (either Debug|iPhone or Debug|iPhoneSimulator):

and run your application:

To set breakpoints, you use the usual MonoDevelop UI, just click on the left-hand side of the editor, or use Command-/:

The debugger offers the usual watch windows, and also allows you to navigate object state by hovering over the value in the IDE:

The Technology

Although this was developed for the iPhone, I should probably have said that this is Mono's new soft debugger engine.

The iPhone is once again a challenging platform to get debugging working on. Since Apple has not published the information necessary to implement something like GDB or Mono's own MDB and we are not going to reverse engineer the protocol, instead we created a new way of debugging Mono applications.

The Mono Debugger for the iPhone platform uses a soft-debugger. Unlike traditional debuggers that act like a fully controlling entity over the Mono process, the soft-debugger is actually a debugger that is built into the Mono process itself. MonoDevelop communicates with this debugger over a compact protocol, similar to what has been done in the past with Java debuggers:

We are providing a new library, the Mono.Debugger.Soft.dll that encapsulates the protocol and offers an API that developers can use to debug a remote Mono process.

On systems where we have access to breakpoints the soft debugging engine will use the standard operating system facilities to single step and set breakpoints.

But on systems like the iPhone and some video game consoles where there is no way to modify memory without special privileges we had to resort to a different technique. The Mono static compiler inserts special code at every sequence point that checks for single stepping or breakpoints. The code generated during these debug builds is larger, but it allows us to support the iPhone without having to resort to undocumented APIs in any form or shape.

MonoDevelop and the iPhone

When the user selects an application for debugging, MonoDevelop configures the application to contact MonoDevelop on startup and link the debugger to it, starts listening on a couple of ports (one for the debugging protocol, and one for redirecting standard output/standard error) and waits for the application on either the iPhone simulator or the iPhone to contact it.

Upon contact, the debugger handshake takes place and operation continues. For the simulator, this takes place with a local socket; for the device, this happens over a TCP/IP socket over WiFi.

One of the nice side effects of this approach to debugging is that it is possible to distribute binaries to testers (using the Ad-Hoc distribution model) and debug problems on a user's iPhone over the network.

To support this scenario, when you build applications with the "Debug|iPhone" configuration, MonoDevelop will modify your application's Settings file on the iPhone.

This allows your beta-testers to enable debugging and connect to your debugger for inspection. This is what the settings looks like:

The first port is the port where MonoDevelop will be listening to. The second port is the port where the standard output and standard error will be redirected to.

All of the Console.Output and Console.Error output will be sent to this port when debugging is enabled on the application.

Pros and Cons

There are pros and cons to the soft debugger approach.

Soft debuggers are unable to do low-level debugging or debug Mono itself (mixed-mode debugging) and they are unable to stop applications at will, all it can do is request politely that the application stops, and when the Mono runtime decides that it is a good time to stop, it will stop.

On the pro side, the soft debugger is easier to port and is more robust as it eliminates some dead-lock situations. These can happen when the runtime has been forcefully stopped by the debugger, and then the debugger invokes methods on the target. These methods in turn might require an internal lock that is currently held by a different thread.

The Protocol

The protocol used between the debugging agent running inside the Mono process and a debugger is implemented in the debugger-agent.c file.

Availability

MonoTouch customers will be offered the update the next time they start MonoDevelop or if they manually "Check for Updates" on the MonoDevelop IDE.

Users of MonoTouch for the iPhone simulator can get the it from the trial page.

The source code for the soft-debugger is available on SVN. The API to communicate with the Mono runtime is available in the Mono.Debugger.Soft assembly and the debugger itself has already been integrated into the Mono 2.6 branch and Mono trunk on SVN.

This should prove useful to other users of Mono that might benefit from a soft debugger like Unity3D or Second Life.

Screenshots

Brent has a nice gallery of screenshots of the debugger in action on his MonoTouch 1.2 with debugging released! post.

April IM logs: