As part of the Moonlight Beta release, I wanted to devote a few blog posts to exploring the features in Moonlight and how we implemented those Silverlight features in Moonlight.

Before I get started on today's topic, we would like to get some feedback from our users to find out which platforms they would like us to support with packages and media codecs. Please fill out our completely platform and media codec survey.

Moonlight 1.0 is an implementation of the Silverlight 1.0 API. It is an entirely self-contained plugin written in C++ and does not really provide any built-in scripting capabilities. All the scripting of an embedded Silverlight component is driven by the browser's Javascript engine. This changes with the 2.0 implementation, but that is a topic of a future post.

The Silverlight/Moonlight Developer View.

One of the most important features of the Silverlight/Moonlight web plugin is to support audio and video playback.

Silverlight takes an interesting approach to video and audio playback. In Silverlight the video can be treated like any other visual component (like rectangles, lines) this means that you can apply a number of affine transformations to the video (flip, rotate, scale, skew), have the video be composed with other elements, add a transparency layer to it or add a clipping path to it.

This is the simplest incarnation of a video player in XAML:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <MediaElement x:Name="ExampleVideo"
		  Source="TDS.wmv"
		  Width="320" Height="240"
		  AutoPlay="true"/>
</Canvas>
	

The result looks like this when invoked with when embedded in a web page (or when using the mopen1 command that I am using to take the screenshots):

The MediaElement has a RenderTransform property that we can use to apply a transformation to it, in this case, we are going to skew the video by 45 degrees:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Source="TDS.wmv" Width="320" Height="240">
	   <MediaElement.RenderTransform>
	     <SkewTransform CenterX="0" CenterY="0" AngleX="45" AngleY="0" />
	   </MediaElement.RenderTransform>
	</MediaElement>
</Canvas>
	

The result looks like this:

But in addition to the above samples, MediaElements can be used as brushes to either fill or stroke other objects.

This means that you can "paint" text with video, or use the same video source to render the information in multiple places on the screen at the same time. You do this by referencing the MediaElement by name as a brush when you paint your text.

This shows how we can fill an ellipse with the video brush:

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Opacity="0.0" Source="TDS.wmv" Width="320" Height="240"/>

	<Ellipse Width="320" Height="240" >
	   <Ellipse.Fill>
 	      <VideoBrush SourceName="ExampleVideo"/>
	   </Ellipse.Fill>
	</Ellipse>
</Canvas>

This looks like this:

You can also set the stroke for an ellipse. In the following example we use one video for the stroke, and one video for the fill. I set the stroke width to be 30 to make the video more obvious.

<Canvas xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
	<MediaElement x:Name="ExampleVideo" AutoPlay="true" Opacity="0.0" Source="TDS.wmv" Width="320" Height="240"/>
	<MediaElement x:Name="launch" AutoPlay="true" Opacity="0.0" Source="launch.wmv" Width="320" Height="240"/>

	<Ellipse Width="320" Height="240" StrokeThickness="30">
	   <Ellipse.Fill>
 	      <VideoBrush SourceName="ExampleVideo"/>
	   </Ellipse.Fill>
	   <Ellipse.Stroke>
 	      <VideoBrush SourceName="launch"/>
	   </Ellipse.Stroke>
	</Ellipse>
</Canvas>
	

Notice that in the examples above I have been using AutoPlay="true". Silverlight provides fine control over how the media is played as well as a number of events that you can listen to from Javascript for various events (for example, you get events for buffering, for the media being opened, closed, paused, or when you hit a "marker" on your video stream).

Streaming, Seeking and Playback

Depending on the source that you provide in the MediaElement, Moonlight will determine the way the video will be played back.

The simplest way of hosting a video or audio file is to place the audio or video file in a web server, and then have Moonlight fetch the file by specifying the Source attribute in the MediaElement. You do not need anything else to start serving videos.

Seeking on the standard Web server scenario: When the programmer requests the media to be played (either by calling the Play method on the MediaElement, or because the MediaElement has AutoPlay set to true) Moonlight will start buffering and play the video back.

If the user wants to seek backwards, or forward, Moonlight will automatically take care of this. In the case where the user fast-forwards to a place in the file that has yet to be downloaded, playback will pause until then.

Seeking with an enhanced media server: If your server implements the Windows Media Streaming HTTP extension if the user seeks to a point in the file beyond the data that has been downloaded, it will send a special message to the server to seek. The server will start sending video from the new position. The user will get the playback started immediately without having to wait. The details of the protocol are documented in the MS-WMSP specification. This is enabled by using the "mms://" prefix for your media files instead of using the "http://" prefix.

Notice that even if it says "mms", Silverlight and Moonlight do not actually speak to an MMS server, they merely replace that with "http" and speak http/streaming to the server.

The extension is pretty simple, it is basically a "Pragma" header on the HTTP requests that contains the stream-time=POSITION value. Our client-side implementation is available here.

You can use IIS, or use the mod_streaming to enhance the video experience for your end users.

This basically means that you can stream videos on the cheap, all you need is a Linux box, two wires, and a 2HB pencil.

Adaptive Streaming

Another cool feature of the Adaptive Streaming support in Moonlight is that the server can detect the client throughput, and depending on the bandwidth available, it can send a high bitrate video, or a low bitrate video. This is a server side feature.

This feature was demoed earlier this year at Mix 08:

I am not aware of an adaptive streaming module for Apache.

Supported Media Formats in Moonlight 1.0

Although Moonlight 1.0 exposes the Silverlight 1.0, Moonlight 1.0 ships a 2.0 media stack (minus the DRM pieces). This means that Moonlight ships with support for the media codecs that are part of Silverlight 2.0 and supports adaptive streaming. This is what we are shipping:

Video:

• WMV1: Windows Media Video 7
• WMV2: Windows Media Video 8
• WMV3: Windows Media Video 9
• WMVA: Windows Media Video Advanced Profile, non-VC-1
• WMVC1: Windows Media Video Advanced Profile, VC-1

Audio:

• WMA 7: Windows Media Audio 7
• WMA 8: Windows Media Audio 8
• WMA 9: Windows Media Audio 9
• WMA 10: Windows Media Audio 10
• MP3: ISO/MPEG Layer-3
  • Input: ISO/MPEG Layer-3 data stream
  • Channel configurations: mono, stereo
  • Sampling frequencies: 8, 11.025, 12, 16, 22.05, 24, 32, 44.1, and 48 kHz
  • Bit rates: 8-320 kbps, variable bit rate
  • Limitations: "free format mode" (see ISO/IEC 11172-3, sub clause 2.4.2.3) is not supported.

We also support server-side playlists.

For more information see the Silverlight Audio and Video Overview document on MSDN.

Media Pipeline

When we first prototyped Moonlight we used the ffmpeg media pipeline. A media pipeline looks like this:

Charts by the taste-impaired.

Originally ffmpeg handled everything for us: fetching media, demultiplexing it, decoding it and scaling it.

Since we needed much more control over the entire pipeline, we had to write our own, one that was tightly integrated with Moonlight.

Today if you download Moonlight's source code you can build it with either the ffmpeg codecs or you can let Moonlight fetch the binary Microsoft Media Pack and use Microsoft's codecs on Linux.

Microsoft Media Pack

The Microsoft Media Pack is a binary component that contains the same code that Microsoft is using on their Silverlight product.

The Moonlight packages that we distribute do not actually have any media codecs built into them.

The first time that Moonlight hits a page that contains media, it will ask you whether you want to install the Microsoft Media Pack which contains the codecs for all of the formats listed before.

Today Microsoft offers the media codecs for Linux on x86 and Linux x86-64 platforms. We are looking for your feedback to find out for which other platforms we should ship binary codecs.

Tests

No animals were harmed in the development of the Moonlight Video Stack. To ensure that our pipeline supported all of the features that Microsoft's Silverlight implementation supports we used a number of video compliance test that Microsoft provided us with as part of the joint Microsoft-Novell collaboration.

In addition to Microsoft's own tests, we created our own set of open source tests. All of these tests are available from the moon/test/media module. This includes the videos that are specially encoded with all the possible combinations and features used as well as XAML files and associated javascript.

I wrote a small tool that exports my Groupwise Calendar to Google Calendar.

This tool only runs on Windows as it is using the Groupwise COM APIs to fetch the calendar data. I would love to have this work on Linux if someone knows how to get to these from Unix.

You will need the Google Calendar assemblies (Google.GData.AccessControl, Google.GData.Calendar, Google.GData.Extenions) and the Groupwise Assemblies (GroupwiseTypeLibrary, GroupWiseCommander) and a text file that contains your passwords (called `passwords').

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Google.GData.Calendar;
using Google.GData.Client;
using Google.GData.Extensions;
using Google.Accounts;
using System.Threading;
using System.IO;

namespace CalendarExporter
{
    class Program
    {
	const string google_email = "[email protected]";
	const string groupwise_login = "YOURNAME";
	
        static void Main(string[] args)
        {
            var f = File.OpenRead("passwords");
            var reader = new StreamReader(f);
            var google_passowrd = reader.ReadLine();
            var groupwise_password = reader.ReadLine();

            new Thread(delegate() {
                Thread.Sleep(1000 * 120);
                Console.Error.WriteLine("Timing out");
                Environment.Exit(1);
            }).Start();

            ClientLoginRequest login = new ClientLoginRequest();
            login.AccountType = "GOOGLE";
            login.Email = google_email;
            login.Password = google_password;
            login.Service = "cl";
            login.Source = "YOURNAMECo-CalendarPush-1";

            var token = login.Login();

            CalendarService cs = new CalendarService("YOURNAMECo-CalendarPush-1");
            cs.SetAuthenticationToken(token.Auth);


            CalendarQuery cq = new CalendarQuery();
            cq.Uri = new Uri("http://www.google.com/calendar/feeds/default/owncalendars/full");
            CalendarFeed resultFeed = cs.Query(cq);
            CalendarEntry gw_at_google = null;
            foreach (CalendarEntry entry in resultFeed.Entries) {
                if (entry.Title.Text == "Groupwise Calendar") {
                    entry.Delete();
                    break;
                }
                
            }
            gw_at_google = new CalendarEntry();
            gw_at_google.Title.Text = "Groupwise Calendar";
            gw_at_google.Summary.Text = "This is the syncrhonized calendar at Novell's Groupwise server";
            gw_at_google.TimeZone = "America/New_York";
            gw_at_google.Hidden = false;
            gw_at_google.Color = "#2952a3";
            gw_at_google.Location = new Where("", "", "Boston");
            gw_at_google.Selected = true;
            Uri postUri = new Uri("http://www.google.com/calendar/feeds/default/owncalendars/full");
            CalendarEntry cal = (CalendarEntry)cs.Insert(postUri, gw_at_google);

            string calurl = cal.EditUri.Content;
            int p = calurl.LastIndexOf ('/');
            string code = calurl.Substring (p);

            
            Uri edit_uri = new Uri ("http://www.google.com/calendar/feeds" + code + "/private/full");
            GroupwareTypeLibrary.GWSession2Class gsc = null;
            try
            {
                gsc = new GroupwareTypeLibrary.GWSession2Class();
            }
            catch
            {
                Console.WriteLine("Did not regsvr the file c:\novell\groupwise\gwcma1.dll and is this program x86-only?");
                return;
            }
            var account = gsc.Login(groupwise_login, "", groupwise_password, null, null);
            var path_to_host = account.PathToHost;

            //alendar calendar = new iCalendar();

            
            int count = 0, skipped =0;
            foreach (GroupwareTypeLibrary.Message m in account.Calendar.Messages)
            {
                if (!m.ClassName.StartsWith ("GW.MESSAGE.APPOINTMENT"))
                    continue;

                GroupwareTypeLibrary.Appointment2 app = (GroupwareTypeLibrary.Appointment2) m;

                // Ignore appointments that are older than 15 days.
                if (app.EndDate < DateTime.Now - TimeSpan.FromDays(7)) {
                    skipped++;
                    continue;
                }

                var ee = new EventEntry();
                ee.Title.Text = app.Subject.PlainText;
                ee.Content.Content = app.BodyText.PlainText;
                ee.Locations.Add (new Where () { ValueString = app.Place });
                ee.Times.Add(new When(app.StartDate, app.EndDate));
                ee.EventVisibility = app.Private ?
                    EventEntry.Visibility.PRIVATE : EventEntry.Visibility.PUBLIC;

                cs.Insert (edit_uri, ee);
            }
            
            Console.WriteLine("Done2");
            Environment.Exit(0);
        }
    }
}
	

You will also need the Login.cs which is some sample code that I found on the tubes for doing Google Account authentication.

We have released the first beta of Moonlight 1.0.

This release supports the Microsoft Media Pack for playing back video and audio files. These are the same video and audio decoders that Microsoft uses in Silverlight 2.0.

Check our Moonlight roadmap for details on upcoming versions.

You can try some of the sites tests that we used to test Moonlight.

Here are some Silverlight 1.0 materials:

• How To Videos on Silverlight 1.0
• Essential Silverlight Training.
• Silverlight 1.0 Unleashed. This is my favorite Silverlight 1.0 book, one of the books that we used to get some of the Moonlight team members up to speed.

You can also read the Silvelright's XAML vocabulary description and its XAML Foundation Specification.

As part of SUSE 11, Mono needs to run on the PowerPC in 64 bit mode. The effort was bootstrapped with some early work from Andreas Faerber.

It was fun to watch Mark's daily commits progress of the port, the tests referenced here are the basic runtime tests that we use to check for regressions and to get a port up and running, it is a good roadmap for how a port comes to life:

	* mini-ppc64.c, cpu-ppc64.md: Fixed some opcodes.  PPC64
	passes basic.exe now.

	---

	* cpu-ppc64.md: Fixed a few instruction lengths.

	* mini-ppc64.c: Don't emit SETLRET.  Now PPC64 passes
	basic-float.exe.

	---


	* decompose.c: Decompose carry and overflow add on PPC64 like
	on other 64 bit archs.  Don't decompose sub at all on PPC64.

	* mini-ppc64.c, exceptions-ppc64.c, tramp-ppc64.c,
	cpu-ppc64.md: 	Several fixes and new opcodes.  Now PPC64 runs (but doesn't
	pass) basic-long.exe.

	---
	
	* ppc/ppc-codegen.h: Use ppc_load_reg instead of ppc_ld in
	ppc_load_func to fix the 2 bit shift.

	---

	* mini-ppc64.c, mini-ppc64.h, cpu-ppc64.md: Several fixes.
	Now PPC64 passes basic-long.exe.

	---

	* ppc/ppc-codegen.h: Make ppc_is_[u]imm16() work with 64 bit
	values.

	---

	* mini-ppc64.h, cpu-ppc64.md: Fixed caller/callee saved
	floating point regs.  Now PPC64 passes basic-calls.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, exceptions-ppc64.c,
	tramp-ppc64.c, cpu-ppc64.md: Several fixes.  PPC64 now runs objects.exe.

	---

	* mini-ppc64.c, tramp-ppc64.c: Small fixes.  PPC64 now runs
	arrays.exe and basic-math.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, exceptions-ppc64.c,
	cpu-ppc64.md: Several fixes.  PPC64 now runs exceptions.exe and
	devirtualization.exe.

	---

	* mini-ppc64.c: Several fixes.  PPC64 now runs iltests.exe.

	---

	* mini-ppc64.c, mini-ppc64.h, tramp-ppc64.c: Disable generic
	code sharing.  PPC64 now passes generics.exe.

	---

	* basic-long.cs: New test case.

	---

	* mini-ppc64.c, mini-ppc64.h, tramp-ppc64.c, cpu-ppc64.md:
	Several	fixes.  PPC64 now passes most of the runtime regressions.
	
	

Followed by today's tweet:

The bootstrap means that the Mono JIT is actually doing a full build of Mono's compilers and class libraries and can be built on the target platform.

Update: Mark has posted a great picture of Jim Purbrick from Second Life, the man behind Mono running on Second Life.

The first Unity3D on Linux screenshot:

The above program was built on MacOS, the result copied to Linux and then executed using the LinuxPlayer. This is still very basic, the port is yet far from done.

I followed Joachim's advise and added a tiny script to update the cube on the screen. See the video of the cubes in action: ogg and wmv.

A couple of years ago I wrote an enthusiastic review of Brad Abrams and Krzysztof Cwalina's Framework Design Guidelines, a book that I absolutely love.

The book is a great compendium of best-practices for building software, traps and pitfalls to avoid.

But most importantly, it is the best source to learn the idioms and patterns used in the .NET Frameworks. Learning these idioms will have you writing code like the native C# speakers in no time.

I was incredibly honored when Brad asked me earlier this year to write the foreword for the second edition of the Framework Design Guidelines.

The second edition tracks the evolution of .NET and they apply as well to Mono. For instance, it now contains LINQ design patterns, extension methods patterns and DependencyProperties (used in WPF and Silverlight).

Last week we branched Moonlight for the 1.0 release, full with the licensed Microsoft Codecs and started our release process for Moonlight Beta 1 to be available in the next few days. This release is not yet published on our web site, watch this space.

The Moonlight engine team has now resumed our work on Moonlight 2.0, the version that will track Silverlight 2.0.

In the meantime, while the GUI team was busy completing 1.0, the Mono core team has been working on the security framework for Moonlight, the networking stack (Silverlight allows Socket connections using policy files) and web services (System.ServiceModel, a subset of WCF).

The security system is the trickier and is the one that has received the most attention. We started early on last year in to implementing this, as we knew it would end up burning a lot of cycles to get it right.

Our hero has posted the initial work partition for the upcoming GUI work on Moonlight 2.0.

Moonlight is a blast, and who knows, maybe with our static compilation engine we might be able to deliver Silverlight on the iPhone.

Update: Fixed some links, corrected some text.

Shawn Burke announced the Silverlight Toolkit and it is licensed under the open source MS-PL. The code is available here complete with unit tests (check Ning's blog on the unit testing framework).

With the Silverlight Toolkit they are taking a new approach to shipping new controls in an effort to move swiftly and deliver the controls to people at the right time. Their previous approach was to ship the Toolkit when every component was ready, and completely fleshed out.

Now they will be shipping the Toolkit with controls that might have different levels of quality (and they are clearly flagged in the documentation). Shawn explains the new "Quality Bands" model that they are using in his post.

You can try the components on the web. The charting control can be tried out with the ChartBuilder (check David's blog for details on the ChartBuilder):

The source code for the Toolkit and the Controls is great to learn how to use Silverlight and it is great for people that need to tweak them for their own applications. When it comes to these controls, you no longer need Microsoft to make small changes for you or the small bug fixes that impact your application.

Themes: An interesting control container in Silverlight is the theming control. You wrap your code around this, and it will let you skin your control with XAML and define the animations and interactions with XAML and the Visual State Manager:

Some of these themes reminded me of the Gtk+ themes from 1998. Back in the days of Enlightenment and the "Cheese Pixmap" theme were hot. Mehdi explains how the themes work and Jafar explains the ImplicitStyleManager, the foundation for themes.

Shawn's Talk

Shawn's talk at the PDC was very interesting. I did not get to see it during the conference, but I watched it in the comfort of home (wmv, mp4 and slide deck).

I wanted to thank everyone that helped get Barack Obama elected. Those that endorsed Obama passionately, those that videocasted, blogged, improved Obama's web site, donated to his campaign, wrote, discussed and voted on Tuesday to get him elected.

Barack does not only represent a change of direction for public policy, he is a truly brilliant candidate.

Some cool links on Barack:

• Larry Lessig: 20 minutes or so on why I am for Barack Obama.
• Tim O'Reilly's Why I Support Barack Obama.
• The Obama I know gives a glimpse of how Obama works.
• The Letter of Endorsement from 76 American Nobel Prize winners. winners.
• For the most cynics among us, you can watch the Chomsky on Obama interview which is as good as an endorsement as you can get from Noam.
• Beautiful pictures from the campaign trail from the Boston Globe.

I was surprised that the Obama campaign already launched their Change.Gov (thanks Nat) web site. You can now see how the team operates in real life, and you can share your story and you can share your vision of where America should go. The blog is here.

The above starts to deliver on the promise he had made during the campaign.

Got a cool collection of pictures about Obama or the reaction to the results? Please post it in the comments.

Inflamatory or misinformed comments will be deleted pronto.

Another nice piece of technology that we showed at the PDC was static compilation, the feature behind allowing Mono to run on the iPhone in a fully legit way (no jail-breaking):

Screenshot from the Unity IDE.

Although Mono has supported batch compilation from CIL into native code in the past there was a small amount of code that was still generated dynamically. The intention originally was to help reduce startup and increase code sharing across multiple processes.

Code sharing is important once you have a handful of Mono processes running on the system. Instead of having to JIT the same code once per Mono instance for say "System.Console.WriteLine", the code laid out in essentially a shared object.

Our generated code uses many of the concepts found on the ELF file format to ensure that we can share code and that the code pages are read-only and not written to. This means that methods are invoked through a program linkage table instead of directly (to cope with the shared libraries being loaded at different addresses).

The Extra Mile

Although we are not certified XBox360 developers yet (we have yet to find the right person at Microsoft to talk to) we know from some of our users that have ported Mono to the XBox360 that JITing is not an option on that platform.

The XBox360 seems to have the same security-imposed limitations that the iPhone has, it is not possible for a Just-in-Time compiler to run in the platform as either the license terms or the kernel do not allow writable pages to become executable pages.

During the last few months we developed a static compilation mode for Mono. First we did this for the 1.0 profile, and now we are working on the 2.0 profile (so that we can support static compilation of generics). The work to support the 2.0 profile is reusing Mark's work on generic code sharing, which I found out to be a very nice synergy of efforts internally.

This means that it is now possible compile code from CIL to native code and not even ship the JIT compiler in your final program (saving some precious kilobytes from the final executable).

To do this, you must:

• Use Mono 2.0.1 at least.
• Request that Mono performs a full AOT compilation by using: mono --aot=full program.exe. That will generate your static executable. This executable still needs the runtime for things like garbage collection, threading and other runtime services.
• You then run your executable with Mono: mono --full-aot program.exe
• Optionally: build a new Mono on a separate location that removes the JIT engine by configuring Mono like this: configure --enable-minimal=jit. This will reduce your deployment by a few hundred Ks as the code generation and JIT engines are stripped out.
• Optionally: build a smaller set of libraries by using the Mono Linker (this is the tool that we use for turning Mono's 3.5 APIs into the Silverlight 2.0 APIs).
• Optionally: strip out the CIL code from the assemblies. We still require the assemblies for their rich metadata, but the actual CIL instructions can be safely removed. The new cil-strip tool built on top of Mono.Cecil can further shrink your deployed executables.

Developers interested in trimming down Mono can look into our documentation for more features that can be removed by using the --enable-minimal option.

Of course, once you remove the JIT you will not be able to use any dynamically generated code. This means no Reflection.Emit dynamically and at least for the time being or no IronPython/IronRuby.

John Lam told me at the PDC that they are looking into bringing static compilation for IronPython/IronRuby/DLR back, so this might just be a short-lived limitation.

For those interested in using Mono on the iPhone today the situation is a bit painful right now. You must run Mono on the target system to do the batch compilation and send the data back to assembly it on the host before you send the code back to the iPhone to run.

If you are wondering how did the demo go so smoothly at the PDC, the reason is that I was using Unity. Unity modified their local copy of Mono to be hardwired to do cross compilation to that exact platform. A more general purpose solution is needed to allow arbitrary platform-to-platform cross compilation, and we hope that this will be available in the future.

If you must quench your thirst for C# on the iPhone today your best choice is to use Unity's product and start building games instead of the enterprise application you always dreamed of.

From the Unity's Video Sample

If your boss demands that line of application running on the iPhone, you have a perfect excuse to learn the Unity gaming APIs and deliver the most glorious multi-touch, 3D-transformed line of business application to ever grace this world full with spinning AI for your "Sort By Customer Last Name" button.