Pre-Mix 08: Moonlight Updates

We have been hard at work in Moonlight and I keep postponing when to blog some updates about our work every few days hoping to cram some new feature in the blog post.

And every few days turn into every few weeks and the next thing you know what should have been published a month ago ends up being my pre-Mix post.

Some of the most important changes in the last two months have been:

Moonlight Media Pipeline: So far we had been using ffmpeg's pipeline to process media. This means that ffmpeg was in charge of detecting the media formats, locating the codecs, demultiplexing the data, and decoding the data into video and audio frames.

The ffmpeg pipeline was fine as a sample pipeline, but we needed a few more things. We needed more control over the pipeline to implement things like media streaming and supporting seek operations over HTTP; We want to be able to relicense the code under non-LGPL terms (for people that can not use the LGPL or some commercial uses) and we need to plug Microsoft's Media Pack media decoders.

Rolf rewrote our multimedia pipeline so that the code on SVN no longer depends on ffmpeg's pipeline and only uses ffmpeg's decoders for video and audio:

The end goal is to plug Microsoft's Media Pack (this contains the codecs video and audio) into Moonlight:

The installers that we currently distribute for Linux do not contain the ffmpeg media codecs. If you need to test media you will need to compile the code from source.

Streaming: With our new pipeline we are now able to stream media. In the past Moonlight downloaded the media file into the local file system, and only when the entire file was downloaded we would start the playback.

Moonlight can now start playback as soon as there is enough buffered data. This works for media that is only available over HTTP. Fernando has been working on the support for media that supports the MMS-over-HTTP (controlling seeking for example). Luckily, the specs for HTTP-based streaming to Windows Media servers were released last week (also the specs for the ASF container format, which should help validate our implementation).

Test Tools: As part of our collaboration with Microsoft we received various tools that are used to test Silverlight. We could not reuse the tools directly for Moonlight (too many Win32-isms, too hard to plug into Mozilla/Linux) so Jackson rewrote all of that code for Linux.

The test tools include plugins that expose Javascript APIs to control Silverlight and the browser, simulate user events and take snapshots of the screen. Microsoft has given us permission to open source our implementation of these tools.

This should help us in creating our own tests, and in checking our own code for regressions beyond the test suites that we will have received from them.

The code should hit SVN soon.

Mozilla Installers: We created some installers for Mozilla that should offer a one-click install experience for users on Linux.

Verifier: Rodrigo continues to work on Mono's CIL verifier and on strengthening the image loader, work that will not be used immediately for our Silverlight 1.0 support, but that will be important for 2.0 (when Mono is required).

Historically Mono had not been used to execute untrusted code as we mostly were a runtime for desktop and server applications and errors in the images fell directly in the "doctor it hurts when I do this" category.

But with Silverlight 2.0 the story changes, Mono will be executing potentially hostile code so this work is mandatory. Additionally, Sebastien has been improving Gendarme and his Monoxide tool to help in the audit process and we have created a security audit plan for the runtime. This work, like the verifier work is a work in progress.

In addition to using it for Silverlight, the verifier work will enable the SecondLife folks to allow the execution of binary code in their simulators instead of being limiting SecondLife to use LSL-based scripts.

Windowless Support: As it turns out Windowless rendering is used by many Silverlight applications. This is where Silverlight content seamlessly blends with the HTMl content. This is a must-have for things like Tafiti, the XamlPad, and the Vista simulator and nice-to-have for some other sites.

The good news is that Chris has implemented the support for it:

The bad news is that this is limited to Firefox 3.0, the 2.0 edition does not have support for Windowless rendering.

See Chris' blog entry for the details about the challenges and limitations of Moonlight/Windowless in Firefox/Linux.

Bug weeks: With the test suite running on Linux we have been focusing on passing all the Microsoft tests that we can and implementing the major features missing (like Windowless support).

Silverlight 2.0 Other than the JIT support for Silvelright 2.0 at this point we have not done any work on it (well there are 3 classes stubbed privately).

There are two reasons for this: the updated 2.0 API is not public and although we have access to it, it is a bit of a mess to try to keep two separate trees (public and private) to support this and since Mix is just around the corner, we will just wait until next week.

The second reason is that we want to focus on shipping 1.0, completing the media pack integration and working on the configuration aspects of Moonlight (auto-update configuration for instance).

Our Priorities

Everyone on the team has been working very hard to get all the pieces in place, and we are getting closer to completion. Every time we fix a bug, it has a nice ripple effect of fixing various other issues that we had seen in some sites at once.

Our priority currently is to ship Moonlight 1.0, and this means more or less:

• Integrate the Microsoft Media Pack.
• Complete our media pipeline.
• Fix all the bugs exposed by the test suites.
• Investigate some of the problems with some video card/driver configurations that seem to burn a lot of cycles.
• Improve the performance for heavily animated scenes.

Am excited about the Mix conference, looking forward to see the demos and what gets announced. And obviously very excited about my own session on Thursday to talk about Moonlight and Mono.

Time to go to sleep. Marek and myself are on a 7am flight and it will be hard to make it.