Microsoft is trying to succeed where web services failed before with their new Open Data Protocol (oData).

A few years ago, I had an epiphany when listening to Don Box talk about data services and web services. Web services and web sites are merely public facades to your internal databases. Web sites expose your database rendered as HTML while web services expose them in some sort of data format, defined on a case-by-case basis.

With Web Services, developers need to define what bits of information should be exposed and how it is exposed (this applies to both SOAP and REST-based services). The problem is that developers might not have the time to spend on exposing every possible bit of interesting information that you maintain in your databases. Most importantly, there are very few web services that provide server-side queries making the information harder to mine.

oData is a little bit different, for a given oData endpoint (like http://odata.netflix.com/Catalog/CatalogTitles) you can request that the server do some work for you, for example, you can pass a query that will limit the results to only those that you are interested in, and you can also specify the information that you are interested in.

For example:

• Movies released this year ($filter=ReleaseYear eq 2010).
• Movies released this year, that have bank in the synopsis ($filter=substringof('bank',Synopsis) and ReleaseYear eq 2010).

It seems like Microsoft is doing the right things to get oData to be adopted. It reused Atom as the exchange format, made the services REST-based, it made client libraries available for many developer platforms and placed the spec under the OSP.

Consuming oData sources

Microsoft is taking a very Google-y approach with oData. They have created client libraries for a wide range of platforms to encourage adoption of this new way of exposing data.

In addition to the JavaScript, Java, PHP, .NET and Objective-C bindings, Microsoft has announced that they will be open sourcing the .NET client library under the Apache 2 license. I was pleasantly surprised to learn that they were open sourcing this code, as we can consume it right away on Mono.n

So this is good news for everyone that wants to consume the information. The bindings for .NET are in particular great since you get strongly typed client-side code that you can use to invoke remote servers.

Producing oData: the need to open source System.Data.Services

This is trickier, and I believe that if Microsoft wants to bootstrap the oData universe they need to seed this space not only with some existing services but also with an open source OData server implementation for Unix.

I should further add that they should not wait a single minute and open their server-side .NET implementation if they want to accelerate the oData adoption.

Let me explain why.

Although the client libraries are great step to drive the adoption of the protocol for clients, it will do very little to unlock the data that sits today out in the web behind Linux servers running PHP, Java, Ruby or Python-based applications.

At the end of the day, the client side code is a relatively simple parser for an XML file format. The server side on the other hand is much more complicated to get right.

The server side requires a complete implementation of the query syntax, selection as well as access control and transaction support required to expose the data safely.

It is clearly possible to implement the oData server technology, IBM did it for WebSphere, but this is an unnecessary wait. Placing a standard under the OSP and documenting it is not enough to drive open source or even third party implementations.

If Microsoft were to open source their server side implementation of oData, we could overnight allow Linux users to expose their data in a format that can be mined. Linux users would only need to run a Mono front-end to the System.Data.Services library to expose the data that currently lives in their servers and is being served by Joomla, Wordpress, Rails, Django front-ends to become available as data services.

Witness what happened with the M modeling language: great idea, OSP-covered, and yet the steep work required to implement it means that not a single implementation has been created in the 18 months since the project went public.

Update: I stand corrected, Douglas Purdy points out that there is an open source implementation of M built with Javascript here.

Ok, so not the best example, risking another egg in my face, I want to say that chances are that jsmeta is not a complete implementation of M.

More on oData

A detailed overview is available on the oData site.

The best intro to oData that I have seen is Douglas Pourdy tour-de-force during the second day keynote as he walks through all the pieces in a span of 20 minutes.

The MIX sessions on oData are packed with great information. The transaction and access control requirements on the server-side support are discussed on the How to create a feed for that session.

If you want to explore oData services, you can use this Silverlight application to build your queries.

Laura and myself will become parents sometime this summer. This will be our first baby.

A friend told me that "Most kids turn ok, despite of their parents" which I interpreted as a green light for raising our daughter with a touch of Dali-like surrealism.

Also, as much as I love the Summer conference circuit, I will be skipping on most travel from May to October and spend some quality time nesting.

Mark Probst found and squashed one of the hardest bugs in our upcoming garbage collector. Pablo from Codice has been testing the new GC under stress and posted a nice message to the list.

Plenty of great feedback on deprecating old libraries and tools from Mono. We will have a lighter distribution. As things are coming together so fast and we are now well into the features we had planned for 3.0, we might end up just calling the next version 3.0 instead o 2.8.

Andreia got Moonlight running on Chrome.

At the core of the iPhone UI lies the UITableView, a powerful table rendering widget that is used by pretty much every application on the iPhone. The UITableView is a powerful widget that can render its data in many different ways based on how you configure the widget, these are all variations of the UITableView:

This is one powerful widget.

The contents of the UITableView are rendered by calling into a developer supplied set of routines that provide the data on demand. The protocol includes queries like "How many sections?", "How many rows in section N?", "What is the title for section N?" as well as callbacks to provide the actual contents of a cell. Although the widget is pretty powerful, creating UIs with it is a chore and a pain to maintain. The pain to maintain and the repetitive nature of the process leads to developers either spending too much time customizing each view, bare minimum configuration and lack of polish on certain configurations. As I ported many of the Objective-C samples to C# I found myself repeating the same process over and over.

My fingers developed calluses, and at night I kept thinking that there should be a better way. But at the time, I was just doing a line-by-line port, I was not really ready to build a new API on top of it.

Recently, when my favorite twitter client on the iPhone butchered its UI I figured I would write my own Twitter client. The first step was to create the account configuration for my twitter account. As you can imagine, the configuration is done with a variation of the UITableView. And once again I found myself setting up the model, responding to view events, sprinkling switch and if statements liberally all through my source code and just plainly not having any fun writing the code. This is how MonoTouch.Dialog was born.

I wanted to reflection to allow a class to be mapped to a dialog box, something that would allow me to write a C# class and have it mapped to a UITableView, something like this:

class TwitterConfig {
    [Section ("Account")]
      [Entry] string Username;
      [Password] string Password;

   [Section ("Settings")]
      bool AutoRefresh;
      bool AutoLoad;
      bool UseTwitterRetweet;
}

Instead of starting with the Reflection code to deal with this, I first created an in memory representation of the entire dialog. The idea was that the Reflection code would then be a bridge that could use the engine code.

The engine code is built on the idea that each row could be a specific kind of widget. It could contain text, a switch box, a text editing line, a slider control, a calendar picker or any other kind of user created control. I call these "Elements" and I created a bunch of these:

• BooleanElement: rendered with a UISwitch
• FloatElement: rendered with a slider.
• HtmlElement: when tapped, starts the web browser on a url.
• StringElement: used to render plain strings.
• MultilineElement: A multi-line string element.
• RadioElement: a radio-item, to select one of many choices.
• CheckboxElement: like the BooleanElement, but uses a checkbox instead of a UISwitch.
• ImageElement: Allows the user to pick an image or take a photo.
• EntryElement: text entry element.
• DateTimeElement, DateElement, TimeElement: pickers for dates and times, dates and times.

The MonoTouch.Dialog follows the Apple HIG for the iPhone and implements as much of the UI policy allowing me to focus on the actual application instead of focusing on the administrivia.

Although the UITableView is built on the powerful Model/View/Controller setup that would allow it to scale efficiently to large data sets, most configuration pages and data entry pages do not require this complexity.

Another feature is that it takes care of all the bookkeeping required to do text entry without any work from the programmer: accepting keyboard input, automatically switching to the next entry line on return, aligning all entry lines in a section, dismissing the keyboard with the last input is reached.

Here is a sample of the API in action:

var root = new RootElement ("Settings") {
  new Section (){
    new BooleanElement ("Airplane Mode", false),
  new RootElement ("Notifications", 0, 0) { Notifications }
  new Section (){
    new RootElement ("Sound"), { Sound },
    new RootElement ("Brightness"){ Brightness },
    new RootElement ("Wallpaper"){ Wallpaper }
  },
  new Section () {
    new EntryElement ("Login", "Your login name", "miguel"),
    new EntryElement ("Password", "Your password", "password", true),
    new DateElement ("Select Date", DateTime.Now),
    new TimeElement ("Select Time", DateTime.Now),
  }
}
	

Once the RootElement has been created, this can be passed to a DialogViewController to manage it:

var dv = new DialogViewController (root);
navigation.PushViewController (dv, true);

Reflection API

The reflection API inspects a class and looks for fields that have some special attributes on them.

Here is a sample class and how it is rendered:

class AccountInfo {
    [Section]
    public bool AirplaneMode;

    [Section ("Data Entry", "Your credentials")]

    [Entry ("Enter your login name")]
    public string Login;

    [Caption ("Password"), Password ("Enter your password")]
    public string passwd;

    [Section ("Travel options")]
    public SeatPreference preference;
}
	

As you can see, enumerations (SeatPreference) are automatically turned into a radio selection that uses the UINavigationController to navigate and the captions are computed from the field names, a behavior that you can customize with the [Caption] attribute.

The attributes that are attached to each instance variable control how things are rendered and can be used to specify a variety of attributes like captions, images and overriding the defaults from MonoTouch.Dialog.

Some Samples

Code:

new RootElement ("Settings") {
  new Section (){
    new BooleanElement ("Airplane Mode", false),
    new RootElement ("Notifications", 0, 0) { Notifications }
  new Section (){
    new RootElement ("Sound"), { Sound },
    new RootElement ("Brightness"){ Brightness },
    new RootElement ("Wallpaper"){ Wallpaper }
  },
  new Section () {
    new EntryElement ("Login", "Your login name", "miguel"),
    new EntryElement ("Password", "Your password", "password", true),
    new DateElement ("Select Date", DateTime.Now),
    new TimeElement ("Select Time", DateTime.Now),
  }
}
	

LINQ and MonoTouch.Dialog

Craig has written a great Conference application for the Mix 2010 conference. I helped him reduce the code size of the application by removing all of the repetitive code required to set up UITableViews for various pieces of the application with MonoTouch.Dialog. Since the conference application deals with a database schedule, I extended MonoTouch.Dialog to work better with LINQ.

In the same spirit of the System.Xml.Linq API that allows you to create XML documents with nested LINQ statements, you can use MonoTouch.Dialog to create your entire UIs.

For Craig's application, I wrote a SessionElement that allows sessions to be "starred" and shows both the title of the session as well as the location of the session.

This code constructs the entire UI for the "My Schedule" tab. The data is populated on demand (Apple recommends that all views are loaded lazily)

public class FavoritesViewController : DialogViewController {
  public FavoritesViewController () : base (null) { }

  public override void ViewWillAppear (bool animated)
  {
    var favs = AppDelegate.UserData.GetFavoriteCodes();
    Root = new RootElement ("Favorites") {
      from s in AppDelegate.ConferenceData.Sessions
        where favs.Contains(s.Code)
        group s by s.Start into g
        orderby g.Key
        select new Section (MakeCaption ("", g.Key)) {
          from hs in g
            select (Element) new SessionElement (hs)
        }
    };
  }
}
	

That is it. The entire source code.

So use any of the two models that you enjoy the most: Reflection for quick one-offs and quick data-entry or the Element API for more advanced user interfaces but without having to spend your life writing boilerplate code.

I hope that this helps guys spend more time improving their applications, and less time writing boilerplate code.

MonoTouch.Dialog is not perfect and does not have every possible bell and whistle that you might want to add. Although I do welcome patches for certain features, feel free to fork the code and patch at will to suit your needs.

We just landed the support on MonoDevelop's trunk to support developing applications that target MeeGo powered devices.

MeeGo is a Linux-based operating system designed to run on mobile computers, embedded systems, TVs and phones. Developers would not typically use a MeeGo device as a development platform, but as a deployment platform.

So it made sense for us to leverage the work that we have done in MonoDevelop to support the iPhone to support MeeGo. Unlike MonoTouch, we are not limited to running on a Mac, you can use MonoDevelop on Windows, Linux and OSX to target Meego Devices.

Developers would continue using their Linux Workstation, their Windows PC, or their Mac to develop and test and the resulting cross-platform binary can be deployed and debugged remotely over the wire using Mono's Soft Debugger.

In this video, I interview Michael Hutchinson as he demostrates the developer experience:

The MonoDevelop/Mono that we will be supporting is entirely Gtk# based, both during development as well as during deployment.

Every once in a while I need to debug some Mono program that does not come with a solution. Either a program that was compiled using a Makefile or an executable that I installed with RPM on my system.

Sometimes I would end up cretaing MonoDevelop solution that contained every source file, command line option and resource that I meticulously copied from the Makefile. In other words, I was in for a world of pain just to be able to use MonoDevelop's awesome debugger.

Lluis offered this solution, and I am blogging it hoping that it will help someone else in the future. Follow these steps to debug a Mono executable and set breakpoints on the source code or class libraries source code:

1. Create a Generic Solution

Select File/New Solution and select Generic Solution:

2. Open the Project Properties

Double click on your project, in my case I called the project "compiler":

3. Add an Execute Custom Command

In your project select Custom Commands:

Add a custom Execute command by selecting it from the "Select a Project Operation" option and point to your Mono executable:

4. Load source files

Use File/Open to load the source file where you want to set a breakpoint (the executable source or some class library source) and set your breakpoints:

Then use Run/Debug to start your program in debugging mode (Linux and Windows users can use F5, MacOS X users can use Command-Return).

Every once in a while I would run into someone that will ask me what exactly we are up to in Mono. As Mono becomes larger, and various big projects "land" into the trunk, we can no longer do releases on a monthly basis. Some of the work that we do is inherently very attractive, things like new features, new libraries, new UIs and new frameworks. But probably more important are the efforts to turn our code into programming system products: fixing bugs, testing it, packaging it, supporting it, writing documentation, test suites, improving error handling, scaling the software, making it faster, slimmer and backporting bug fixes.

I wanted to give my readers a little bit of an insight of the various things that we are doing at Novell in my team. This is just focused on the work that we do at Novell, and not on the work of the larger Mono community which is helping us fill in the blanks in many areas of Mono.

MonoDevelop work

We just released MonoDevelop 2.2, a major upgraded to our IDE story, and the backbone that allows developers on Linux to debug various kinds of Mono-based applications. With support for the new Soft debugging engine, it has allowed us to support debugging ASP.NET web sites, ASP.NET web services, Silverlight applications, Gtk# and Console applications with minimal effort. Since the soft debugger leverages Mono's JIT engine, porting the soft debugger to a new architecture is very simple.

MonoDevelop 2.2 major goal was to create a truly cross platform IDE for .NET applications. We are off to a solid start with Linux, Windows and OSX support as well as solid support for C#, VB, Vala and Python.

We are now turning our attention to MonoDevelop 2.4. This new release will incorporate many new UI touch ups which I will blog about separately.

MeeGo/Moblin Support

We have been working closely with the MeeGo (previously Moblin) team at Novell to offer a streamlined developer experience for developers on Windows, Mac and Linux to target MeeGo devices.

Developers will be able to develop, test and deploy from their favorite platform software for MeeGo devices.

Mono Service Pack

A service pack release of Mono's enterprise supported ASP.NET release is ahead of us and we will be upgrading Mono from release 2.4 to release 2.6.

This will bring to our customers support for all of the new features in Mono 2.6 with the added benefit that it has gone through four months of extra testing and polish.

As part of this effort we are also upgrading the MonoTools for Visual Studio to support the new Mono Soft Debugger.

Runtime Infrastructure

Mono's runtime is being upgraded in various ways: we continue to work on integrating LLVM [1], productize our new copying garbage collector that can compact the heap, and make Mono scale better on multi-core systems with the integration of ParallelFX into Mono as well as re-architecting thread management and thread pools in Mono.

A big upgrade for Mono 2.8 will be the support for obfuscated assemblies that insert junk in dead blocks. This is a feature that we never had, but with many Silverlight applications being deployed with these options we started work on this.

We are working to improve our support for F# and together with various groups at Microsoft we are working to improve Mono's compatibility with the CLR to run IronPython, IronRuby and F# flawlessly in Mono. Supporting F# will require some upgrades to the way that Mono works to effectively support tail call optimizations. [1] LLVM: better use LLVM to produce better code, use it in more places where the old JIT was still required and expand its use to be used for AOT code.

Mono for Mobile Devices

We recently shipped Mono for the iPhone and we continue to develop and improve that platform. Our goal is to provide developers with a great experience, so we are doing everything in our power to make sure that every wish and whim of the iPhone developer community is satisfied. We are working to expand our API coverage, write helper libraries to assist developers, tune existing .NET libraries to run on Mobile devices, reduce startup time, and reduce executable sizes.

But we have also just started an effort to ship MonoDroid: Mono for the Android platform. This will include a comprehensive binding to the Java APIs, but accessible through the JIT-compiled, 335-powered runtime engine.

Our vision is to allow developers to reuse their engine and business logic code across all mobile platforms and swapping out the user interface code for a platform-specific API. MonoTouch for iPhone devices and the Monodroid APIs for Android devices.

Head-less Tasks

A big part of Mono's effort is in the development kit: the compiler, the tools and the server side components.

Mono has now a complete C# 4.0 implementation that will be ready to ship with the next version of Mono. Anyone can try it today by building Mono from SVN. We are also porting our C# compiler to work with Microsoft's Reflection.Emit to enable us to run our C# Interactive Shell in Silverlight applications and to allow .NET developers to embed our compiler in their applications to support C# Eval.

Our MSBuild implementation is very robust these days, and it will be fully supported in Mono 2.8 (and we will be backporting it to Mono 2.6 as well).

On the ASP.NET front, we are working with third party vendors to certify that their controls work with Mono's ASP.NET (we will have some tasty announcements soon) and we are also catching up to the new features that are coming with .NET 4.0.

WCF has turned out to be one of the most requested features. We had historically only paid attention to WCF for its Silverlight/Moonlight use cases, but as time goes by, more and more users are moving to WCF. We are working on completing our WCF support.

On the ADO.NET front our major focus has been to complete the support for LINQ to SQL as part of the DbLinq project that we are contributing to. At this point we have no plans to implement Entity Frameworks due to the large scope of that project.

Moonlight 3

I do not need to say much about Moonlight 3. Moonlight 3 is one of our most visible projects right now due to the adoption of Silverlight and Smooth Streaming.

By the first week of Feburary there had been 610,000 downloads of Moonlight 2.0 for Linux. This is only counting the downloads since the official release on December.

Policy Changes

Mono 2.6 was the last release of Mono to support the .NET 1.0 API profile. With Mono 2.8 we are going to drop the class library support for 1.0 and ship both 3.5 and 4.0 assemblies.

With Mono 2.8 we are also switching the default tools and compiler to be 4.0-based as opposed to be based on the 3.5 profile. We will be doing a release of Mono 2.8 a couple of months after .NET 4.0 ships.

Ciao!

The above probably reflects the main focus of the team in the past three months. There are many community driven efforts that are very cool and that deserve their own space and a future blog post. Things like the amazing work that was done on Qyoto and the Synapse IM client come to mind.

We look forward to a great year ahead of us.

Everyone knows that in this day an age a software product is not complete until it offers a Desktop UI, a Web UI, and a front-end on the Appstore.

We access beautiful web sites, we purchase 0.99 apps on our phones and install gorgeous native software on our desktops.

The sysadmin community, once the backbone of Linux adoption, keeps asking "but what about me?". Indeed. What about them?

What are we doing about these heroes? The heroes that ssh in the middle of the night to a remote server to fix a database; The heroes that remove a chubby log file clogging the web server arteries; The very same heroes that restore a backup after we drag and dropped the /bin directory into the trashcan?

They are a rare breed in danger of extinction, unable to transition into a GUI/web world. Trapped in a vt100 where they are forced to by conditions to a small 80x24 window (or 474 by 188 with 6 pixel fonts on a 21 inch flat screen display).

These fragile creatures need our attention.

Today I am doing my part, my 25 cents to help improve their lives.

I am releasing Mono Curses 0.4 a toolkit to create text-based applications using C# or your favorite CLR language.

The combination of C#'s high-level features, Mono's libraries, Mono/.NET third party library ecosystem and the beautifully designed gui.cs, we can bring both hope and change to this community. Hope and change in the form of innovative text-based applications that run comfortably in 80x24 columns.

What is gui.cs

We know that hardcore sysadmins will want full control over what gets placed on the screen, so at the core of mono-curses we expose a C# curses binding.

On top of this, we provide a widget set called "gui.cs". gui.cs was introduced in 2007 enjoying unprecedented public acclaim among a circle of five friends of mine. Eight months after its introduction, it experienced an outstanding 100% growth when a second application was written using it.

Today, gui.cs is the cornerstone of great work-in-progress applications that any decade now will see the light of day. Including a new and riveting version of the Midnight Commander:

With only 3% of features implemented progress is clearly unstoppable!

I believe in dogfooding my own software before I unleash it to the world:

On a typical 21" sysadmin setup: 474x188 with the "Sysadmin Choice" award winning 6 pixel font.

Valentine's Day

So in this Valentine's Day, after you are tired of spending quality time with your grandmother, making out with your significant other or a stranger you just met at the checkout line in Costco, consider donating. Donate some of your time towards building some nice applications for your favorite sysadmin. God knows that he spent the whole day monitoring the dmesg output, looking for a SATA controller failure and keeping an eye on /var/log/secure, waiting for your ex to deface your wordpress installation.

And you have a choice, you can use Boo, IronRuby, IronPython, F# for building your app. VB.NET is also available if you want to teach your sysadmin a lesson in humility.

Get inspired today with some of the old screenshots.

The amazing Moonlight Team lead by Chris Toshok just released Preview 2 of Moonlight 3 just in time for the Winter Olympics' broadcast:

The player has some nice features like live streaming, Tivo-like "jump-back", accelerated playback and slow motion and it does this using Smooth Streaming which adjusts the quality of your video feed based on your bandwidth availability.

Thanks to Tom Taylor, Brian Goldfarb and the rest of the team at Microsoft for assisting us with test suites and early access to some of the technologies in use at NBC Olympics. With their help we were able to make sure that Moonlight 3 would work on time for the event (with full 24 hours and 14 minutes still to burn!).

As usual, the team did a great job, considering that we had to implement in record time plenty of Silverlight 3 features for Moonlight.

	MOONLIGHT_OVERRIDES=effects=no firefox
	

We have just released our first preview of Moonlight 3.0.

This release contains many updates to our 3.0 support, mostly on the infrastructure level necessary to support the rest of the features.

In the release:

• MP4 demuxer support. The demuxer is in place but there are no codecs for it yet (unless you build from source code and configure Moonlight to pick up the codecs from ffmpeg).
• Initial work on UI Virtualization.
• Platform Abstraction Layer: the Moonlight core is now separated from the windowing system engine. This should make it possible for developers to port Moonligh to other windowing/graphics systems that are not X11/Gtk+ centric.
• The new 3.0 Binding/BindingExpression support is in.
• Many updates to the 3.0 APIs

The above is in addition to some of the Silverlight 3.0 features that we shipped with Moonlight 2.0.

For the adventurous among you, our SVN version of Moonlight contains David Reveman's pixel shader support:

From Silverlight Parametric Pixel Shader.