A Night at the Movies

by Miguel de Icaza

One of the advantages of going to the movies in Boston is that the experience is enhanced by comments from expert graduates and undergraduates from the many famous universities in town.

We recently went to see the new James Bond movie, "Casino Royale" at one of Boston's largest movie theaters. We arrived with plenty of time to spare, and we managed to get some great seats: middle, center-front.

There were four or five Harvard students sitting behind us. Properly dressed, well groomed, and with impeccable haircuts. On our left, an MIT student with his date. He reminded me of the comic guy from the Simpsons.

MIT guy provided some valuable data throughout the movie.

When the first bullet was fired and the character dies, he solemnly informed his date "That is unrealistic" and he gave some details to back up his assertion "a bullet fired in that direction and speed would not push the body in that angle, let alone throw a guy from a chair".

This was a premonition of the things to come. He had found a mission, a goal worth standing up for. During the rest of the movie he kept us informed about which parts were realistic and which ones were not. For the next two hours and a half, we were treated to a string of assessments "realistic" and "not realistic".

Every once in a while he would also comment on the technological developments in this new Bond movie, "that technology actually exists.". He complements it with, "they are actually different". I was hoping he would say to his date at some point "That myth has been busted", but I waited in vain.

I probably could have learned more from this MIT fellow about poker and the probabilities in the game, but the movie was distracting me from his fascinating lecture on the math and strategy behind poker.

The Harvard guys were definitely more social, more outgoing. And they also shared with the rest of us their movie evaluations.

Although they mostly focused on the various Bond girls. Unlike the MIT guy that went into longer discussions and technical explorations, they limited their comments to short sentences, they had developed an advanced taste for succinct statements: "Yeah Baby" was a favorite, a few "I want some of that" and sometimes they just used a few guttural sounds.

They also shared with us their jet-setting background as they addressed the local culture "That is totally Venice".

If I had to guess, I would guess they were not English Majors.

What I found interesting is that the well-groomed Harvard folks giggled every time a girl was on screen. If the scene contained some erotic material, love declarations or kissing, the giggling usually turned into chatter. I learned a lot from them. Four out of five would "totally do her" and one of them would "Also quit the job for her", perhaps he was a secret agent wearing a Harvard sweat shirt yearning for a better undercover assignment.

Posted on 06 Dec 2006


Mono Migration Analysis Tool: Updated.

by Miguel de Icaza

An updated version of the Mono Migration Analysis has been released, you can download this from here.

This new release from Jonathan Pobst has some important updates:

  • It tracks the API of Mono 1.2.2 (just released today).
  • It will automatically update to new API releases as we make them, so you can keep an eye on the progress we make as we publish new versions of Mono without having to update your Moma installation.
  • It optionally allows you to send us feedback and information about the software you are analyzing: you can tell us more about your product, your target and the time frame that you are thinking of to port to Linux.

In the last week since the release of Moma, we have received 524 application submissions, from 299 different IP addresses (196 of them were single submissions, 48 were two submissions, 28 were three and a few folks sent more than that).

Posted on 05 Dec 2006


OpenOffice Forks?

by Miguel de Icaza

Groklaw is running a sensationalistic headline today:

Novell "Forking" OpenOffice.org

Well, if there are any Novell supporters left, here's something else to put in your pipe and smoke it. Novell is forking OpenOffice.org.

There will be a Novell edition of OpenOffice.org and it will support Microsoft OpenXML. (The default will be ODF, they claim, but note that the subheading mentions OpenXML instead.) I am guessing this will be the only OpenOffice.org covered by the "patent agreement" with Microsoft. You think?

Facts barely matter when they get in the way of a good smear. The comments over at Groklaw are interesting, in that they explore new levels of ignorance.

Let me explain.

We have been working on OpenOffice.Org for longer than anyone else has. We were some of the earliest contributors to OpenOffice, and we are the largest external contributor to actual code to OpenOffice than anyone else.

We have for years maintained go-ooo.org (as well as its previous incarnations) a site where we encouraged new developers to join the OpenOffice effort, and worked to lower the barrier for contributors by creating tutorials, pre-compiled images and provide tools for contributors to work on it (some of this content is now being migrated to OpenOffice's new Wiki system).

For years we have been shipping a patched version of OpenOffice because the release schedule of OpenOffice did not match our release schedule. In the very same way that Linux distributions have to ship patches against vanilla packages because the release schedule of those packages does not necessarily match the release schedule of a distribution.

The work at go-ooo.org started in the Ximian days, when we were an independent startup, and we did quite a lot of work to make OpenOffice better integrate with the Linux desktop, upgrading its aging pieces and did quite some work on improving its performance.

Our patches have been published in here (see for example) for the longest time. And plenty of them have already been merged upstream.

But technically, Ximian never shipped a vanilla OpenOffice, we always shipped an improved version of it (with bug fixes, with backports or new features). This is nothing new.

Today we ship modified versions of OpenOffice to integrate GStreamer, 64-bit fixes, integrate with the GNOME and KDE file choosers, add SVG importing support, add OpenDMA support, add VBA support, integrate Mono, integrate fontconfig, fix bugs, improve performance and a myriad of others. The above url contains some of the patches that are pending, but like every other open source project, we have published all of those patches as part of the src.rpm files that we shipped, and those patches have eventually ended up in every distribution under the sun.

But the problem of course is not improving OpenOffice, the problem is improving OpenOffice in ways that PJ disapproves of. Improving OpenOffice to support an XML format created by Microsoft is tantamount to treason.

And of course, the code that we write to interop with Office XML is covered by the Microsoft Open Specification Promise (Update: this is a public patent agreement, this has nothing to do with the Microsoft/Novell agreement, and is available to anyone; If you still want to email me, read the previous link, and read it twice before hitting the send button).

I would reply to each individual point from PJ, but she either has not grasped how open source is actually delivered to people or she is using this as a rallying cry to advance her own ideological position on ODF vs OfficeXML.

Debating the technical merits of one of those might be interesting, but they are both standards that are here to stay, so from an adoption and support standpoint they are a no-brainer to me. The ideological argument on the other hand is a discussion as interesting as watching water boil. Am myself surprised at the spasms and epileptic seizures that folks are having over this.

Btw, I believe the translator that people are discussing is built with C# and XSLT and is available here. I wonder some of the posters on the Groklaw thread are going to have a stroke over the fact that the software is hosted at source forge.

Posted on 04 Dec 2006


Gulev: Free Software Conference in Cancun, Mexico

by Miguel de Icaza

Mexico is in a turmoil over the questionable elections this year.

Fighting erupted among the representatives over the new appointed president. The new president was sworn-in in a rush, the international guests barely could sit before they had to whisk him out. Link

The upside is that what is typically an incredibly boring TV broadcast that goes for a few hours was reduced to less than five minutes.

I will be talking at the GULEV conference in Cancun next weekend. The GULEV conference has been running for a few years and it has always been incredibly fun. It is usually hosted in the port city of Veracruz.

The last time I attended, my friend Arturo had just broken the screen of his powerbook. He traveled to Veracruz with an external monitor, underwear and a toothbrush.

Posted on 02 Dec 2006


Pato y Mancha

by Miguel de Icaza

Posted on 01 Dec 2006


Top 1619

by Miguel de Icaza

There are 1,619 API calls that folks have reported that are either flagged with a [MonoTODO] attribute or are not implemented in Mono yet.

The list was compiled out of the 244 Moma submissions available as of 8pm.

A lot of these are 2.0 calls. The API lists are available here:

I will be updating those urls with API updates as we move along.

Posted on 28 Nov 2006


Mono Migration Analyzer, Results

by Miguel de Icaza

Within 12 hours of announcing Moma we received 114 submissions generated by the tool detailing what features were missing.

There were a number of interesting discoveries, these are:

False Positives: One of the most commonly reported error were method calls and property accesses to the System.Net.WebRequest class. These turned out to be incorrect, since this class is an abstract class that happens to have stubbed all those methods and throws NotImplementedExceptions as a mechanism to ensure that implementors get those methods right.

Developers will always be using a subclass that is created by the WebRequest.Create factory method (It makes me wonder why the methods were not flagged as abstract)

New or overwritten methods: In a number of places, overwritten methods are showing up in the report. For example, a very common one is Exception.GetType(), or with Label::set_Text (the Label.Text setter), in both cases the problem is caused by 2.0 introducing a new method that overrides the base definition.

The code, if compiled with Mono would have produced the correct result, but the binaries would not have worked as the member references were done to newer classes. Most of these are innocuous, and we already introduced the missing APIs for the most commonly reported problems.

Outdated MonoTODOs: We have plenty of miss-used MonoTODOs, these are the messages that we used internally to flag that something in a method is incomplete, and ideally, it should provide an explanation of why the flag was set.

But this is an attribute that we have used a bit recklessly over the past few years, and has had a number of meanings, from valid uses like "This method only implements half the specification" to less useful instances like "I do not like this API", to "Someone should optimize this routine" to "One day I would like to look improve this". So plenty of MonoTODOs were hints aimed at the source code maintainer, and not really to the developer that consumes the API.

Eliminating all of the bogus MonoTODOs is an ongoing process, but we might still get a few erroneous reports for a while that might confuse developers.

In particular, a lot of people hit on various methods in DataSet. None of those errors are correct, they are implemented, but they had old "MonoTODO" flags left behind.

Limitations in the .NET API: In a number of cases, developers have resorted to P/Invoke into the Win32 API, and the results are very interesting, the top reasons for using P/Invoke have been:

  • Message dispatching: SendMessage, PostMessage.
  • GDI/DC operations: GetDC, CreateBrush, CreateRgn, SelectObject, DrawText.
  • Window operations: SetWindowPos, SetForegroundWindow ShowWindow, GetWindowRect, ClientToScreen, MoveWindow and Hook handling.
  • Allocation: GlobalAlloc
  • Fonts and Printing Dialogs.
  • Shell access: to get recently used files, icons for files and a handful of others.

There are of course many more, but these are the majority of the uses.

Some of those APIs are easily translated into portable calls (Message dispatching, window operations, allocation) while others are very hard (GDI/DC operations) and some others are better if replaced as a chunk with alternative code.

Our next goal will be to provide a portability assembly that will take care of those operations that are easily translated, so P/Invoke calls would be replaced with calls into this portability assembly. On Windows, we will continue to P/Invoke the same methods, while on other platforms we would route the request through Mono's internals.

In the case of Shell access, the best thing to do is to provide a cross-platform API that does the integration with the user shell, and provide an API that developers can migrate to. This would be effectively a "complement" for APIs that are today not available for Windows.Forms and even be a lot simpler for many people to use than resorting to P/Invoking.

Lack of Comments: The first version of Moma lacked support for entering some comments from the person submitting the report, so all we got in our hands are the list of unimplemented methods, but we do not know much about the applications that are using it.

When we ran this over applications we had, we were able to identify pieces that can be replaced as units based on the class names where they were used (this is part of the report that people get, but not part of the information transmitted to us). For example, a commercial application that we want to port (with the assistance of the owners) has neatly isolated things like printing dialogs in its own file.

Cecil and Obfuscated Assemblies: Some of the applications that were processed were obfuscated. But the P/Invoke signatures also seem obfuscated, am not sure how those are actually resolved at runtime.

Moma's version: at some point, we will need to enable Moma to download new API definitions from our site, to allow developers to check back if their applications are ready without having to download a new copy of Moma.

Accounting: There are a number of limitations on the accounting and statistics that can be done with the results.

It is hard to tell with precision how many of these are .NET 1.1 vs .NET 2.0, which ones are C# and VB and which ones are ASP.NET vs Windows.Forms. This is caused because Moma only uploads lists of missing methods, it does not upload other information that might be confidential, so Moma erred on the side of safety and removed data that was not necessary for us.

The accounting for ASP.NET is further limited, since Moma does work on assemblies, and not on .aspx files. Those that used ASP.NET are most likely helper code that is used in ASP.NET applications.

Another challenge with the data is that each submission contains the report for all the assemblies submitted. The end user might have chosen a single program, a single project (made up of many executables and libraries) or multiple programs.

This is not a problem, but it is worth keeping in mind when you read the results below.

Results

From the submitted applications, ten of them work out of the box without any changes; Another 30 contained reports that could be ignored (MonoTODOs that should not have been there, the NotImplementeds that did not exist). Almost all of them contain calls to methods that are either easy to implement, or were already implemented.

At least 29 of the applications are 2.0-based (by looking at Label::set_Text and Exception::GetType) and 16 of them are VisualBasic applications. One of the applications embeds IKVM.

32 applications were ASP.NET applications; 56 used Windows.Forms; 32 use System.Data and 44 used System.Drawing (see the above caveats about interpreting this data; This is based on assemblies and submissions, not on projects).

P/Invoke usage, from the 114 applications submitted:

  • 67 do not contain any P/Invoke calls.
  • 9 contain between 1 and 3 P/Invoke calls.
  • 10 contain between 4 and 10 P/Invoke calls.
  • 9 contain between 11 and 20 P/Invoke calls.
  • 5 contain between 22 and 48 P/Invoke calls.
  • 6 contain between 55 and 100 P/Invoke calls.
  • 5 contain between 102 and 180 P/Invoke calls.
  • 3 container more than 200 P/Invoke calls
    In the three cases, about half of the P/Invokes are to their own libraries.

(details).

So roughly half of the .NET applications can port without any concerns about P/Invoke. The remaining 40% are easily portable (those that have less than 48 P/Invokes) another 5% would take a few weeks, and maybe a Linux/Mono expert to assist on the port. The last 5% will require some large refactoring to work on Linux.

The application that uses the most P/Invoke that we have received so far seems to be some kind of designer and also happens to use the System.Security.CodeAccessPermission::Assert a lot.

Update: Sebastien comments that this is most likely a case of badly designed software, it is intended to avoid lots of stack walks to check for permissions. The above was probably copy/pasted from some recipe as an optimization that was useful in the 1.x frameworks. It is not required for 2.0.

From the received applications, 15 of them need more work on VisualBasic. Although most of the Visual Basic applications reported NotImplementedExceptions for a number of their methods in CompilerServices, that turned out to be false positives, which leaves us with only 24 methods missing in the VB.NET runtime

Unsupported Technologies

There are a few features that we do not support, and do not plan to support, in Mono. These technologies are either being phased out (EnterpriseServices, System.Messaging, to be replaced by WCF/Olive), or they would require a lot of work that we are currently not planning on doing (COM).

EnterpriseServicesis was used only by two applications, and all they seem to miss is a call to ServicedComponent::Dispose(). I suspect these are generated by a tool, considering the lack of any other methods references (and that we know are flagged).

System.Messaging is another of the libraries that we have not implemented and it showed up in only three applications.

COM, showed up in three applications.

Update

By the time this post went up this morning, we had received 171 submissions.

Posted on 28 Nov 2006


MonoDevelop 1.0 Planning

by Miguel de Icaza

Lluis Sánchez has posted his tentative release plan for MonoDevelop 1.0.

Posted on 27 Nov 2006


Mono Migration Analyzer 1.0

by Miguel de Icaza

Update: Moma 1.0 reports some false positives, see the post here for some explanations.

Jonathan Pobst wrote a very nice tool, the Mono Migration Analyzer (Moma) a tool that can be used by Windows .NET developers to evaluate whether their software will need any modifications to run on the Mono supported platforms, here is the welcome screen:

Moma works by analyzing all the instructions that the code contains and all the references to external types and assemblies (.exe and .dll files). The analysis is done using Cecil, a library used to examine, introspect and manipulate ECMA CIL images.

The first step is to select the assemblies to analyze:

During the analysis, Moma will look for any references to methods, fields, properties or events that the program does on external assemblies and report any of the following problems:

  • Using un-implemented classes, methods or accessing missing properties or fields.
    This would happen for example if a piece of functionality that developers expect is not yet available on Mono (most likely, 2.0 features).
  • Calls to methods, references to fields or types that have been flagged with the special "MonoTODO" attribute.
    Class libraries in Mono, when incomplete, are flagged with an attribute, like this:
    	[MonoTODO ("In Mono this is controlled by the --share-code flag")]
    	public LoaderOptimization LoaderOptimization {
    		// implementation details.
    	}
    		

    If your program uses the above property, Moma will provide a warning with the above text.
  • P/Invoke invocations. P/Invoke has two uses: to call into your own unmanaged libraries, or to call into Win32 APIs that are not exposed in the .NET API.
    Moma will report all uses of P/Invoke (uses, not declarations; A lot of people include a lot of declarations that they never use).
  • Any methods in Mono whose signature exists, but that throws a NotImplementedException when called.

After analyzing your code, Moma will then present a summary of the issues found in your application:

The report is a summary of the issues identified by Moma. You can get a detailed report of issues by clicking on "View Detailed Report". This new report is what you will be using when you go through your application making sure that your code is portable, the report looks like this:

Finally, you can help us prioritize which classes and features are most important, you can do so by submitting the report to our team:

Early Experiences

I used Moma in a couple of applications from third party vendors that we are interested in bringing to Linux. The results so far are very promising as we are not missing that much.

The majority of the problems so far have been in sections of code that P/Invoke to get to some features not available in the framework, things like preferences, recently used documents, CPU detection and a handful of features to get information about windows, items on the screen and a few more.

To help port those applications we will be creating a portability assembly that will provide an API that Windows developers can call and will provide the proper native equivalent on Linux, Windows and OSX.

Getting Moma

You can download Moma 1.0.1 from here.

Update New version is here.

Posted on 27 Nov 2006


More on Mexico's Election Fraud

by Miguel de Icaza

My father has a follow-up to his previous studies where he estimates the number of fake votes introduced in the July election. The programs to run the study at home are here.

The interesting result is that statistically 3,700,000 were inserted into the election results (to match the actual published results), with the majority of the inserted votes going to the party that officially won (by a less than 220,000 votes).

Posted on 24 Nov 2006


« Newer entries | Older entries »