Mono Migration Analyzer, Results

by Miguel de Icaza

Within 12 hours of announcing Moma we received 114 submissions generated by the tool detailing what features were missing.

There were a number of interesting discoveries, these are:

False Positives: One of the most commonly reported error were method calls and property accesses to the System.Net.WebRequest class. These turned out to be incorrect, since this class is an abstract class that happens to have stubbed all those methods and throws NotImplementedExceptions as a mechanism to ensure that implementors get those methods right.

Developers will always be using a subclass that is created by the WebRequest.Create factory method (It makes me wonder why the methods were not flagged as abstract)

New or overwritten methods: In a number of places, overwritten methods are showing up in the report. For example, a very common one is Exception.GetType(), or with Label::set_Text (the Label.Text setter), in both cases the problem is caused by 2.0 introducing a new method that overrides the base definition.

The code, if compiled with Mono would have produced the correct result, but the binaries would not have worked as the member references were done to newer classes. Most of these are innocuous, and we already introduced the missing APIs for the most commonly reported problems.

Outdated MonoTODOs: We have plenty of miss-used MonoTODOs, these are the messages that we used internally to flag that something in a method is incomplete, and ideally, it should provide an explanation of why the flag was set.

But this is an attribute that we have used a bit recklessly over the past few years, and has had a number of meanings, from valid uses like "This method only implements half the specification" to less useful instances like "I do not like this API", to "Someone should optimize this routine" to "One day I would like to look improve this". So plenty of MonoTODOs were hints aimed at the source code maintainer, and not really to the developer that consumes the API.

Eliminating all of the bogus MonoTODOs is an ongoing process, but we might still get a few erroneous reports for a while that might confuse developers.

In particular, a lot of people hit on various methods in DataSet. None of those errors are correct, they are implemented, but they had old "MonoTODO" flags left behind.

Limitations in the .NET API: In a number of cases, developers have resorted to P/Invoke into the Win32 API, and the results are very interesting, the top reasons for using P/Invoke have been:

  • Message dispatching: SendMessage, PostMessage.
  • GDI/DC operations: GetDC, CreateBrush, CreateRgn, SelectObject, DrawText.
  • Window operations: SetWindowPos, SetForegroundWindow ShowWindow, GetWindowRect, ClientToScreen, MoveWindow and Hook handling.
  • Allocation: GlobalAlloc
  • Fonts and Printing Dialogs.
  • Shell access: to get recently used files, icons for files and a handful of others.

There are of course many more, but these are the majority of the uses.

Some of those APIs are easily translated into portable calls (Message dispatching, window operations, allocation) while others are very hard (GDI/DC operations) and some others are better if replaced as a chunk with alternative code.

Our next goal will be to provide a portability assembly that will take care of those operations that are easily translated, so P/Invoke calls would be replaced with calls into this portability assembly. On Windows, we will continue to P/Invoke the same methods, while on other platforms we would route the request through Mono's internals.

In the case of Shell access, the best thing to do is to provide a cross-platform API that does the integration with the user shell, and provide an API that developers can migrate to. This would be effectively a "complement" for APIs that are today not available for Windows.Forms and even be a lot simpler for many people to use than resorting to P/Invoking.

Lack of Comments: The first version of Moma lacked support for entering some comments from the person submitting the report, so all we got in our hands are the list of unimplemented methods, but we do not know much about the applications that are using it.

When we ran this over applications we had, we were able to identify pieces that can be replaced as units based on the class names where they were used (this is part of the report that people get, but not part of the information transmitted to us). For example, a commercial application that we want to port (with the assistance of the owners) has neatly isolated things like printing dialogs in its own file.

Cecil and Obfuscated Assemblies: Some of the applications that were processed were obfuscated. But the P/Invoke signatures also seem obfuscated, am not sure how those are actually resolved at runtime.

Moma's version: at some point, we will need to enable Moma to download new API definitions from our site, to allow developers to check back if their applications are ready without having to download a new copy of Moma.

Accounting: There are a number of limitations on the accounting and statistics that can be done with the results.

It is hard to tell with precision how many of these are .NET 1.1 vs .NET 2.0, which ones are C# and VB and which ones are ASP.NET vs Windows.Forms. This is caused because Moma only uploads lists of missing methods, it does not upload other information that might be confidential, so Moma erred on the side of safety and removed data that was not necessary for us.

The accounting for ASP.NET is further limited, since Moma does work on assemblies, and not on .aspx files. Those that used ASP.NET are most likely helper code that is used in ASP.NET applications.

Another challenge with the data is that each submission contains the report for all the assemblies submitted. The end user might have chosen a single program, a single project (made up of many executables and libraries) or multiple programs.

This is not a problem, but it is worth keeping in mind when you read the results below.

Results

From the submitted applications, ten of them work out of the box without any changes; Another 30 contained reports that could be ignored (MonoTODOs that should not have been there, the NotImplementeds that did not exist). Almost all of them contain calls to methods that are either easy to implement, or were already implemented.

At least 29 of the applications are 2.0-based (by looking at Label::set_Text and Exception::GetType) and 16 of them are VisualBasic applications. One of the applications embeds IKVM.

32 applications were ASP.NET applications; 56 used Windows.Forms; 32 use System.Data and 44 used System.Drawing (see the above caveats about interpreting this data; This is based on assemblies and submissions, not on projects).

P/Invoke usage, from the 114 applications submitted:

  • 67 do not contain any P/Invoke calls.
  • 9 contain between 1 and 3 P/Invoke calls.
  • 10 contain between 4 and 10 P/Invoke calls.
  • 9 contain between 11 and 20 P/Invoke calls.
  • 5 contain between 22 and 48 P/Invoke calls.
  • 6 contain between 55 and 100 P/Invoke calls.
  • 5 contain between 102 and 180 P/Invoke calls.
  • 3 container more than 200 P/Invoke calls
    In the three cases, about half of the P/Invokes are to their own libraries.

(details).

So roughly half of the .NET applications can port without any concerns about P/Invoke. The remaining 40% are easily portable (those that have less than 48 P/Invokes) another 5% would take a few weeks, and maybe a Linux/Mono expert to assist on the port. The last 5% will require some large refactoring to work on Linux.

The application that uses the most P/Invoke that we have received so far seems to be some kind of designer and also happens to use the System.Security.CodeAccessPermission::Assert a lot.

Update: Sebastien comments that this is most likely a case of badly designed software, it is intended to avoid lots of stack walks to check for permissions. The above was probably copy/pasted from some recipe as an optimization that was useful in the 1.x frameworks. It is not required for 2.0.

From the received applications, 15 of them need more work on VisualBasic. Although most of the Visual Basic applications reported NotImplementedExceptions for a number of their methods in CompilerServices, that turned out to be false positives, which leaves us with only 24 methods missing in the VB.NET runtime

Unsupported Technologies

There are a few features that we do not support, and do not plan to support, in Mono. These technologies are either being phased out (EnterpriseServices, System.Messaging, to be replaced by WCF/Olive), or they would require a lot of work that we are currently not planning on doing (COM).

EnterpriseServicesis was used only by two applications, and all they seem to miss is a call to ServicedComponent::Dispose(). I suspect these are generated by a tool, considering the lack of any other methods references (and that we know are flagged).

System.Messaging is another of the libraries that we have not implemented and it showed up in only three applications.

COM, showed up in three applications.

Update

By the time this post went up this morning, we had received 171 submissions.

Posted on 28 Nov 2006


MonoDevelop 1.0 Planning

by Miguel de Icaza

Lluis Sánchez has posted his tentative release plan for MonoDevelop 1.0.

Posted on 27 Nov 2006


Mono Migration Analyzer 1.0

by Miguel de Icaza

Update: Moma 1.0 reports some false positives, see the post here for some explanations.

Jonathan Pobst wrote a very nice tool, the Mono Migration Analyzer (Moma) a tool that can be used by Windows .NET developers to evaluate whether their software will need any modifications to run on the Mono supported platforms, here is the welcome screen:

Moma works by analyzing all the instructions that the code contains and all the references to external types and assemblies (.exe and .dll files). The analysis is done using Cecil, a library used to examine, introspect and manipulate ECMA CIL images.

The first step is to select the assemblies to analyze:

During the analysis, Moma will look for any references to methods, fields, properties or events that the program does on external assemblies and report any of the following problems:

  • Using un-implemented classes, methods or accessing missing properties or fields.
    This would happen for example if a piece of functionality that developers expect is not yet available on Mono (most likely, 2.0 features).
  • Calls to methods, references to fields or types that have been flagged with the special "MonoTODO" attribute.
    Class libraries in Mono, when incomplete, are flagged with an attribute, like this:
    	[MonoTODO ("In Mono this is controlled by the --share-code flag")]
    	public LoaderOptimization LoaderOptimization {
    		// implementation details.
    	}
    		

    If your program uses the above property, Moma will provide a warning with the above text.
  • P/Invoke invocations. P/Invoke has two uses: to call into your own unmanaged libraries, or to call into Win32 APIs that are not exposed in the .NET API.
    Moma will report all uses of P/Invoke (uses, not declarations; A lot of people include a lot of declarations that they never use).
  • Any methods in Mono whose signature exists, but that throws a NotImplementedException when called.

After analyzing your code, Moma will then present a summary of the issues found in your application:

The report is a summary of the issues identified by Moma. You can get a detailed report of issues by clicking on "View Detailed Report". This new report is what you will be using when you go through your application making sure that your code is portable, the report looks like this:

Finally, you can help us prioritize which classes and features are most important, you can do so by submitting the report to our team:

Early Experiences

I used Moma in a couple of applications from third party vendors that we are interested in bringing to Linux. The results so far are very promising as we are not missing that much.

The majority of the problems so far have been in sections of code that P/Invoke to get to some features not available in the framework, things like preferences, recently used documents, CPU detection and a handful of features to get information about windows, items on the screen and a few more.

To help port those applications we will be creating a portability assembly that will provide an API that Windows developers can call and will provide the proper native equivalent on Linux, Windows and OSX.

Getting Moma

You can download Moma 1.0.1 from here.

Update New version is here.

Posted on 27 Nov 2006


More on Mexico's Election Fraud

by Miguel de Icaza

My father has a follow-up to his previous studies where he estimates the number of fake votes introduced in the July election. The programs to run the study at home are here.

The interesting result is that statistically 3,700,000 were inserted into the election results (to match the actual published results), with the majority of the inserted votes going to the party that officially won (by a less than 220,000 votes).

Posted on 24 Nov 2006


Compiz Roadmap

by Miguel de Icaza

David has posted a roadmap for Compiz. It is all about the core engine, not much about plugins.

It also seems like a new repository for plugins will be created.

Also, he released a new version of Compiz, it is much smoother than the Compiz I have installed on my laptop.

Posted on 23 Nov 2006


Perl/Linux

by Miguel de Icaza

A Linux distribution that contains only 3 binaries compiled with the C compiler: the kernel, libc and perl.

Someone needs to do a Mono/Linux distribution and follow the steps of these guys ;-)

Posted on 23 Nov 2006


Gaza

by Miguel de Icaza

Uri Avneri on the ongoing massacre in Gaza.

Gideon Levy:

"Gaza threatens to become Chechnya. There are thousands of wounded, disabled and shell-shocked people in Gaza, unable to receive any treatment. Those on respirators are liable to die due to the frequent power outages since Israel bombed the power plant. Tens of thousands of children suffer from existential anxiety, while their parents are unable to provide help. They are witnesses to sights that even Gaza's old-timers have never seen before".

Posted on 22 Nov 2006


Update: Mono on the Nokia 770

by Miguel de Icaza

Paolo has blogged his updates on running Mono on the Nokia 770, with the techincal details about the port.

Posted on 20 Nov 2006


Top 10 reasons against DRM

by Miguel de Icaza

From Reddit, Top 10 Reasons against DRM.

And a nice new discovery Defective by Design.Org.

Posted on 19 Nov 2006


AjaxPro in Mono

by Miguel de Icaza

Michael Schwarz has a tutorial for Windows users that are using the Mono VMware image on how to run their ASP.NET and AjaxPro applications on Mono.

The tutorial is a step-by-step tutorial on how to move your files from Windows to Linux and accessing it from there.

At the end of the article, he focuses on running the application with XSP, I left a comment on how to run it with Apache:

Running xsp in a console is similar to running Microsoft "Cassini" server, it is a small web server, and it is not very powerful.

The VMWare image comes with both the small server, and Apache integration (which would be equivalent to running your ASP.NET with IIS on the Windows world).

To run your applications with Apache, instead of xsp, just put your whole ASP.NET application in the /srv/www/htdocs directory.

To go there, just open the file manager, type Control-L, that will open the location bar, and enter /srv/www/htdocs

Copy the files there, and you are done, say you the directory MyWeb20App in /srv/www/htdocs, now you can browse to that location by going to:

http://localhost/MyWeb20App

Posted on 18 Nov 2006


« Newer entries | Older entries »