Open Source Contribution Etiquette

by Miguel de Icaza

Some developers, when faced with fixing, or adding a feature to an open source project are under the mistaken impression that the first step before any fixing takes place, or before adding a new feature takes place is to make the code "easier for them" to work on.

"Easier for them" usually is a combination of renaming methods, fields, properties, locals; Refactoring of methods, classes; Gratuitous split of code in different files, or merging of code into a single file; Reorganization by alphabetical order, or functional order, or grouping functions closer to each other, or having helper methods first, or helper methods last. Changing indentation, aligning variables, or parameters or dozen other smaller changes.

This is not how you contribute to an open source project.

When you contribute fixes or new features to an open source project you should use the existing coding style, the existing coding patterns and stick by the active maintainer's choice for his code organization.

The maintainer is in for the long-haul, and has been working on this code for longer than you have. Chances are, he will keep doing this even after you have long moved into your next project.

Sending a maintainer a patch, or a pull request that consists of your "fix" mixed with a dozen renames, refactoring changes, variable renames, method renames, file splitting, layout changing code is not really a contribution, it is home work.

The maintainer now has to look at your mess of a patch and extract the actual improvement, wasting precious time that could have gone to something else. This sometimes negates the effort of your "contribution".

If you really have an urge to refactor the code, first of all, discuss the changes with the maintainer with the rationale for the changes. If the maintainer agrees with the changes, make sure that you keep your refactoring and changes independent from code fixes, it makes reviewing the code a lot simpler.

The alternative, to keep your fork, is usually a guarantee that your effort will be wasted, and wont help other users. People have tried to do this. It is attempted every year, by hunders of developers who in tbe back of their minds are thinking "I can do better" and "I wont make the same mistakes". After 18 years doing open source I can probably think of a handful of project forks that have survived and flourished. Out of hundreds of such failures. So the odds are not good.

So respect the original coding style, and if you want to make refactoring changes, discuss this with the maintainer.

Posted on 31 Dec 2010


For your OOXML Conspiracy Theories

by Miguel de Icaza

The staff at Groklaw has never really tolerated any dissent when it came to OOXML. They spent years advocating against OOXML only to have OOXML emerge not only stronger, but also with an ISO stamp of approval.

Today they tried to insinuate that my involvement and opinion on OOXML was somehow the result of the 2006 Microsoft/Novell agreement.

Their conspiracy theory falls apart as our active involvement on OOXML goes back to the year 2005, 11 months before there is any agreement between Microsoft and Novell.

My interest in the file format interop problem goes back to 1998-1999 when I wrote the Gnumeric spreadsheet and both Michael and his brother started contributing a plugin for reading and writing Excel files.

My involvement with OOXML started in 2005, when Jack Messman was still Novell's CEO and the company was in the middle of various legal disputes with Microsoft. Not the best environment for collaboration between companies.

ECMA was starting a new working group to look into standardizing OOXML, and since we were already members of ECMA (as part of our work on C# and CLI) I recommended that we should participate in the effort to come up with solid documentation that we sorely needed for improving OpenOffice's interoperability story. ECMA had been great in particular for the CLI, in large part thanks to Sam Ruby at IBM who pushed for the file format to be specified (the original drafts did not document the actual assembly file format, only the instruction set).

In 2005, one of our major goals was to make Linux suitable for enterprise desktop deployments, and interoperability with Microsoft protocols and file formats was key to this strategy. We were the major contributors to OpenOffice outside of Sun (and perhaps still are) and what mattered to us was to get a good spec we could use to fix customer issues that prevented us from deploying the Linux desktop to enterprise customers.

For years, we had been reverse engineering Word and Excel. This was our chance of getting important information on the file formats. Our work in this area today benefits every OpenOffice and Gnumeric users.

So we attended and participated in the ECMA OOXML meetings starting with the initial meeting on December 15th. I blogged about this publicly on November 2005, and so did Novell which on the same month Novell blogged about our participation on the Open Invention Network (we were founding members) and was actively promoting OpenOffice.. The minutes of this meeting and every other meeting ever since are available to all ECMA members.

It was our team that pushed to get the entire Formula Spec in OOXML back in 2006 (those 700 pages of formula specs, the ones that actually make spreadsheets work) as well as filing piles of issues as we prototyped the work with Gnumeric and OpenOffice. And all of this happened before any Novell-Microsoft agreement.

My involvement after setting up the initial participation was superficial, as the actual hackers working on OpenOffice and Gnumeric took over (Jody Goldberg and Michael Meeks). These were important years for Mono, and that is where my energy has been going since about 2002.

The 2006 agreement with Microsoft did not impact much of my work, despite Mono being something where interop could really be helped.

The ECMA work on OOXML brought hackers together and allowed our teams to interact as people looking for some shared goals instead of interacting as foes. In my experience face-to-face meetings, like the ECMA working groups, help smooth out human relationships that might have been poisoned by preconceived biases.

But the interop agreement certainly allowed other collaborations and meetings in other areas with Microsoft, for example, it lead to various components used by Mono to become open source, and to our Moonlight/Silverlight collaboration.

So two full months into having signed an agreement with Microsoft, I wrote my first pro-OOXML post, largely based on a news report that I felt was misguided. We had been working on this at this point for a year, and clearly people with no actual office experience had already formed an opinion.

In retrospect, had I known that double standards, hypocrisy and character assassination would become the tool of choice of the anti-OOXML crowd I would not have said a thing.

The energy that went into stopping OOXML could have been better used in actually completing the formula spec for ODF, which almost four years later is still not part of the ISO spec. In the eyes of the ISO world, it remains an "implementation specific" work. But "advocacy" is a little bit like watching the TV, it is relatively easy. While actually working on improving open source, or open standards is equivalent to going to work. It requires skills, time and longs hours of difficult work (particularly if you are working on the OpenOffice code base).

As for the March agreement of 2010, it is absolutely brilliant. Microsoft is funding our OpenOffice team to develop open source code that will improve the OOXML import and export capabilities and we help drive the OOXML standard forward based on the experience that we will gain from doing this work.

If you do not like us doing this work, there is an easy solution for you: do not open or save files in OOXML format with OpenOffice.

See what I did there Trevor? I found out what bothered you emotionally, decomposed the problem, and BAM! I provided you with a solution. It is called teamwork, Trevor.

Michael Meeks from our OpenOffice team provides more color as he was the one actively working on this.

Posted on 21 Dec 2010


Brazil

by Miguel de Icaza

Tomorrow we are flying to Porto Alegre in Brazil to spend two weeks in a nice, warm climate tasting delicious foods and hanging out with good friends.

If you are a Porto Alegre-based Linux-ista, Mono-ista, Android-ista, MacOS-hacker-ista, .NET-ista, C#-ista, or, iPhonista and would like to get together for coffee, lunch or dinner, drop me an email.

Posted on 16 Dec 2010


MonoReports: Report Designer and Reporting Engine

by Miguel de Icaza

This is quite cool, Tomasz Kubacki has released a report engine and designer that runs on Mono:

MonoReports has a nice Gtk# based GUI designer that runs on Linux, Mac and Windows and can generate reports that you can later run GUI-less.

From Tomasz announcement:

  • Simple layouting - if control in section is growable and will grow due to assigned data, engine will do layouting to make report look properly
  • Page breaking - Monoreports engine will break or keep together report sections whatever is needed
  • Generating and running reports from designer and code.
  • PDF export
  • Reporting engine is not tightly coupled with gtk/cairo stuff, therefore it's reasonably easy to write new export backends (e.g. html, or xls for example).

He also made a six minute video walkthrough of MonoReports's features.

Posted on 09 Dec 2010


Mono: What we are Cooking

by Miguel de Icaza

Although everything we do is public in some form or another, folks that are not keeping track of things might be wondering what the Mono team at Novell has been up to.

Here are some of the projects that we have been working on, and that we expect to release in the next three months, some sooner, some later. You are welcome to join us in testing, all you need is to get comfortable building Mono from git.

Bundling F#, IronPython, IronRuby and UnityScript: Now that all of these languages are open source, we want more people to use them, and we want to remove any friction that there might be in getting started. So we are doing a push to provide packages on Linux and bundle with our OSX installer all of them.

The first three languages are from Microsoft, the third one is from Unity, and it is a strongly-typed and class-based incarnation of JavaScript. The lightweight syntax of Javascript, with the components necessary to produce optimal native code.

Getting F# to build and run on Mono was a challenge on its own. This is not a language that many of us were familiar with, but we are now at a point where things are baked. We should be uploading our modified version of F# to the fsharp organization on GitHub.

MonoDevelop's Git support: MonoDevelop 2.6 will come with Git support. Originally Lluis prototyped this by calling out to the system Git, but this is not very elegant and also hard to make reliable and work smoothly across Windows, Linux and MacOS. So Lluis used db4object's tool to convert Java source code into C# source code to bring Eclipse's jGit into Mono as NGit. Now MonoDevelop has a full managed Git implementation that works the same on Windows, Linux and MacOS.

MonoDevelop's Online Template System: To help developers get up and running with any kind of interesting .NET project, we are going to make MonoDevelop use an online gallery system and we are going to open it up for contributions.

New Profiler: our new profiler is a complete new implementation that obsoletes the old logging profiler, the old heap-shot profiler and the old heap-buddy profiler and our old statistical profiler into a single profiler that does it all, does it better, and does it well.

This new profiler was already used to pinpoint bugs and performance problems in our own web server, our Parallel Frameworks and inspired Alan to write a leak detector for Moonlight.

MonoDevelop's Profiler GUI: The companion to our new profiler. Currently it only has a CPU profiling mode, but in the future we will add a GUI for memory profiling as well.

Mono on Android: we are very close to shipping Mono on Android. The experience right now is very close to what we want.

This has taken longer than we anticipated, but mostly because we are providing a full binding to the Android APIs. Not just a subset. This is made possible by Google publishing the Android API contract in XML form.

WCF: Our WCF implementation so far has been the Silverlight client profiler and its mirror on the server. This is clearly not sufficient for many of our users, so we have redoubled our efforts to fill in all the gaps in WCF. Mono 2.10 will have better coverage for WCF, but it wont be complete. That is still some ways off.

MonoDevelop support for MonoDroid: Currently our entire toolchain is Visual Studio based, just because we figured this is where the majority of developers would be. We are hard at work to bring the same experience to MonoDevelop users on MacOS and Linux.

Upgrade MonoTouch to Mono 2.8: Currently our MonoTouch product is based on our older Mono 2.6 release, and there are too many features in Mono 2.8 that users want.

One feature in particular that we are working on is the use of LLVM's optimizing compiler for Mono on the iPhone. As MonoTouch is entirely batch-compiled and has no JIT, we can use the LLVM backend of Mono instead of our codegen backend to generate smaller and faster code.

And this is only weeks after we pushed our jumbo 3.2.2 release.

Deploy Cecil/Light: Cecil is JB's library that we use every time we need to process ECMA CIL images. Debuggers, compilers, assembly browsers, linkers, mergers, spliters, injectors, decompilers and many other tools in Mono are built with Cecil.

JB has revamped Cecil to be a lot lighter memory-wise, and has improved its API. This new version has been already deployed on most of our master trees in GitHub and will be available to everyone on the next iteration.

C# Compiler to use IKVM Engine: Our C# compiler has historically used System.Reflection.Emit as its code generation engine. We are adding a second backend to the compiler, this time Jeroen Frijters's IKVM.Reflection engine.

This will eliminate the need to have our compilers "bound" to an mscorlib profile. This means that we will no longer need various hacks that we have in place to build special compilers to target special profiles of Mono. Just one compiler for all API profiles.

VB Compiler using Cecil Engine: Rolf updated the Visual Basic compiler to use Cecil as its backend engine.

Upgrade our Online API documentation: a very much needed upgrade to our documentation system is ongoing, based on Jonathan's Kipunji ASP.NET MVC documentation engine and Jackson's hard work to productize it. We are going to move our documentation to this new system. You can preview our new documentation system here: MonoMac's Documentation Test Site.

Hopefully, this time, we will make our documentation editable on the web.

Moonlight GPU Acceleration and Perspective 3D transforms: this project has been underway for a little more than a year and I blogged about the GPU Acceleration recently.

GPU acceleration is based on the fine work from Gallium, and also has made Moonlight's engine more reusable as a general purpose accelerated framework.

Moonlight's RichText control: this is basically a word-processor on a widget. And who better to work on this than Chris Toshok who originally wrote XWord for Unix almost 13 years ago and which lead to the creation of LessTif.

Moonlight Test Suite: We are up to 47.77% of the Silverlight 4 test suite passing. This is a really big number considering that we only started work on Silverlight 4 features very recently. We have been making progress at about 3% every week.

Moonlight Beta: as the stars of codecs, tests and MS-PL code align, we are getting ready to do the first public beta of Moonlight 4 early next year.

Moonlight Platform Abstraction Layer: We did a proof of concept port of Moonlight to Android and OSX to improve Moonlight's Platform Abstraction Layer. It should now be possible to reuse the Moonlight engine in other platforms and operating systems. Think of this as a "porting kit". You get the source, get it running elsewhere and if it breaks, you get to keep both pieces.

MonoMac: we just released MonoMac 0.4 and we are now adding support for creating self-contained application bundles that developers can redistribute themselves or even submit to the Apple AppStore for MacOS.

MonoMac has been a lot of work, mostly, due to popular contributions. There is a nice emerging community of new contributors to Mono that has sparked in the last few weeks.

Garbage Collector Performance Tuning: with Mono 2.8 we shipped our new copying and precise collector and we made it easy for users to try it out (mono --with-gc=sgen). With the information we have collected, we are now improving the reliability and performance of the collector.

Sgen already helped us get up to 30% performance boosts on ASP.NET workloads, and the new results are nothing short of amazing. We can not wait to share this with the world (or you can try it out today by building Mono from master).

System.Xaml: The venerable desktop version of the XAML parser is coming to Mono in our next release.

Precise Stack Scanning for SGen: we have also added support for precise stack scanning that will be useful in a few workloads.

One surprising thing that we found out, and this will come as a shocker to many --it certainly was for me--, precise stack scanning over conservative scanning requires a lot of extra memory to keep stack maps. Unless you have some particular need to use precise stack scanning, you will be better off memory-wise and CPU-wise using conservative stack scanning.

Concurrent GC with SGen: as you can guess, SGen has been great for Mono, and more concurrent configurations of GC scanning will be supported in our next release. Some were disabled in 2.8, some are new.

Moonlight and DeepZoom: we had a decent DeepZoom implementation, but now we have a fabulous one. And by fabulous I mean, we pass Microsoft's tests :-)

Intellisense for Android's markup: Atsushi created a DTD for us to bundle with MonoDroid to let users at least edit their Android UI XML with auto-complete in Visual Studio.

ParallelFX: our parallel FX in Mono 2.8 was the first time that we distributed it. We have been working on improving its performance, distribution and balancing algorithms.

Update: both the ParallelExtra samples gallery and Microsoft Biology Foundation are now runnable.

Monodevelop Addins: Lluis launched the Beta for MonoDevelop's Addins system. You can use this to publish your MonoDevelop add-ins to your users, maintain multiple versions of them and get cute statistics on your add-in.

This is the app-store of MonoDevelop addins, except they are all free. For example, this shows that in the last 7 days, 144 users installed the MonoMac addin:

And that is what has kept us busy since the releases of Mono 2.8, Moonlight 2, MonoDevelop 2.4 and MonoTouch 3.2.

January and February will be busy months for us as we release betas for Moonlight 3/4, MonoDevelop 2.6, Mono 2.10 and the official launch of MonoDroid 1.0.

Posted on 09 Dec 2010


Gtk# designer on OSX

by Miguel de Icaza

Happy to see that Gtk+ and Gtk# on MacOS are complete enough that the MonoDevelop GUI designer can be used there to build UIs.

In the past, we had to use Linux to do all of our GUI design. Now our Gtk+ GUI design on OSX is self-hosting:

You need the latest Mono and MonoDevelop to get this working.

Posted on 08 Dec 2010


Mono Introspect: Binding GObject-based APIs for use in Mono

by Miguel de Icaza

Alan McGovern, the hacker behind the amazing Moonlight GC tracking device has started work on a tool to bind the new Gtk+ 3.0-based APIs that use GObject instrospection for Mono consumption.

Check out his project hosted in Github's mono-introspect module.

Posted on 06 Dec 2010


Beautiful Hack: Using Mono's Profiler to find Hard Memory Leaks

by Miguel de Icaza

Alan McGovern of MonoTorrent, Moonlight and Mono Introspect fame has written a blog post explaining how he used the new Mono Profiling interface to write a custom memory leak detector for Moonlight.

His post is a step-by-step document on how he created a new loadable profiling module that the Mono runtime uses. He then registers for listening to profiling events for the GC roots (MONO_PROFILE_GC_ROOTS) and then tracks the GC handle use.

This is how he found a difficult memory leak involving Mono's VM, the browser Javascript VM and the C++ code that backed every Moonlight object.

Hard core hacking reading.

Posted on 06 Dec 2010