Usability Disaster Story

In December, someone asked me about how many Mono downloads we had per month to estimate the size of Mono users. With software like Mono the download numbers do not mean much, because most of our users get their software through their distribution, package channels or as a bundled executable.

Since we are short on space on the machine hosting the files, we do not keep logs for too long, they are rotated and discarded pretty quickly, but we could at least do some guesswork based on a week worth of downloads. The data would not have been very useful, as this story takes place on December 27th, but I was still curious.

What I discovered with horror as I looked through the logs was that people using Windows and MacOS were downloading software that was two years old. Mono 1.1.7 (released sometime in May of 2005) was the most popular download.

Only 5% of the Mac downloads were actually getting the latest version, 95% was downloading this two year old version. I don't have my notes handy for Windows, but they were similarly abysmal [Update: found the notes, they were 95% as well].

The Linux downloads for RPMs were mostly fine (probably because they had to select the architecture from a separate list). But a whole 1/3rd of the people downloading the Mono "Universal Installer" were getting the wrong Mono as well (Update: clarified).

Using the referrer logs, we tracked down a few problems to people that had linked for their software directly to a particular version of Mono, but this could not explain the popularity of this aging Mono 1.1.7 release, and the logs were very odd, people apparently were selecting this from our list of options.

This made no sense.

Can you spot the problem?

I clicked on our links, they were fine. They downloaded the proper software, the logs matched the download. I was puzzled.

Only after a while I realized what was going on:

Since we are using MediaWiki for our web site and we were adding cute little icons for each platform, clicking on the image actually took users to the page for the image. In the Windows case, this was the page:

	http://www.mono-project.com/Image:Mono_icon_windows.gif
	

Which happens to contain the image, and some automatically generated content for the image. Some of the generated content lists all of pages that *link* to the image:

Now, that looks like a messy page, but most users were picking the largest number they found listed there, which happened to be an old legacy page of ours for the release "1.1.7". Going to this page, would be a very similar download page to the main one, and people at this point figured out "do not click on the image, click on the text". Except now they were clicking on the downloader for this ancient release.

I have no idea for how long this has been going on, but this is a colossal disaster.

We came up with a quick fix, luckily, you can replace the markup [[Image:Blah.png]] in MediaWiki with [url] and if the url points to an image, it will insert an unclickable image into the page.

But the quick fix could not address another problem: we probably had the worst download page in the history of human kind. It was incredibly bad, and you can still see it on the archives.

In the past few weeks, we wrote the beginnings of a new download page. Users will now be directed to this page, you can see it here. Thanks to the whole team in Provo that put together this new system and the folks in x-m-l that provided feedback on it.

There is still some work left to do on it, but the entry page is no longer the disaster that it was. Luckily since the correctly on December 27th or so, people have finally been downloading our latest Mono.

We of course feel terrible to all of the 95% of the users that wanted to try Mono on Windows and the Mac and ended up using a two year old version of Mono.