Miguel de Icaza

Need for Exercises

miguel@gnome.org (Miguel de Icaza) — Thu, 25 Apr 2013 16:21:00 -0500

For many years, I have learned various subjects (mostly programming related, like languages and frameworks) purely by reading a book, blog posts or tutorials on the subjects, and maybe doing a few samples.

In recent years, I "learned" new programming languages by reading books on the subject. And I have noticed an interesting phenomenon: when having a choice between using these languages in a day-to-day basis or using another language I am already comfortable with, I go for the language I am comfortable with. This, despite my inner desire to use the hot new thing, or try out new ways of solving problems.

I believe the reason this is happening is that most of the texts I have read that introduce these languages are written by hackers and not by teachers.

What I mean by this is that these books are great at describing and exposing every feature of the language and have some clever examples shown to you, but none of these actually force you to write code in the language.

Compare this to Scheme and the book "Structure and Interpretation of Computer Programs". That book is designed with teaching in mind, so at the end of every section where a new concept has been introduced, the authors have a series of exercises specifically tailored to use the knowledge that you just gained and put it to use. Anyone that reads that book and does the exercises is going to be a guaranteed solid Scheme programmer, and will know more about computing than from reading any other book.

In contrast, the experience of reading a modern computing book from most of the high-tech publishers is very different. Most of the books being published do not have an educator reviewing the material, at best they have an editor that will fix your English and reorder some material and make sure the proper text is italicized and your samples are monospaced.

When you finish a chapter in a modern computing book, there are no exercises to try. When you finish it, your choices are to either take a break by checking some blogs or keep marching in a quest to collect more facts on the next chapter.

During this process, while you amass a bunch of information, at some neurological level, you have not really mastered the subject, nor gained the skills that you wanted. You have merely collected a bunch of trivia which most likely you will only put to use in an internet discussion forum.

What books involving an educator will do is include exercises that have been tailored to use the concepts that you just learned. When you come to this break, instead of drifting to the internet you can sit down and try to put your new knowledge to use.

Well developed exercises are an application of the psychology of Flow ensuring that the exercise matches the skills that you have developed and they guide you through a path that keeps you in an emotional state ranging that includes control, arousement and joy (flow).

Anecdote Time

Back in 1988 when I first got the first edition of the "C++ Language", there were a couple of very simple exercises in the first chapter that took me a long time to get right and they both proved very educational.

The first exercises was "Compile Hello World". You might think, that is an easy one, I am going to skip that. But I had decided that I was going to do each and every single of one of the exercises in the book, no matter how simple. So if the exercise said "Build Hello World", I would build Hello World, even if I was already seasoned assembly language programmer.

It turned out that getting "Hello World" to build and run was very educational. I was using the Zortech C++ compiler on DOS back, and getting a build turned out to be almost impossible. I could not get the application to build, I got some obscure error and no way to fix it.

It took me days to figure out that I had the Microsoft linker in my path before the Zortech Linker, which caused the build to fail with the obscure error. An important lesson right there.

On Error Messages

The second exercise that I struggled with was a simple class. The simple class was missing a semicolon at the end. But unlike modern compilers, the Zortech C++ compiler at the time error message was less than useful. It took a long time to spot the missing semicolon, because I was not paying close enough attention.

Doing these exercises trains your mind to recognize that "useless error message gobble gobble" actually means "you are missing a semicolon at the end of your class".

More recently, I learned in this same hard way that the F# error message "The value or constructor 'foo' is not defined" really means "You forgot to use 'rec' in your let", as in:

let foo x =
   if x == 1
     1
   else
     foo (x-1)

That is a subject for another post, but the F# error message should tell me what I did wrong at a language level, as opposed to explaining to me why the compiler is unable to figure things out in its internal processing of the matter.

Plea to book authors

Nowadays we are cranking books left and right to explain new technologies, but rarely do these books get the input from teachers and professional pedagogues. So we end up accumulating a lot of information, we sound lucid at cocktail parties and might even engage in a pointless engineering debate over features we barely master. But we have not learned.

Coming up with the ideas to try out what you have just learned is difficult. As you think of things that you could do, you quickly find that you are missing knowledge (discussed in further chapters) or your ideas are not that interesting. In my case, my mind drifts into solving other problems, and I go back to what I know best.

Please, build exercises into your books. Work with teachers to find the exercises that match the material just exposed and help us get in the zone of Flow.

Introducing MigCoin

miguel@gnome.org (Miguel de Icaza) — Sat, 13 Apr 2013 03:29:00 -0500

Non-government controlled currency systems are now in vogue. Currencies that are not controlled by some government that might devalue your preciously earned pesos at the blink of an eye.

BitCoin is powered by powerful cryptography and math to ensure a truly digital currency. But it poses significant downsides, for one, governments can track your every move, and every transaction is stored on each bitcoin, making it difficult to prevent a tax audit in the future by The Man.

Today, I am introducing an alternative currency system that both keeps the anonymity of your transactions, and is even more secure than the crypto mumbo jumbo of bitcoins.

Today, I am introducing the MigCoin.

Like bitcoins, various MigCoins will be minted over time, to cope with the creation of value in the world.

Like bitcoins, the supply of MigCoins will be limited and will eventually plateau. Like bitcoin, the MigCoin is immune to the will of some Big Government bureaucrat that wants to control the markets by printing or removing money from circulation. Just like this:

Projected number of Bitcoins and MigCoins over time.

Unlike bitcoins, I am standing by them and I am not hiding behind a false name.

Like BitCoins, MigCoins come with a powerful authentication system that can be used to verify their authenticity. Unlike BitCoins, they do not suffer from this attached "log" that Big Brother and the Tax Man can use to come knocking on your door one day.

How does this genius of a currency work? How can you guarantee that governments or rogue entities wont print their own MigCoins?

The answer is simple my friends.

MigCoins are made of my DNA material.

Specifically, spit.

Every morning, when I wake up, for as long as I remain alive, I will spit on a glass. A machine will take the minimum amount of spit necessary to lay down on a microscope slide, and this is how MigCoins are minted.

Then, you guys send me checks, and I send you the microscope slides with my spit.

To accept MigCoins payments all you have to do is carry a DNA sequencer with you, put the microscope slide on it, press a button, and BAM! 10 minutes later you have your currency validated.

To help accelerate the adoption of MigCoins, I will be offering bundles of MigCoins with the Ilumina MiSeq Personal DNA sequencer:

Some might argue that the machine alone is 125,000 dollars and validating one MigCoin is going to set me back 750 dollars.

Three words my friends: Economy of Scale.

We are going to need a few of you to put some extra pesos early on to get the prices to the DNA machines down.

Early Adopters of MigCoins

I will partner with visionaries like these to get the first few thousands sequencers built and start to get the prices down. Then we will hire that guy ex-Apple guy that was CEO of JC Penney to get his know-how on getting the prices of these puppies down.

Like Bitcoin, I expect to see a lot of nay-sayers and haters. People that will point out flaws on this system. But you know what?

The pace of innovation can not be held back by old-school economists that "don't get it" and pundits on CNN trying to make a quick buck. Hater are going to hate. 'nuff said.

Next week, I will be launching MigXchange, a place where you can trade your hard BitCoins for slabs of spit.

Join the revolution! Get your spit on!

Exclusive! What we know about the Facebook Phone

miguel@gnome.org (Miguel de Icaza) — Fri, 29 Mar 2013 16:54:00 -0500

We obtained some confidential information about the upcoming Facebook Phone. Here is what we know about it so far:

The FacebookPhone will be free (no contract) but will pause your call every 30 seconds to play an ad for 20
— Miguel de Icaza (@migueldeicaza) March 29, 2013

Everyone will get an FacebookPhone to use as a honeypot trap for unwanted calls.
— Miguel de Icaza (@migueldeicaza) March 29, 2013

@migueldeicaza it will also force a call to a person you haven't talked to in 10 years for every 5 calls you make.
— Jonathan Chambers (@jon_cham) March 29, 2013

@migueldeicaza it'll use the ambient light sensor to detect when your not listening to the ad on your phone and just restart it
— Alex Trott (@AlexTrott_) March 29, 2013

@migueldeicaza and ad calls cannot be muted, refused and the disconnect button will be suspended for the period of the call!...
— Sumit Maitra (@sumitkm) March 29, 2013

@migueldeicaza the proximity sensor will be used to switch between earphone and speakerphone if it detects you've put the phone down!
— Sumit Maitra (@sumitkm) March 29, 2013

@migueldeicaza Before you're able to call, you must finish a game of Farmville. When done, it asks "Do you wish to call *related friend*?"
— Marco Kuiper (@marcofolio) March 29, 2013

The FacebookPhone will charge you 100 dollars to dial people who you have not friended
— Miguel de Icaza (@migueldeicaza) March 29, 2013

@migueldeicaza After the call, a transcript and recording will be posted to your timeline, with the other party tagged.
— Chris Howie (@cdhowie) March 29, 2013

@migueldeicaza it will have a single hardware button -'Like'
— Martin Topping (@eMartinTopping) March 29, 2013

The FacebookPhone has no lock code, as privacy is just an illusion
— Miguel de Icaza (@migueldeicaza) March 29, 2013

@migueldeicaza When you call someone, their phone will ring on the lowest volume unless you pay to "Promote" the call.
— Brent Schooley (@brentschooley) March 29, 2013

@migueldeicaza It will also change the UI every week to expose features you don't use
— Shmueli Englard (@Shmuelie) March 29, 2013

@migueldeicaza "The application 'My Friend Secrets' would like the following permissions: * Eavesdrop on all of your FacebookPhone calls."
— Chris Howie (@cdhowie) March 29, 2013

How I ended up with Mac

miguel@gnome.org (Miguel de Icaza) — Tue, 05 Mar 2013 19:49:00 -0500

While reading Dave Winer's Why Windows Lost to Mac post, I noticed many parallels with my own experience with Linux and the Mac. I will borrow the timeline from Dave's post.

I invested years of my life on the Linux desktop first as a personal passion (Gnome) and when while awoken for two Linux companies (my own, Ximian and then Novell). During this period, I believed strongly in dogfooding our own products. I believed that both me and my team had to use the software we wrote and catch bugs and errors before it reached our users. We were pretty strict about it: both from an ideological point of view, back in the days of all-software-will-be-free, and then practically - during my tamer business days. I routinely chastised fellow team members that had opted for the easy path and avoided our Linux products.

While I had Macs at Novell (to support Mono on MacOS), it would take a couple of years before I used a Mac regularly. In some vacation to Brazil around 2008 or so, I decided to only take the Mac for the trip and learn to live with the OS as a user, not just as a developer.

Computing-wise that three week vacation turned out to be very relaxing. Machine would suspend and resume without problem, WiFi just worked, audio did not stop working, I spent three weeks without having to recompile the kernel to adjust this or that, nor fighting the video drivers, or deal with the bizarre and random speed degradation that my ThinkPad suffered.

While I missed the comprehensive Linux toolchain and userland, I did not miss having to chase the proper package for my current version of Linux, or beg someone to package something. Binaries just worked.

From this point on, using the Mac was a part-time gig for me. During the Novell layoffs, I returned my laptop to Novell and I was left with only one Linux desktop computer at home. I purchased a Mac laptop and while I fully intended to keep using Linux, the dogfooding driver was no longer there.

Dave Winer writes, regarding Windows:

Back to 2005, the first thing I noticed about the white Mac laptop, that aside from being a really nice computer, there was no malware. In 2005, Windows was a horror. Once a virus got on your machine, that was pretty much it. And Microsoft wasn't doing much to stop the infestation. For a long time they didn't even see it as their problem. In retrospect, it was the computer equivalent of Three Mile Island or Chernobyl.

To me, the fragmentation of Linux as a platform, the multiple incompatible distros, and the incompatibilities across versions of the same distro were my Three Mile Island/Chernobyl.

Without noticing, I stopped turning on the screen for my Linux machine during 2012. By the time I moved to a new apartment in October of 2012, I did not even bother plugging the machine back and to this date, I have yet to turn it on.

Even during all of my dogfooding and Linux advocacy days, whenever I had to recommend a computer to a single new user, I recommended a Mac. And whenever I gave away computer gifts to friends and family, it was always a Mac. Linux just never managed to cross the desktop chasm.

The Making of Xamarin Studio

miguel@gnome.org (Miguel de Icaza) — Fri, 22 Feb 2013 19:42:00 -0500

We spent a year designing the new UI and features of Xamarin Studio (previously known as MonoDevelop).

I shared some stories of the process on the Xamarin blog.

After our launch, we open sourced all of the work that we did, as well as our new Gtk+ engine for OSX. Lanedo helps us tremendously making Gtk+ 2.x both solid and amazing on OSX (down to the new Lion scrollbars!). All of their work has either been upstreamed to Gtk+ or in the process of being upstreamed.

"Reality Distortion Field"

miguel@gnome.org (Miguel de Icaza) — Wed, 07 Nov 2012 20:57:00 -0500

"Reality Distortion Field" is a modern day cop out. A tool used by men that lack the intellectual curiosity to explain the world, and can deploy at will to explain excitement or success in the market place. Invoking this magical super power saves the writer from doing actual work and research. It is a con perpetuated against the readers.

The expression originated as an observation made by those that worked with Steve to describe his convincing passion. It was insider joke/expression which has now been hijacked by sloppy journalists when any subject is over their head.

The official Steve Jobs biography left much to be desired. Here a journalist was given unprecedented access to Steve Jobs and get answers to thousands of questions that we have to this day. How did he approach problems? Did he have a method? How did he really work with his team? How did he turn his passion for design into products? How did he make strategic decisions about the future of Apple? How did the man balance engineering and marketing problems?

The biography has some interesting anecdotes, but fails to answer any of these questions. The biographer was not really interested in understanding or explaining Steve Jobs. He collected a bunch of anecdotes, stringed them together in chronological order, had the text edited and cashed out.

Whenever the story gets close to an interesting historical event, or starts exploring a big unknown of Steve's work, we are condescendingly told that "Steve Activated the Reality Distortion Field".

Every. Single. Time.

Not once did the biographer try to uncover what made people listen to Steve. Not once did he try to understand the world in which Steve operated. The breakthroughs of his work are described with the same passion as a Reuters news feed: an enumeration of his achievements glued with anecdotes to glue the thing together.

Consider the iPhone: I would have loved to know how the iPhone project was conceived. What internal process took place that allowed Apple to gain the confidence to become a phone manufacturer. There is a fascinating story of the people that made this happen, millions of details of how this project was evaluated and what the vision for the project was down to every small detail that Steve cared about.

Instead of learning about the amazing hardware and software engineering challenges that Steve faced, we are told over and over that all Steve had to do was activate his special super power.

The biography in short, is a huge missed opportunity. Unprecedented access to a man that reshaped entire industries and all we got was some gossip.

The "Reality Distortion Field" is not really a Steve Jobs super-power, it is a special super power that the technical press uses every time they are too lazy to do research.

Why do expensive and slow user surveys, or purchase expensive research from analysts to explain why some product is doing well, or why people are buying it when you can just slap a "they activated the Reality Distortion Field and sales went through the roof" statement in your article.

As of today, a Google News search for "Reality Distortion Field Apple" reports 532 results for the last month.

Perhaps this is just how the tech press must operate nowadays. There is just no time to do research as new products are being unveiled around the clock, and you need to deliver opinions and analysis on a daily basis.

But as readers, we deserve better. We should reject these explanations for what they are: a cheap grifter trick.

Mono 3.0 is out

miguel@gnome.org (Miguel de Icaza) — Mon, 22 Oct 2012 15:15:00 -0500

After a year and a half, we have finally released Mono 3.0.

Like I discussed last year, we will be moving to a more nimble release process with Mono 3.0. We are trying to reduce our inventory of pending work and get new features to everyone faster. This means that our "master" branch will remain stable from now on, and that large projects will instead be developed in branches that are regularly landed into our master branch.

What is new

Check our release notes for the full details of this release. But here are some tasty bits:

C# Async compiler
Unified C# compiler for all profiles
4.5 Async API Profile
Integrated new Microsoft's Open Sourced stacks:
- ASP.NET MVC 4
- ASP.NET WebPages
- Entity Framework
- Razor
- System.Json (replaces our own)
New High performance Garbage Collector (SGen - with many performance and scalability improvements)
Metric ton of runtime and class library improvements.

Also, expect F# 3.0 to be bundled in our OSX distribution.

The Sophisticated Procrastinator - Volume 1

miguel@gnome.org (Miguel de Icaza) — Mon, 22 Oct 2012 01:03:00 -0500

Let me share with you some links that I found interesting in the past few weeks. These should keep the most diligent person busy for a few hours.

Software Reads

Talbot Crowell's Introduction to F# 3.0 slides from Boston CodeCamp.

Bertrand Meyer (The creator of Eiffel, father of good taste in engineering practices) writes Fundamental Duality of Software Engineering: on the specifications and tests. This is one of those essays where every idea is beautifully presented. A must read.

Good article on weakly ordered CPUs.

MonkeySpace slide deck on MonoGame.

David Siegel shares a cool C# trick, switch expressions.

Oak: Frictionless development for ASP.NET MVC.

Simon Peyton Jones on video talks about Haskell, past, present and future. A very tasty introductory talk to the language. David Siegel says about this:

Simon Peyton-Jones is the most eloquent speaker on programming languages. Brilliant, funny, humble, adorable.

Rob Pike's talk on Concurrency is not Parallelism. Rob is one of the crisper minds in software development, anything he writes, you must read, everything he says, you must listen to.

Answering the question of what is the fastest way to access properties dynamically: DynamicMethod LINQ expressions, MethodInfo. Discussion with Eric Maupin.

OpenGL ES Quick Reference Card, plus a good companion: Apple's Programming Guide.

Interesting Software

SparkleShare, the open source file syncing service running on top of Git released their feature-complete product. They are preparing for their 1.0 release. SparkleShare runs on Linux, Mac and Windows. Check out their Release Notes.

Experts warn that Canonical might likely distribute a patched version that modifies your documents and spreadhseets to include ads and Amazon referal links.

Pheed a twitter competitor with a twist.

Better debugging tools for Google Native Client.

Touch Draw comes to MacOS, great vector drawing application for OSX. Good companion to Pixelmator and great for maintaining iOS artwork. It has great support for structured graphics and for importing/exporting Visio files.

MonoGame 3D on the Raspberry Pi video.

Fruit Rocks a fun little game for iOS.

@Redth, the one man factory of cool hacks has released:

PassKitSharp, a library to generate, maintain, process Apple's Passbook files written in C#
Zxing.Mobile, an open source barcode library built on top of ZXing (Zebra Crossing) runs on iOS and Android.
PushSharp, A server-side library for sending Push Notifications to iOS (iPhone/iPad APNS), Android (C2DM and GCM - Google Cloud Message), Windows Phone, Windows 8, and Blackberry devices.

Coding on Passbook: Lessons Learned.

Building a Better World

Phil Haack blogs about MonkeySpace

Patrick McKenzie writes Designing First Run Experiences to Delight Users.

Kicking the Twitter habit.

Twitter Q&A with TJ Fixman, writer for Insomniac Games.

Debunking the myths of budget deficits: Children and Grandchildren do not pay for budget deficits, they get interest on the bonds.

Cool Stuff

Live updates on HoneyPots setup by the HoneyNet Project.

Updated Programming F# 3.0, 2nd Edition is out. By Chris Smith, a delightful book on F# has been updated to cover the new and amazing type providers in F#.

ServiceStack now has 113 contributors.

News

From Apple Insider: Google may settle mobile FRAND patent antitrust claim.

The Salt Lake City Tribune editorial board endorses Obama over Romney:

In considering which candidate to endorse, The Salt Lake Tribune editorial board had hoped that Romney would exhibit the same talents for organization, pragmatic problem solving and inspired leadership that he displayed here more than a decade ago. Instead, we have watched him morph into a friend of the far right, then tack toward the center with breathtaking aplomb. Through a pair of presidential debates, Romney’s domestic agenda remains bereft of detail and worthy of mistrust.
Therefore, our endorsement must go to the incumbent, a competent leader who, against tough odds, has guided the country through catastrophe and set a course that, while rocky, is pointing toward a brighter day. The president has earned a second term. Romney, in whatever guise, does not deserve a first.

From Blue States are from Scandinavia, Red States are from Guatemala the author looks at the differences in policies in red vs blue states, and concludes:

Advocates for the red-state approach to government invoke lofty principles: By resisting federal programs and defying federal laws, they say, they are standing up for liberty. These were the same arguments that the original red-staters made in the 1800s, before the Civil War, and in the 1900s, before the Civil Rights movement. Now, as then, the liberty the red states seek is the liberty to let a whole class of citizens suffer. That’s not something the rest of us should tolerate. This country has room for different approaches to policy. It doesn’t have room for different standards of human decency.

Esquire's take on the 2nd Presidential Debate.

Dave Winer wrote Readings from News Execs:

There was an interesting juxtaposition. Rupert Murdoch giving a mercifully short speech saying the biggest mistake someone in the news business could make is thinking the reader is stupid. He could easily have been introducing the next speaker, Bill Keller of the NY Times, who clearly thinks almost everyone who doesn't work at the NY Times is stupid.

What do you know, turns out that Bill Moyers is not funded by the government nor does he get tax money, like many Republicans like people to believe. The correction is here.

Twitter Quotes

Joseph Hill

"Non-Alcoholic Sparkling Beverage" - Whole Foods' $7.99 name for "bottle of soda".

Jonathan Chambers

Problem with most religious people is that their faith tells them to play excellently in game of life, but they want to be the referees.

Hylke Bons on software engineering:

"on average, there's one bug for every 100 lines of code" this is why i put everything on one line

Waldo Jaquith:

If government doesn't create jobs, isn't Romney admitting that his campaign is pointless?

Alex Brown

OH "It is a very solid grey area" #sc34 #ooxml

Jo Shields

"I don't care how many thousand words your blog post is, the words 'SYMBIAN WAS WINNING' mean you're too high on meth to listen too.

Jeremy Scahill on war monger Max Boots asks the questions

Do they make a Kevlar pencil protector? Asking for a think tanker.
Max Boot earned a Purple Heart (shaped ink stain on his shirt) during the Weekly Standard War in 1994.

Tim Bray

"W3C teams with Apple, Google, Mozilla on WebPlatform"... or we could all just sponsor a tag on StackOverflow.

David Siegel

Most programmers who claim that types "get in the way" had a sucky experience with Java 12 years ago, tried Python, then threw the baby out.

Outrage Dept

How Hollywood Studios employ creative accounting to avoid sharing the profits with the participants. If you were looking at ways to scam your employees and partners, look no further.

Startvation in Gaza: State forced to release 'red lines' document for food consumption.

Dirty tricks and disturbing trends: Billionaire warn employees that if Obama is reelected, they will be facing layoffs.

Israeli Children Deported to South Sudan Succumb to Malaria:

Here we are today, three months later, and within the last month alone, these two parents lost two children, and the two remaining ones are sick as well. Sunday is already in hospital with malaria, in serious condition, and Mahm is sick at home. “I’ve only two children left,” Michael told me today over the phone. The family doesn’t have money to properly treat their remaining children. The hospitals are at full capacity and more people leave them in shrouds than on their own two feet. I ask you, beg of you to help me scream the story of these children and their fate, dictated by the heartless, immoral Israeli government.

When Suicide is Cheaper the horrifying tales of Americans that can not afford health care.

Paul Ryan is not that different from Todd Akin, when it comes to women rights.

Interesting Discussions, Opinions and Articles

A Windows 8 critique: someone is not very happy with it.

On "meritocracy": what is wrong with it.

Fascinating read on the fast moving space of companies: Intimate Portrait of Innovation, Risk and Failure Through Hipstamatic's Lens.

Kathy Sierra discusses sexism in the tech world. How she changed her mind about it, and the story that prevented her from seeing it.

Response @antirez's sexist piece.

Chrystia Freeland's The Self Destruction of the 1% percent is a great article, which touches on the points of her book of Plutocrats

Sony's Steep Learning Process a look at the changing game with a focus on Sony's challenges.

Entertainment

One Minute Animated Primers on Major Theories on Religion.

Cat fact and Gif provides Cat facts with a gif that goes with it. The ultimate resource of cat facts and gifs.

Drowning Good Ideas with Bloat. The tale of pkg.m4.

miguel@gnome.org (Miguel de Icaza) — Sat, 20 Oct 2012 15:08:00 -0500

The gnome-config script was a precursor to pkg-config, they are tools used that you can run and use to extract information about the flags needed to compile some code, link some code, or check for a version. gnome-config itself was a pluggable version of Tcl's tclConfig.sh script.

The idea is simple: pkg-config is a tiny little tool that uses a system database of packages to provide version checking and build information to developers. Said database is merely a well-known directory in the system containing files with the extension ".pc", one per file.

These scripts are designed to be used in shell scripts to probe if a particular software package has been installed, for example, the following shell script probes whether Mono is installed in the system:

# shell script

if pkg-config --exists mono; then
    echo "Found Mono!"
else
    echo "You can download mono from www.go.mono.com"
fi

It can also be used in simple makefiles to avoid hardcoding the paths to your software. The following makefile shows how:

CFLAGS = `pkg-config --cflags mono`
LIBS   = `pkg-config --libs mono`

myhost: myhost.c

And if you are using Automake and Autoconf to probe for the existence of a module with a specific version and extract the flags needed to build a module, you would do it like this:

AC_SUBST(MONO_FLAGS)
AC_SUBST(MONO_LIBS)
if pkg-config --atleast-version=2.10 mono; then
    MONO_FLAGS=`pkg-config --cflags mono`
    MONO_LIBS=`pkg-config --libs mono`
else
   AC_MSG_ERROR("You need at least Mono 2.10")
fi

There are two main use cases for pkg-config.

Probing: You use the tool to pobe for some condition about a package and taking an action based on this. For this, you use the pkg-config exit code in your scripts to determine whether the condition was met. This is what both the sample automake and the first script show.

Compile Information: You invoke the tool which outputs to standard output the results. To store the result or pass the values, you use the shell backtick (`). That is all there is to it (example: version=`pkg-config --version`).

The tool is so immensely simple that anyone can learn every command that matters in less than 5 minutes. The whole thing is beautiful because of its simplicity.

The Siege by the Forces of Bloat

Perhaps it was a cultural phenomenon, perhaps someone that had nothing better to do, perhaps someone that was just trying to be thorough introduced one of the most poisoning memes into the pool of ideas around pkg-config.

Whoever did this, thought that the "if" statement in shell was a complex command to master or that someone might not be able to find the backtick on their keyboards.

And they hit us, and they hit us hard.

They introduced pkg.m4, a macro intended to be used with autoconf, that would allow you to replace the handful of command line flags to pkg-config with one of their macros (PKG_CHECK_MODULES, PKG_CHECK_EXISTS). To do this, they wrote a 200 line script, which replaces one line of shell code with almost a hundred. Here is a handy comparison of what these offer:

# Shell style
AC_SUBST(MONO_LIBS)
AC_SUBST(MONO_CFLAGS)
if pkg-config --atleast-version=2.10 mono; then
   MONO_CFLAGS=`pkg-config --cflags mono`
   MONO_LIBS=`pkg-config --libs mono`
else
   AC_MSG_ERROR(Get your mono from www.go-mono.com)
fi

#
# With the idiotic macros
#
PKG_CHECK_MODULES([MONO], [mono >= 2.10],[], [
   AC_MSG_ERROR(Get your mono from www.go-mono.com)
])

#
# If you do not need split flags, shell becomes shorter
#
if pkg-config --atleast-version=2.10 mono; then
   CFLAGS="$CFLAGS `pkg-config --cflags mono`"
   LIBS="$LIBS `pkg-config --libs mono`"
else
   AC_MSG_ERROR(Get your mono from www.go-mono.com)
fi

The above shows the full benefit of using a macro, MONO is a prefix that will have LIBS and CFLAGS extracted. So the shell script looses. The reality is that the macros only give you access to a subset of the functionality of pkg-config (no support for splitting -L and -l arguments, querying provider-specific variable names or performing macro expansion).

Most projects, adopted the macros because they copy/pasted the recipe from somewhere else, and thought this was the right way of doing things.

The hidden price is that saving that few lines of code actually inflicts a world of pain on your users. You will probably see this in your forums in the form of:

Subject: Compilation error

I am trying to build your software, but when I run autogen.sh, I get
the following error:

checking whether make sets $(MAKE)... yes
checking for pkg-config... /usr/bin/pkg-config
./configure: line 1758: syntax error near unexpected token FOO,'
./configure: line 1758:PKG_CHECK_MODULES(FOO, foo >= 2.9)'

And then you will engage in a discussion that in the best case scenario helps the user correctly configure his ACLOCAL_FLAGS, create his own "setup" script that will properly configure his system, and your new users will learn the difference between running a shell script and "sourcing" a shell script to properly setup his development system.

In the worst case scenario, the discussion will devolve into how stupid your user is for not knowing how to use a computer and how he should be shot in the head and taken out to the desert for his corpse to be eaten by vultures; because, god dammit, they should have googled that on their own, and they should have never in the first place have installed two separate automake installations in two prefixes, without properly updating their ACLOCAL_FLAGS or figured out on their own that their paths were wrong in the first place. Seriously, what moron in this day and age is not familiar with the limitations of aclocal and the best practices to use system-wide m4 macros?

Hours are spent on these discussions every year. Potential contributors to your project are driven away, countless hours that could have gone into fixing bugs and producing code are wasted, you users are frustrated. And you saved 4 lines of code.

The pkg.m4 is a poison that is holding us back.

We need to end this reign of terror.

Send pull requests to eliminate that turd, and ridicule anyone that suggests that there are good reasons to use it. In the war for good taste, it is ok to vilify and scourge anyone that defends pkg.m4.

Why Mitt does not need an Economic Plan

miguel@gnome.org (Miguel de Icaza) — Thu, 04 Oct 2012 20:00:00 -0500

Mitt Romney does not need to have an economic plan. He does not need to have a plan to cut the deficit or to cut services.

It is now well understood that to get the US out of the recession, the government has to inject money into the economy. To inject money into the economy, the US needs to borrow some money and spend it. Borrowing is also at an all-time low, so the price to pay is very low.

Economists know this, and Republicans know this.

But the Republicans top priority is to get Obama out of office at any cost. Even at the cost of prolonging the recession, damaging the US credit rating and keeping people unemployed.

The brilliance of the Republican strategy is that they have convinced the world that the real problem facing the US is the debt. Four years of non-stop propaganda on newspapers and TV shows have turned everyone into a "fiscal conservative". The propaganda efforts have succeeded into convincing the world that US economic policy should be subject to the same laws of balancing a household budget (I wont link to this idiocy).

The campaign has been a brilliant and has forced the Democrats to adopt policies of austerity, instead of policies of growth. Instead of spending left and right to create value, we are cutting. And yet, nobody has stopped the propaganda and pointed out that growth often comes after spending money. Startups start in the red and are funded for several years before they become profitable; Companies go public and use the IPO to raise capital to grow, and for many years they lose money until their investments pay off and allows them to turn the tide.

So this mood has forced Obama to talk about cuts. He needs to be detailed about his cuts, he needs to be a fiscal conservative.

But Economists and Republicans know what the real fix is. They know they have to spend money.

If Romney is elected to office, he will do just that. He will borrow and spend money, because that is the only way of getting out of the recession. That is why his plan does not need to have any substance, and why he can ignore the calls to get more details, because he has no intention to follow up with them.

Obama made a critical mistake in his presidency. He decided to compromise with Republicans, he was begging to be loved by Republicans and in the process betrayed his own base and played right into the Republican's plans.

Mono 2.11.4 is out

miguel@gnome.org (Miguel de Icaza) — Tue, 02 Oct 2012 19:23:00 -0500

A couple of weeks ago we released Mono 2.11.4; I had not had time to blog about it.

Since our previous release a month before, we had some 240 commits, spread like this:

488 files changed, 28716 insertions(+), 6921 deletions(-)

Among the major updates in this release:

Integrated the Google Summer of Code code for Code Contracts.
Integrated the Google Summer of Code code for TPL's DataFlow.
Plenty of networking stack fixes and updates (HTTP stack, web services stack, WCF)
Improvements to the SGen GC.
TPL fixes for AOT systems like the iPhone.
Debugger now supports batched method invocations.

And of course, a metric ton of bug fixes all around.

Head over to Mono's Download Page to get the goods. We would love to hear about any bugs to have a great stable release.

TypeScript: First Impressions

miguel@gnome.org (Miguel de Icaza) — Mon, 01 Oct 2012 20:35:00 -0500

Today Microsoft announced TypeScript a typed superset of Javascript. This means that existing Javascript code can be gradually modified to add typing information to improve the development experience: both by providing better errors at compile time and by providing code-completion during development.

As a language fan, I like the effort, just like I pretty much like most new language efforts aimed at improving developer productivity: from C#, to Rust, to Go, to Dart and to CoffeeScript.

A video introduction from Anders was posted on Microsoft's web site.

The Pros

Superset of Javascript allows easy transition from Javascript to typed versions of the code.
Open source from the start, using the Apache License.
Strong types assist developers catch errors before the deploy the code, this is a very welcome addition to the developer toolchest. Script#, Google GWT and C# on the web all try to solve the same problem in different ways.
Extensive type inference, so you get to keep a lot of the dynamism of Javascript, while benefiting from type checking.
Classes, interfaces, visibility are first class citizens. It formalizes them for those of us that like this model instead of the roll-your-own prototype system.
Nice syntactic sugar reduces boilerplate code to explicit constructs (class definitions for example).
TypeScript is distributed as a Node.JS package, and it can be trivially installed on Linux and MacOS.
The adoption can be done entirely server-side, or at compile time, and requires no changes to existing browsers or runtimes to run the resulting code.

Out of Scope

Type information is erased when it is compiled. Just like Java erases generic information when it compiles, which means that the underling Javascript engine is unable to optimize the resulting code based on the strong type information.

Dart on the other hand is more ambitious as it uses the type information to optimize the quality of the generated code. This means that a function that adds two numbers (function add (a,b) { return a+b;}) can generate native code to add two numbers, basically, it can generate the following C code:

double add (double a, double b)
{
    return a+b;
}

While weakly typed Javascript must generated something like:

JSObject add (JSObject a, JSObject b)
{
    if (type (a) == typeof (double) &&
	type (b) == typeof (double))
	return a.ToDouble () + b.ToDouble ();
    else
	JIT_Compile_add_with_new_types ();
}

The Bad

The majority of the Web is powered by Unix.

Developers use MacOS and Linux workstations to write the bulk of the code, and deploy to Linux servers.

But TypeScript only delivers half of the value in using a strongly typed language to Unix developers: strong typing. Intellisense, code completion and refactoring are tools that are only available to Visual Studio Professional users on Windows.

There is no Eclipse, MonoDevelop or Emacs support for any of the language features.

So Microsoft will need to convince Unix developers to use this language merely based on the benefits of strong typing, a much harder task than luring them with both language features and tooling.

There is some basic support for editing TypeScript from Emacs, which is useful to try the language, but without Intellisense, it is obnoxious to use.

Free Market Fantasies

miguel@gnome.org (Miguel de Icaza) — Fri, 07 Sep 2012 15:20:00 -0500

This recording of a Q&A with Noam Chomsky in 1997 could be a Q&A session done last night about bailouts, corporate wellfare, and the various distractions that they use from keeping us in the dark, like caring about "fiscal responsibility".

Also on iTunes and Amazon.

2012 Update: Running C# on the Browser

miguel@gnome.org (Miguel de Icaza) — Thu, 06 Sep 2012 19:00:00 -0500

With our push to share the kernel of your software in reusable C# libraries and build a native experience per platform (iOS, Android, WP7 on phones and WPF/Windows, MonoMac/OSX, Gtk/Linux) one component that is always missing is what about doing a web UI that also shares some of the code.

Until very recently the answer was far from optimal, and included things like: put the kernel on the server and use some .NET stack to ship the HTML to the client.

Today there are two solid choices to run your C# code on the browser and share code between the web and your native UIs.

JSIL

JSIL will translate the ECMA/.NET Intermediate Language into Javascript and will run your code in the browser. JSIL is pretty sophisticated and their approach at running IL code on the browser also includes a bridge that allows your .NET code to reference web page elements. This means that you can access the DOM directly from C#.

You can try their Try JSIL page to get a taste of what is possible.

Saltarelle Compiler

The Saltarelle Compiler takes a different approach. It is a C# 4.0 compiler that generates JavaScript instead of generating IL. It is interesting that this compiler is built on top of the new NRefactory which is in turn built on top of our C# Compiler as a Service.

It is a fresh, new compiler and unlik JSIL it is limited to compiling the C# language. Although it is missing some language features, it is actively being developed.

This compiler was inspired by Script# which is a C#-look-alike language that generated Javascript for consuming on the browser.

Native Client

I left NativeClient out, which is not fair, considering that both Bastion and Go Home Dinosaurs are both powered by Mono running on Native Client.

The only downside with Native Client today is that it does not run on iOS or Android.

What Killed the Linux Desktop

miguel@gnome.org (Miguel de Icaza) — Wed, 29 Aug 2012 16:09:00 -0500

True story.

The hard disk that hosted my /home directory on my Linux machine failed so I had to replace it with a new one. Since this machine lives under my desk, I had to unplug all the cables, get it out, swap the hard drives and plug everything back again.

Pretty standard stuff. Plug AC, plug keyboard, plug mouse but when I got to the speakers cable, I just skipped it.

Why bother setting up the audio?

It will likely break again and will force me to go on a hunting expedition to find out more than I ever wanted to know about the new audio system and the drivers technology we are using.

A few days ago I spoke to Klint Finley from Wired who wrote the article titled OSX Killed Linux. The original line of questioning was about my opinion between Gnome 3's shell, vs Ubuntu's Unity vs Xfte as competing shells.

Personally, I am quite happy with Gnome Shell, I think the team that put it together did a great job, and I love how it enabled the Gnome designers -which historically only design, barely hack- to actually extend the shell, tune the UI and prototype things without having to beg a hacker to implement things for them. It certainly could use some fixes and tuning, but I am sure they will address those eventually.

What went wrong with Linux on the Desktop

In my opinion, the problem with Linux on the Desktop is rooted in the developer culture that was created around it.

Linus, despite being a low-level kernel guy, set the tone for our community years ago when he dismissed binary compatibility for device drivers. The kernel people might have some valid reasons for it, and might have forced the industry to play by their rules, but the Desktop people did not have the power that the kernel people did. But we did keep the attitude.

The attitude of our community was one of engineering excellence: we do not want deprecated code in our source trees, we do not want to keep broken designs around, we want pure and beautiful designs and we want to eliminate all traces of bad or poorly implemented ideas from our source code trees.

And we did.

We deprecated APIs, because there was a better way. We removed functionality because "that approach is broken", for degrees of broken from "it is a security hole" all the way to "it does not conform to the new style we are using".

We replaced core subsystems in the operating system, with poor transitions paths. We introduced compatibility layers that were not really compatible, nor were they maintained. When faced with "this does not work", the community response was usually "you are doing it wrong".

As long as you had an operating system that was 100% free, and you could patch and upgrade every component of your operating system to keep up with the system updates, you were fine and it was merely an inconvenience that lasted a few months while the kinks were sorted out.

The second dimension to the problem is that no two Linux distributions agreed on which core components the system should use. Either they did not agree, the schedule of the transitions were out of sync or there were competing implementations for the same functionality.

The efforts to standardize on a kernel and a set of core libraries were undermined by the Distro of the Day that held the position of power. If you are the top dog, you did not want to make any concessions that would help other distributions catch up with you. Being incompatible became a way of gaining market share. A strategy that continues to be employed by the 800 pound gorillas in the Linux world.

To sum up: (a) First dimension: things change too quickly, breaking both open source and proprietary software alike; (b) incompatibility across Linux distributions.

This killed the ecosystem for third party developers trying to target Linux on the desktop. You would try once, do your best effort to support the "top" distro or if you were feeling generous "the top three" distros. Only to find out that your software no longer worked six months later.

Supporting Linux on the desktop became a burden for independent developers.

But at this point, those of us in the Linux world still believed that we could build everything as open source software. The software industry as a whole had a few home runs, and we were convinced we could implement those ourselves: spreadsheets, word processors, design programs. And we did a fine job at that.

Linux pioneered solid package management and the most advance software updating systems. We did a good job, considering our goals and our culture.

But we missed the big picture. We alienated every third party developer in the process. The ecosystem that has sprung to life with Apple's OSX AppStore is just impossible to achieve with Linux today.

The Rise of OSX

When OSX was launched it was by no means a very sophisticated Unix system. It had an old kernel, an old userland, poor compatibility with modern Unix, primitive development tools and a very pretty UI.

Over time Apple addressed the majority of the problems with its Unix stack: they improved compatibility, improved their kernel, more open source software started working and things worked out of the box.

The most pragmatic contributors to Linux and open source gradually changed their goals from "an world run by open source" to "the open web". Others found that messing around with their audio card every six months to play music and the hardships of watching video on Linux were not worth that much. People started moving to OSX.

Many hackers moved to OSX. It was a good looking Unix, with working audio, PDF viewers, working video drivers, codecs for watching movies and at the end of the day, a very pleasant system to use. Many exchanged absolute configurability of their system for a stable system.

As for myself, I had fallen in love with the iPhone, so using a Mac on a day-to-day basis was a must. Having been part of the Linux Desktop efforts, I felt a deep guilt for liking OSX and moving a lot of my work to it.

What we did wrong

Backwards compatibility, and compatibility across Linux distributions is not a sexy problem. It is not even remotely an interesting problem to solve. Nobody wants to do that work, everyone wants to innovate, and be responsible for the next big feature in Linux.

So Linux was left with idealists that wanted to design the best possible system without having to worry about boring details like support and backwards compatibility.

Meanwhile, you can still run the 2001 Photoshop that came when XP was launched on Windows 8. And you can still run your old OSX apps on Mountain Lion.

Back in February I attended FOSDEM and two of my very dear friends were giggling out of excitement at their plans to roll out a new system that will force many apps to be modified to continue running. They have a beautiful vision to solve a problem that I never knew we had, and that no end user probably cares about, but every Linux desktop user will pay the price.

That day I stopped feeling guilty about my new found love for OSX.

Update September 2nd, 2012

Clearly there is some confusion over the title of this blog post, so I wanted to post a quick follow-up.

What I mean with the title is that Linux on the Desktop lost the race for a consumer operating system. It will continue to be a great engineering workstation (that is why I am replacing the hard disk in my system at home) and yes, I am aware that many of my friends use Linux on the desktop and love it.

But we lost the chance of becoming a mainstream consumer OS. What this means is that nobody is recommending a non-technical person go get a computer with Linux on it for their desktop needs (unless you are doing it so for idelogical reasons).

We had our share of chances. The best one was when Vista bombed in the marketplace. But we had our own internal battles and struggles to deal with. Some of you have written your own takes of our struggled in that period.

Today, the various Linux on the desktops are the best they have ever been. Ubuntu and Unity, Fedora and GnomeShell, RHEL and Gnome 2, Debian and Xfce plus the KDE distros. And yet, we still have four major desktop APIs, and about half a dozen popular and slightly incompatible versions of Linux on the desktop: each with its own curated OS subsystems, with different packaging systems, with different dependencies and slightly different versions of the core libraries. Which works great for pure open source, but not so much for proprietary code.

Shipping and maintaining apps for these rapidly evolving platforms is a big challenge.

Linux succeeded in other areas: servers and mobile devices. But on the desktop, our major feature and our major differentiator is price, but comes at the expense of having a timid selection of native apps and frequent breakage. The Linux Hater blog parodied this on a series of posts called the Greatest Hates.

The only way to fix Linux is to take one distro, one set of components as a baseline, abadone everything else and everyone should just contribute to this single Linux. Whether this is Canonical's Ubutu, or Red Hat's Fedora or Debian's system or a new joint effort is something that intelligent people will disagree until the end of the days.

Mono 2.11.3 is out

miguel@gnome.org (Miguel de Icaza) — Mon, 13 Aug 2012 19:04:00 -0500

This is our fourth preview release of Mono 2.11. This version includes Microsoft's recently open sourced EntityFramework and has been updated to match the latest .NET 4.5 async support.

We are quite happy with over 349 commits spread like this:

 514 files changed, 15553 insertions(+), 3717 deletions(-)

Head over to Mono's Download Page to get the goods.

Hiring: Documentation Writer and Sysadmin

miguel@gnome.org (Miguel de Icaza) — Sat, 11 Aug 2012 16:17:00 -0500

We are growing our team at Xamarin, and we are looking to hire both a documentation writer and a system administrator.

For the documentation writer position, you should be both familiar with programming and API design and be able to type at least 70 wpms (you can check your own speed at play.typeracer.com). Ideally, you would be based in Boston, but we can make this work remotely.

For the sysadmin position, you would need to be familiar with Unix system administration. Linux, Solaris or MacOS would work and you should feel comfortable with automating tasks. Knowledge of Python, C#, Ruby is a plus. This position is for working in our office in Cambridge, MA.

If you are interested, email me at: miguel at xamarin.

XNA on Windows 8 Metro

miguel@gnome.org (Miguel de Icaza) — Thu, 19 Apr 2012 22:07:00 -0500

The MonoGame Team has been working on adding Windows 8 Metro support to MonoGame.

This will be of interest to all XNA developers that wanted to target the Metro AppStore, since Microsoft does not plan on supporting XNA on Metro, only on the regular desktop.

The effort is taking place on IRC in the #monogame channel on irc.gnome.org. The code is being worked in the develop3d branch of MonoGame.

Contributing to Mono 4.5 Support

miguel@gnome.org (Miguel de Icaza) — Fri, 13 Apr 2012 21:54:00 -0500

For a couple of weeks I have been holding off on posting about how to contribute to Mono, since I did not have a good place to point people to.

Gonzalo has just updated our Status pages to include the differences betwee .NET 4.0 to .NET 4.5, these provide a useful roadmap for features that should be added to Mono.

This in particular in the context of ASP.NET 4.5, please join us in mono-devel-list@lists.ximian.com.

Modest Proposal for C#

miguel@gnome.org (Miguel de Icaza) — Wed, 11 Apr 2012 15:31:00 -0500

This is a trivial change to implement, and would turn what today is an error into useful behavior.

Consider the following C# program:

struct Rect {
	public int X, Y, Width, Height;
}

class Window {
	Rect bounds;

	public Rect Bounds {
		get { return bounds; }
		set {
			// Some code that needs to run when the	property is set
			WindowManager.Invalidate (bounds);
			WindowManager.Invalidate (value);
			bounds = value;
		}
	}
}

Currently, code like this:

Window w = new Window ();
w.Bounds.X = 10;

Produces the error:

Cannot modify the return value of "Window.Bounds.X" because it is not a variable

The reason is that the compiler returns a copy of the "bounds" structure and making changes to the returned value has no effect on the original property.

If we had used a public field for Bounds, instead of a property, the above code would compile, as the compiler knows how to get to the "Bounds.X" field and set its value.

My suggestion is to alter the C# compiler to turn what today is considered an error when accessing properties and doing what the developer expects.

The compiler would rewrite the above code into:

Window w = new Window ();
var tmp = w.Bounds;
tmp.X = 10;
w.Bounds = tmp;

Additionally, it should cluster all of the changes done in a single call, so:

Window w = new Window ();
w.Bounds.X = 10;
w.Bounds.Y = 20;

Will be compiled as:

Window w = new Window ();
var tmp = w.Bounds;
tmp.X = 10;
tmp.Y = 20;
w.Bounds = tmp;

To avoid calling the setter for each property set in the underlying structure.

The change is trivial and wont break any existing code.

Can JITs be faster?

miguel@gnome.org (Miguel de Icaza) — Wed, 04 Apr 2012 15:53:00 -0500

Herb Sutter discusses in his Reader QA: When Will Better JITs save Managed Code?:

In the meantime, short answer: C++ and managed languages make different fundamental tradeoffs that opt for either performance or productivity when they are in tension.
[...]
This is a 199x/200x meme that’s hard to kill – “just wait for the next generation of (JIT or static) compilers and then managed languages will be as efficient.” Yes, I fully expect C# and Java compilers to keep improving – both JIT and NGEN-like static compilers. But no, they won’t erase the efficiency difference with native code, for two reasons.
First, JIT compilation isn’t the main issue. The root cause is much more fundamental: Managed languages made deliberate design tradeoffs to optimize for programmer productivity even when that was fundamentally in tension with, and at the expense of, performance efficiency. (This is the opposite of C++, which has added a lot of productivity-oriented features like auto and lambdas in the latest standard, but never at the expense of performance efficiency.) In particular, managed languages chose to incur costs even for programs that don’t need or use a given feature; the major examples are assumption/reliance on always-on or default-on garbage collection, a virtual machine runtime, and metadata.

This is a pretty accurate statement on the difference of the mainstream VMs for managed languages (.NET, Java and Javascript).

Designers of managed languages have chosen the path of safety over performance for their designs. For example, accessing elements outside the boundaries of an array is an invalid operation that terminates program execution, as opposed to crashing or creating an exploitable security hole.

But I have an issue with these statements:

Second, even if JIT were the only big issue, a JIT can never be as good as a regular optimizing compiler because a JIT compiler is in the business of being fast, not in the business of generating optimal code. Yes, JITters can target the user’s actual hardware and theoretically take advantage of a specific instruction set and such, but at best that’s a theoretical advantage of NGEN approaches (specifically, installation-time compilation), not JIT, because a JIT has no time to take much advantage of that knowledge, or do much of anything besides translation and code gen.

In general the statement is correct when it comes to early Just-in-Time compilers and perhaps reflects Microsoft's .NET JIT compiler, but this does not apply to state of the art JIT compilers.

Compilers are tools that convert human readable text into machine code. The simplest ones perform straight forward translations from the human readable text into machine code, and typically go through or more of these phases:

Optimizing compilers introduce a series of steps that alter their inputs to ensure that the semantics described by the user are preserved while generating better code:

An optimization that could be performed on the high-level representation would transform the textual "5 * 4" in the source code into the constant 20. This is an easy optimization that can be done up-front. Simple dead code elimination based on constant folding like "if (1 == 2) { ... }" can also be trivially done at this level.

An optimization on the medium representation would analyze the use of variables and could merge subexpressions that are computed more than once, for example:

	int j = (a*b) + (a*b)

Would be transformed by the compiler into:

	int _tmp = a * b;
	int j = _tmp + _tmp;

A low-level optimization would alter a "MULTIPLY REGISTER-1 BY 2" instruction into "SHIFT REGISTER-1 ONE BIT TO THE LEFT".

JIT compilers for Java and .NET essentially break the compilation process in two. They serialize the data in the compiler pipeline and split the process in two. The first part of the process dumps the result into a .dll or .class files:

The second step loads this file and generates the native code. This is similar to purchasing frozen foods from the super market, you unwrap the pie, shove it in the oven and wait 15 minutes:

Saving the intermediate representation and shipping it off to a new system is not a new idea. The TenDRA C and C++ compilers did this. These compilers saved their intermediate representation into an architecture neutral format called ANDF, similar in spirit to .NET's Common Intermediate Language and Java's bytecode. TenDRA used to have an installer program which was essentially a compiler for the target architecture that turned ANDF into native code.

Essentially JIT compilers have the same information than a batch compiler has today. For a JIT compiler, the problem comes down to striking a balance between the quality of the generated code and the time it takes to generate the code.

JIT compilers tend to go for fast compile times over quality of the generated code. Mono allows users to configure this threshold by allowing users to pick the optimization level defaults and even lets them pick LLVM to perform the heavy duty optimizations on the code. Slow, but the generated code quality is the same code quality you get from LLVM with C.

Java HotSpot takes a fascinating approach: they do a quick compilation on the first pass, but if the VM detects that a piece of code is being used a lot, the VM recompiles the code with all the optimization turned on and then they hot-swap the code.

.NET has a precompiler called NGen, and Mono allows the --aot flag to be passed to perform the equivalent process that TenDRA's installer did. They precompile the code tuned for the current hardware architecture to avoid having the JIT compiler spend time at runtime translating .NET CIL code to native code.

In Mono's case, you can use the LLVM optimizing compiler as the backend for precompiling code, which produces great code. This is the same compiler that Apple now uses on Lion and as LLVM improves, Mono's generated code improves.

NGen has a few limitations in the quality of the code that it can produce. Unlike Mono, NGen acts merely as a pre-compiler and tests suggest that there are very limited extra optimizations applied. I believe NGen's limitations are caused by .NET's Code Access Security feature which Mono never implemented [1].

[1] Mono only supports the CoreCLR security system, but that is an opt-in feature that is not enabled for desktop/server/mobile use. A special set of assemblies are shipped to support this.

Optimizing JIT compilation for Managed Languages

Java, JavaScript and .NET have chosen a path of productivity and safety over raw performance.

This means that they provide automatic memory management, arrays bounds checking and resource tracking. Those are really the elements that affect the raw performance of these languages.

There are several areas in which managed runtimes can evolve to improve their performance. They wont ever match the performance of hand-written assembly language code, but here are some areas that managed runtimes can work on to improve performance:

>Alias analysis is simpler as arrays are accessed with array operations instead of pointer arithmetic.

Intent: with the introduction of LINQ in C#, developers can shift their attention from how a particular task is done to expressing the desired outcome of an operation. For example:

var biggerThan10 = new List;
for (int i = 0; i < array.Length; i++){
    if (array [i] > 10)
       biggerThan10.Add (i);
}

Can be expressed now as:

var biggerThan10 = x.Where (x => x > 10).Select (x=>x);
	
// with LINQ syntax:
var biggerThan10 = from x in array where x > 10 select x;

Both managed compilers and JIT compilers can take advantage of the rich information that is preserved to turn the expressed intent into an optimized version of the code.

Extend VMs: Just like Javascript was recently extended to support strongly typed arrays to improve performance, both .NET and Java can be extended to allow fewer features to be supported at the expense of safety.

.NET could allow developers to run without the CAS sandbox and without AppDomains (like Mono does).

Both .NET and Java could offer "unsafe" sections that would allow performance critical code to not enforce arrays-bounds-optimization (at the expense of crashing or creating a security gap, this can be done today in Mono by using -O=unsafe).

.NET and Mono could provide allocation primitives that allocate objects on a particular heap or memory pool:

	var pool = MemoryPool.Allocate (1024*1024);

	// Allocate the TerrainMesh in the specified memory pool
	var p = new pool, TerrainMesh ();

	[...]
	
	// Release all objects from the pool, all references are
	// nulled out
	//
	Assert.NotEquals (p, null);
	pool.Destroy ();
	Assert.Equals (p, null);

Limiting Dynamic Features: Current JIT compilers for Java and .NET have to deal with the fact that code can be extended dynamically by either loading code at runtime or generating code dynamically.

HotSpot leverages its code recompiled to implement sophisticated techniques to perform devirtualization safely.

On iOS and other platforms it is not possible to generate code dynamically, so code generators could trivially devirtualize, inline certain operations and drop features from both their runtimes and the generated code.

More Intrinsics: An easy optimization that JIT engines can do is map common constructs into native features. For example, we recently inlined the use of ThreadLocal<T> variables. Many Math.* methods can be inlined, and we applied this technique to Mono.SIMD.

Microsoft's new Open Sourced Stacks

miguel@gnome.org (Miguel de Icaza) — Wed, 28 Mar 2012 20:20:00 -0500

Yesterday Microsoft announced that another component of .NET would be open sourced. The entire ASP.NET MVC stack is now open source, including the Razor Engine, System.Json, Web API and WebPages.

With this release, they will start accepting external contributions to these products and will be running the project like other open source projects are.

Mono and the new Stacks

We imported a copy of the git tree from Codeplex into GitHub's Mono organization in the aspnetwebstack module.

The mono module itself has now taken a dependency on this module, so the next time that you run autogen.sh in Mono, you will get a copy of the aspnetwebstack inside Mono.

As of today, we replaced our System.Json implementation (which was originally built for Moonlight) and replaced it with Microsoft's implementation.

Other libraries like Razor are next, as those are trivially imported into Mono. But ASP.NET MVC 4 itself will have to wait since it depends on extending our own core ASP.NET stack to add asynchronous support.

Our github copy will contain mostly changes to integrate the stack with Mono. If there are any changes worth integrating upstream, we will submit the code directly to Microsoft for inclusion. If you want to experiment with ASP.NET Web Stack, you should do this with your own work and work directly with the upstream maintainers.

Extending Mono's ASP.NET Engine

The new ASP.NET engine has been upgraded to support C# 5.0 asynchronous programming and this change will require a number of changes to the core ASP.NET.

We currently are not aware of anyone working on extending our ASP.NET core engine to add these features, but those of us in the Mono world would love to assist enthusiastic new developers of people that love async programming to bring these features to Mono.

Mono 2.11.0 is out

miguel@gnome.org (Miguel de Icaza) — Thu, 22 Mar 2012 14:33:00 -0500

After more than a year of development, we are happy to announce Mono 2.11, the first in a series of beta releases that will lead to the next 2.12 stable release.

Continuous Integration

To assist those helping us with testing the release, we have setup a new continuous build system that builds packages for Mac, OpenSUSE and Windows at http://wrench.mono-project.com/Wrench.

Packages

To test drive Mono 2.11 head to our our downloads page and select the "Alpha" section of the page to get the packages for Mac, Windows or Linux.

The Linux version is split up in multiple packages.

The Windows version ships with Gtk+ and Gtk#

The Mac version ships with Gtk+, Gtk#, F#, IronPython and IronRuby and comes in two versions: Mono Runtime Environment (MRE) and the more complete Mono Development Kit (MDK).

At this stage, we recommend that users get the complete kit.

Runtime Improvements in Mono 2.11

There are hundreds of new features available in this release as we have accumulated them over a very long time. Every fix that has gone into the Mono 2.10.xx series has been integrated into this release.

In addition, here are some of the highlights of this release.

Garbage Collector: Our SGen garbage collector is now considered production quality and is in use by Xamarin's own commercial products.

The collector on multi-CPU systems will also distribute various tasks across the CPUs, it is no longer limited to the marking phase.

The guide Working with SGen will help developers tune the collector for their needs and discusses tricks that developers can take advantage of.

ThreadLocal<T> is now inlined by the runtime engine, speeding up many threaded applications.

Full Unicode Surrogate Support this was a long standing feature and has now been implemented.

C# 5.0 -- Async Support

Mono 2.11 implements the C# 5.0 language with complete support for async programming.

The Mono's class libraries have been updated to better support async programming. See the section "4.5 API" for more details.

C# Backend Rewrite

The compiler code generation backend was rewritten entirely to support both IKVM.Reflection and System.Reflection which allowed us to unify all the old compilers (mcs, gmcs, dmcs and smcs) into a single compiler: mcs. For more information see Backend Rewrite.

The new IKVM.Reflection backend allows the compiler to consume any mscorlib.dll library, instead of being limited to the ones that were custom built/crafted for Mono.

In addition, the compiler is no longer a big set of static classes, instead the entire compiler is instance based, allowing multiple instances of the compiler to co-exist at the same time.

Compiler as a Service

Mono's Compiler as a Service has been extended significantly and reuses the compiler's fully instance based approach (see Instance API for more details).

Mono's compiler as a service is still a low-level API to the C# compiler. The NRefactory2 framework --shared by SharpDevelop and MonoDevelop-- provides a higher level abstraction that can be -- used by IDEs and other high-level tools.

C# Shell

Our C# interactive shell and our C# API to compile C# code can in addition to compiling expressions and statements can now compile class definitions.

4.5 API

4.5 Profile Mono now defaults to the 4.5 profile which is a strict superset of the 4.0 profile and reuses the same version number for the assemblies.

Although .NET 4.5 has not yet been officially released, the compiler now defaults to the 4.5 API, if you want to use different profile API you must use the -sdk:XXX switch to the command line compiler.

Because 4.5 API is a strict superset of 4.0 API they both share the same assembly version number, so we actually install the 4.5 library into the GAC.

Some of the changes in the 4.5 API family include:

New Async methods
WinRT compatibility API
Newly introduced assemblies: System.Net.Http, System.Threading.Tasks.Dataflow

The new System.Net.Http stack is ideal for developers using the C# 5.0 async framework.

Debugging

The GDB support has been extended and can pretty print more internal variables of Mono as well as understanding SGen internals.

The soft debugger has seen a large set of improvements:

Single stepping is now implemented using breakpoints in most cases, speeding it up considerably.
Calls to System.Diagnostics.Debugger:Log()/Break () are now routed to the debugger using new UserLog/UserBreak event types.
S390x is now supported (Neale Ferguson).
MIPS is now supported.
Added new methods to Mono.Debugger.Soft and the runtime to decrease the amount of packets transmitted between the debugger and the debuggee. This significantly improves performance over high latency connections like USB.

Mac Support

Mac support has been vastly extended, from faster GC by using native Mach primitives to improves many features that previously only worked on Linux to extending the asynchronous socket support in Mono to use MacOS X specific primitives.

New Ports

We have completed the Mono MIPS port.

Performance

As a general theme, Mono 2.11 has hundreds of performance improvements in many small places which add up.

Mono and Google Summer of Code

miguel@gnome.org (Miguel de Icaza) — Fri, 16 Mar 2012 19:41:00 -0500

Students, get your pencils ready for an intense summer of hacking with the Google Summer of Code and Mono!

Check out the Mono organization Summer of Code Project site.

Cross Platform Game Development in C#

miguel@gnome.org (Miguel de Icaza) — Fri, 16 Mar 2012 16:33:00 -0500

If you missed the live session on Cross Platform Game Development in C# from AltDevConf you can now watch presentation.

You can also check the videos for all the AltDevConf presentations.

Working With SGen

miguel@gnome.org (Miguel de Icaza) — Mon, 05 Mar 2012 19:18:00 -0500

As SGen becomes the preferred garbage collector for Mono, I put together the Working With SGen document. This document is intended to explain the options that as a developer you can tune in SGen as well as some practices that you can adopt in your application to improve your application performance.

This document is a complement to the low-level implementation details that we had previously posted.

Gtk+ and MacOS X

miguel@gnome.org (Miguel de Icaza) — Mon, 05 Mar 2012 16:20:00 -0500

We have released a new build of Mono 2.10.9 (Beta) with the latest version of Gtk+2 containing dozens of bug fixes done by the Lanedo team to improve the quality of Mono/Gtk+ apps on OSX.

This is still a beta release, please take it out for a spin, we are almost ready to graduate this as our stable Mono package.

Phalanger's PHP on Mono/.NET Updates

miguel@gnome.org (Miguel de Icaza) — Mon, 05 Mar 2012 16:18:00 -0500

The Phalanger developers have published an updated set of benchmarks of their PHP compiler running on top of .NET vs PHP and Cached PHP, and the results are impressive:

There are two cases on the language shootout where they are slower than PHP (out of eighteen cases) and they are also slower on eight of thirtyone microbenchmarks.

But in general with real applications like WordPress and MediaWiki, the performance gains are impressive.

C# for Gaming: Slides

miguel@gnome.org (Miguel de Icaza) — Sat, 11 Feb 2012 22:12:00 -0500

You can now get the Slides for my Mono for Game Development talk.

Or you can go straight to the resources for the talk.

C# for Gaming: AltDevConf This Weekend

miguel@gnome.org (Miguel de Icaza) — Thu, 09 Feb 2012 19:59:00 -0500

It is a great honor to participate this weekend on the online AltDevConf conference. This is an online two-day event

Our goal is twofold: To provide free access to a comprehensive selection of game development topics taught by leading industry experts, and to create a space where bright and innovative voices can also be heard. We are able to do this, because as an online conference we are not subject to the same logistic and economic constrains imposed by the traditional conference model.

I will be participating in the talk on Cross Platform Game Development using C# with Matthieu Laban and Philippe Rollin.

You can register here for our session on Saturday at 3pm Eastern Standard Time, noon Pacific Time, and 9pm Paris Time.

If you are located in the Paris time zone, that means that you get to enjoy the talk sipping a tasty hot chocolate with some tasty baguettes.