I have the E-M5 and have been very impressed with it. The in-body stabilisation is little short of superb and seems to work quicker than the in-lens IS on my Canon lenses (and it is great for my legacy Leicas as well

); the IQ is as good as my Canon 7D - and the noise performance of the Oly is definitely better. There are even reviews that compare favourably against the Canon 5DII: obviously it will suffer when blown up to 40" prints but I think that at A3 and less I would defy anyone to tell the difference. Measurebating websites put the dynamic range up towards the top of the pile, beating many DSLRs.
My one hassle with the E-M5 is the interface. I find the Panasonic cameras far more intuitive and being a haphazard shooter I am changing functions a lot, and as the Olympus cameras are very menu-driven this can be irritating - but I should add that I have not yet set up the custom menus so I should be able to solve that one. Having said all that, the E-M5 is still the MFT I take with me instead of my Panasonic GX-1 so go figure.
The price of the E-M5 is plummeting at the moment: on Amazon it is now below £700 which makes it an absolute bargain IMO.
Oly lenses are superb, and because they do not have IS built in they are generally small, light and cheap. My set of primes would be Panasonic 20mm, Oly 45mm and Oly 75mm. But several bloggers have the two Panasonic lenses (12-35 f2.8 and 35-100 f2.8) as a do-it-all kit and I think that is the way I am headed, with the Oly 60mm macro for portrait/macro.
Until now, the problem with MFTs has been that the contrast detection AF is not best for tracking moving subjects. With the E-M1 they introduced hybrid phase+contrast detect AF it seems they have sorta solved it, and on one website I frequent one poster figures the E-M1 + 75-300 is as good as his Canon 7D +100-400 lens for tracking birds (in flight!) and bikes. Until I take the plunge with the E-M1 I will keep my Canon 7D for wildlife and use the MFTs for virtually everything else.