Matt, I had a EP3 before getting my Fuji X pro. I was waiting on the Oly and may still get it as a backup. the menus are the same and I can tell you they are no big deal to learn, in fact, to me easier than my Nikon's were. The Fuji even moreso. If you get the good glass like the 45 1.8, the 12.20 you can get some really nice stuff. Believe it or not (because its not a fast lens), the 74-300, in essence a 150-600 lens is quite remarkable also, and really nice in good light.
I don't know how many blogs you have been to but check out Robin Wongs site. He does some great stuff and can really show off the 12-50 kit lens also.
I fondled the camera and they changed the button outlay from the EP3 I had, but you will be right at home soon. I purchased for a travel camera but now my Fuji goes with me everywhere. I think the EM-5 is really nice, and love the fact you can change the highlights/shadows in the viewfinder....very cool.
I may still pick one up but I think the Oly is a cool system. It was the cmaera I was waiting for I just fell for the Fuji harder. You can use plenty of adapters and your Nikon glass, and i have close focus 5T and 6T I used as well.
If it speaks to you...go for it. It's the best of the m43.