I think it's the perfectionist character of most Japanese that is their downfall in trying to master English. Japanese can't learn English because of their culture of being a 'perfectionist'. This, coupled with the 'hazukashi' feeling when they make mistakes, make it very difficult for them to learn the language, as they keep on apologizing over every little thing. Those who mastered the language probably did away with those barriers.
Recently, I heard a distinguished speaker in the 15th anniversary celebration at NAIST say that, Japanese as with other non-native speakers, are actually communicating in 'broken' English with each other. So there should be no feeling of inadequacies when they speak the language.
I know that Malaysians with their Manglish and Singaporeans with their Singlish may be taking things a little too far. Native English speakers may have a hard time trying to understand words like 'cannot meh', 'so, so lar'. Similarly, if you study Japanese, I am sure that you will come across many borrowed words like ケーキ(cake), of テレビ(TV) etc, that may be confusing to English speakers.
But then again, I may be wrong. I see many examples of Japanese people trying really hard to master English. Let's hope that the people who make fun of them, are kind enough to correct these errors.