I usually don't have a problem with "broken" English, but I've also spent a lot of time listening to it being spoken by non-native speakers (many of whom learned by ear rather than formal education).
For a number of English speakers, though, there is a growing movement to "refuse" to understand something that is not in what they consider proper English due to "lazy" English Lykofos mentions. Most of the time, I fall in that category, as it really annoys me when native English speakers can't take the time to try to communicate clearly.
At the same time, if it's clear there's an effort or I see mistakes that are more typical of a non-native speaker, I give them full leeway. I'll ignore the guy that isn't trying, but help the guy who is.
(Much will depend on my mood at the time, though.
)