This is so true. Same with "Don't take this the wrong way but ..." or "I don't mean to be rude but ..."
Also, I thought "Offence" was incorrect, but apparently that's the spelling used in Britain. Which is just stupid. No offense, of course...
Thanks to DustBunny Mike!