To be honest, I'm completely divided on the matter. For the most parts, I watch anime off the internet in Japanese (well, duh, I have to), and DVDs I buy in Engling, this being the dub. But occasionally the English dub is just too horrible to bear - best example being Stratos4 probably.
One thing is for certain, though. Once I start a series in a certain language, I won't change back, simply because I find myself getting used to the characters. For example, Mahoromatic: I watched the first two episodes off the internet, decided I really liked it and bought the English DVDs. I haven't gottent around to watching them yet, but when I do, it'll be in Japanese with subs.