Coast-to-coast I've heard people say "my tap water is fine, the best in the country". While the source may be pure, the end result often is not.
For example, San Francisco's water comes from Hetch Hetchy which is in Yosemite. It's a pristine source, but we aren't drinking from the source. It has to go through plenty of miles of toxicity before it ever comes out of the tap.
Recently, I got a counter top system for my Brooklyn apartment, and about two months later the flow was very slow. The company asked me if I lived in NY, and I asked them how they knew that. Well, it turns out we have the largest amount of dirt and sediment of anywhere in the US. They sent me some new filters and a different attachment to filter sediment. I was grossed out by how filthy the filter was after such a short time.
Then there is the fact that most water is contaminated with pharmaceutical drugs. I doubt your Brita filter is remotely equipped to handle this level of toxicity. Speaking of those pitcher systems, they really don't filter out much. They do improve the taste, but the chlorine or chloramine, fluoride, and heavy metals remain.
So what type of filters do remove contaminants? Well, reverse osmosis gets everything out, but the water is then devoid of beneficial minerals. The RO systems are expensive and they waste a lot of water.
Carbon block filters (like Brita, but bigger) come in many shapes and sizes, and they can remove most contaminants without getting rid of all the minerals. Ceramic pitchers are another effective method.
Check out, Aquasana, Multi-pure, and General Ecology.