That's a fair and somewhat interesting question -- how do I decide since I find I disagree w/ CR's ratings so frequently? In considering it, I've come to two conclusions --
First, I really buy very little that I don't already have strong opinions upon and likes/dislikes so that I rarely have a quandry in selecting a product -- I already know what I want so just go get it.
On the second where there is some uncertainty, in thinking about what I've done over the last several years, I find I have made a final selection between competing products somewhat like the other poster noted by comparing online user comments at Amazon or similar forums. Products w/ quite a number of DOA's or other comments are avoided and otherwise I simply select based on what I want/see in the literature I can get.
I will agree that the one place where CR still does have some credibility w/ me is in the long-term repair histories -- but again, in many instances they're either so generic as to be meaningless or my personal experience doesn't seem to match the overall data. So, I might avoid a particular model from a particular manufacturer, but if I have used that manufacurer's products for quite some time w/ satisfaction, I'm not likely to switch until I have a personal experience that.
On one specific issue, paint testing -- the problem there is that while CR's tests may be perfect and there of some value, in the real world it's similar to real estate's mantra of "location, location, ..." -- it's all in the "preparation, preparation, preparation." The best paint poorly applied on an improperly prepared surface will peform more poorly than almost the worst paint well- applied on a good surface. Hence, the anecdotal stories...
In general, I've come to the conclusion that CR is, for the most part, more interested in selling CR than anything else...anybody else getting the stupid "testing results as news" on their local news casts? Talk about a waste of air time and lazy reporting...I've wondered but never bothered to ask--are these sponsored? :(