Probably the single worst attribute of the factory guard is "inconvenience".
Most are difficult and/or time consuming to remove and replace ... thus their use is discouraged by the inconvenience of doing so. And once off, the tendency it to leave them off.
Also, in many cases, factory guards are an integral part of the splitter, so if you remove one, you remove both.
The splitter (or riving knife, as it is known in some countries) is reckoned to be the single best defense against kickback, an occurrence which probably leads to more table saw injuries than any other, particularly for those not used to using a table saw on a regular basis.
Some folks blame the factory guards for accidents and say they can't see the blade with one mounted. I've never seen any first hand evidence of that myself, and have yet to see a blade guard I couldn't see the blade through ... but there is a first time for everything.
That said, their use should be voluntary in a home shop and certainly left up to the individual.
IMO, nonuse in a commercial shop is nothing more than lawyer bait.
I personally use an aftermarket, overhead type blade guard that is easily swung up out of the way and does not have the splitter attached to it. IOW, it is "convenient" to use, and takes seconds to remove and replace.
_I_ use it because I still make a good part of my income playing music professionally and the integrity of my fingers are too important to my family for me to not take all reasonable precautions.
I use a standalone, easily removable splitter for the same reasons.
The use, or nonuse, of safety equipment should be a voluntary, personal decision in a one man shop ... there is not much more that can be said about the issue.