It is only irritant if handled or mechanically disturbed. You may have seen the installers wearing masks and overalls while they were working with the stuff. Your "problem" was unfortunately an extreme reaction to handling it unprotected - I don't like handling it either :-). Maybe the installers should have shown you the manufacturers instructions, or mentioned it's a bit itchy.
There is however no problem once its installed and the dust has settled. If you are going to be in contact with it again (say, by going up to the loft and storing things) either quit and send someone else, wear disposable gloves or initally wash your hands/body with cold water to disperse the fibres. Hot water first is bad news, your pores open up and the irritant gets in. Not a recognized long term medical issue though.
I wouldn't bother ripping it all out now!!
By the way, you are coming across here as an screaming mad sectionable person with all the UPPER CASE shouting in your posting title and content? Are you? :-)
-- Adrian C