I don't think fire is an issue except with some Lithium cells. Mercury, which is quite insidiously toxic and can leak into groundwater used to be used in some button cells,[1] but I don't think it is any more. But it is largely metal toxicity which is seen as a risk when batteries are disposed of in landfill or incinerated. I must admit I am pretty ignorant about which current batteries present a serious risk.
[1] Button cells can kill small children if swallowed, so however they are disposed of they should be well packaged so they can't fall out in the house or in the street.