The heart of this is that you want to save and yet be warm.
Insulation, Insulation, Insulation.
Is there any in the walls? Is there any under the floor? How well sealed is your house?
Most houses that are older than 20 years or so are woefully inadequate. Just having insulation in the attic is not enough, far from it.
I noticed a huge difference after I blew cellulose in the walls, and again after adding underfloor insulation. My friends who didn't, all have high heat and cooling bills, and are not as comfortable.
My neighbor across the street with a similarly constructed house, heats with a kero and added no insulation. My bills are a third of his and my house is warm as opposed to his which is downright chilly. Well it isn't always chilly, in the summer it is a sweat farm. Two different strategies.
Another note. Even though no heat goes up the chimney with a kero heater, it consumes air. That air must be drawn into the house from the cold outside air. It really is beter if not only for the combustion products are vented outdoors but if the heater gets it's air not from your living space which must by necessity be drawn from outdoors.
Jeff