If you burn only good dry hard wood then you can get away for longer without sweeping but always remember that a chimney fire with a lot of tar build up is pretty scary and will likely destroy the flue.
A neighbour (retired fireman) had one a few years back. Needed a fire tender to put it out and made an impressive noise a bit like a bunsen burner with a blue flame out the top. His fire crew mates fell about laughing when he opened the door to them. Should have known better.
I have mine swept once a year mainly to remove anything loose and prevent a build up. I burn virtually smoke free so it is generally fine particulate dust.
Official recommendation is twice yearly but I usually have mine done about now prior to the main heating season.
Every now and then it gets reverse swept in summer by a bird falling down the chimney - starlings and sparrows seem to be the most common.