My Dad showed me how to do this, when I was a boy. Take a flash light bulb (PR-4 is good, the old screw in 112 is better, more close to the proper voltage). Then, take about six or seven inch long piece of #10 or #12 solid wire. Wrap one end of the wire around the bulb. Curve the rest of the wire, so it looks like a letter C, or G.
Touch one end of the battery to the lead spot, on the bulb. Touch the other end of the wire to the other end of the battery. If the bulb lights, the battery is OK. Works for AAAA through D cells. Have to bend the wire a bit, for different sizes.
After using this for a while, you can also roughly guess the battery state. New, used, weak, dead. By how bright the light is.