I generally let unrecognized phone numbers just go to voicemail. But if I need to answer-- like to prevent napping grandkids from being awakened, I just pick up the phone and don't say anything.
Usually there's an electronic disconnect click after a few seconds. If it's a person I want to talk to and I say hi. If it's an obvious "Hello Rhett, how are you today?" telemarketer or scammer, I hang up without saying anything.