Frank, I would guess you have two problems. Both have been mentioned already.
First, either the plug cutter is dull or it is turning too slowly. You'll have to watch for burning on cherry, though.
Keep in mind that there is end grain along the sides of the plug. If you can see any of the sides of the plug due to either the plug being oval rather than round or the hole being a bit oversized, you might being seeing a bit of that end grain.
If your counterbore for the plugs isn't deep enough, the tapered plug won't be tight in the hole
Second, you might be seeing a glue line. Although it's probably too late for this project, in the future you might do what boat builders do. Plugs (boat builders call them "bungs") are bedded with the same varnish that will be used on the surface of the brightwork. It holds the bungs securely but allows them to be removed when needed. Since the finish is being used to "glue" them in, there is no glue line to be seen.
Another thing about making plugs is this. Make yourself a jig (basically a fence) for the drill press to hold a piece of stock just wide enough to cut the plugs out of. Rip long pieces of wood for your plugs and cut them against the fence. Set the depth stop on the drill press so that you almost cut through the plug stock. Leave 1/32" or so. This will leave you with a floppy strip of plugs.
When it is time to install the plugs, you can easily handle them because they are in the strip. Dip the end plug into your finish and then stick it in the hole. Tap with a mallet to seat it and break it loose from the strip. Make sure when you are installing plugs the grain direction matches the wood being plugged. This is much easier with this long strip of plugs than handling them individually.
A couple of months back Shop Notes published my drill press jig for cutting plugs in there reader's tips section. They didn't include the part about cutting the plugs deep enough to leave a flexible strip. I asked them about that. They thought it would be too difficult for the average woodworker to do. I'm an average woodworker and I can do it so I thought that was a bit condescending.
Dave