I would use cement board screws:
formatting link
and pick the shortest length that could hold the number and only just penetrate the plank but not protrude longer than that. If the number's aren't raised (i.e. didn't come with their own stand-offs), they may look slanted because the plank is slanted, so you may need to add your own stand-offs to raise the numbers 1/2' to an inch above the surface. The shadow they'll drop will make them look nice, too. Make sure all fasteners and stand-offs are corrosion resistant.
if the numbers are really heavy, like large brass ones can be, you'd use corrosion resistant hollow wall anchors and fasteners instead of just screws so they can grab the back of the board and provide more support. Just keep in mind that there isn't much space behind the plank, use the anchors that protrude least. Â