Check the slope outside. YOu may need to do some grading. In a conventional slab foundation, a foam seal is applied between the sill plate and the concrete foundation. And the weight of the whole wall keeps the seal intact unless really under water. In a pole barn, you're relying on compression at relatively few points to support the walls and roof. There is no real seal under any bottom board, and those are merely there for holding on the siding, and not functional as a support mechanism. THerefore, they don't ever really "seal" out water. Caulking won't really help with a moisture problem as large as you're describing.
YOu have another problem all together. Either the slope outside can't manage to channel away water from the roof, or else the sheeting action of the water running down the building side is seeping in because you don't have a lip on the concrete slab to keep it out and the slab isn't sloped at the edge in any way to keep it out. I'd bet it was the second problem, and the cure for that would be to go back and chip out a drain edge on the lip of the slab (the sloped edge you see where a garage door is) where it meets the siding to keep the water from running in. After you do that, applying a foam sealant between the bottom board and slab might help with the residual, but you have to actively channel the water *away* from the slab first.