Exactly! In my (quite long, 60 years or so) of riding horses they will never cease to surprise one at what frightens them, and what doesn't frighten them.
My current horse is almost totally traffic proof, buses, large loriies, noisy cars, no problem. However he still walks carefully round a manhole cover a few hundred yards from home.
It's mostly things they can't quite 'understand' that throws them, and also things that disappear or hide.
A previous horse of ours found marching soldiers were a very strange animal but would quite happily walk calmly through woods where the same soldiers were firing blanks (I hope they were) at each other. I never quite fathomed out why we were allowed in the woods while exercises were going on.
People who retreated behind trees and hedges to avoid 'frightening the horses' were a real pain, much better just to keep in sight and look like normal human beings.