So until you can cut all that growth down what's the issue with him cutting some branches off from your side of the fence? What's he supposed to do, cut them off where they protrude from his side of the fence? That will look pretty stupid.
However it is my growth, growing from my soil. Is he allowed to determine what saplings or undergrowth come out of my soil. I understand if it protrudes or overhangs on his side of the property line he has rights to prune and trim as he sees fit I would say. He should be able to prune and clip up to a vertical plane that extends halfway into the fence.
The issue I raise is does he have a legal right to deliberately reach over and down to my soil and prune at the ground my "ugly weedy" saplings. He decided that day to determine what I should be growing or not growing.
I don't really know what the answer is, although I wish the guy had introduced himself and asked permission first. I'm not upset at the matter, just wondering who gets what rights. My wife, now she's the one upset with the guy.