I never find a need to use any abrasive but your post begs for a definition of "build"
Oil finishes, linseed, boiled linseed, tung, and Danish oil, aren't meant to build. If you actually manage to put enough on to get a build it is going to be a soft, maybe gooey finish that still provides the minimal finish of non built up oil finishes.
Unlike surface finishes oils are meant to be soft, warm finishes, especially nice on walnut, that maintain the feel of the wood. I saw the box on the binary group. Very nice, but the sheen you have already arrived at is about as good as it gets, which, I think, is just about right for an oil finish..
However, if you want build and or increased protection for the wood you had best be looking at a surface finish, shellac, lacquer, varnish.Those finishes are suppose to build.
Personal opinion, a jewelry box is just fine, in terms of protection, with an oil finish and before changing I'd consider appearance over wood protection.