Dremel type device, slim cutting disk and cut a partial thickness groove in it (I usually do it on the diagonal). When deep enough (but obviously not full thickness if you want to avoid damage to the pipe) insert the tip of a good screwdriver in the slot and twist. This will then snap the olive.
The most important part is to make your cut as clean as possible with vertical sides so that the screwdriver tip gets a good purchase.
Tim