Hi Ben,
I think I know what you are doing wrong;
The piece of tile you want to keep, after the cuttting process, should be butted up against the fence (plastic straight edge) NOT the part that will end up as the off-cut.
Otherwise yes the 'part you want' will rotate in to fill the gap left by the blade (kerf).
Let us know if this helps...
Reader.