Been doing site work for 20 years and I've walked away from two trenches in my career, both times because my gut said no. Here's what I tell the new guys now, wish someone had told me at 22.
First, look up before you look down. Any wall of dirt taller than you, sitting straight up and down with no shoring, no benching, nothing cut back at an angle, is a killer. It doesn't matter if it's been standing all week. Ground that's been open for days gets weaker, not stronger, expecially after any rain or a freeze-thaw cycle. Check where the spoil pile is too. If they pushed the dug-out dirt right to the edge, that's extra weight pushing down on soil that's alredy fighting to hold itself up. Second, watch the crew, not just the hole. No ladder within reach of the workers, no hard hats on the guys in the bottom, a foreman who waves off questions, that's your real answer about how the job is run. A good site has a competent person who actually checks the trench every morning and after every rain.
Last thing. If your stomach drops when you walk into a spot, listen to it. That feeling has kept me out of two collapses, and I've seen a wall let go about ninety seconds after I would've been standing under it. You can find another job. You can't find another you.