Thanks for all the information so far. I felt I did not do a good job of
describing the situation so it's a bit tricky. I took more pictures so
hopefully will have a better idea what we are dealing with.
The shelf, when fully assembled, looks like this:
Each aluminum joint is either a "L" (corner), "T" (top or bottom) or "X"
(interior). For example, a "T" joint looks like this:
See how one side where I have a red circle? the pin broke from that one.
The shelf panels look like this at the corner, where these joints are to be
"embedded". Here is an image of a corner where the hole is fine.
Here is a panel where the hole is jammed with a broken off pin. The hole
itself is seated deep into the groove.
When I tried to drill, it did not even make a dent. The groove is narrow so
it's hard to get a vice grip in there.