If there's any screws to take the plastic casing off the controller, you can try that. You might need a special screwdriver, it depends. Then the thing lodged in there should come out. Otherwise idk, pull really hard?

My controller is also somewhat broken, it has a ring that cracked and broke in half, but it works fine since the ring isn't part of the controls.