USB mouse is not recognised

I've had this issue with my framework laptop where the usba dongle for my wireless mouse is not recognised until I take it, and it's expansion card out a bunch of times. This always takes a long time of fiddling and I feel like doing this every single day might damage the expansion slots.

If I run LSUSB the dongle shows up like:

Bus 005 Device 002: ID 32ac:0002 [unknown] [unknown]

Did anyone else have this issue? I have no idea what it could be personally. I'd prefer the expansion slots just work like regular usb ports.