Inspiron Startup Freezes With USB Devices

When I plug a new USB device into my Dell Inspiron 530s and restart it, it freezes up the Dell startup screen before Windows starts. I have tried it in different ports and everything. Although, some of my other USB devices work fine while plugged in on startup but others do not. Even a BIOS update fixes nothing. What might be the cause of this problem and how can I solve it?

Thank you,
Stephen Hobson