I don't really see it. It could, sure. Anything you do to a computer that doesn't include keeping it powered off could compromise it. But realistically, I wouldn't worry about it.
USB is a a serial bus designed to replace the mass of cables that you have with keyboards, mice, modems and so on. As you no doubt know, you can even put disk drives and other reasonably fast devices on it as well. This is even more viable with USB2.
But as a security threat, USB doesn't create one in and of itself. If someone already has potentially compromising access to a firewall (like the root password), then USB could make it easier for them to do bad things. But they already could do bad things.
An old saying in computer security is that physical access is all. Someone who can put their hot little hands on a computer can do much more to it than someone who has to use a network. Put your firewall behind a locked door. Give keys to only a limited number of people. Better yet, make it an electronic lock that makes an audit record for everyone who badges in. In such a situation, USB isn't much of a worry at all.
For more information on this topic, visit these other SearchSecurity.com resources:
This was first published in February 2003