Honeypots have many different uses, and detection of botnets is one of the possible uses.
It might be difficult, however, for a honeypot to detect a P2P botnet. An advanced P2P botnet uses encryption and only talks to registered peers, whereas standard botnets use centralized IRC connections for command and control. It would be unlikely that the honeypot could detect such an advanced P2P botnet connection if it weren't a registered peer and implicitly granted access to the botnet network. If the P2P botnet used IP scanning of semi-random IPs on the Internet to identify peers, however, a honeypot could detect this scan connection potentially as a P2P botnet, though it would still have to wait for the botnet to scan for it.
If the honeypot has been customized to emulate a node or another peer in the botnet, then it can be used to analyze the botnet's operations. However, this is a technically complex process that's not recommended for the average security pro. The honeypot could be registered manually or joined in some way to the botnet to analyze the operations of the botnet. This type of advanced analysis has been done by security researchers from UC Berkley (.pdf) targeting the MegaD botnet . The researchers reverse engineered the protocol used by MegaD and setup a honeypot to observe the operations of the botnet. This type of analysis is used in the technical part of the takedown of a botnet, but requires significant efforts and may not be an effective use.
This was first published in November 2010