The pktlen error that occurs would seem to point to some problem with the two firewalls handling the respective encryption protocols. Are you aware of any known problems with VPN tunneling between these two firewall products or between one and another?
The real answer for you is to contact your tech support. Both Watchguard and Checkpoint have good tech support. VPN debugging is a real pain in the neck, and e-mail isn't the best way to do it.
Looking at your logs, there is indeed some problem there, and I think your guess that there's a parameter mismatch is a good one. (Which only means I'd guess the same thing -- it doesn't mean it's the right guess.) But you really need to talk to vendor tech support. Starting with Watchguard is good, because their box is the one complaining, but the problem could be on the Checkpoint box, too, so keep that in mind.
When I've had VPN problems, its often best to put calls into *both* companies. This is irritating because you have to deal with two sets of tech support at the same time and remember which one you'd told what, but if you can keep that straight (keeping notes is a good thing), you double your chance of someone figuring out what to do. All you have to do is flip the right set of switches so that one of the boxes can talk to the other. That's just easier said than done.
Beyond that, if you want to do more research yourself, look at the VPN Users' Mailing List. It's run by a good friend of mine, Tina Bird, and that collection of people has seen a lot.
There are also archives on SecurityFocus, but Tina's Web site has all the pointers. I gave Tina a call and she says that Watchguard is one of their supporting members, and she knows there is Watchguard information in the archives.
For more information on this topic, visit these other SearchSecurity.com resources:
Best Web Links: Firewalls
Scheier's Security Product Round Up: Firewalls still lack multivendor management
The Information Architect: Firewalls: How to choose what's right for you
This was first published in February 2002