IPsec VPN problems with Bintec and Sophos UTM

Some days ago I migrated a customer from two Bintec firewalls to a Sophos UTM HA cluster. The branch offices kept their Bintec routers so I configured site2site VPN tunnels to the Sophos UTM. I had many problems with this VPN tunnels, I got always log entries like these:

2015:11:04-20:01:42 utm-company-2 pluto[3488]: “S_REF_IpsSitHsBranchVpn_0″[22] 80.60.50.40:816 #389: next payload type of ISAKMP Identification Payload has an unknown value: 182
2015:11:04-20:01:42 utm-company-2 pluto[3488]: “S_REF_IpsSitHsBranchVpn_0″[22] 80.60.50.40:816 #389: malformed payload in packet. Probable authentication failure (mismatch of preshared secrets?)
2015:11:04-20:01:42 utm-company-2 pluto[3488]: “S_REF_IpsSitHsBranchVpn_0″[22] 80.60.50.40:816 #389: sending encrypted notification PAYLOAD_MALFORMED to 80.60.50.40:816

My colleague found a knowledgebase article at Sophos. There need to be NAT rules for UDP 500 and 4500 to establish a correct running IPsec VPN tunnel to other firewalls than Bintec devices.

Check the following settings on the Bintec device:

x4000:> ipnatouttable

inx IfIndex(*rw) Protocol(-rw) RemoteAddr(rw)
RemoteMask(rw) ExtAddr(*rw) RemotePort(rw)
RemotePortRange(rw) IntAddr(*rw) IntMask(rw)
IntPort(rw)

ExtPort(rw) ExtMask(rw)
Timeout(rw) 0 0 udp 0.0.0.0
0.0.0.0 0.0.0.0 -1
-1 0.0.0.0 0.0.0.0
4500 4500 255.255.255.255
0

1 0 udp 0.0.0.0
0.0.0.0 0.0.0.0 -1
-1 0.0.0.0 0.0.0.0
500 500 255.255.255.255
0

2 0 esp 0.0.0.0
0.0.0.0 0.0.0.0 -1
-1 0.0.0.0 0.0.0.0
-1 -1 255.255.255.255
0

if the entries are missing, configure them with these commands:

x4000:ipNatOutTable> IfIndex=0 Protocol=udp ExtAddr=0.0.0.0 IntAddr=0.0.0.0 IntPort=500 ExtPort=500
x4000:ipNatOutTable> IfIndex=0 Protocol=udp ExtAddr=0.0.0.0 IntAddr=0.0.0.0 IntPort=4500 ExtPort=4500

and enable “Sync SAs With Local Ifc” in the IPsec advanced settings. After this changes, we’ve got no problems anymore.