Debian 10 Upgrade breaks IPsec VPN from Windows 10 Clients

by jrklein Uncategorized

When I upgraded to Debian 10, IPsec VPN connections from Windows 10 clients to our Debian 10 VPN server stopped working.

It seems that Windows is configured to use MODP_1024 (DH Group 2), but Debian 10 is requiring MODP_2048 (DH Group 14) or better.

Here’s the relevant output from /var/log/daemon.log showing the “received proposals” from Windows 10 and the default “configured proposals” on Debian 10:

Dec  2 14:47:26 ip-172-xx-xx-xx ipsec[4443]: 11[CFG] received proposals: IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, IKE:3DES_CBC/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, IKE:3DES_CBC/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, IKE:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, IKE:AES_CBC_128/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, IKE:AES_CBC_192/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, IKE:AES_CBC_192/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, IKE:AES_CBC_192/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, IKE:AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, IKE:AES_CBC_256/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, IKE:AES_GCM_16_128/PRF_HMAC_SHA1/MODP_1024, IKE:AES_GCM_16_128/PRF_HMAC_SHA2_256/MODP_1024, IKE:AES_GCM_16_128/PRF_HMAC_SHA2_384/MODP_1024, IKE:AES_GCM_16_256/PRF_HMAC_SHA1/MODP_1024, IKE:AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_1024, IKE:AES_GCM_16_256/PRF_HMAC_SHA2_384/MODP_1024
Dec  2 14:47:26 ip-172-xx-xx-xx ipsec[4443]: 11[CFG] configured proposals: IKE:AES_CBC_128/AES_CBC_192/AES_CBC_256/AES_CTR_128/AES_CTR_192/AES_CTR_256/CAMELLIA_CBC_128/CAMELLIA_CBC_192/CAMELLIA_CBC_256/3DES_CBC/HMAC_SHA2_256_128/HMAC_SHA2_384_192/HMAC_SHA2_512_256/AES_XCBC_96/AES_CMAC_96/HMAC_SHA1_96/PRF_AES128_XCBC/PRF_AES128_CMAC/PRF_HMAC_SHA2_256/PRF_HMAC_SHA2_384/PRF_HMAC_SHA2_512/PRF_HMAC_SHA1/ECP_256/ECP_384/ECP_521/ECP_256_BP/ECP_384_BP/ECP_512_BP/CURVE_25519/CURVE_448/MODP_3072/MODP_4096/MODP_6144/MODP_8192/MODP_2048, IKE:AES_CCM_16_128/AES_CCM_16_192/AES_CCM_16_256/AES_GCM_16_128/AES_GCM_16_192/AES_GCM_16_256/AES_CCM_8_128/AES_CCM_8_192/AES_CCM_8_256/AES_CCM_12_128/AES_CCM_12_192/AES_CCM_12_256/AES_GCM_8_128/AES_GCM_8_192/AES_GCM_8_256/AES_GCM_12_128/AES_GCM_12_192/AES_GCM_12_256/PRF_AES128_XCBC/PRF_AES128_CMAC/PRF_HMAC_SHA2_256/PRF_HMAC_SHA2_384/PRF_HMAC_SHA2_512/PRF_HMAC_SHA1/ECP_256/ECP_384/ECP_521/ECP_256_BP/ECP_384_BP/ECP_512_BP/CURVE_25519/CURVE_448/MODP_3072/MODP_4096/MODP_6144/MODP_8192/MODP_2048
Dec  2 14:47:26 ip-172-xx-xx-xx ipsec[4443]: 11[IKE] no matching proposal found, trying alternative config

Here’s the same list of “received proposals” in an easier to read format. Notice all of the proposals received from Windows 10 are requesting MODP_1024 (DH Group 2).

IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, 
IKE:3DES_CBC/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, 
IKE:3DES_CBC/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, 
IKE:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, 
IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, 
IKE:AES_CBC_128/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, 
IKE:AES_CBC_192/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, 
IKE:AES_CBC_192/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, 
IKE:AES_CBC_192/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, 
IKE:AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024, 
IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024, 
IKE:AES_CBC_256/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024, 
IKE:AES_GCM_16_128/PRF_HMAC_SHA1/MODP_1024, 
IKE:AES_GCM_16_128/PRF_HMAC_SHA2_256/MODP_1024, 
IKE:AES_GCM_16_128/PRF_HMAC_SHA2_384/MODP_1024, 
IKE:AES_GCM_16_256/PRF_HMAC_SHA1/MODP_1024, 
IKE:AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_1024, 
IKE:AES_GCM_16_256/PRF_HMAC_SHA2_384/MODP_1024

The default IPsec configuration in Debian 10 appears to have dropped support for MODP_1024. I could have configured IPsec on Debian Linux to support this weaker proposal by configuring the esp/ike parameters in /etc/ipsec.conf. By default, the esp/ike proposal(s) you configure would be in addition to the default server proposals, unless you explicitly tell the server to only use your proposal(s), so configuring the additional proposal shouldn’t break compatibility with other clients.

Instead, I opted to configure the Windows 10 clients to use the stronger MODP_2048 proposal. We ran the following Windows PowerShell command on the Windows 10 clients to modify the VPN connection configuration. 

Set-VpnConnectionIPsecConfiguration -ConnectionName “VPN NAME” -EncryptionMethod AES256 -IntegrityCheckMethod SHA256 -DHGroup Group14 -CipherTransformConstants AES256 -AuthenticationTransformConstants SHA256128 -PfsGroup PFS2048

The relevant parameters were either “DHGroup” and “PfsGroup”, which have option names “Group14” and “PFS2048” respectively. I believe both options are MODP_2048, so it’s unclear which one needed to be changed.

You can learn more about this Windows 10 PowerShell command on the Set-VpnConnectionIPsecConfiguration PowerShell documentation page.

After applying the configuration change, Windows 10 was able to immediately connect to the VPN server on Debian 10.

The relevant output from /var/log/daemon.log was as follows, indicating that the client and server were able to agree on a proposal:

Dec  2 16:21:51 ip-172-xx-xx-xx charon: 09[CFG] selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048

Here’s the same “selected proposals” in an easier to read format:

IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048

Debian 10 is configured to support proposals that are stronger than this option. Windows is capable of supporting proposals stronger than what I configured. If you would like to configure a stronger proposal, explore the Set-VpnConnectionIPsecConfiguration PowerShell documentation page and the section below.

Comparing Windows and Debian Proposal Names

Here are relevant parameters for “Set-VpnConnectionIPsecConfiguration” along with the Windows option and the corresponding Debian Linux ipsec option (if known).

I am documenting my findings for future reference since the open source ecosystem is becoming very aggressive about phasing out insecure protocols.

Microsoft Documentation for Set-VpnConnectionIPsecConfiguration

Set-VpnConnectionIPsecConfiguration

-ConnectionName
 This must match the name of your existing Windows VPN connection

Phase 1 Parameters

-EncryptionMethod

Windows vs Debian Linux
 DES
 DES3    3DES_CBC
 AES128    AES_CBC_128
 AES192    AES_CBC_192
 AES256    AES_CBC_256
 GCMAES128    AES_GCM_16_128
 GCMAES256    AES_GCM_16_256

-IntegrityCheckMethod

Windows vs Debian Linux
 MD5
 SHA1    PRF_HMAC_SHA1
 SHA256    PRF_HMAC_SHA2_256
 SHA384    PRF_HMAC_SHA2_384

-DHGroup

Windows vs Debian Linux
 None
 Group1
 Group2    MODP_1024
 Group14    MODP_2048
 ECP256
 ECP384
 Group24

-CipherTransformConstants

Windows vs Debian Linux
 DES
 DES3
 AES128        AES_CBC_128
 AES192        AES_CBC_192
 AES256        AES_CBC_256
 GCMAES128    AES_GCM_16_128
 GCMAES192
 GCMAES256    AES_GCM_16_256
 None

Phase 2 Parameters

-AuthenticationTransformConstants

Windows vs Debian Linux
 MD596
 SHA196        HMAC_SHA1_96
 SHA256128    HMAC_SHA2_256_128
 GCMAES128
 GCMAES192
 GCMAES256
 None

-PfsGroup

Windows vs Debian Linux
 None
 PFS1
 PFS2
 PFS2048        MODP_2048 ??
 ECP256
 ECP384
 PFSMM
 PFS24

Thanks for Visiting!

Hopefully this helps you get back up and running right away. If this did help you resolve an issue connecting Windows 10 to your Debian 10 (or other Linux) IPsec server, can you please post a short thanks/reply below so I know this this helped someone out? 🙂 

Let me know if you have any corrections or details to add.