1244547 – NetworkManager 1.0.6-2 Regression - All VPN connections get incorrect MTU and are broken

Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1244547 - NetworkManager 1.0.6-2 Regression - All VPN connections get incorrect MTU and are broken

Summary: NetworkManager 1.0.6-2 Regression - All VPN connections get incorrect MTU and...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Fedora
Classification:	Fedora
Component:	NetworkManager
Sub Component:
Version:	22
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	unspecified
Target Milestone:	---
Assignee:	Lubomir Rintel
QA Contact:	Fedora Extras Quality Assurance
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2015-07-19 22:12 UTC by Steve Storey
Modified:	2015-10-04 22:53 UTC (History)
CC List:	15 users (show)
Fixed In Version:	NetworkManager-1.0.6-6.fc22
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Clones:	1267004 (view as bug list)
Environment:
Last Closed:	2015-10-04 22:53:02 UTC
Type:	Bug
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description Steve Storey 2015-07-19 22:12:16 UTC

Description of problem:

My ethernet is configured with an MTU of 1500. With NM 1.0.2, I would connect to my Cisco VPN with via OpenConnect (from KDE Plasma desktop, via plasma-nm-openconnect), and on connecting the MTU of the newly created vpn0 interface would be set to 1406 (as there are 94 bytes of overhead in the openconnect frames). With NM 1.0.4 the MTU of the vpn0 interface is set to 1500, and thus the connection doesn't work for any comms with packets that require being split. I'm certain it's not related to changes in the upstream server, as downgrading to 1.0.2 also fixes the problem, and it also doesn't occur on a machine without updates-testing enabled yet.

Version-Release number of selected component (if applicable):

NetworkManager-1.0.4-1.fc22.x86_64
NetworkManager-adsl-1.0.4-1.fc22.x86_64
NetworkManager-bluetooth-1.0.4-1.fc22.x86_64
NetworkManager-config-connectivity-fedora-1.0.4-1.fc22.x86_64
NetworkManager-devel-1.0.4-1.fc22.x86_64
NetworkManager-glib-1.0.4-1.fc22.x86_64
NetworkManager-glib-devel-1.0.4-1.fc22.x86_64
NetworkManager-l2tp-0.9.8.7-3.fc22.x86_64
NetworkManager-libnm-1.0.4-1.fc22.x86_64
NetworkManager-openconnect-1.0.2-1.fc22.x86_64
NetworkManager-openswan-1.0.2-1.fc22.x86_64
NetworkManager-openvpn-1.0.2-2.fc22.x86_64
NetworkManager-pptp-1.1.0-1.20150428git695d4f2.fc22.x86_64
NetworkManager-pptp-gnome-1.1.0-1.20150428git695d4f2.fc22.x86_64
NetworkManager-team-1.0.4-1.fc22.x86_64
NetworkManager-vpnc-1.0.2-1.fc22.x86_64
NetworkManager-vpnc-gnome-1.0.2-1.fc22.x86_64
NetworkManager-wifi-1.0.4-1.fc22.x86_64
NetworkManager-wwan-1.0.4-1.fc22.x86_64
kernel-4.1.2-200.fc22.x86_64
plasma-nm-openconnect-5.3.2-1.fc22.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Start VPN and connect as usual
2. Run ifconfig

Actual results:
p2p1 interface has an MTU of 1500
vpn0 interface has an MTU of 1500

Expected results:

p2p1 interface has an MTU of 1500
vpn0 interface has an MTU of 1406

Additional info:

I have a crude shell script that works around the issue for me in /etc/NetworkManager/dispatcher.d/pre-up.d/:

#!/bin/bash
#
# Forcefully setup the MTU of the vpn0 device as being 94 lower than the
# MTU of the device responsible for the default gateway if the current
# MTU is too high to allow openconnect VPNs to work properly

TARGET_DEVICE="vpn0"
REQUIRED_MTU_OVERHEAD=94

#echo "Event $1 $2" >> /tmp/test.log
if [ "$2" != "pre-up" ]; then
    exit 0
fi

if [[ "$1" == "$TARGET_DEVICE" ]]; then
        # Grab the default device name
        default_gw_device=$(route -n | grep ^0\. | sed 's/.*UG[ 0-9]*//')
        default_gw_mtu=$(ifconfig | grep "$default_gw_device" | sed 's/.*mtu //')
        max_vpn_mtu=$(expr $default_gw_mtu - $REQUIRED_MTU_OVERHEAD)
        current_mtu=$(ifconfig | grep "$1" | sed 's/.*mtu //')
        if (( $max_vpn_mtu < $current_mtu )); then
                #echo Setting MTU for $1 to $max_vpn_mtu
                ip link set "$1" mtu "$max_vpn_mtu"
        fi
fi

Comment 1 lionghostshop 2015-08-01 16:02:13 UTC

It seems that my pppe has the same error message for wrong MTU.

Comment 2 lionghostshop 2015-08-05 12:54:31 UTC

It seems that the problem is gone now. I don't know why.

Comment 3 lionghostshop 2015-08-05 12:58:47 UTC

It seems too early to say. It is very unstable. The wired network often gets disconnected.

Comment 4 Luc des Trois Maisons 2015-08-07 04:35:34 UTC

In my experience the MTU for the VPN pseudo-interface is being set to 1500 regardless of the MTU for the underlying carrier interface.  For what the opinion of a passer-by is worth, it wasn't clear to me from the provided description (thank you for the script, btw) that the MTU setting wasn't simply missing a subtraction for overhead.

Comment 5 Fedora Admin XMLRPC Client 2015-08-18 15:00:26 UTC

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 6 Julien HENRY 2015-09-04 09:14:03 UTC

I just wanted to confirm I have the same issue. I have to manually update the MTU of my VPN connection:

# ifconfig vpn0 mtu 1402

I found this optimal value using ping -f

Comment 7 Michael Cronenworth 2015-09-23 00:22:19 UTC

The maximum, and typical, MTU on Cisco AnyConnect connections is 1406. I'm seeing this same issue here after updating NetworkManager.

Comment 8 marianne@tuxette.fr 2015-09-24 11:46:27 UTC

Same bug here in fedora 23, manually changing the MTU work fine

NetworkManager-l2tp-0.9.8.7-4.fc23.x86_64
NetworkManager-adsl-1.0.6-5.fc23.x86_64
NetworkManager-team-1.0.6-5.fc23.x86_64
NetworkManager-glib-1.0.6-5.fc23.x86_64
NetworkManager-pptp-1.1.0-2.20150428git695d4f2.fc23.x86_64
NetworkManager-libnm-1.0.6-5.fc23.x86_64
NetworkManager-config-connectivity-fedora-1.0.6-5.fc23.x86_64
NetworkManager-wwan-1.0.6-5.fc23.x86_64
NetworkManager-vpnc-gnome-1.0.6-3.fc23.x86_64
NetworkManager-wifi-1.0.6-5.fc23.x86_64
NetworkManager-openvpn-1.0.6-3.fc23.x86_64
NetworkManager-bluetooth-1.0.6-5.fc23.x86_64
NetworkManager-vpnc-1.0.6-3.fc23.x86_64
NetworkManager-pptp-gnome-1.1.0-2.20150428git695d4f2.fc23.x86_64
NetworkManager-openvpn-gnome-1.0.6-3.fc23.x86_64
NetworkManager-1.0.6-5.fc23.x86_64
NetworkManager-openconnect-1.0.2-2.fc23.x86_64

Comment 9 Andy Campbell 2015-09-24 12:29:06 UTC

Started getting VPN issues on F22 after NetworkManager updates I picked up today,

Following were upgraded ...

 NetworkManager                            x86_64  1:1.0.6-5.fc22  
 NetworkManager-adsl                       x86_64  1:1.0.6-5.fc22  
 NetworkManager-bluetooth                  x86_64  1:1.0.6-5.fc22  
 NetworkManager-config-connectivity-fedora x86_64  1:1.0.6-5.fc22  
 NetworkManager-glib                       x86_64  1:1.0.6-5.fc22  
 NetworkManager-libnm                      x86_64  1:1.0.6-5.fc22  
 NetworkManager-team                       x86_64  1:1.0.6-5.fc22  
 NetworkManager-wifi                       x86_64  1:1.0.6-5.fc22  
 NetworkManager-wwan                       x86_64  1:1.0.6-5.fc22  

Luckily I had another PC I hadn't upgraded to check against

I can resolve by setting the mtu to match the PC that was still working, mtu was set to 1500.

ifconfig vpn0 mtu 1300 

VPN is Cisco AnyConnect.

Comment 10 todd_lewis 2015-09-25 01:29:43 UTC

Confirming, I'm seeing the same problem: mtu on vpn0 is too high.

Comment 11 Thomas Sondergaard 2015-09-25 09:57:10 UTC

(In reply to Andy Campbell from comment #9)
> Started getting VPN issues on F22 after NetworkManager updates I picked up
> today,

I am seeing the same issue after upgrading NetworkManager to 1.0.6. MTU ends up being 1500 and the connection doesn't work. Reducing the MTU to 1400 makes it work.

Comment 12 Daniele Viganò 2015-09-25 10:06:01 UTC

With NetworkManager 1.0.6 my PPTP VPN MTU is fine:

ppp0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1456

but I'm experiencing this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1266440

I don't know if the bugs are related.

Comment 13 fednuc 2015-09-25 16:18:36 UTC

Same bug with a workaround patch on Arch Linux:

https://bugs.archlinux.org/task/46157

Also, worth noting that manually setting the MTU with e.g. "ip link set mtu 1200 dev vpn0" is not a complete workaround. It seems that immediately after the VPN interface is brought up, secondary parameters such as search domains are communicated, and in my case some of these end up only partially configured (some search domains missing etc.) after bringing up the VPN since the MTU issue was introduced.

Comment 14 Marcel Ziswiler 2015-09-25 23:20:18 UTC

For me today's update to NetworkManager-1:1.0.6-5.fc22.x86_64 made VPN half way fail as well. I got really strange server errors reported from various of our infrastructure servers. Firing up Wireshark showed TCP reassembly errors probably due to MTU mismatch! Just DNF downgrading NetworkManager made it all work again. Let me know if you need more details and please specify what those might comprise.

Comment 15 Robert L 2015-09-26 18:31:29 UTC

I can confirm the issue using NetworkManager-1.0.6-5.fc22. MTU is 1500 and reducing manually to 1406 works.

Comment 16 Lubomir Rintel 2015-09-28 19:03:57 UTC

Fix in review in https://bugzilla.gnome.org/show_bug.cgi?id=754781

Comment 17 Fedora Update System 2015-09-28 21:00:26 UTC

NetworkManager-1.0.6-6.fc22 NetworkManager-openswan-1.0.6-2.fc22 NetworkManager-openvpn-1.0.6-3.fc22 NetworkManager-vpnc-1.0.6-3.fc22 network-manager-applet-1.0.6-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-10143

Comment 18 Fedora Update System 2015-09-28 21:01:30 UTC

NetworkManager-1.0.6-6.fc22 NetworkManager-openswan-1.0.6-2.fc22 NetworkManager-openvpn-1.0.6-3.fc22 NetworkManager-vpnc-1.0.6-3.fc22 network-manager-applet-1.0.6-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-10143

Comment 19 marianne@tuxette.fr 2015-09-29 15:07:12 UTC

Can you also push the update for fedora 23 ?

Comment 20 Lubomir Rintel 2015-09-30 10:38:28 UTC

Yeah, done.

Comment 21 marianne@tuxette.fr 2015-09-30 11:29:50 UTC

I can't find the update in bodhi (I wanted to test and give karma to help it reach users faster). Do you have the bodhi URL (like it is given for fedora 22 by Fedora Uppdate System) ?

Comment 22 Marcel Ziswiler 2015-09-30 11:36:25 UTC

I manually installed it from https://bodhi.fedoraproject.org/updates/FEDORA-2015-10143 but it was missing libnm-gtk:

error: Failed dependencies:
	libnm-gtk.so.0(libnm_gtk_1_0_6)(64bit) is needed by NetworkManager-openvpn-1:1.0.6-3.fc22.x86_64
	libnm-gtk.so.0(libnm_gtk_1_0_6)(64bit) is needed by NetworkManager-openvpn-gnome-1:1.0.6-3.fc22.x86_64
	libnm-gtk.so.0(libnm_gtk_1_0_6)(64bit) is needed by NetworkManager-vpnc-gnome-1:1.0.6-3.fc22.x86_64

I then just dash-dash-nodepsed that away and my VPN is working again. Hope this gets released through regular update channels soon. Thanks!

Comment 23 marianne@tuxette.fr 2015-10-01 18:36:40 UTC

Updated packages available in Fedora repos (not fedora-updates-testing, I don't know why) are not working for me.
Update don't appear in bodhi 
I'm using the openconnect client (Cisco Anyconnect VPN server)

actual packages version :

NetworkManager-wifi-1.0.6-6.fc23.x86_64
NetworkManager-l2tp-0.9.8.7-4.fc23.x86_64
NetworkManager-adsl-1.0.6-6.fc23.x86_64
NetworkManager-bluetooth-1.0.6-6.fc23.x86_64
NetworkManager-pptp-1.1.0-2.20150428git695d4f2.fc23.x86_64
NetworkManager-config-connectivity-fedora-1.0.6-6.fc23.x86_64
NetworkManager-team-1.0.6-6.fc23.x86_64
NetworkManager-wwan-1.0.6-6.fc23.x86_64
NetworkManager-vpnc-gnome-1.0.6-3.fc23.x86_64
NetworkManager-openvpn-1.0.6-3.fc23.x86_64
NetworkManager-vpnc-1.0.6-3.fc23.x86_64
NetworkManager-1.0.6-6.fc23.x86_64
NetworkManager-pptp-gnome-1.1.0-2.20150428git695d4f2.fc23.x86_64
NetworkManager-openvpn-gnome-1.0.6-3.fc23.x86_64
NetworkManager-libnm-1.0.6-6.fc23.x86_64
NetworkManager-glib-1.0.6-6.fc23.x86_64
NetworkManager-openconnect-1.0.2-2.fc23.x86_64

Comment 24 Julien HENRY 2015-10-01 19:50:13 UTC

Any chance to backport the fix on Fedora 21?

Comment 25 Fedora Update System 2015-10-02 03:49:46 UTC

NetworkManager-1.0.6-6.fc22, NetworkManager-openswan-1.0.6-2.fc22, NetworkManager-openvpn-1.0.6-3.fc22, NetworkManager-vpnc-1.0.6-3.fc22, network-manager-applet-1.0.6-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update NetworkManager-openswan network-manager-applet NetworkManager NetworkManager-openvpn NetworkManager-vpnc'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-10143

Comment 26 Steve Storey 2015-10-02 16:51:53 UTC

I can confirm that the updated packages work fine for me - thank you! This bug can be closed as far as I'm concerned - Do I close it?

Comment 27 Michael Cronenworth 2015-10-02 17:43:07 UTC

The bug will be automatically closed once it has been released to the stable branch.

Comment 28 Fedora Update System 2015-10-04 22:52:28 UTC

NetworkManager-1.0.6-6.fc22, NetworkManager-openswan-1.0.6-2.fc22, NetworkManager-openvpn-1.0.6-3.fc22, NetworkManager-vpnc-1.0.6-3.fc22, network-manager-applet-1.0.6-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.