Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1656157 - NetworkManager 1.12.6 breaks brcmfmac43241b4-sdio wifi (WPA2)
Summary: NetworkManager 1.12.6 breaks brcmfmac43241b4-sdio wifi (WPA2)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Beniamino Galvani
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-04 20:52 UTC by Hans de Goede
Modified: 2018-12-15 03:18 UTC (History)
10 users (show)

Fixed In Version: NetworkManager-1.12.6-2.fc29 NetworkManager-1.12.6-3.fc29
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-15 03:18:39 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)
NetworkManager.service log (37.83 KB, text/plain)
2018-12-04 20:52 UTC, Hans de Goede 		no flags Details
wpa_supplicant.service log (2.33 KB, text/plain)
2018-12-04 20:53 UTC, Hans de Goede 		no flags Details
NetworkManager.service log (589.83 KB, text/plain)
2018-12-06 14:07 UTC, Hans de Goede 		no flags Details
wpa_supplicant.service log (203.44 KB, text/plain)
2018-12-06 14:07 UTC, Hans de Goede 		no flags Details
[PATCH] device: always take device down when changing MAC for wifi devices (1.66 KB, patch)
2018-12-06 16:10 UTC, Beniamino Galvani 		no flags Details | Diff
[PATCH v2] device: always take device down when changing MAC for wifi devices (1.59 KB, patch)
2018-12-10 10:31 UTC, Beniamino Galvani 		no flags Details | Diff
Show Obsolete (3) View All

Description Hans de Goede 2018-12-04 20:52:27 UTC 
Created attachment 1511466 [details]
NetworkManager.service log

After updating from NetworkManager 1.12.4 to 1.12.6 the wifi on a Lenovo Miix 2 8, using a brcmfmac43241b4-sdio nic, no longer connects.

Instead it errors out that credentials are required even if a passwd is specified on the nmcli command line.

Downgrading to 1.12.4 restores wifi connectivity.

I'm attaching journalctl -b0 -u Networkmanager.service and wpa_supplicant.service output, let me know if you need any more logs.

Note there are a lot of mywifi connections configured in NM because this is from an USB SSD which I used to test various devices with the latest Fedora, so for each new device/MAC there is a new connection.
Comment 1 Hans de Goede 2018-12-04 20:53:13 UTC 
Created attachment 1511467 [details]
wpa_supplicant.service log
Comment 2 Hans de Goede 2018-12-04 20:54:27 UTC 
p.s. I've already tried upgrading wpa_supplicant to 2.6-20.fc29.x86_64 this does not help.
Comment 3 Beniamino Galvani 2018-12-04 21:34:11 UTC 
Hi,

can you please try adding this at the end of /etc/NetworkManager/NetworkManager.conf:

[device-mac-addr-change-wifi]
wifi.scan-rand-mac-address=no

and restart the NetworkManager service?

I suspect the regression is caused by commit

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/commit/b81648fc6c272e0f2bf3116168844737eecfa354

because wpa_supplicant assumes that the MAC address can only change when the interface is down. To verify this can you please capture wpa_supplicant logs without the above workaround after adding the '-dd' option OTHER_ARGS in /etc/sysconfig/wpa_supplicant (and restarting the service)? Also, NetworkManager logs with trace level would be helpful (set level=trace in the [logging] service of /etc/NetworkManager/NetworkManager.conf). Thanks.
Comment 4 Hans de Goede 2018-12-06 14:04:37 UTC 
Created attachment 1512145 [details]
NetworkManager.service log
Comment 5 Hans de Goede 2018-12-06 14:05:08 UTC 
Created attachment 1512146 [details]
wpa_supplicant.service log
Comment 6 Hans de Goede 2018-12-06 14:07:00 UTC 
Created attachment 1512148 [details]
NetworkManager.service log
Comment 7 Hans de Goede 2018-12-06 14:07:26 UTC 
Created attachment 1512149 [details]
wpa_supplicant.service log
Comment 8 Hans de Goede 2018-12-06 14:09:44 UTC 
Hi,

I've attached new logs of a fresh boot with the new NM + connect failure using the requested debug options.

After gathering these logs I've added:

[device-mac-addr-change-wifi]
wifi.scan-rand-mac-address=no

And restarted NM, I can confirm that this fixes / works around the problem.

Regards,

Hans
Comment 9 Beniamino Galvani 2018-12-06 16:10:53 UTC 
Created attachment 1512179 [details]
[PATCH] device: always take device down when changing MAC for wifi devices
Comment 10 Thomas Haller 2018-12-06 17:44:04 UTC 
(In reply to Beniamino Galvani from comment #9)
> Created attachment 1512179 [details]
> [PATCH] device: always take device down when changing MAC for wifi devices

was_taken_down is left uninitalized.


Also, while a suitable workaround, this is ultimately a bug in supplicant. It should be at least reported, preferably fixed. Also, it may not be a suitable workaround for macsec or 802-1x.
Comment 11 Beniamino Galvani 2018-12-10 10:31:55 UTC 
Created attachment 1513004 [details]
[PATCH v2] device: always take device down when changing MAC for wifi  devices

(In reply to Thomas Haller from comment #10)
> Also, while a suitable workaround, this is ultimately a bug in supplicant.
> It should be at least reported, preferably fixed.

I think the patch should be applied to NM in any case; even if we try to submit wpa_supplicant patches to improve this (which is not a trivial task), they will probably take long time (months?) to appear in a stable release. If they are accepted, which is not granted.

> Also, it may not be a suitable workaround for macsec or 802-1x.

For MACsec and 802.1X Ethernet we create the supplicant interface only when the connection is being activated and we do it in stage2, after the MAC was changed in stage1. So there isn't any MAC change while wpa_supplicant is managing the interface.

For Wi-Fi it is different because we need to scan and so the supplicant interface is created when the device is managed. Then we change the MAC on activation in stage1.
Comment 12 Thomas Haller 2018-12-10 11:02:09 UTC 
v2 lgtm
Comment 13 Thomas Haller 2018-12-10 14:52:17 UTC 
Merged upstream:

master:  https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=29e8f6d5a17c3dbfd11655338cd0ffc61e1fc91b
nm-1-14: https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=b2686110ef9cfcd87470d65872abf4adb9dab37c
nm-1-12: https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=0624814af17fb66826fda0e901c3d73a292ff37e


and fixed in NetworkManager-1.12.6-2.fc29
Comment 14 Fedora Update System 2018-12-11 13:21:08 UTC 
NetworkManager-1.12.6-3.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-3edec78f47
Comment 15 Fedora Update System 2018-12-12 03:33:20 UTC 
NetworkManager-1.12.6-3.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-3edec78f47
Comment 16 Hans de Goede 2018-12-13 13:19:52 UTC 
I can confirm that the 1.12.6-3 build fixes this (wifi no longer working on a device with a brcmfmac43241b4-sdio nic).
Comment 17 Fedora Update System 2018-12-15 03:18:39 UTC 
NetworkManager-1.12.6-3.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.
Note You need to log in before you can comment on or make changes to this bug.