Bug 2188280

Please attach a full log ('journalctl -b -o short-monotonic --no-hostname').

from journal log:
kernel: bridge0: port 1(vnet0) entered blocking state
kernel: bridge0: port 1(vnet0) entered disabled state
kernel: bridge0: port 1(vnet0) entered blocking state
kernel: bridge0: port 1(vnet0) entered forwarding state
kernel: IPv6: ADDRCONF(NETDEV_CHANGE): bridge0: link becomes ready
NetworkManager[676]: <info>  [1681982573.9089] device (bridge0): carrier: link connected
NetworkManager[676]: <info>  [1681982573.9195] device (bridge0): bridge port vnet0 was attached
NetworkManager[676]: <info>  [1681982573.9197] dhcp4 (bridge0): activation: beginning transaction (timeout in 45 seconds)
kernel: bridge0: port 2(vnet1) entered blocking state
kernel: bridge0: port 2(vnet1) entered disabled state
kernel: bridge0: port 2(vnet1) entered blocking state
kernel: bridge0: port 2(vnet1) entered forwarding state
NetworkManager[676]: <info>  [1681982574.4039] device (bridge0): bridge port vnet1 was attached
kernel: bridge0: port 3(vnet2) entered blocking state
kernel: bridge0: port 3(vnet2) entered disabled state
kernel: bridge0: port 3(vnet2) entered blocking state
kernel: bridge0: port 3(vnet2) entered forwarding state
NetworkManager[676]: <info>  [1681982574.8577] device (bridge0): bridge port vnet2 was attached
kernel: bridge0: port 4(enp3s0f0) entered blocking state
kernel: bridge0: port 4(enp3s0f0) entered disabled state
kernel: bridge0: port 4(enp3s0f0) entered blocking state
kernel: bridge0: port 4(enp3s0f0) entered forwarding state
NetworkManager[676]: <info>  [1681982576.9356] device (bridge0): attached bridge port enp3s0f0
NetworkManager[676]: <info>  [1681982583.7999] dhcp4 (bridge0): state changed new lease, address=x.x.x.219
NetworkManager[676]: <info>  [1681982583.8004] policy: set 'bridge0' (bridge0) as default for IPv4 routing and DNS
systemd-resolved[612]: bridge0: Bus client set search domain list to: fredhs.net
systemd-resolved[612]: bridge0: Bus client set default route setting: yes
systemd-resolved[612]: bridge0: Bus client set DNS server list to: x.x.x.51, x.x.x.52
NetworkManager[676]: <info>  [1681982583.8464] device (bridge0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
NetworkManager[676]: <info>  [1681982583.8517] device (bridge0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
NetworkManager[676]: <info>  [1681982583.8547] device (bridge0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
NetworkManager[676]: <info>  [1681982583.8574] device (bridge0): Activation: successful, device activated.

Looks like NetworkManager add virtual nic to bridge before physical nic.That explain some of the behavior.

*full log*, please.

Created attachment 1958541 [details]
requested log

with MacAddressPolicy=none, the bridge's MAC address is determined by kernel. Which chooses it based on the attached ports, and in particular on the order in which ports get attached.

In your case, the bridge and enp3s0f0 interfaces are configured by NetworkManager, while the vnet[012] interfaces are attached by somebody else (libvirt?)

That somebody is faster than NetworkManager, hence the first interface to be attached is vnet0.

> Predictable mac is used for dhcp request. This was one of main reasons for this change.

What you actually get with MacAddressPolicy=none (and no explicit configuration of the bridge's MAC address otherwise), is that the MAC address of the bridge is *one* of its ports. The distinction whether the port is a virtual nic does not matter to kernel.

Maybe a simple solution is to actually configure the MAC address that you want? Like `nmcli connection modify bridge0 ethernet.cloned-mac-address a8:60:b6:1d:86:17` ?

it's slightly more involved...

Kernel will assign the "lowest" MAC address to the bridge. This happened, you can see "a8:*" was assigned, which is sorted before the others.

However, before that happened, NetworkManager already started doing DHCP on that interface, at a time when it still had the other MAC address. Hence, the DHCP request uses a different MAC address.


For a short time, there was a patch in NetworkManager to restart DHCP when the MAC address changes ([1]). But that caused other problems (like the IP address changing, when a port gets attached), so it was reverted. I think the solution is still to either configure the bridge's "ethernet.cloned-mac-address" or "ipv4.dhcp-client-id" explicitly.

[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/98197386078a471ed993d009e34e08a6aa7c22d9

One problem that https://fedoraproject.org/wiki/Changes/MAC_Address_Policy_none addresses, is that when you setup a bridge in your data-center, then the bridge's MAC address and DHCP client ID will be based on *one* of the known, physical MAC addresses of that machine. So you can configure your DHCP server based on that knowledge and based on the inventory of the machine.

It does not solve the problem, that if you don't specify the MAC address, that it will in all cases automagically choose the desired configuration. It seems, you need some explicit configuration here.

`nmcli connection modify bridge0 ethernet.cloned-mac-address a8:60:b6:1d:86:17` mitigate my issue. This will add following to /etc/NetworkManager/system-connections/bridge0.nmconnection

[ethernet]
cloned-mac-address=A8:60:B6:1D:86:17

I read you already have had a long discussion about "MAC Address Policy none". Probably not wise to rehash it.

The bridge was created with cockpit, so it may be good idea to tell the cockpit team to set mac when bridge is created.

It is ok for me to close the ticket, and leave it as a documentation.

Thank you both for the investigation.
I'd say that this is a poster child for why MACAddressPolicy=persistent was introduced.
But as it was said, we already had a long discussion about this.

Since everything is working as designed, let's close this.