(In reply to Florian Weimer from comment #0)
>
> The symptoms are very similar to bug 1848943.

yes, it's exactly the same issue as bug 1848943.


So, historically with dracut's network module, ip=none means to configure static addresses (if any).
The dracut module would however not reset "accept_ra" sysctl, so this mode also meant to do autoconf.

Now we use NetworkManager's dracut module.

Originally, we set either ipv6.method=link-local or ipv6.method=disabled, depending on whether static IPv6 addresses are configured.
That was the regression in bug 1848943, and the fix was to set ipv6.method=auto.

But that causes different problems, for example NetworkManager will wait up to 30 seconds to get an IPv6 address (unless it already got an IPv4 address earlier). That is undesirable (bug 1883958). Also, it seems wrong to configure ip=none and still have NetworkManager doing autoconf. So, we decided to break bug-compatibility and do [1].

[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/fc7c83cbdd629f2c3573c471573577277b16e9c7


This is the reason for this bug and for regressing bug 1848943. The proposed solution how to handle this with 8.4 was to document it in the release notes. This is tracked by bug 1883958.


Yes, it's a change in behavior and it's painful. If you think this must not be done, then please discuss.




What makes this especially painful is that ipv6.method=disabled in NetworkManager means to set /proc/sys/net/ipv6/conf/$IFNAME/disable_ipv6. That means, if you later run your scripts to setup the interface, you would have to enable it first -- which may not be obvious.

Thanks Florian.

I was asking, because the command line in comment 1 does not have ip=...:none, so I was confused whether this is the same issue.

But the latest run (comment 9), under "Installation" it seems to have command line 

>  DASD=120,121,122,123 MACADDR=02:DE:BD:BE:EF:83 RUNKS=1 ip=10.16.69.42::10.16.71.254:21:ibm-z-131.rhts.eng.bos.redhat.com:enc8000:none::02:DE:BD:BE:EF:83 ks=http://beaker.engineering.redhat.com/kickstart/7777424 ksdevice=bootif nameserver=10.11.5.19 netbootloader= ramdisk_size=40000 rd.znet=qeth,0.0.8000,0.0.8001,0.0.8002,layer2=1,portname=z-131,portno=0 ro

But when I then look at dmesg logfile, it says:

[    0.241539] Kernel command line: root=/dev/mapper/rhel_ibm--z--131-root crashkernel=auto rd.dasd=0.0.0120 rd.dasd=0.0.0121 rd.dasd=0.0.0122 rd.dasd=0.0.0123 rd.lvm.lv=rhel_ibm-z-131/root rd.lvm.lv=rhel_ibm-z-131/swap rd.znet=qeth,0.0.8000,0.0.8001,0.0.8002,layer2=1,portname=z-131,portno=0 BOOT_IMAGE=0

With this configuration, I don't think that NetworkManager should do any network setup in initrd.



Could you install my ssh-key on that machine? "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGO5ve3AN8ynbd6/0DfG0Vm9mVxBKvO0oVERpkqj+sfO thaller". Or just share the password... Or should I somehow be able to see that in beaker? Where?

Thomas, I've added your key.

The root password is the default one, shown under User Preferences in Beaker.

Thanks for investigating this issue.

autopsy report:

the machine is currently running. The interface has disable_ipv6=1, as there is a profile active with ipv6.method=disabled.

That profile is stored in ifcfg format:

>>
# cat /etc/sysconfig/network-scripts/ifcfg-enc8000
MACADDR=02:DE:BD:BE:EF:83
SUBCHANNELS=0.0.8000,0.0.8001,0.0.8002
NETTYPE=qeth
PORTNAME=z-131
OPTIONS="layer2=1 portno=0"
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=10.16.69.42
PREFIX=21
GATEWAY=10.16.71.254
DNS1=10.11.5.19
DEFROUTE=yes
DHCP_HOSTNAME=ibm-z-131.rhts.eng.bos.redhat.com
DHCP_VENDOR_CLASS_IDENTIFIER=anaconda-Linux
IPV4_FAILURE_FATAL=yes
IPV6_DISABLED=yes
IPV6INIT=no
NAME=enc8000
UUID=af51879d-ad25-4c8d-adba-3983d2316743
DEVICE=enc8000
ONBOOT=yes
MULTI_CONNECT=1
DEVTIMEOUT=60
<<

also, the kernel command line is

  Kernel command line: root=/dev/mapper/rhel_ibm--z--131-root crashkernel=auto rd.dasd=0.0.0120 rd.dasd=0.0.0121 rd.dasd=0.0.0122 rd.dasd=0.0.0123 rd.lvm.lv=rhel_ibm-z-131/root rd.lvm.lv=rhel_ibm-z-131/swap rd.znet=qeth,0.0.8000,0.0.8001,0.0.8002,layer2=1,portname=z-131,portno=0 BOOT_IMAGE=0

and (of course), NetworkManager was not running in initrd.



So, I think what happend is that during installation NetworkManager was running in initrd, and with ip=...:none it created a profile in /run with ipv6.method=disabled. Then, Anaconda modified that profile and persisted it to disk -- so that the only profile on the installed system ends up IPv6 disabled too.


@bgalvani, WDYT?

Yes, that matches what I described here:

https://bugzilla.redhat.com/show_bug.cgi?id=1848943#c4

(In reply to Beniamino Galvani from comment #13)
> Yes, that matches what I described here:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1848943#c4

ok.

Then I think this works as expected...

- I guess, it makes sense that anaconda copies the profile from the installation over to the installed system (whatever it is).

- I guess, it also makes sense that we now treat ip=...:none as IPv6 disabled. It's unfortunate change in behavior, but having IPv6 enable also seems wrong on such a system.


The fix would be to update the kernel command line during boot. Or to modify (and reactivate) the profile on the running system.



Does that sound acceptable? bug 1883958 is tracking this and we will to document this change in behavior in the 8.4 documentation.
The alternative is to revert the patch again (and instead WONTFIX bug 1883958 in rhel-8).

I *suspect* the change in behavior disproportionately impacts Red Hat due to the Beaker provisioning system we use. Most customers are likely less impacted. Since this is down to installer behavior, in-place upgrades are not affected.

If that is actually true, changing behavior in 8.4 and documenting it should be acceptable.

But I think we still need to adjust our Beaker provisioning process to restore the old behavior (see the internal thread referenced in comment 7).

Closing as this is a result of a breaking change in NM that will be documented in the RHEL 8.4 release.